Add support for VT paging operations (#16615)

This PR adds support for multiples pages in the VT architecture, along
with new operations for moving between those pages: `NP` (Next Page),
`PP` (Preceding Page), `PPA` (Page Position Absolute), `PPR` (Page
Position Relative), and `PPB` (Page Position Back).

There's also a new mode, `DECPCCM` (Page Cursor Coupling Mode), which
determines whether or not the active page is also the visible page, and
a new query sequence, `DECRQDE` (Request Displayed Extent), which can be
used to query the visible page.

## References and Relevant Issues

When combined with `DECCRA` (Copy Rectangular Area), which can copy
between pages, you can layer content on top of existing output, and
still restore the original data afterwards. So this could serve as an
alternative solution to #10810.

## Detailed Description of the Pull Request / Additional comments

On the original DEC terminals that supported paging, you couldn't have
both paging and scrollback at the same time - only the one or the other.
But modern terminals typically allow both, so we support that too.

The way it works, the currently visible page will be attached to the
scrollback, and any content that scrolls off the top will thus be saved.
But the background pages will not have scrollback, so their content is
lost if it scrolls off the top.

And when the screen is resized, only the visible page will be reflowed.
Background pages are not affected by a resize until they become active.
At that point they just receive the traditional style of resize, where
the content is clipped or padded to match the new dimensions.

I'm not sure this is the best way to handle resizing, but we can always
consider other approaches once people have had a chance to try it out.

## Validation Steps Performed

I've added some unit tests covering the new operations, and also done a
lot of manual testing.

Closes #13892
Tests added/passed

Author: j4james

.github/actions/spelling/expect/expect.txt

@@ -406,6 +406,7 @@ DECNKM
 DECNRCM
 DECOM
 decommit
+DECPCCM
 DECPCTERM
 DECPS
 DECRARA
@@ -414,6 +415,7 @@ DECREQTPARM
 DECRLM
 DECRPM
 DECRQCRA
+DECRQDE
 DECRQM
 DECRQPSR
 DECRQSS
@@ -2123,6 +2125,7 @@ XIn
 XManifest
 XMath
 xorg
+XPan
 XResource
 xsi
 xstyler
@@ -2142,6 +2145,7 @@ YCast
 YCENTER
 YCount
 YLimit
+YPan
 YSubstantial
 YVIRTUALSCREEN
 YWalk

src/cascadia/TerminalCore/Terminal.hpp

@@ -131,8 +131,7 @@ class Microsoft::Terminal::Core::Terminal final :
     // These methods are defined in TerminalApi.cpp
     void ReturnResponse(const std::wstring_view response) override;
     Microsoft::Console::VirtualTerminal::StateMachine& GetStateMachine() noexcept override;
-    TextBuffer& GetTextBuffer() noexcept override;
-    til::rect GetViewport() const noexcept override;
+    BufferState GetBufferAndViewport() noexcept override;
     void SetViewportPosition(const til::point position) noexcept override;
     void SetTextAttributes(const TextAttribute& attrs) noexcept override;
     void SetSystemMode(const Mode mode, const bool enabled) noexcept override;

src/cascadia/TerminalCore/TerminalApi.cpp

@@ -34,14 +34,9 @@ Microsoft::Console::VirtualTerminal::StateMachine& Terminal::GetStateMachine() n
     return *_stateMachine;
 }
 
-TextBuffer& Terminal::GetTextBuffer() noexcept
+ITerminalApi::BufferState Terminal::GetBufferAndViewport() noexcept
 {
-    return _activeBuffer();
-}
-
-til::rect Terminal::GetViewport() const noexcept
-{
-    return til::rect{ _GetMutableViewport().ToInclusive() };
+    return { _activeBuffer(), til::rect{ _GetMutableViewport().ToInclusive() }, !_inAltBuffer() };
 }
 
 void Terminal::SetViewportPosition(const til::point position) noexcept

src/cascadia/UnitTests_TerminalCore/SelectionTest.cpp

@@ -44,6 +44,11 @@ namespace TerminalCoreUnitTests
             VERIFY_ARE_EQUAL(selection, expected);
         }
 
+        TextBuffer& GetTextBuffer(Terminal& term)
+        {
+            return term.GetBufferAndViewport().buffer;
+        }
+
         TEST_METHOD(SelectUnit)
         {
             Terminal term{ Terminal::TestDummyMarker{} };
@@ -394,7 +399,7 @@ namespace TerminalCoreUnitTests
             const auto burrito = L"\xD83C\xDF2F";
 
             // Insert wide glyph at position (4,10)
-            term.GetTextBuffer().GetCursor().SetPosition({ 4, 10 });
+            GetTextBuffer(term).GetCursor().SetPosition({ 4, 10 });
             term.Write(burrito);
 
             // Simulate click at (x,y) = (5,10)
@@ -417,7 +422,7 @@ namespace TerminalCoreUnitTests
             const auto burrito = L"\xD83C\xDF2F";
 
             // Insert wide glyph at position (4,10)
-            term.GetTextBuffer().GetCursor().SetPosition({ 4, 10 });
+            GetTextBuffer(term).GetCursor().SetPosition({ 4, 10 });
             term.Write(burrito);
 
             // Simulate click at (x,y) = (5,10)
@@ -440,11 +445,11 @@ namespace TerminalCoreUnitTests
             const auto burrito = L"\xD83C\xDF2F";
 
             // Insert wide glyph at position (4,10)
-            term.GetTextBuffer().GetCursor().SetPosition({ 4, 10 });
+            GetTextBuffer(term).GetCursor().SetPosition({ 4, 10 });
             term.Write(burrito);
 
             // Insert wide glyph at position (7,11)
-            term.GetTextBuffer().GetCursor().SetPosition({ 7, 11 });
+            GetTextBuffer(term).GetCursor().SetPosition({ 7, 11 });
             term.Write(burrito);
 
             // Simulate ALT + click at (x,y) = (5,8)
@@ -496,7 +501,7 @@ namespace TerminalCoreUnitTests
 
             // Insert text at position (4,10)
             const std::wstring_view text = L"doubleClickMe";
-            term.GetTextBuffer().GetCursor().SetPosition({ 4, 10 });
+            GetTextBuffer(term).GetCursor().SetPosition({ 4, 10 });
             term.Write(text);
 
             // Simulate double click at (x,y) = (5,10)
@@ -540,7 +545,7 @@ namespace TerminalCoreUnitTests
 
             // Insert text at position (4,10)
             const std::wstring_view text = L"C:\\Terminal>";
-            term.GetTextBuffer().GetCursor().SetPosition({ 4, 10 });
+            GetTextBuffer(term).GetCursor().SetPosition({ 4, 10 });
             term.Write(text);
 
             // Simulate click at (x,y) = (15,10)
@@ -568,7 +573,7 @@ namespace TerminalCoreUnitTests
 
             // Insert text at position (4,10)
             const std::wstring_view text = L"doubleClickMe dragThroughHere";
-            term.GetTextBuffer().GetCursor().SetPosition({ 4, 10 });
+            GetTextBuffer(term).GetCursor().SetPosition({ 4, 10 });
             term.Write(text);
 
             // Simulate double click at (x,y) = (5,10)
@@ -597,7 +602,7 @@ namespace TerminalCoreUnitTests
 
             // Insert text at position (21,10)
             const std::wstring_view text = L"doubleClickMe dragThroughHere";
-            term.GetTextBuffer().GetCursor().SetPosition({ 4, 10 });
+            GetTextBuffer(term).GetCursor().SetPosition({ 4, 10 });
             term.Write(text);
 
             // Simulate double click at (x,y) = (21,10)
@@ -685,7 +690,7 @@ namespace TerminalCoreUnitTests
 
             // Insert text at position (4,10)
             const std::wstring_view text = L"doubleClickMe dragThroughHere";
-            term.GetTextBuffer().GetCursor().SetPosition({ 4, 10 });
+            GetTextBuffer(term).GetCursor().SetPosition({ 4, 10 });
             term.Write(text);
 
             // Step 1: Create a selection on "doubleClickMe"

src/cascadia/UnitTests_TerminalCore/TerminalApiTest.cpp

@@ -152,7 +152,8 @@ void TerminalApiTest::CursorVisibility()
     VERIFY_IS_TRUE(term._mainBuffer->GetCursor().IsOn());
     VERIFY_IS_TRUE(term._mainBuffer->GetCursor().IsBlinkingAllowed());
 
-    term.GetTextBuffer().GetCursor().SetIsVisible(false);
+    auto& textBuffer = term.GetBufferAndViewport().buffer;
+    textBuffer.GetCursor().SetIsVisible(false);
     VERIFY_IS_FALSE(term._mainBuffer->GetCursor().IsVisible());
     VERIFY_IS_TRUE(term._mainBuffer->GetCursor().IsOn());
     VERIFY_IS_TRUE(term._mainBuffer->GetCursor().IsBlinkingAllowed());

src/host/outputStream.cpp

@@ -52,25 +52,16 @@ StateMachine& ConhostInternalGetSet::GetStateMachine()
 }
 
 // Routine Description:
-// - Retrieves the text buffer for the active output buffer.
+// - Retrieves the text buffer and virtual viewport for the active output
+//   buffer. Also returns a flag indicating whether it's the main buffer.
 // Arguments:
 // - <none>
 // Return Value:
-// - a reference to the TextBuffer instance.
-TextBuffer& ConhostInternalGetSet::GetTextBuffer()
+// - a tuple with the buffer reference, viewport, and main buffer flag.
+ITerminalApi::BufferState ConhostInternalGetSet::GetBufferAndViewport()
 {
-    return _io.GetActiveOutputBuffer().GetTextBuffer();
-}
-
-// Routine Description:
-// - Retrieves the virtual viewport of the active output buffer.
-// Arguments:
-// - <none>
-// Return Value:
-// - the exclusive coordinates of the viewport.
-til::rect ConhostInternalGetSet::GetViewport() const
-{
-    return _io.GetActiveOutputBuffer().GetVirtualViewport().ToExclusive();
+    auto& info = _io.GetActiveOutputBuffer();
+    return { info.GetTextBuffer(), info.GetVirtualViewport().ToExclusive(), info.Next == nullptr };
 }
 
 // Routine Description:

src/host/outputStream.hpp

@@ -32,8 +32,7 @@ class ConhostInternalGetSet final : public Microsoft::Console::VirtualTerminal::
     void ReturnResponse(const std::wstring_view response) override;
 
     Microsoft::Console::VirtualTerminal::StateMachine& GetStateMachine() override;
-    TextBuffer& GetTextBuffer() override;
-    til::rect GetViewport() const override;
+    BufferState GetBufferAndViewport() override;
     void SetViewportPosition(const til::point position) override;
 
     void SetTextAttributes(const TextAttribute& attrs) override;

src/terminal/adapter/DispatchTypes.hpp

@@ -531,6 +531,7 @@ namespace Microsoft::Console::VirtualTerminal::DispatchTypes
         ATT610_StartCursorBlink = DECPrivateMode(12),
         DECTCEM_TextCursorEnableMode = DECPrivateMode(25),
         XTERM_EnableDECCOLMSupport = DECPrivateMode(40),
+        DECPCCM_PageCursorCouplingMode = DECPrivateMode(64),
         DECNKM_NumericKeypadMode = DECPrivateMode(66),
         DECBKM_BackarrowKeyMode = DECPrivateMode(67),
         DECLRMM_LeftRightMarginMode = DECPrivateMode(69),

src/terminal/adapter/ITermDispatch.hpp

@@ -49,6 +49,12 @@ class Microsoft::Console::VirtualTerminal::ITermDispatch
     virtual bool DeleteCharacter(const VTInt count) = 0; // DCH
     virtual bool ScrollUp(const VTInt distance) = 0; // SU
     virtual bool ScrollDown(const VTInt distance) = 0; // SD
+    virtual bool NextPage(const VTInt pageCount) = 0; // NP
+    virtual bool PrecedingPage(const VTInt pageCount) = 0; // PP
+    virtual bool PagePositionAbsolute(const VTInt page) = 0; // PPA
+    virtual bool PagePositionRelative(const VTInt pageCount) = 0; // PPR
+    virtual bool PagePositionBack(const VTInt pageCount) = 0; // PPB
+    virtual bool RequestDisplayedExtent() = 0; // DECRQDE
     virtual bool InsertLine(const VTInt distance) = 0; // IL
     virtual bool DeleteLine(const VTInt distance) = 0; // DL
     virtual bool InsertColumn(const VTInt distance) = 0; // DECIC

src/terminal/adapter/ITerminalApi.hpp

@@ -39,9 +39,15 @@ namespace Microsoft::Console::VirtualTerminal
 
         virtual void ReturnResponse(const std::wstring_view response) = 0;
 
+        struct BufferState
+        {
+            TextBuffer& buffer;
+            til::rect viewport;
+            bool isMainBuffer;
+        };
+
         virtual StateMachine& GetStateMachine() = 0;
-        virtual TextBuffer& GetTextBuffer() = 0;
-        virtual til::rect GetViewport() const = 0;
+        virtual BufferState GetBufferAndViewport() = 0;
         virtual void SetViewportPosition(const til::point position) = 0;
 
         virtual bool IsVtInputEnabled() const = 0;

src/terminal/adapter/InteractDispatch.cpp

@@ -108,7 +108,7 @@ bool InteractDispatch::WindowManipulation(const DispatchTypes::WindowManipulatio
         _api.ShowWindow(false);
         return true;
     case DispatchTypes::WindowManipulationType::RefreshWindow:
-        _api.GetTextBuffer().TriggerRedrawAll();
+        _api.GetBufferAndViewport().buffer.TriggerRedrawAll();
         return true;
     case DispatchTypes::WindowManipulationType::ResizeWindowInCharacters:
         // TODO:GH#1765 We should introduce a better `ResizeConpty` function to
@@ -135,7 +135,7 @@ bool InteractDispatch::WindowManipulation(const DispatchTypes::WindowManipulatio
 bool InteractDispatch::MoveCursor(const VTInt row, const VTInt col)
 {
     // First retrieve some information about the buffer
-    const auto viewport = _api.GetViewport();
+    const auto viewport = _api.GetBufferAndViewport().viewport;
 
     // In VT, the origin is 1,1. For our array, it's 0,0. So subtract 1.
     // Apply boundary tests to ensure the cursor isn't outside the viewport rectangle.