Skip to content

Add a means by which UIA clients can determine whether a build of Windows Terminal does or doesn't support notifications #13601

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
codeofdusk opened this issue Jul 26, 2022 · 8 comments
Assignees
Labels
Area-Accessibility Issues related to accessibility Area-TerminalControl Issues pertaining to the terminal control (input, selection, keybindings, mouse interaction, etc.) Issue-Task It's a feature request, but it doesn't really need a major design. Product-Terminal The new Windows Terminal.
Milestone

Comments

@codeofdusk
Copy link
Contributor

codeofdusk commented Jul 26, 2022

Description of the new feature/enhancement

#12358 adds support for UIA notifications, allowing screen readers and other UIA clients to easily determine exactly what new text to report without doing any of the heavy lifting themselves (diffing, etc..). However, as not all terminals support notifications, UIA clients need a way to determine when this support is available and when they should fall back to performing diffing themseles. (if clients diff even when notifications are available, we risk doing extra work and/or double-reporting incoming text).

Proposed technical implementation details (optional)

  • In Use UIA notifications for text output #12358 and over email at various times, we discussed using a different class name (TermCtrl2) for situations when notifications are available. NVDA supports this new ID today, but there were concerns from JAWS.
  • We also discussed sending an "API version" of sorts through a custom property. This is necessary since wt and conhost versions aren't necessarily tied to particular Windows releases/aren't reliable for feature detection.
@codeofdusk codeofdusk added the Issue-Feature Complex enough to require an in depth planning process and actual budgeted, scheduled work. label Jul 26, 2022
@ghost ghost added Needs-Tag-Fix Doesn't match tag requirements Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting labels Jul 26, 2022
@codeofdusk
Copy link
Contributor Author

codeofdusk commented Jul 26, 2022

Blocking nvaccess/nvda#13781.

CC @carlos-zamora, @LeonarddeR.

@LeonarddeR
Copy link

It could also be considered to add an automationID (it is currently empty)

@zadjii-msft zadjii-msft added Area-Accessibility Issues related to accessibility Needs-Discussion Something that requires a team discussion before we can proceed labels Jul 26, 2022
@zadjii-msft zadjii-msft added this to the Terminal v1.16 milestone Jul 26, 2022
@codeofdusk
Copy link
Contributor Author

It could also be considered to add an automationID (it is currently empty)

@LeonarddeR That idea was rejected during initial implementation.

@carlos-zamora carlos-zamora removed this from the Terminal v1.16 milestone Jul 26, 2022
@carlos-zamora
Copy link
Member

Chatted with @codeofdusk. This is a feature/task. We'll chat more about it at sync but the short version is this:

  • In a world where Terminal doesn't send notifications for text output...
    • NVDA reads the text buffer, performs a diff on what was stored, and reads out what changed.
    • Diffing is (unsurprisingly) pretty expensive! This is a common practice among screen readers.
  • Today, Terminal sends notifications for text output to tell the screen reader exactly what is relevant...
    • NVDA would normally read these notifications, resulting in a "local echo" where it would read the notification and the result of the diff. Not good! So NVDA straight up ignores those notifications.
    • That said, these notifications uncovered a massive opportunity for screen readers! What if they just trusted the notification instead of diffing themselves? That would be a huge performance enhancement!
  • The Problem: NVDA needs some kind of check to know if Terminal will send out these notifications. If that check succeeds, then they know they can trust our notifications, and they can avoid diffing altogether.
    • NOTE: NVDA already does these "checks" on UIA. There's 3 "API Levels":
      • 0 --> "GetVisibleRange" returns one range per line of text --> initial UIA implementation (pre-UIA provider refactor)
      • 1 --> "GetVisibleRange" returns one range total --> after my UIA provider refactor
      • 2 --> text formatting (i.e. color/font) is exposed --> most recent UIA implementation

@zadjii-msft zadjii-msft added Product-Terminal The new Windows Terminal. Issue-Task It's a feature request, but it doesn't really need a major design. and removed Issue-Feature Complex enough to require an in depth planning process and actual budgeted, scheduled work. Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting Needs-Discussion Something that requires a team discussion before we can proceed labels Aug 15, 2022
@ghost ghost removed the Needs-Tag-Fix Doesn't match tag requirements label Aug 15, 2022
@zadjii-msft zadjii-msft added this to the Terminal v1.16 milestone Aug 15, 2022
@zadjii-msft
Copy link
Member

discussion: sure let's do it

@carlos-zamora
Copy link
Member

@codeofdusk here's an idea actually. So, all versions of Windows Terminal in active support going back to v1.12 send notifications. At this point, why not just assume that if NVDA is connected to Windows Terminal, it has support for notifications?

The key part here is "connected to Windows Terminal" because Conhost won't support notifications. So if the class name is TermControl, you know you're connected to Windows Terminal, and by extension, you know we support notifications.

CC @DHowett

@DHowett
Copy link
Member

DHowett commented Aug 16, 2022

Conhost won't support notifications

Until SV2

@carlos-zamora
Copy link
Member

Conhost won't support notifications

Until SV2

Nah. Conhost doesn't use the UiaRenderer. So the whole notifications thing doesn't affect Conhost. It operates the same way it always has (I believe it's attached to the Write API but don't quote me on that one, I'd have to look into it).

@zadjii-msft zadjii-msft added the Area-TerminalControl Issues pertaining to the terminal control (input, selection, keybindings, mouse interaction, etc.) label Dec 19, 2022
@carlos-zamora carlos-zamora modified the milestones: Terminal v1.17, Backlog Jan 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Accessibility Issues related to accessibility Area-TerminalControl Issues pertaining to the terminal control (input, selection, keybindings, mouse interaction, etc.) Issue-Task It's a feature request, but it doesn't really need a major design. Product-Terminal The new Windows Terminal.
Projects
None yet
Development

No branches or pull requests

5 participants