Improve Automatic Speech Output for Screen Readers

[slannon97]

I'm totally blind and use Lectrote on Windows 11 with the NVDA screen reader. Most of the time, the output from my commands is automatically read. However, there are times when NVDA appears to start reading information from the top of the screen after entering a command instead of the output from that command. This means I have to manually read the output. Is there a way to fix this, such as using https://github.com/dkager/tolk or possibly customizing how the program handles live regions? I'm not sure how the current implementation works, though it might be using ARIA live regions. Perhaps the easiest thing to do is always use a live region containing the latest output information which the screen reader should automatically read. I'm not a programmer, so I'm not sure how much information I can provide, but please let me know if you need anything, and I'll do my best to provide it.

[slannon97]

I'm still experiencing this issue with version 1.5. Lectrote should automatically read the output from the last command entered, or immediately after loading a new game file. Is this possible to implement using ARIA live regions? This would also allow menu items to be automatically read. For example, when playing the 350 point Inform port of Colossal Cave Adventure and typing help, the screen reader doesn't read anything when pressing n and p to navigate through the menu of options. If this isn't a Lectrote issue, I can open it in the NVDA GitHub repository, but a system for automatic reading would be helpful for all screen readers on all platforms. I've also tried using Orca on Linux, and the output is read very strangely. I can of course manually review the screen, but this isn't always practical.

[erkyrath]

I'm afraid I haven't looked at this recently.

Lecrote is entirely implemented in Javascript and HTML, so ARIA is the only available solution.

The current implementation uses ARIA live regions, but in a simplistic way. The story window area has aria-live: polite; aria-atomic: false; aria-relevant: additions. The status window (which includes the Colossal Cave menu) does not.

I could try to improve the ARIA handling, but I don't have a good sense for how it should work.

[slannon97]

I'm not a programmer, but from my very limited research into those ARIA attributes, it looks like it should be reading only the updated content if the atomic setting is set to false and relevant is set to additions. I don't understand why it sometimes works correctly and other times it doesn't. Adding the attributes to the status window would be very helpful. I can navigate in those game menus and content is automatically read after pressing enter, but I have no idea what options I'm selecting when I press n, p, or enter to go into submenus.
Perhaps the best solution is to set the live attribute to assertive. This usually isn't recommended because it interrupts what's being spoken, but in this case, that's not a bad thing since I want to immediately know what happens after executing a command, and can't see the screen since i have no usable vision at all. This might also solve the problem where the intro text isn't always read when loading a game for the first time or after clearing its save data.

Let me know if providing an audio recording would be helpful. You can of course test using NVDA as well, but it might be helpful.

[slannon97]

Version 1.51 doesn't fix the problem. When I go into help menus, it now reads all the menu items in a large chunk every time when pressing n and p rather than just the one that's selected. Can you make it to where the live region tells the screen reader to announce the currently selected item only?

[erkyrath]

Sorry -- I did not change anything to do with ARIA or the screen arrangement in 1.5.0 or 1.5.1.

[slannon97]

These automatic reading issues are still present in the latest version. It appears to work fine for a while, but then NVDA starts reading information from the beginning of the output rather than the latest information that was generated from the last command. It's also still very difficult to use menus where you press n or p to move between options. The screen reader reads the entire menu when you press these commands rather than telling you what option you've moved to. Manually reading the output is a workaround and can be useful, but it shouldn't be required to play. I'm curious if my suggestion of using a live region containing the latest output information would solve the issue?