Bug 160982 - LibreOffice doesn't work with Microsoft Narrator
Summary: LibreOffice doesn't work with Microsoft Narrator
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
3.3.0 release
Hardware: All Windows (All)
: medium normal
Assignee: Michael Weghorn
URL: https://listarchives.libreoffice.org/...
Whiteboard: target:25.2.0
Keywords: accessibility
Depends on:
Blocks: a11y-Windows
  Show dependency treegraph
 
Reported: 2024-05-07 20:29 UTC by Ana G
Modified: 2024-10-31 07:33 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ana G 2024-05-07 20:29:38 UTC
Description:
Using Windows 11 Narrator, I was not able to use Libre Office. When I pressed enter on the desktop icon, I couldn't put focus on items like Writer, Calc, etc: Narrator didn't "see" anything. Then I switched screen readers (to NVDA) to open Writer, and using Narrator again, I received no spoken feedback while typing, when using Narrator reading commands, or when trying to access the menus via the alt key. In other words, Libre Office is completely unusable with Narrator. Please improve accessibility for blind users by making Libre Office work with MS Narrator.

Steps to Reproduce:
1. In Windows, start Narrator with ctrl+windows+enter.
2. Open Libre Office.
3. Try using Libre Office with Narrator commands.

Actual Results:
Screen reader is unable to focus on items in initial screen/menu so no useful spoken feedback is provided. Within the individual apps, no spoken feedback is provided for text/data entry and readback, and focus issue appears to persist in menus.

Expected Results:
I expect to hear menu content as I navigate. I epxect to interact with menus and menu items. I expect to hear text/data during and after entry. In other words, I expect to use Libre Office with Narrator as I do with other screen readers, like NVDA.


Reproducible: Always


User Profile Reset: Yes

Additional Info:
It should be usable by a totally blind person using MS Narrator.
Comment 1 m_a_riosv 2024-05-07 20:45:41 UTC
Have you enabled Menu/Tools/Options/LibreOffice/Accessibility?
Comment 2 V Stuart Foote 2024-05-07 21:54:21 UTC
LibreOffice provides no assistive technology support for Microsoft's UI Automation framework. And MS has chosen to not support IAccessible2 in Narator, just spotty support for legacy MSAA.

LibreOffice fully instruments IAccessible2 API calls on Windows, which NVDA parses as "accessible events" reasonably well. JAWS a bit less so.

For other supported os/DE LibreOffice's cross platform assistive technology support uses NS Accessibility and ATK/AT-SPI on macOS and Linux respective.

Situation on Windows os/DE will not change, as UI Automation framework is not compatible with LibreOffices assistive technology. 

Simply put just use NVDA for reasonable AT experience with LibreOffice.
Comment 3 Michael Weghorn 2024-05-08 05:46:28 UTC
Comment 2 describes the situation well.

It's not completely impossible for LO to one day support UIA, but the clear focus for Windows is currently IAccessible2, so using a screen reader that supports that is currently the only option to get a good experience with LibreOffice.
Comment 4 Michael Weghorn 2024-10-18 16:49:08 UTC
(In reply to V Stuart Foote from comment #2)
> LibreOffice provides no assistive technology support for Microsoft's UI
> Automation framework. And MS has chosen to not support IAccessible2 in
> Narator, just spotty support for legacy MSAA.

It turns out that Windows provides a MSAA to UIA proxy that makes (at least some) information available to UIA without LO having to implement native UIA support. This however wasn't working "out of the box" because Narrator (other than third-party screen readers) doesn't report the SPI_GETSCREENREADER [1] parameter.

When explicitly enabling the accessible bridge by setting an environment variable SAL_FORCE_IACCESSIBLE2=1 before starting LO, Narrator works in general, e.g. announces menus, toolbar buttons etc. just fine when they receive focus.

Pending Gerrit change to also enable the accessible bridge automatically when Narrator or another AT requests information:
https://gerrit.libreoffice.org/c/core/+/175172

Once that change is in, I'll close this bug as fixed. For any particular issues that do not get reported by Narrator, feel free to create separate bug reports. (But as Stuart noted, support may still remain limited for now due to the reasons mentioned earlier.)





[1] https://learn.microsoft.com/en-us/windows/win32/winauto/screen-reader-parameter
Comment 5 Commit Notification 2024-10-19 06:49:42 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/bfbaeb8192447265bdd78d1be4990947d135eb6e

tdf#160982 wina11y: Drop extra screen reader check to enable a11y bridge

It will be available in 25.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 6 Ana G 2024-10-20 14:46:06 UTC
Thank you so much! I can't wait to test 25.2.0.
Comment 7 V Stuart Foote 2024-10-20 16:19:51 UTC
Checked Narrator with 2024-10-20 nightly, AT functions at least as well as the old JAB AT.

Demonstrates how much better NVDA with IAccessible2 support is--of course don't run them both or you get double soundings.

Version: 25.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 341f3618866c622fa511f153a1b48621e04ddd17
CPU threads: 8; OS: Windows 10 X86_64 (10.0 build 19045); UI render: Skia/Vulkan; VCL: win
Locale: en-US (en_US); UI: en-US
Calc: CL threaded
Comment 8 Mike Kaganski 2024-10-30 13:58:34 UTC
(In reply to Commit Notification from comment #5)
> Michael Weghorn committed a patch related to this issue.
> It has been pushed to "master":
> 
> https://git.libreoffice.org/core/commit/
> bfbaeb8192447265bdd78d1be4990947d135eb6e
> 
> tdf#160982 wina11y: Drop extra screen reader check to enable a11y bridge

FTR: exactly after this change, on Windows, with some assistive technology active, I see in Basic IDE, that pressing and holding Enter to add multiple new paragraphs, the current line with the cursor goes beyond the window (previously, the cursor was always at the bottom, and the window expectedly scrolled to keep it visible).
Comment 9 Michael Weghorn 2024-10-30 14:17:16 UTC
(In reply to Mike Kaganski from comment #8)
> FTR: exactly after this change, on Windows, with some assistive technology
> active, I see in Basic IDE, that pressing and holding Enter to add multiple
> new paragraphs, the current line with the cursor goes beyond the window
> (previously, the cursor was always at the bottom, and the window expectedly
> scrolled to keep it visible).

I think that's likely some pre-existing issue that didn't get triggered earlier if the assistive technology doesn't set the SPI_GETSCREENREADER parameter. In that case, the LO a11y bridge on Windows wasn't activated before this commit, but is now (which I think generally makes sense, as something is requesting information via the a11y API).

The same is most likely true for the crash/assert also seen in that scenario (discussed on IRC). With NVDA active (which does set the SPI_GETSCREENREADER param), I see crashes with much older LO versions (at least somewhere in the 7.2 Windows bibisect repo range, but it doesn't seem to be 100% reliable there, so not sure whether spending more time on trying to bibisect is really useful).

I'm taking a look at that.
Comment 10 Michael Weghorn 2024-10-31 07:33:55 UTC
(In reply to Michael Weghorn from comment #9)
> I think that's likely some pre-existing issue that didn't get triggered
> earlier if the assistive technology doesn't set the SPI_GETSCREENREADER
> parameter. In that case, the LO a11y bridge on Windows wasn't activated
> before this commit, but is now (which I think generally makes sense, as
> something is requesting information via the a11y API).
> 
> The same is most likely true for the crash/assert also seen in that scenario
> (discussed on IRC). With NVDA active (which does set the SPI_GETSCREENREADER
> param), I see crashes with much older LO versions (at least somewhere in the
> 7.2 Windows bibisect repo range, but it doesn't seem to be 100% reliable
> there, so not sure whether spending more time on trying to bibisect is
> really useful).
> 
> I'm taking a look at that.

Tracked in bug 163697 now.