Bug 136760 - AccessibilityUnusable coordinates announced
Summary: AccessibilityUnusable coordinates announced
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
6.4.5.2 release
Hardware: x86-64 (AMD64) Windows (All)
: medium enhancement
Assignee: Michael Weghorn
URL:
Whiteboard: target:7.6.0
Keywords: accessibility
: 140732 (view as bug list)
Depends on:
Blocks: a11y-Windows
  Show dependency treegraph
 
Reported: 2020-09-14 19:49 UTC by Karl-Heinz Arkenau
Modified: 2023-03-20 21:26 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Karl-Heinz Arkenau 2020-09-14 19:49:54 UTC
Description:
Using screenreader NVDA the cooidinates announced in LibreOffice are unsuable because only towonly two  numbers are announced without description what thes numbers are saying.

Steps to Reproduce:
1 Open Writer.
2. Working with screenreader NVDA hit NVDA + Delete. 
3.

Actual Results:
The Screeader announces a two part number, withou anyone knoing what this is about to be.

Expected Results:
A clear position shound be announced (lef, right, high and low) position should be announced.


Reproducible: Always


User Profile Reset: Yes


OpenGL enabled: Yes

Additional Info:
Using NVDA 2020.2
Comment 1 juergenkohler23 2020-09-16 10:23:57 UTC
Using Windows 10, LibreOffice 7.0 and NVDA 2020.2 the problem still occurs.

My exact operating system information:

Windows
Edition	Windows 		10 Home
Version				1909
Operating system build		18363.1016

LibreOffice
Version:	7.0.0.3 (x64)
Build ID: 8061b3e9204bef6b321a21033174034a5e2ea88e
CPU-Threads: 	4; BS: Windows 10.0 Build 18363; UI-Render: Skia/Vulkan; VCL: win
Locale: 	de-DE (de_DE); UI: de-DE
Calc: 		threaded
Comment 2 Michael Weghorn 2021-10-14 06:24:12 UTC
Corresponding issue in NVDA issue tracker: https://github.com/nvaccess/nvda/issues/11696
Comment 3 Michael Weghorn 2023-03-10 10:12:46 UTC
Pasting my comment from https://github.com/nvaccess/nvda/issues/11696#issuecomment-1463575359 :

From a first analysis, I haven't seen any way to expose/retrieve the information at what position in the page the cursor currently is via the accessibility interfaces (`IAccessible2` in the case of LibreOffice).

The `IAccessibleComponent` interface has a way to [get the position in pixels relative to the parent](https://accessibility.linuxfoundation.org/a11yspecs/ia2/docs/html/interface_i_accessible_component.html#a8709fdbf3f51ae079af274854a8bffd4) , but that's not really helpful here. From a first look, the Writer page itself also has no a11y object in the a11y tree.

What NVDA seems to do for MS Word is to use the Microsoft Office COM API to interact with MS Office, not the accessibility APIs:
https://github.com/nvaccess/nvda/blob/d2d4bfce2d31de8fca3c6df16caf93350b26374d/source/NVDAObjects/window/winword.py#L597-L614

It queries for the `wdHorizontalPositionRelativeToPage` and `wdVerticalPositionRelativeToPage` attributes described here:
https://learn.microsoft.com/en-us/office/vba/api/word.wdinformation

This page has some more infos on using the COM with NVDA and Word:
https://github.com/nvaccess/nvda/wiki/Using-COM-with-NVDA-and-Microsoft-Word

So if there's no way to retrieve the information via the accessibility APIs, I see no "easy" way of implementing this to work with LibreOffice Writer as well.

What *might* work is to not use the accessibility APIs, but LibreOffice's UNO API (which also has a Python binding), but adding support for doing this would presumably be quite a lot of work.

Possibly, there are ways to retrieve the required information (or retrieve information from which the position in the page could then be calculated). Potentially helpful UNO interfaces might be (but that would need a closer look):

* `XTextViewCursor::getPosition()` to retrieve cursor pos relative to top left position of the first page of the document
* `XPageCursor::getPage()` can be used to retrieve the page that the cursor is currently on
* `offapi/com/sun/star/style/PageProperties.idl` to access properties of the page,...
* ...

Looking into the implementation of `SwXTextViewCursor::getPosition` (which is the LO-internal implementation of one of the interfaces) might give some more ideas.

If that's an approach to investigate further, it would probably be a good idea to start with a macro to run from within LO to see whether it's possible to calculate the position from the information accessible via the UNO API.
Comment 4 Michael Weghorn 2023-03-10 10:37:40 UTC
Based on the analysis in comment 3, I currently see nothing that can be done on the LO side, so I'm closing this as NOTOURBUG and suggest to keep tracking this in the corresponding NVDA issue for now:
https://github.com/nvaccess/nvda/issues/11696

From what I can see, LO generally seems to do "the right thing" according to the IAccessible2 specification, and reports the coordinates in screen pixels.

If there are any further insights or information I have missed in my analysis or it turns out that something else is needed on LO side, this can of course be reopened.
Comment 5 Karl-Heinz Arkenau 2023-03-12 16:39:06 UTC
*** Bug 154149 has been marked as a duplicate of this bug. ***
Comment 6 Michael Weghorn 2023-03-13 19:43:50 UTC
(In reply to Michael Weghorn from comment #4)
> Based on the analysis in comment 3, I currently see nothing that can be done
> on the LO side, so I'm closing this as NOTOURBUG and suggest to keep
> tracking this in the corresponding NVDA issue for now:
> https://github.com/nvaccess/nvda/issues/11696
> 
> From what I can see, LO generally seems to do "the right thing" according to
> the IAccessible2 specification, and reports the coordinates in screen pixels.
> 
> If there are any further insights or information I have missed in my
> analysis or it turns out that something else is needed on LO side, this can
> of course be reopened.

See the further discussion in the NVDA issue, including a quick demo in https://github.com/nvaccess/nvda/issues/11696#issuecomment-1465006145 : Custom attributes for the document object might be a way to move forward.

-> Reopening this ticket to keep track of this.
Comment 7 Michael Weghorn 2023-03-13 19:44:14 UTC
*** Bug 140732 has been marked as a duplicate of this bug. ***
Comment 8 Michael Weghorn 2023-03-17 12:37:33 UTC
Based on the discussion in the NVDA issue, I have now submitted a change to LibreOffice's Gerrit and a PR For NVDA with a potential implementation:
https://gerrit.libreoffice.org/c/core/+/149051
https://github.com/nvaccess/nvda/pull/14727

Now waiting for feedback/suggestions on these.
Comment 9 Commit Notification 2023-03-20 21:22:13 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/4a93739e619fd4f4ec369932e923554d71297900

tdf#136760 sw a11y: Provide page-relative cursor pos via doc attr

It will be available in 7.6.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 10 Michael Weghorn 2023-03-20 21:26:02 UTC
Both, the NVDA pull request as well as the LibreOffice change have been merged now, so a page-relative position is reported in Writer when using the current development versions of LibreOffice and NVDA.

Example for a locale that uses imperial units:
"cursor positioned 0.79 inches from left edge of page, 0.79 inches from top edge of page"
Otherwise, centimetres are used instead of inches.

-> Closing as FIXED