Bug 158030 - Consider exposing the end-user-consumable coordinates via {row,col}indextext object attribute
Summary: Consider exposing the end-user-consumable coordinates via {row,col}indextext ...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
unspecified
Hardware: All All
: medium normal
Assignee: Michael Weghorn
URL:
Whiteboard: target:24.2.0 target:24.8.0
Keywords:
Depends on:
Blocks: a11y-Linux
  Show dependency treegraph
 
Reported: 2023-11-01 16:50 UTC by Joanmarie Diggs
Modified: 2024-02-29 11:43 UTC (History)
1 user (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Joanmarie Diggs 2023-11-01 16:50:23 UTC
Calc currently exposes the coordinates as the accessible name of the cell ("A1", "B1", etc.)

What Google Sheets does is to expose no name on the cell itself and use the aria-rowindextext and aria-coindextext attributes instead*. See https://w3c.github.io/aria/#aria-rowindextext

This solution makes it possible for the accessible name of the cell to be set to something more name-like while still making it possible for screen readers to present the human-friendly/on-screen cell coordinates.

There may be old ATs that still expect the coordinates to be exposed as the name, so I'm not sure how safe it would be to remove the coordinates from the name. But hopefully nothing will break if you also expose the coordinates as indicated in the ARIA spec.

Making this change would facilitate removing some LO-specific handling for presenting coordinates, row labels, and column labels.

*Technically Sheets currently uses aria-rowtext and aria-coltext, which are not a thing. The ARIA feature was created for this use case and it appears Google Sheets needs to be updated/fixed.
Comment 1 Michael Weghorn 2023-11-01 20:50:56 UTC
Setting to NEW, since this generally sounds reasonable.

The Core Accessibility API Mappings 1.2 suggest to map the "aria-rowindextext" attribute to "rowindextext" for all of IAccessible2, UIA and AT-SPI: [1]
It might therefore even be possible to just implement that right in the cell object rather than having to map to different attributes on different platforms.

IMHO the problem/blocker is that only Gtk 3 currently seems to support setting object attributes on Linux, while Gtk 4 and Qt don't, see bug 155447 comment 2 for more details.



[1] https://www.w3.org/TR/core-aam-1.2/#ariaRowIndexText
Comment 2 Joanmarie Diggs 2023-11-06 12:37:04 UTC
I just filed https://gitlab.gnome.org/GNOME/gtk/-/issues/6196. Let's see what they say.
Comment 3 Michael Weghorn 2023-11-10 18:40:12 UTC
(In reply to Joanmarie Diggs from comment #2)
> I just filed https://gitlab.gnome.org/GNOME/gtk/-/issues/6196. Let's see
> what they say.

Thanks!
Corresponding report for Qt and first suggestion for an implementation, also inspired by the discussion the Gtk issue so far:

https://bugreports.qt.io/browse/QTBUG-119057

https://codereview.qt-project.org/c/qt/qtbase/+/517525 (new interface)
https://codereview.qt-project.org/c/qt/qtbase/+/517526 (AT-SPI bridge)
Comment 4 Joanmarie Diggs 2023-11-13 17:44:43 UTC
Nice!

BTW, I just landed a change in Orca master which causes it to look for rowindextext and colindextext for Calc. Hopefully that will allow you to test your prototype.

Currently if I up arrow in Calc, I see this in my debugging output:

> (timestamp) - AXTable: Coordinates label for [table cell: 'C1']: 

For Google sheets, I see this:

> (timestamp) - AXTable: Coordinates label for [table cell]: 
> (timestamp) - AXTable: Updated coordinates label based on [table row]: C1

The reason why is that the ARIA rowindextext property is supported on both cell and row. Related aside: Orca currently takes an all-or-nothing approach: Either both indices are provided or it doesn't use an alternative label for the coordinates. Whether or not Orca should be that picky is TBD. :)

Anyhoo, Sheets exposes accessible row objects, so Orca checks there for the rowindextext before giving up. The rowindextext is indeed exposed by Sheets on the row, and we got the colindextext from the cell, so we're good.

What I expect to see in the case of Calc, which does not expose accessible row objects, is:

> (timestamp) - AXTable: Coordinates label for [table cell: 'C1']: C1

assuming you continue to expose the coordinates as the accessible name. If you stop doing that, I'd expect:

> (timestamp) - AXTable: Coordinates label for [table cell]: C1

If you have a local implementation and don't see that in your debug.out from Orca, let's talk. :)
Comment 5 Michael Weghorn 2023-11-14 11:41:51 UTC
(In reply to Joanmarie Diggs from comment #4)
> BTW, I just landed a change in Orca master which causes it to look for
> rowindextext and colindextext for Calc. Hopefully that will allow you to
> test your prototype.
> 
> (...)
> 
> If you have a local implementation and don't see that in your debug.out from
> Orca, let's talk. :)

Thanks, works as described for me with a pending Gerrit change that implements reporting of these attributes:
https://gerrit.libreoffice.org/c/core/+/159412

gtk3 "just works", since it just forwards objects attributs to ATK unchanged.

qt6 works with the pending Qt changes and an updated https://gerrit.libreoffice.org/c/core/+/159309 that just forwards custom attributes as is for now (discussing adding new ones to Qt might make more sense once the currently pending changes have been reviewed).

gtk4 didn't have a mapping to AT-SPI2 yet, but should in theory work with https://gerrit.libreoffice.org/c/core/+/159412 and a corresponding Gtk change: https://gitlab.gnome.org/GNOME/gtk/-/merge_requests/6563
(However, event handling isn't implemented yet and LO currently freezes when trying to interact with Calc tables: https://gitlab.gnome.org/GNOME/gtk/-/issues/6204 )

Anyway, since there seems to be a clear way forward for these attributes at least, I'll merge those changes, but would recommend to keep the fallback path in Orca for now.
Comment 6 Joanmarie Diggs 2023-11-14 12:52:45 UTC
Cool!!

Related aside which you probably already thought about, but just in case: The reason Orca master now reliably presents the presence and contents of a formula for Calc cells is because this is now exposed as an object attribute on the cell. (That wasn't always the case AFAIK.)

If Gtk will only allow official properties, "formula" should become one of them if we don't want this to break again.

I think formula is worthy of consideration as an official property. A calculator app's display might use it for instance.
Comment 7 Michael Weghorn 2023-11-14 13:13:58 UTC
(In reply to Joanmarie Diggs from comment #6)
> Related aside which you probably already thought about, but just in case:
> The reason Orca master now reliably presents the presence and contents of a
> formula for Calc cells is because this is now exposed as an object attribute
> on the cell. (That wasn't always the case AFAIK.)

Thanks for the hint. I wouldn't have noticed yet, but an early Orca master adopter actually just mentioned that this works reliably now (bug 155634 comment 6) which made me notice the Orca commit. Really nice! :-)


> If Gtk will only allow official properties, "formula" should become one of
> them if we don't want this to break again.
> 
> I think formula is worthy of consideration as an official property. A
> calculator app's display might use it for instance.

Sounds good to me. Do you want to suggest it there?
(If there's an OK from the Gtk maintainers, I'm happy to come up with a suggestion for an implementation if that helps.)

FWIW, the Windows IAccessible2 API doc even uses formula as its only example for object attributes, which kind of emphasizes its usefulness:
file:///home/michi/development/git/IAccessible2/docs/html/interface_i_accessible2.html#a0d51b0c189a000ee3b6ddf7f68da2009
Comment 8 Commit Notification 2023-11-15 09:11:11 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/c2165ec33c5549dcc494ad9197727e096678fc3c

tdf#158030 sc a11y: Report cell coords via {row,col}indextext obj attr

It will be available in 24.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 9 Commit Notification 2023-11-15 09:11:13 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/df9e08cd765731451bf58b2d23590db7263d49fe

tdf#158030 gtk4 a11y Map {col,row}indextext obj attributes

It will be available in 24.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 10 Michael Weghorn 2023-11-15 09:18:38 UTC
Closing as fixed, since the LibreOffice side is implemented. Qt/Gtk 4 aspects are tracked in the corresponding upstream projects.

IIRC, at least NVDA currently expects the cell name to be the coordinates, so I'd rather leave the accessible name unchanged for now.
Comment 11 Commit Notification 2024-02-29 11:35:26 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/1b6963cda0a209548249dd43c5c1fa705ef32bcd

tdf#158030 qt a11y: Implement new QAccessibleAttributesInterface

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Michael Weghorn 2024-02-29 11:43:53 UTC
(In reply to Michael Weghorn from comment #3)
> Corresponding report for Qt and first suggestion for an implementation, also
> inspired by the discussion the Gtk issue so far:
> 
> https://bugreports.qt.io/browse/QTBUG-119057
> 
> https://codereview.qt-project.org/c/qt/qtbase/+/517525 (new interface)
> https://codereview.qt-project.org/c/qt/qtbase/+/517526 (AT-SPI bridge)

The first Qt change (adding the interface) is merged now and I expect the AT-SPI bridge one to follow soon, have merged the LO change using the new interface now, so this should all work with Qt >= 6.8.