Bug 140762 - No text spoken by screen reader for "Borders" dropdown button items in Calc's formatting toolbar (NVDA on Windows)
Summary: No text spoken by screen reader for "Borders" dropdown button items in Calc's...
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: UI (show other bugs)
Version:
(earliest affected)
7.2.0.0.alpha0+
Hardware: All Windows (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:7.2.0
Keywords: accessibility
Depends on:
Blocks: a11y
  Show dependency treegraph
 
Reported: 2021-03-02 14:22 UTC by Michael Weghorn
Modified: 2021-03-04 16:40 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
Screencast showing the behaviour (1.11 MB, video/x-matroska)
2021-03-02 14:22 UTC, Michael Weghorn
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Weghorn 2021-03-02 14:22:49 UTC
Created attachment 170180 [details]
Screencast showing the behaviour

Steps to reproduce (see also attached screencast):

0. enable NVDA screen reader
1. start LO Calc
2. Press F6 a few times to enable the Calc formatting toolbar
3. press "tab" a few times to navigate to the "Borders (Shift to override)" dropdown button
4. Press space or Enter key to activate
   * result: a popup opens that offers the different border types
5. press arrow keys to change border types

Actual behaviour:
NVDA screen reader does not say anything in steps 4 and 5

Expected behaviour:

NVDA should announce what border style is currently selected in steps 4 and 5

Version: 7.2.0.0.alpha0+ (x64) / LibreOffice Community
Build ID: 3078044653f1e1b0a055c47eeb8d27834c07268a
CPU threads: 4; OS: Windows 10.0 Build 19042; UI render: Skia/Raster; VCL: win
Locale: en-US (en_US); UI: en-US
Calc: threaded
Comment 1 Michael Weghorn 2021-03-02 14:24:30 UTC
Originally reported on the a11y mailing list:
https://listarchives.libreoffice.org/global/accessibility/msg00966.html

I can reproduce -> setting to NEW

Some further notes/comments:

* When hovering over the items using the mouse in step 5 instead of using the keyboard, NVDA says "item 1", "item 2", "item 3", etc. for the individual choices  (i.e. it announces texts, but they're not helpful).
    * and it also announces something when using the keyboard afterwards
* When using gtk3 VCL plugin on Linux and the Orca screen reader, Orca says "item 1", "item 2", "item 3", etc. in step 5 when navigating using the arrow keys
* The behaviour is similar for other toolbar items, e.g. the "Border Style" combobutton is affected just the same way.
Comment 2 V Stuart Foote 2021-03-02 16:09:33 UTC
Have the general case of bug 113488 to expose the descriptive text when 'accessible events' are focus triggered, but then individual GUI widgets can require some refactoring to expose their controls meaningfully to AT, as here.

Tackle the 'accessible event' firing via keyboard navigation, so movement sequences and focus states for the elements in the widget(s).
Comment 3 Commit Notification 2021-03-04 07:15:50 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/0cbd8a5d4571cdf3601ca8deb64d734d35e234f9

tdf#140762  Move RIDs related to borders to include/svx

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 4 Commit Notification 2021-03-04 07:16:00 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/6065e4f400390702080c7de3bcc3de670453a347

tdf#140762 Set texts for items in "Borders" dropdown button

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 5 Commit Notification 2021-03-04 07:17:11 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/23548f4ff51e28fec6751604a0a28197d6680773

Related tdf#140762 Add texts for "Border style" toolbar item

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 6 Michael Weghorn 2021-03-04 10:22:58 UTC
With the above 3 commits, the descriptive texts are set for the items in the "Borders" and "Border Type" dropdown buttons in Calc's formatting toolbar.

Those are shown as tooltips when hovering using the mouse on both, Linux (tested with kf5 and gtk3) and Windows.

The Orca screen reader on Linux with the gtk3 VCL plugin speaks those texts when navigating between the items using the arrow keys (left, right, up, down). (It doesn't say the value of the initially selected entry when pressing space or Enter, though.)

For Windows with NVDA, texts are not spoken when navigating using the arrow keys at first. They are spoken when hovering over the items using the mouse. And after doing so, the item text is also announced by NVDA when using the arrow keys, so it seems some initial activation/focus is missing. (?)

 
(In reply to V Stuart Foote from comment #2)
> Have the general case of bug 113488 to expose the descriptive text when
> 'accessible events' are focus triggered, but then individual GUI widgets can
> require some refactoring to expose their controls meaningfully to AT, as
> here.
> 
> Tackle the 'accessible event' firing via keyboard navigation, so movement
> sequences and focus states for the elements in the widget(s).

Thanks for that info! Looking at bug 113488:

(In reply to V Stuart Foote from bug 113488 comment #0)
> By default mouseover events expose a more descriptive tooltip, or an
> extended tooltip--when offline local help is installed and has an article
> pointing at the button.
> 
> And with Assistive Technology (AT) support enabled, on mouseover of a button
> control the descriptive tooltip is sounded by AT instead of the button label.

That sounds like it *could* explain the behavior here is well, but...

> 
> When keyboard navigation is used: e.g. F10, F6, TAB/ShiftTab, or cursor
> UP/DOWN/LEFT/RIGHT--the AT sounds *only* the abbreviated button label.  Same
> for menu label, and context menu label which also sound just the short label.
> 
> If possible, for better accessibility support, keyboard navigation should
> offer the same audible/visual rendering of the more descriptive tooltip(s)
> provided with mouseover event of buttons.

... for the specific case of "Borders" and "Border style" toolbar items (with NVDA on Windows), it seems that the up/down/left/right keys does lead to AT announcing the same as the tooltip shows, but only after some "initial activation" by hovering over one of the items so it's tooltip is shown once, s. above. Maybe initial focus needs to be handled differently or somesuch. This might need further investigation.

Any quick ideas?

The above are just my first thoughts on this. I've just started trying to get a bit familiar with AT. Will try to take a closer look at the bugs you've mentioned at some point to at least get a deeper understanding.
Comment 7 Michael Weghorn 2021-03-04 15:57:36 UTC
I've taken a look at what happens on Win for the "Borders" dropdown button. As far as I understand it so far, the XAccessibleEventListener responsible for notifying AT is only set once mouse hovering happens.

For that reason, the call to 'ImplHasAccessibleListeners()' in 'ValueSet::SelectItem' [1] returns false if that hasn't happened yet, and no a11y event is triggered.

For the mouse hovering case, the XAccessibleEventListener is set, backtrace:

> 1   ValueSetAcc::addAccessibleEventListener valueacc.cxx              678  0x7ff8cffb5e37 
> 2   AccObjectWinManager::InsertAccObj       AccObjectWinManager.cxx   725  0x7ff8cccb6a10 
> 3   AccObjectManagerAgent::InsertAccObj     AccObjectManagerAgent.cxx 148  0x7ff8cccbdc38 
> 4   CMAccessible::GetChildInterface         MAccessible.cxx           1398 0x7ff8cc8ec486 
> 5   CMAccessible::accHitTest                MAccessible.cxx           993  0x7ff8cc8f1e3e 
> 6   CMAccessible::accHitTest                MAccessible.cxx           994  0x7ff8cc8f1e95 
> 7   CMAccessible::accHitTest                MAccessible.cxx           994  0x7ff8cc8f1e95 
> 8   CMAccessible::accHitTest                MAccessible.cxx           994  0x7ff8cc8f1e95 
> 9   CMAccessible::accHitTest                MAccessible.cxx           994  0x7ff8cc8f1e95 
> 10  CMAccessible::accHitTest                MAccessible.cxx           994  0x7ff8cc8f1e95 
> 11  CMAccessible::accHitTest                MAccessible.cxx           994  0x7ff8cc8f1e95 
> 12  NdrSendReceive                          RPCRT4                         0x7ff90ac18fd3 
> 13  NdrStubCall2                            RPCRT4                         0x7ff90abbc6c3 
> 14  NdrStubCall3                            RPCRT4                         0x7ff90abbb294 
> 15  CStdStubBuffer_Invoke                   combase                        0x7ff90aff297c 
> 16  CStdStubBuffer_Invoke                   RPCRT4                         0x7ff90abfa92b 
> 17  CoWaitForMultipleObjects                combase                        0x7ff90afced43 
> 18  CoWaitForMultipleObjects                combase                        0x7ff90afceace 
> 19  HSTRING_UserSize                        combase                        0x7ff90aff84b6 
> 20  CoGetCurrentLogicalThreadId             combase                        0x7ff90af5e6b3 
> ...

and after that has happened once, the listener is already set when above code passage is reached again when using the keyboard afterwards, which explains the behaviour described earlier.

The registration of the a11y listener is specific to the "platform" implementation, i.e. it's different for Win ('winaccessibility/' subdir) than it is for the gtk3 case, where the listener is registered earlier.
As a side note, it doesn't happen for the gtk3 case with env var 'SAL_NO_MOUSEGRABS=1' set, which makes debugging more complicated...

Therefore, from how I understand it so far, the case covered here is different from what tdf#113488 covers.

That's just a first analysis of someone who is mostly using Linux. As of now, I don't have any specific plan to continue looking into this right now (but might still come back to it at some point in time in the future). If anyone wants to work on it, please feel free. :-)

[1] https://git.libreoffice.org/core/+/23548f4ff51e28fec6751604a0a28197d6680773/svtools/source/control/valueset.cxx#794
Comment 8 V Stuart Foote 2021-03-04 16:40:46 UTC
Michael, thanks for the fix!

Caolán and Tomaž have kept after these facets of AT, but there are lots of places in the UI that need similar rework (e.g. provision of SVX strings) to make the widgets meaningful to AT--rather than just sounding generic item IDs.

(In reply to Michael Weghorn from comment #7)
...
> Therefore, from how I understand it so far, the case covered here is
> different from what tdf#113488 covers.
> 

That is fair, bug 113488 is about a difference in accessible event & focus handling between keyboard only navigation vs what happens on mouse over or mouse click.

Here once triggered, the focus driven accessible events did sound with keyboard movement--they just had no meaningful descriptions. 

That is fixed now, but still have the issue of difference between mouse-over or click versus keyboard focus. 

Focus requiring a mouse action is detrimental to AT. Keyboard only movements need to provide similar accessible event triggers.  And that is the crux of bug 113488 -- never silent, and best to expose the more descriptive text with every keyboard movement inside a widget.