Bug 153806 - Improve keyboard interaction for Special Characters dialog
Summary: Improve keyboard interaction for Special Characters dialog
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: UI (show other bugs)
Version:
(earliest affected)
6.4.7.2 release
Hardware: All All
: medium enhancement
Assignee: Michael Weghorn
URL:
Whiteboard: target:7.6.0
Keywords: accessibility
Depends on:
Blocks: a11y, Accessibility Special-Character
  Show dependency treegraph
 
Reported: 2023-02-24 13:54 UTC by Michael Weghorn
Modified: 2023-07-27 08:42 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Weghorn 2023-02-24 13:54:59 UTC
(Based on Colomban's comment/workaround while implementing a11y UI tests, s. https://gerrit.libreoffice.org/c/core/+/142260/22/sw/qa/extras/accessibility/dialogs.cxx )

Interacting with the "Special characters" dialog using the keyboard is not particularly intuitive, which is bad for a11y.

# Example 1 (from Colomban's UI test, s. link above):

1) start Writer, open dialog using "Insert" -> "Special characters" menu item
2) type "copyright" in the search box
3) press the Tab key a few times until the entry for the copyright symbol is highlighted

Result:
* The UI elements update to show that this item has been selected (e.g. the character is shown on the right hand side, and it says "COPYRIGHT SIGN" below that).
* However, the "Insert" button is still inactive, i.e. it's not possible to press "Tab" more often to get to that button and insert the character.

What works however is (note the extra step 4, that's rather unexpected and which I saw only when looking into the current implementation):

1) start Writer, open dialog using "Insert" -> "Special characters" menu item
2) type "copyright" in the search box
3) press the Tab key a few times until the entry for the copyright symbol is highlighted
4) press the space key to activate the "Insert" button
5) press Tab key more times to select the "Insert" button
6) press Enter key to insert the character and close the dialog

# Example 2:

* It's possible to insert a character from the table of characters without closing the dialog by double-clicking on the character using the mouse (or right-clicking, then selecting "Insert into document" in the context menu)
* However, there seems to be no way to do so using the keyboard instead.

# Example 3:

Contrary to the characters in the "table", the characters in the "Recent Characters" and "Favorite Characters" section *can* be inserted using the keyboard:

1) press Tab key until the desired character is focused
2) press either space or Enter key

This is IMHO inconsistent.

# Expected behavior:

The behavior should be more consistent and the dialog easily usable using the keyboard.

Suggestion:

* for example 1: enable the "Insert" key right when a character in the table is selected using the Tab key
* insert currently focused character from the table when pressing the space or Enter key, just as is the case for "Recent characters"/"Favorite characters" as well


Version: 7.6.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 6d404f9ccad3ac6a7ffb83876ae7a656bc2d4547
CPU threads: 12; OS: Linux 6.1; UI render: default; VCL: kf5 (cairo+xcb)
Locale: en-GB (en_GB.UTF-8); UI: en-US
Calc: threaded

Version: 7.6.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 6d404f9ccad3ac6a7ffb83876ae7a656bc2d4547
CPU threads: 12; OS: Linux 6.1; UI render: default; VCL: gtk3
Locale: en-GB (en_GB.UTF-8); UI: en-US
Calc: threaded
Comment 1 Michael Weghorn 2023-02-24 14:13:53 UTC
@Heiko: Any thoughts/opinion on this?

(In reply to Michael Weghorn from comment #0)
> Suggestion:
> 
> * for example 1: enable the "Insert" key right when a character in the table
> is selected using the Tab key
> * insert currently focused character from the table when pressing the space
> or Enter key, just as is the case for "Recent characters"/"Favorite
> characters" as well

Suggested Gerrit change implementing that behavior:
https://gerrit.libreoffice.org/c/core/+/147657
Comment 2 Stéphane Guillou (stragu) 2023-02-24 15:52:53 UTC
I agree that the current situation needs improvement, at the very least allowing inserting directly from the search results with the keyboard, just like in the favourite list.

Interestingly, back in 6.3, I was able to insert from the search results by using Space bar, then Enter (which would also close the dialog). Since 6.4, I can't do that. Some kind of regression along the way?

Version: 6.3.6.2
Build ID: 2196df99b074d8a661f4036fca8fa0cbfa33a497
CPU threads: 8; OS: Linux 5.15; UI render: default; VCL: gtk3; 
Locale: en-AU (en_AU.UTF-8); UI-Language: en-US
Calc: threaded

Version: 6.4.7.2
Build ID: 639b8ac485750d5696d7590a72ef1b496725cfb5
CPU threads: 8; OS: Linux 5.15; UI render: default; VCL: gtk3; 
Locale: en-AU (en_AU.UTF-8); UI-Language: en-US
Calc: threaded
Comment 3 Michael Weghorn 2023-02-24 23:49:37 UTC
(In reply to Stéphane Guillou (stragu) from comment #2)
> I agree that the current situation needs improvement, at the very least
> allowing inserting directly from the search results with the keyboard, just
> like in the favourite list.
> 
> Interestingly, back in 6.3, I was able to insert from the search results by
> using Space bar, then Enter (which would also close the dialog). Since 6.4,
> I can't do that. Some kind of regression along the way?

Reading this, another idea might be to distinguish between space and Enter key and have Enter key close the dialog in addition, as happens e.g. in the Insert -> Hyperlink dialog, too, so maybe sth like this (but then consistently for both, characters in the character table (like search results) *and* the favorites/recently used ones), so:

* space key inserts the character, but leaves the dialog open
* Enter inserts the character and closes the dialog

(And still make putting focus into the search results make the "Insert" button usable right away.)
What do you think?
Comment 4 Heiko Tietze 2023-02-27 09:25:16 UTC
Navigating in the table per tab is wrong. The "copy" search results © and ℗ allow using left/right arrow, if the list is larger up/down work as well. And Insert becomes enabled.

So what we have to do is to accept a select per tab.

Another aspect on this dialog is the favorite chars, where only tab is accepted to traverse through the content; arrow keys are ignored. One has to tab through all items to finally continue at the next control.

Another a11y improvement for the dialog could be to add a label for the character list so we have a mnemonic for quick access. Spoils the design a bit, however.
Comment 5 V Stuart Foote 2023-02-27 13:36:50 UTC
(In reply to Michael Weghorn from comment #3)
> (In reply to Stéphane Guillou (stragu) from comment #2)
> > I agree that the current situation needs improvement, at the very least
> > allowing inserting directly from the search results with the keyboard, just
> > like in the favourite list.
> > 
> > Interestingly, back in 6.3, I was able to insert from the search results by
> > using Space bar, then Enter (which would also close the dialog). Since 6.4,
> > I can't do that. Some kind of regression along the way?
> 
> Reading this, another idea might be to distinguish between space and Enter
> key and have Enter key close the dialog in addition, as happens e.g. in the
> Insert -> Hyperlink dialog, too, so maybe sth like this (but then
> consistently for both, characters in the character table (like search
> results) *and* the favorites/recently used ones), so:
> 
> * space key inserts the character, but leaves the dialog open
> * Enter inserts the character and closes the dialog
> 
> (And still make putting focus into the search results make the "Insert"
> button usable right away.)
> What do you think?

+1, differentiate action of <Space> (to perform the Insert leaving SCD open for subsequent picks) and <Enter> (to perform the Insert and close the SCD).
Comment 6 V Stuart Foote 2023-02-27 13:49:08 UTC
(In reply to Heiko Tietze from comment #4)
> Navigating in the table per tab is wrong. The "copy" search results © and ℗
> allow using left/right arrow, if the list is larger up/down work as well.
> And Insert becomes enabled.
> 
> So what we have to do is to accept a select per tab.
> 
> Another aspect on this dialog is the favorite chars, where only tab is
> accepted to traverse through the content; arrow keys are ignored. One has to
> tab through all items to finally continue at the next control.
> 
> Another a11y improvement for the dialog could be to add a label for the
> character list so we have a mnemonic for quick access. Spoils the design a
> bit, however.

NO! The <Tab> should continue to reposition in the UI. When in the character chart cursor <U,D,L,R> *alone* should perform the movement--but the Insert button, along with the focus "accessible action" for sounding the character name must activate.

This is the same as movements between the clustered elements of the SB.

And core keyboard navigation of <F10>, <F6>, <TAB>/<Shift><TAB>, and cursor <U,D,L,R>, and <ESC> is *maintained* with appropriate focus events firing.

What is wrong with the SCD is its "Insert" action while in the character chart UI is not correctly linked.  

As enhancement: a shortcut to directly place UI focus into the character chart would be helpful, as would a shortcut to place directly in the search field.

Likewise as enhancement: differentiating <Space> action from <Enter> action for completing the character pick via keyboard.
Comment 7 V Stuart Foote 2023-02-27 14:05:13 UTC
(In reply to Heiko Tietze from comment #4)

My last NO was for an "accept per <TAB>" action.
 
> Another aspect on this dialog is the favorite chars, where only tab is
> accepted to traverse through the content; arrow keys are ignored. One has to
> tab through all items to finally continue at the next control.
> 

This is a bug and a regression, needs bibisection. The Cursor movements should move between elements on the Recents and Favorites bar (with same selection options and accessible event triggers)--both in the SCD and on the Split button.  Linkage has changed as I'm pretty sure this worked correctly, is bad at 7.3.7 (that I have immediately to hand). 

The <L> or <U> cursor takes the focus elsewhere but <TAB> positions at the start of the character bar, Favorites or Recents. While <R> or <D> does not respond. So cursor not doing nothing, movement is just not correctly linked.
Comment 8 Stéphane Guillou (stragu) 2023-02-27 14:42:47 UTC
Correct me if I'm wrong, but I thought Heiko simply meant that leaving the character grid with a tab should have the last highlighted character selected (which is already the case for the grid, but not for Favourites and Recents).

I agree with all proposed improvements, with consistent behaviour in the fill list, search results, recent characters and favorite characters.

Looking at another dialog that could allow inserting several things: AutoText.
In that dialog, clicking the Insert button, double-clicking an item, pressing the spacebar or return key all do the same thing: insert and close.

The special character dialog is different in that a mouse action (double-click) can insert without closing, and there should be a way to do the same thing without the mouse. (Which in my opinion also means it should be non-modal, although I just saw Heiko had arguments against in bug 56301.)

(In reply to V Stuart Foote from comment #7)
> (In reply to Heiko Tietze from comment #4)
> > Another aspect on this dialog is the favorite chars, where only tab is
> > accepted to traverse through the content; arrow keys are ignored. One has to
> > tab through all items to finally continue at the next control.
> > 
> 
> This is a bug and a regression, needs bibisection. The Cursor movements
> should move between elements on the Recents and Favorites bar (with same
> selection options and accessible event triggers)--both in the SCD and on the
> Split button.  Linkage has changed as I'm pretty sure this worked correctly,
> is bad at 7.3.7 (that I have immediately to hand). 

6.0.0.3 already tabbed through all favourites. Not sure when Favourites were introduced. 5.4 didn't have favourites. (testing on Linux)
Comment 9 V Stuart Foote 2023-02-27 16:58:27 UTC
Your right, Favorites and Recents came in on the SCD at 6.0 for GSOC 2017, but the navigation <TAB> and Cursor (U,D,L,R) was not implemented for the Character bar, only mouse click.

Split button came in at 7.0 IIRC.

The move at 5.4 -> 6.0 dropped the edit bar, making the SCD *non-modal*. Bug 115477 remain open to restore--necessary for string picks (both mouse or keyboard) and for polyglot input when no IME is in use.
Comment 10 cwendling 2023-02-28 13:51:08 UTC
FWIW, IMO what would be expected would be:

- <Enter> should validate the dialog and perform the default action (likely, Insert) *consistently*
- Insert should either *always* be active when there are > 0 matches in the table, OR whenever the table gains focus and has > 0 matches
- <Tab> should behave as it does (cycle through UI elements with a fine-ish granularity, e.g. between buttons and such, but not individual table cells), but in the recent/favorite it should *probably* not cycle through each char (but like on the main table, go out of it)
- Reason for not tabbing through the whole table is that it's a nightmare.  Basically people <Tab> through to get to the next useful control, having to <Tab> a million times because there's many options is a huge pain.
- <U,D,L,R> should move around the table(s) content (current behavior seems alright, although in recent/favorite <U,D> leaves the table (which is consistent as them probably not being tables but individual widgets

Additionally:

- Recent/favorite should probably be tables like the main one (which behaves fine)
- Recent/favorite items should have proper names attached to them, as currently they are not accessible at all (the screen reader doesn't see a name, so can only tell the user it's a "drawing area")

Note that the main issue currently is that Insert doesn't get enabled when the focus enters the table, but only when clicking, or using one of <Space,U,D,L,R>.  (OP's #1)

#3 is also confusing because <Enter> doesn't behave consistently, and the user might even not realize the character has been entered, and activate Insert again.
Comment 11 Heiko Tietze 2023-03-16 10:21:33 UTC
(In reply to cwendling from comment #10)
> - <Enter> should validate the dialog and perform the default action (likely,
> Insert) *consistently*
> - Insert should either *always* be active when there are > 0 matches in the
> table, OR whenever the table gains focus and has > 0 matches

The patch makes Insert active and does not close the dialog via Enter.

> - <Tab> should behave as it does...

Tab iterates over Recent and Favorite Characters instead cursor lef/right. This should be addressed somehow. Michael, do you want to integrate this in the current patch?
Comment 12 Michael Weghorn 2023-03-17 06:40:38 UTC
Thanks for all the valuable input!

(In reply to Heiko Tietze from comment #11)
> (In reply to cwendling from comment #10)
> > - <Enter> should validate the dialog and perform the default action (likely,
> > Insert) *consistently*
> > - Insert should either *always* be active when there are > 0 matches in the
> > table, OR whenever the table gains focus and has > 0 matches
> 
> The patch makes Insert active and does not close the dialog via Enter.

I agree Enter should close the dialog and will update the patch accordingly.

> > - <Tab> should behave as it does...
> 
> Tab iterates over Recent and Favorite Characters instead cursor lef/right.
> This should be addressed somehow. Michael, do you want to integrate this in
> the current patch?

I think that can be handled separately.
I'll probably leave that aspect and some of the other aspects discussed here for "sometime later" (as I find time, unless anybody else has the time to look into this in the meantime).
Comment 13 Michael Weghorn 2023-04-01 20:46:28 UTC
(In reply to Michael Weghorn from comment #12)
> > The patch makes Insert active and does not close the dialog via Enter.
> 
> I agree Enter should close the dialog and will update the patch accordingly.

While looking into this, I decided to split the new Enter key handling from the current patch into a new one, to keep the scope of the single changes smaller/clearer:
https://gerrit.libreoffice.org/c/core/+/149920
Comment 14 Commit Notification 2023-04-03 15:00:09 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/cb29cd301d6a493870d8efc233965658ae532206

tdf#153806 cui: Deduplicate code for special char dialog handlers

It will be available in 7.6.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 15 Commit Notification 2023-04-03 15:01:12 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/d1f5b3c5cb226fd5f58fe2cdc8be9d1330fad6a2

tdf#153806 a11y: Improve keyboard interaction in special char dialog

It will be available in 7.6.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 16 Commit Notification 2023-04-03 15:01:14 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/fa7fc141e49bf1a5e1fc30645d559c19205d76a7

tdf#153806 a11y: Insert special char + close dialog on return key

It will be available in 7.6.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 17 Commit Notification 2023-04-03 15:02:17 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/1f437264084fd173116965fa4d856aeafdfe7a98

tdf#153806 a11y: Allow opening context menu in special char dlg using keyboard

It will be available in 7.6.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 18 Commit Notification 2023-04-13 04:54:36 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/fc893169d37ae78d34b6a127bc920bc5b887711a

tdf#153806 sw a11y test: Drop obsolete workaround

It will be available in 7.6.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 19 Commit Notification 2023-04-23 04:20:23 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/20c86bd1059b54915190ce2e3dd39206d293aced

tdf#153806 a11y Allow opening context menu of fav/recent char w/ keyboard

It will be available in 7.6.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 20 Stéphane Guillou (stragu) 2023-07-25 20:35:30 UTC
Michael, do you think this can be closed as "fixed" and maybe further improvement handled in new reports?
I tested in 7.6.0.1 and the issues from the Description are resolved:

Version: 7.6.0.1 (X86_64) / LibreOffice Community
Build ID: 776eaf34564cbf3f034a0ba1fd1d5c32ff9ccf1c
CPU threads: 8; OS: Linux 5.15; UI render: default; VCL: gtk3
Locale: en-AU (en_AU.UTF-8); UI: en-US
Calc: threaded

One issue that remains in my opinion is the inconsistency in the Enter key behaviour: inserts and closes in the table, but only inserts in the Recent and Favourites sections. (Arrow keys now work to navigate in the section.)

I also think that Tab should jump from Recent to Favorite directly instead of going through all the characters, but that's already tracked by bug 109215.
Comment 21 Michael Weghorn 2023-07-27 08:42:06 UTC
(In reply to Stéphane Guillou (stragu) from comment #20)
> Michael, do you think this can be closed as "fixed" and maybe further
> improvement handled in new reports?
> I tested in 7.6.0.1 and the issues from the Description are resolved:
> 
> Version: 7.6.0.1 (X86_64) / LibreOffice Community
> Build ID: 776eaf34564cbf3f034a0ba1fd1d5c32ff9ccf1c
> CPU threads: 8; OS: Linux 5.15; UI render: default; VCL: gtk3
> Locale: en-AU (en_AU.UTF-8); UI: en-US
> Calc: threaded
> 
> One issue that remains in my opinion is the inconsistency in the Enter key
> behaviour: inserts and closes in the table, but only inserts in the Recent
> and Favourites sections. (Arrow keys now work to navigate in the section.)
> 
> I also think that Tab should jump from Recent to Favorite directly instead
> of going through all the characters, but that's already tracked by bug
> 109215.

Thanks for retesting and your suggestion, Stéphane!
I think that totally makes sense, since both of the above aspects need adapting the key handling for the recent/favorite character controls, or maybe reworking the used controls more after all.
I've added a corresponding comment in tdf#109215. Let's close this one here and keep the remaining aspects there (and open separate tickets as necessary if that one is done and anything else remains to be done).