Bug 93430 - Cannot get accessible text attributes for 'Not in dictionary' entry
Summary: Cannot get accessible text attributes for 'Not in dictionary' entry
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: UI (show other bugs)
Version:
(earliest affected)
unspecified
Hardware: Other All
: medium normal
Assignee: Tamás Zolnai
QA Contact:
URL:
Whiteboard: target:5.4.0 target:5.3.0.1 target:5.2.5
Keywords: accessibility
Depends on:
Blocks: Spell-Checking a11y
  Show dependency treegraph
 
Reported: 2015-08-14 16:21 UTC by Joanmarie Diggs
Modified: 2017-02-09 15:10 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Accerciser : tee for accessing "Not In Dictionary" element (165.97 KB, image/png)
2016-05-04 11:00 UTC, Alex ARNAUD
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Joanmarie Diggs 2015-08-14 16:21:33 UTC
Steps to reproduce:

1. Launch Writer and type 'Hello worrld' in a new document

2. Start spellcheck

3. Launch Accerciser

4. In Accerciser's tree on the left, select the document paragraph with the error, then in Accerciser's iPython console execute:

   acc.queryText().getAttributeRun(0)

5. In Accerciser's tree on the left, select the 'Not in dictionary' text entry, then in Accerciser's iPython console execute:

   acc.queryText().getAttributeRun(0)

Expected results: Both step 4 and 5 would result in a listing of text attributes.

Actual results: Step 4 results in a listing of text attributes, but no text attributes are listed after performing step 5 (see below).

Impact: One of the things Orca should present is the word that is misspelled (in this case "worrld"). Sighted users can identify the error because it is indicated in bold, red text. Orca could perform the same task by looking at the text attributes -- if they were exposed via ATK/AT-SPI2. They're not.

--------

Sample output after step 4:

[['text-decoration:none',
  'variant:normal',
  'strikethrough:none',
  'invisible:false',
  'language:en-us',
  'fg-color:0,0,0',

  ...

  'weight:400',
  'style:normal',
  'mm-to-pixel-ratio:0.263158'],
 0,
 6]

--------

Sample output after step 5:

[[], 0, 0]
Comment 1 Buovjaga 2015-09-12 16:29:00 UTC
I'm only seeing Accerciser itself in the tree view.
I'm running KDE. Is there something else I should do?

Manjaro Linux KDE Plasma 5, 64-bit
Version: 5.0.1.2.0+
Build ID: 5.0.1.2 Arch Linux build-1
Locale: fi-FI (fi_FI.UTF-8)
Comment 2 Buovjaga 2015-09-12 16:48:45 UTC
Now I tried with Ubuntu with Unity and I saw various other apps, but still not LibO :(
It's weird as I've used Accerciser with this Ubuntu & LibO before.

Ubuntu 15.04 64-bit 
Version: 5.0.1.2
Build ID: 00m0(Build:2)
Locale: en-US (en_US.UTF-8)
Comment 3 Alex ARNAUD 2016-05-03 10:31:36 UTC
(In reply to Buovjaga from comment #1)
> I'm only seeing Accerciser itself in the tree view.
> I'm running KDE. Is there something else I should do?
> 
> Manjaro Linux KDE Plasma 5, 64-bit
> Version: 5.0.1.2.0+
> Build ID: 5.0.1.2 Arch Linux build-1
> Locale: fi-FI (fi_FI.UTF-8)

Are you using LibreOffice GTK version ? I don't know if the others are accessible.

Best regards.
Comment 4 Buovjaga 2016-05-03 13:00:06 UTC
(In reply to Alex ARNAUD from comment #3)
> Are you using LibreOffice GTK version ? I don't know if the others are
> accessible.

Thanks, now I could make it work. How do I find the paragraph from the tree, though? I looked for a long time, but could not find it.
Comment 5 Alex ARNAUD 2016-05-04 11:00:12 UTC
Created attachment 124837 [details]
Accerciser : tee for accessing "Not In Dictionary" element

(In reply to Buovjaga from comment #4)
> (In reply to Alex ARNAUD from comment #3)
> > Are you using LibreOffice GTK version ? I don't know if the others are
> > accessible.
> 
> Thanks, now I could make it work. How do I find the paragraph from the tree,
> though? I looked for a long time, but could not find it.

Dear Buovjaga,
I join a screen shot that shows you the tree.
In I sum up : 
1) Open LibreOffice Writer
2) Type "Hello worrld" and press enter
3) Open Spell Check (F7)
4) Launch Accerciser
5) Go to soffice
6) Go to Spell Check
7) Unfold all the unique elements
8) Unfold the element with the role "PANEL"
9) You'll see a element "Not in dictionary" with the role "LABEL"

Best regards.
Comment 6 Buovjaga 2016-05-05 16:27:49 UTC
Step 4 I could not do as I could not locate paragraph, but confirm that step 5 gives:
In [1]: acc.queryText().getAttributeRun(0)
Out[1]: [[], 0, 0]

64-bit, KDE Plasma 5
Build ID: 5.1.2.2 Arch Linux build-1
CPU Threads: 8; OS Version: Linux 4.5; UI Render: default; 
Locale: fi-FI (fi_FI.UTF-8)
Comment 7 Yousuf Philips (jay) 2016-09-13 14:47:12 UTC
My suggestion from bug 102044: "When the spelling dialog is opened and when a new mispelt word is highlighted, the cursor should return to the 'not in dictionary' text area so that a user can easily modify the word and a screen reader can read the mispelt word, as presently a a11y users cant know which of the text area words are mispelt."

The above idea is how MS word does it, while i had thought of an earlier alternative idea of duplicating the mispelt word in the 'not in dictionary' label and as the label is linked with the paragraph, the screen reader should be able to read it. Alternatively a variant of this idea could be done with a hidden label.
Comment 8 Joanmarie Diggs 2016-09-13 19:51:10 UTC
Why did you unblock the Orca bug? I (Orca maintainer) still would like this fixed properly.
Comment 9 Joanmarie Diggs 2016-09-13 20:15:01 UTC
By the way, and to be clear, Orca has spellcheck support that includes customizable presentation of the misspelled word, the context of the error, and the suggestion. So hidden labels and such designed to try to make Orca say what you think it should is not how to solve this problem. Exposing what is on the screen (in this case the bold and red text that sighted users see) so that Orca can do its job as a screen reader and present what is on the screen is the desired solution.
Comment 10 Yousuf Philips (jay) 2016-09-14 00:57:19 UTC
(In reply to Joanmarie Diggs from comment #8)
> Why did you unblock the Orca bug? I (Orca maintainer) still would like this
> fixed properly.

I am organizing bugs that are cross-platform into a main a11y meta bug and those that are OS specific can stay in their OS specific meta bugs.

(In reply to Joanmarie Diggs from comment #9)
> By the way, and to be clear, Orca has spellcheck support that includes
> customizable presentation of the misspelled word, the context of the error,
> and the suggestion. So hidden labels and such designed to try to make Orca
> say what you think it should is not how to solve this problem. Exposing what
> is on the screen (in this case the bold and red text that sighted users see)
> so that Orca can do its job as a screen reader and present what is on the
> screen is the desired solution.

Yes i was aware that Orca and other screen readers have mispelt word support and i was just making suggestions of how it might be able to be fixed, even if it was temporary until a correct fix was implemented.
Comment 11 Commit Notification 2016-12-04 00:29:51 UTC
Tamás Zolnai committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=17773e1a8dd6e97f57db111819338c418f819cef

tdf#93430: Get run attributes of text objects

It will be available in 5.4.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Tamás Zolnai 2016-12-04 00:36:45 UTC
With the commit above now LibreOffice returns text attributes for different kind of text objects (e.g. text edit controls, labels, etc.) too, not only for paragraphs.
On the other hand for 'Not in dictionary' entry it returns these lines for all characters:
[['text-decoration:none',
  'strikethrough:none',
  'underline:none',
  'weight:0',
  'size:0',
  'scale:0'],
 0,
 12]

It seems text properties are not set properly for this entry. So need some more investigation.
Comment 13 Commit Notification 2016-12-09 09:17:36 UTC
Tamás Zolnai committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=e0d8c3821b8fa1e7d00f7b4a7d007f9cb5c592a5

tdf#93430: Cannot get accessible text attributes for 'Not in dictionary' entry

It will be available in 5.4.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 14 Tamás Zolnai 2016-12-09 12:17:17 UTC
With the second commit, now you can see a difference in character properties of the red text and the normal text:
In [1]: acc.queryText().getAttributeRun(0) # normal text
Out[1]: 
[['text-decoration:none',
  'scale:0',
  'weight:0',
  'strikethrough:none',
  'underline:none',
  'fg-color:46,52,54', # dark grey character color
  'size:0'],
 0,
 12]
In [1]: acc.queryText().getAttributeRun(6) # misspelled word's character
Out[2]: 
[['text-decoration:none',
  'scale:0',
  'weight:0',
  'strikethrough:none',
  'underline:none',
  'fg-color:255,0,0', # red character color
  'size:0'],
 0,
 12]
I guess, orca can find out which is the misspelled word now using this information.
Comment 15 Commit Notification 2016-12-09 15:59:04 UTC
Tamás Zolnai committed a patch related to this issue.
It has been pushed to "libreoffice-5-3":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=b815cdf1933813a6864350ea1baade0bccee9ec4&h=libreoffice-5-3

tdf#93430: Cannot get accessible text attributes for 'Not in dictionary' entry

It will be available in 5.3.0.1.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 16 Joanmarie Diggs 2016-12-09 16:03:39 UTC
(In reply to Zolnai Tamás from comment #14)
> With the second commit, now you can see a difference in character properties
> of the red text and the normal text:
> In [1]: acc.queryText().getAttributeRun(0) # normal text
> Out[1]: 
> [['text-decoration:none',
>   'scale:0',
>   'weight:0',
>   'strikethrough:none',
>   'underline:none',
>   'fg-color:46,52,54', # dark grey character color
>   'size:0'],
>  0,
>  12]
> In [1]: acc.queryText().getAttributeRun(6) # misspelled word's character
> Out[2]: 
> [['text-decoration:none',
>   'scale:0',
>   'weight:0',
>   'strikethrough:none',
>   'underline:none',
>   'fg-color:255,0,0', # red character color
>   'size:0'],
>  0,
>  12]
> I guess, orca can find out which is the misspelled word now using this
> information.

I'll take a look. But in the meantime, I'm curious about the output above. Are they each taken from the same accessible object and with the same content and formatting?

The reason I ask is that the last two items in the array are expected to be the character offsets corresponding to the range within the string which has the formatting described by the first item in the array. Are characters 0-12 red, or is only a subset of those characters (including character 6) red?
Comment 17 Tamás Zolnai 2016-12-09 16:49:30 UTC
(In reply to Joanmarie Diggs from comment #16)
> (In reply to Zolnai Tamás from comment #14)
> > With the second commit, now you can see a difference in character properties
> > of the red text and the normal text:
> > In [1]: acc.queryText().getAttributeRun(0) # normal text
> > Out[1]: 
> > [['text-decoration:none',
> >   'scale:0',
> >   'weight:0',
> >   'strikethrough:none',
> >   'underline:none',
> >   'fg-color:46,52,54', # dark grey character color
> >   'size:0'],
> >  0,
> >  12]
> > In [1]: acc.queryText().getAttributeRun(6) # misspelled word's character
> > Out[2]: 
> > [['text-decoration:none',
> >   'scale:0',
> >   'weight:0',
> >   'strikethrough:none',
> >   'underline:none',
> >   'fg-color:255,0,0', # red character color
> >   'size:0'],
> >  0,
> >  12]
> > I guess, orca can find out which is the misspelled word now using this
> > information.
> 
> I'll take a look. But in the meantime, I'm curious about the output above.
> Are they each taken from the same accessible object and with the same
> content and formatting?
> 
> The reason I ask is that the last two items in the array are expected to be
> the character offsets corresponding to the range within the string which has
> the formatting described by the first item in the array. Are characters 0-12
> red, or is only a subset of those characters (including character 6) red?

These two examples are run on 'Hello worrld' in the problematic 'Not in dictionary' entry. The first 6 character has the normal character color and the others has red color. So 0 and 12 seems not the proper values.
I checked the code, it is intended to return only the run's (text range with same formatting) first and start index, but this functionality is not implemented in general, only for paragraphs and LO always returns 0 an the text length for other text objects.
I'll check how this can be solved.
Comment 18 Joanmarie Diggs 2016-12-09 17:33:30 UTC
(In reply to Zolnai Tamás from comment #17)

> I'll check how this can be solved.

Thanks! Having the correct offsets is an important part of the fix.
Comment 19 Commit Notification 2016-12-10 12:46:12 UTC
Tamás Zolnai committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=8d52045853cecb716138d51b8b6ad0272a86af86

tdf#93430: Return the right offsets for runs

It will be available in 5.4.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 20 Tamás Zolnai 2016-12-10 12:54:01 UTC
Now I get these lines:
In [1]: acc.queryText().getAttributeRun(0)
Out[1]: 
[['size:0',
  'fg-color:46,52,54',
  'underline:none',
  'weight:0',
  'scale:0',
  'text-decoration:none',
  'strikethrough:none'],
 0,
 6]
In [1]: acc.queryText().getAttributeRun(6)
Out[2]: 
[['size:0',
  'fg-color:255,0,0',
  'underline:none',
  'weight:0',
  'scale:0',
  'text-decoration:none',
  'strikethrough:none'],
 6,
 12]

The offsets are set correctly.
Comment 21 Commit Notification 2016-12-10 20:36:06 UTC
Tamás Zolnai committed a patch related to this issue.
It has been pushed to "libreoffice-5-3":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=b88a5657a95e1b94498ed238e2b4e0fb0418d40b&h=libreoffice-5-3

tdf#93430: Return the right offsets for runs

It will be available in 5.3.0.1.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 22 Commit Notification 2016-12-11 20:54:43 UTC
Tamás Zolnai committed a patch related to this issue.
It has been pushed to "libreoffice-5-2":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=c5b71538e9cd7854f502d20c36396176a49bef2b&h=libreoffice-5-2

tdf#93430: Cannot get accessible text attributes for 'Not in dictionary' entry

It will be available in 5.2.5.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 23 Alex ARNAUD 2017-02-09 15:10:46 UTC
Dear all,

I can confirm that the fix works properly on LibreOffice 5.2.5 GTK2 on Debian GNU/Linux.

Best regards.