Bug 81264 - Calc is not accessible to screen readers if sheet is modified [a11y]
Summary: Calc is not accessible to screen readers if sheet is modified [a11y]
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.3.0.2 rc
Hardware: Other All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:4.4.0 target:4.3.4
Keywords: accessibility
: 74410 81098 (view as bug list)
Depends on:
Blocks: a11y, Accessibility mab4.3
  Show dependency treegraph
 
Reported: 2014-07-12 11:15 UTC by Arnaud Versini
Modified: 2016-10-25 20:08 UTC (History)
9 users (show)

See Also:
Crash report or crash signature:


Attachments
debug out created with "orca --debug" (540.89 KB, application/octet-stream)
2014-07-31 20:17 UTC, chrys87@web.de
Details
accessible event listener (422 bytes, text/x-python)
2014-09-09 16:50 UTC, Joanmarie Diggs
Details
AccProbe session <F10> toggle into and out of Calc sheet -- no Focus (131.04 KB, image/png)
2014-11-03 19:50 UTC, V Stuart Foote
Details
AccProbe session <F10> toggle intoa nd out of Calc sheet -- gains Foucs on tableCells (99.27 KB, image/png)
2014-11-03 20:02 UTC, V Stuart Foote
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Arnaud Versini 2014-07-12 11:15:29 UTC
Hi

Steps to reproduce :
- Activate a screen reader
- Create a new calc document
- Navigate on the sheet with keyboard => Ok it works, coordonates are speaked
- Modify the current sheet
- Navigate => Doesn't work anymore, screen reader doesn't talk anymore about coordonates and contents
- Delete modified cell => No sound, can't have sound on this sheet anymore
- Go to another sheet => works
Comment 1 V Stuart Foote 2014-07-12 13:07:02 UTC
Amaund,

Thanks for posting. 

These are likely loss of focus, and accessibile event issues.

But, we need better details about the version of Linux, which screen reader (Orca we'll assume), and what other navigation you perform on the Calc document that also cause issues.

Very possible this is the same issue as bug 81098 (loss of AT in Calc on Windows) and we'll merge them.
Comment 2 Arnaud Versini 2014-07-12 13:11:32 UTC
Ok, Jean Philippe will tell you all version information on his computer.

But on windows it's all the time, on Linux depending on file content (I know, weird), I will check on mine, we tested on his computer.

Not sure about loss focus because we can move with arrow.
Comment 3 Jean-Philippe MENGUAL 2014-07-12 14:17:51 UTC
(In reply to comment #2)
> Ok, Jean Philippe will tell you all version information on his computer.
> 
> But on windows it's all the time, on Linux depending on file content (I
> know, weird), I will check on mine, we tested on his computer.
> 
> Not sure about loss focus because we can move with arrow.

The bug appears on Ubuntu 12.04, with Orca 3.4 (they are still supported by Ubuntu, as they are LTS). Any movement is affected: up, down, right, left, etc.

Regards,
Comment 4 Arnaud Versini 2014-07-13 11:44:13 UTC
Confirmed also on OpenSuse with Orca 3.12 .

We should restart LO with orca enabled to have calc saying cell coordinates. 
First modification stop screen reader to say wich case we are on.
Comment 5 chrys87@web.de 2014-07-31 20:17:00 UTC
Created attachment 103766 [details]
debug out created with "orca --debug"
Comment 6 chrys87@web.de 2014-07-31 20:19:44 UTC
i add an debug.out of orca. maybe this helps.
this i have done:
0 enable orca with "orca --debug"
1 alt + f2 -> type:"libreoffice --calc"
2 navigate a little arround in the sheet. -> everything works.
3 type "testtest" -> press enter
4 navigate arraound. nothing happens anymore.
Comment 7 chrys87@web.de 2014-08-04 15:04:46 UTC
If some other debugging stuff is needed, just let me know.
I realy wanna help to make calc working again for blind users.
Comment 8 chrys87@web.de 2014-08-04 15:18:02 UTC
I also think the Importent should raised to height here because calc became unusable for blind users with this bug.
Comment 9 Joanmarie Diggs 2014-09-09 16:50:11 UTC
Created attachment 105994 [details]
accessible event listener

The problem appears to be:
1. Missing object:state-changed:focused events when the table regains focus
2. The re-focused table lacks state focused (probably because of item 1).

At the end of this comment is output that was captured using the attached accessible-event listener performing the following steps:

1. Give initial focus to a newly-created sheet
2. Down Arrow twice in that a newly-created sheet
3. Alt+Tab out of Calc and back in
4. Down Arrow in the re-focused sheet

Note that for 4.2.6.3 each time the table gets focus, it claims focus. Also note that each time you down arrow and an active-descendant-changed event is seen, the table emitting that event claims to be focused. In contrast, in 4.3 the state-changed events for gaining focus are absent. And while the sheet initially claims to be focused, once it loses focus it never reclaims it. This is why, as the users have described, Orca stops presenting the events: Users don't want to hear events for objects they are not in.


===============
Calc v. 4.2.6.3
===============

object:state-changed:focused(1, 0, 0)
	source: [table | Sheet Sheet1]
	host_application: [application | soffice]
Event source is focused: True

object:active-descendant-changed(1024, 0, [table cell | Cell A2 ])
	source: [table | Sheet Sheet1]
	host_application: [application | soffice]
Event source is focused: True

object:active-descendant-changed(2048, 0, [table cell | Cell A3 ])
	source: [table | Sheet Sheet1]
	host_application: [application | soffice]
Event source is focused: True

object:state-changed:focused(0, 0, 0)
	source: [table | Sheet Sheet1]
	host_application: [application | soffice]
Event source is focused: False

object:state-changed:focused(1, 0, 0)
	source: [table | Sheet Sheet1]
	host_application: [application | soffice]
Event source is focused: True

object:active-descendant-changed(2048, 0, [table cell | Cell A3 ])
	source: [table | Sheet Sheet1]
	host_application: [application | soffice]
Event source is focused: True

object:active-descendant-changed(3072, 0, [table cell | Cell A4 ])
	source: [table | Sheet Sheet1]
	host_application: [application | soffice]
Event source is focused: True

===============
Calc v. 4.3.1.1
===============

object:active-descendant-changed(1024, 0, [table cell | A2])
	source: [table | Sheet Sheet1]
	host_application: [application | soffice]
Event source is focused: True

object:active-descendant-changed(2048, 0, [table cell | A3])
	source: [table | Sheet Sheet1]
	host_application: [application | soffice]
Event source is focused: True

object:state-changed:focused(0, 0, 0)
	source: [table | Sheet Sheet1]
	host_application: [application | soffice]
Event source is focused: False

object:active-descendant-changed(2048, 0, [table cell | A3])
	source: [table | Sheet Sheet1]
	host_application: [application | soffice]
Event source is focused: False

object:active-descendant-changed(3072, 0, [table cell | A4])
	source: [table | Sheet Sheet1]
	host_application: [application | soffice]
Event source is focused: False
Comment 10 V Stuart Foote 2014-09-11 13:18:49 UTC
*** Bug 81098 has been marked as a duplicate of this bug. ***
Comment 11 Niklas Johansson 2014-09-11 18:53:27 UTC
I did a bisect with 43only, and ran the accessible event listener that Joanmarie supplied. I'll see if I can find the time to narrow it down a bit further, at a quick glance I suspect this commit:
http://cgit.freedesktop.org/libreoffice/core/commit/?id=b41332475783c31136673fb44cf4c411bb0148f8

but let's hope it's something else since the commit is quite big. ;)
 fa2f11e14834fe607f413e0b2c8787df6c7242fd is the first bad commit
commit fa2f11e14834fe607f413e0b2c8787df6c7242fd
Author: Bjoern Michaelsen <bjoern.michaelsen@canonical.com>
Date:   Sat May 10 19:02:08 2014 +0000

    source-hash-5b03bc8a4d92fee2fdfdca4917b321985feb930a
    
    commit 5b03bc8a4d92fee2fdfdca4917b321985feb930a
    Author:     Matúš Kukan <matus.kukan@collabora.com>
    AuthorDate: Mon Dec 2 20:53:54 2013 +0100
    Commit:     Matúš Kukan <matus.kukan@collabora.com>
    CommitDate: Tue Dec 3 08:31:17 2013 +0100
    
        ugly build-fix for libmerged
    
        Change-Id: I6c107ea5e780ea1178759e2ee827f9bd6d5ede3a

:100644 100644 5b641e0949034a17ebe37fcc5721bb8d681f918a a29c686040b42da895dc8d6c7acff088ebe46f0d M	ccache.log
:100644 100644 d58a1fda1b6443fe603edcf1a5852106250dbb98 40217d36816ddd2c5f6899872b7c3fa8df1b1632 M	commitmsg
:100644 100644 cd43b5ba40bab2031582507c959155e323fe54a5 9565d9c5e8aea2b16d832f9908db07614bd9444d M	make.log
:040000 040000 0bd8f3f42aca7a170c71b0cf68638adf30a4cae4 da6593955cc4467f45fc6db11335b763e16f5f10 M	opt
niklas@niklas-bibisect:~/bibisect-43only$ git bisect log
# bad: [a92705c1fabafddd43d175a0714855cd22551232] source-hash-c15927f20d4727c3b8de68497b6949e72f9e6e9e
# good: [6ab7f53af36f13bbefdd4e4fcbd3d1ea432a77d9] source-hash-22029c7e17b4cb48acb058d47ec9c3b6b8b6b294
git bisect start 'latest' 'oldest'
# bad: [bebf9d31c8fe9de96798484288a0fffc4d54917d] source-hash-09e5de8278dd8f13adcf614db35c8a8a04ba8e47
git bisect bad bebf9d31c8fe9de96798484288a0fffc4d54917d
# bad: [20d42d26a12e5ded00b3510d2d9e254e7876dc78] source-hash-c1503da35d8879366da13258837cf0084a536809
git bisect bad 20d42d26a12e5ded00b3510d2d9e254e7876dc78
# bad: [df997ea92f012344541d1cf25eb1ff402e6de210] source-hash-7f5494f3c4bf14209a119c6b21c02e10075503ae
git bisect bad df997ea92f012344541d1cf25eb1ff402e6de210
# bad: [fb7cc8c858ad0d6b018efed2d24ae912c4ed670d] source-hash-c828319753558e25a48ce7808604bcc648f2483d
git bisect bad fb7cc8c858ad0d6b018efed2d24ae912c4ed670d
# bad: [fb7cc8c858ad0d6b018efed2d24ae912c4ed670d] source-hash-c828319753558e25a48ce7808604bcc648f2483d
git bisect bad fb7cc8c858ad0d6b018efed2d24ae912c4ed670d
# bad: [5c5a5d374edf40a9089de1af6e0c184572158d82] source-hash-a0be5278c24efcc9a6f22fe5398d780b0744f8ce
git bisect bad 5c5a5d374edf40a9089de1af6e0c184572158d82
# bad: [5c5a5d374edf40a9089de1af6e0c184572158d82] source-hash-a0be5278c24efcc9a6f22fe5398d780b0744f8ce
git bisect bad 5c5a5d374edf40a9089de1af6e0c184572158d82
# bad: [5c5a5d374edf40a9089de1af6e0c184572158d82] source-hash-a0be5278c24efcc9a6f22fe5398d780b0744f8ce
git bisect bad 5c5a5d374edf40a9089de1af6e0c184572158d82
# good: [37db8f46d051bae7e7a9ce9e68ce1693d909e40c] source-hash-137f872aa8e6e598e7c7ed1ffa4d21e580e22bdb
git bisect good 37db8f46d051bae7e7a9ce9e68ce1693d909e40c
# good: [37db8f46d051bae7e7a9ce9e68ce1693d909e40c] source-hash-137f872aa8e6e598e7c7ed1ffa4d21e580e22bdb
git bisect good 37db8f46d051bae7e7a9ce9e68ce1693d909e40c
# bad: [fa2f11e14834fe607f413e0b2c8787df6c7242fd] source-hash-5b03bc8a4d92fee2fdfdca4917b321985feb930a
git bisect bad fa2f11e14834fe607f413e0b2c8787df6c7242fd
# bad: [fa2f11e14834fe607f413e0b2c8787df6c7242fd] source-hash-5b03bc8a4d92fee2fdfdca4917b321985feb930a
git bisect bad fa2f11e14834fe607f413e0b2c8787df6c7242fd
# good: [0b7262a63fb4f7a664b2db371ce61fe1f74f36c3] source-hash-362c8d67e1cb8920bf179b52c50b5997d32eb296
git bisect good 0b7262a63fb4f7a664b2db371ce61fe1f74f36c3
# first bad commit: [fa2f11e14834fe607f413e0b2c8787df6c7242fd] source-hash-5b03bc8a4d92fee2fdfdca4917b321985feb930a
Comment 12 Commit Notification 2014-09-16 09:42:07 UTC
Niklas Johansson committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=93410b5ba13749cf3663d3d696fe1a14474bf696

fdo#81264 Calc is not accessible to screen readers if sheet is modified [a11y]



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 13 Eike Rathke 2014-09-16 09:47:44 UTC
If this works as intended, please verify on Windows, we should consider a cherry-pick to 4-3
Comment 14 Niklas Johansson 2014-09-17 14:45:56 UTC
Hi, I did some quick checking on Windows with NVDA and from what I can tell the patch don't really seem to have any direct impact on Windows. I'll try to prepare a backport to 4.3 sometime later this week.

Calc still don't speak the cell coordinates on Windows (unless I double click a cell using the mouse, after that it seems to work). I'll try to find time to look into that issue later.
Comment 15 Commit Notification 2014-09-25 08:56:55 UTC
Niklas Johansson committed a patch related to this issue.
It has been pushed to "libreoffice-4-3":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=cc803df1a94bf572bd14ff5402ccbad5674c9bef&h=libreoffice-4-3

fdo#81264 Calc is not accessible to screen readers if sheet is modified [a11y]


It will be available in LibreOffice 4.3.3.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 16 David Goldfield 2014-10-13 10:47:44 UTC
I wanted to report that as of the 9/30/2014 nightly build I am still not receiving accessibility feedback with LibreOffice Calc and NVDA 2014.3 while moving through cells using the arrow keys.  Was the patch in this particular version?
Comment 17 Niklas Johansson 2014-10-15 07:18:34 UTC
(In reply to David Goldfield from comment #16)
> I wanted to report that as of the 9/30/2014 nightly build I am still not
> receiving accessibility feedback with LibreOffice Calc and NVDA 2014.3 while
> moving through cells using the arrow keys.  Was the patch in this particular
> version?

Hi David, the situation on Windows is a bit different then the one described in this bug. After I modified a cell I can actually get updates (well not always, but sometimes). I've been meaning to open a new bug and reference it in here, but time has slipped through my hands, sorry about that.
Comment 18 V Stuart Foote 2014-10-15 14:13:39 UTC
Niklas, *,
(In reply to Niklas Johansson from comment #17)
> Hi David, the situation on Windows is a bit different then the one described
> in this bug. After I modified a cell I can actually get updates (well not
> always, but sometimes). I've been meaning to open a new bug and reference it
> in here, but time has slipped through my hands, sorry about that.

Could always reopen bug 81098 if issues are really not releated.
Comment 19 David Goldfield 2014-11-03 14:42:29 UTC
there is some definite progress regarding this issue. I just tested Calc
Version: 4.4.0.0.alpha1+
Build ID: d9473f25380c627966b4406cc4cdfaafcf44bc37
TinderBox: Win-x86@51-TDF, Branch:MASTER, Time: 2014-11-03_04:24:26
Pressing f2 followed by escape does not allow me to hear cell coordinates or data as I use arrow keys.
However, at that point if I press ctrl-alt-N to exit and restart NVDA and place focus back into the Calc spreadsheet, cell coordinates and data within the cell speak reliably.  Does that information provide any help?
Comment 20 V Stuart Foote 2014-11-03 15:03:45 UTC
David, *,
(In reply to David Goldfield from comment #19)
> However, at that point if I press ctrl-alt-N to exit and restart NVDA and
> place focus back into the Calc spreadsheet, cell coordinates and data within
> the cell speak reliably.  Does that information provide any help?
Thanks helpful.

On Windows 7 sp1, 64-bit en-US with NVDA 2014.2 and a current 4.4.0.0alpha1+ build of master.

Not sure why but the <ctrl><alt>+N cycling NVDA correctly establishes the rendering of IAccessible2 content.  If NVDA is running when LibreOffice is launched the Calc sheet content is not reliably gaining focus, although all the menus and toolbars seem to have no trouble.

Are we assigning an incorrect role to the sheet (a drawing layer) and its cells?

Once we gain focus of the sheet, all seems correct. The relaunch of NVDA seems to do it most efficiently, more so that the <F2> and <esc> I'd been using.
Comment 21 Niklas Johansson 2014-11-03 17:52:14 UTC
I've been trying to debug this and it seems that ScAccessibleSpreadsheet is not created when a spreadsheet is created. I believe this need to be created somewhere to (probably from ScAccessibleDocument). At the moment I'm a bit lost as to what actually broke the behavior and how to fix it.

Sometimes ScAccessibleSpreadsheet is created but something else is missing leaving us with focus events being sent for a cell and then immediately removed. So the output could look something like this.
EVENT_OBJECT_SELECTION name = B2 role = tableCell
EVENT_OBJECT_FOCUS name = B2 role = tableCell
EVENT_OBJECT_SELECTIONREMOVE name = B2 role = tableCell

In Apache OpenOffice 4.1.1 it instead look like this:
EVENT_OBJECT_SELECTION Name=B2 Role=tableCell
EVENT_OBJECT_FOCUS Name=B2 Role=tableCell
EVENT_OBJECT_SELECTIONREMOVE Name=, Role=rootPane

LibreOffice master from 19 December 2013 seems to work as well and the object that is removed was the cell we moved from which seems more correct than the root pane. But there where probably some other issue involved.

I'm hoping to find some windows builds of master between 19 December up until the summer to try and narrow down when the focus issue first occurred. I'm guessing it was introduced somewhere around January but that might be totally wrong. ;)
Comment 22 V Stuart Foote 2014-11-03 19:50:46 UTC
Created attachment 108856 [details]
AccProbe session <F10> toggle into and out of Calc sheet -- no Focus

On Windows 7 sp1, 64-bit en-US with
Accessibility Probe
Version: 1.2.1.1

and

Version: 4.4.0.0.alpha1+
Build ID: d9473f25380c627966b4406cc4cdfaafcf44bc37
TinderBox: Win-x86@51-TDF, Branch:MASTER, Time: 2014-11-03_04:24:26

Attached clip of AccProbe session when no Focus is gained into the table and tableCell -- instead see optionPane and rootPane roles only, and tableCells do not gain focus on keyboard navigation.
Comment 23 V Stuart Foote 2014-11-03 20:02:56 UTC
Created attachment 108857 [details]
AccProbe session <F10> toggle intoa nd out of Calc sheet -- gains Foucs on tableCells

On Windows 7 sp1, 64-bit en-US with
Accessibility Probe
Version: 1.2.1.1

and

Version: 4.4.0.0.alpha1+
Build ID: d9473f25380c627966b4406cc4cdfaafcf44bc37
TinderBox: Win-x86@51-TDF, Branch:MASTER, Time: 2014-11-03_04:24:26

Attached clip of AccProbe session when Focus is gained into the tableCells. 

The table object shows as focusable, and never gains focus--which moves immediately to a tableCell. And on deselection--the rootPane shows "[normal]".

Contrasted to the no-Focus when table is never exposed, just optionPane, and the tableCells never gain focus.
Comment 24 Commit Notification 2014-11-06 09:49:43 UTC
Niklas Johansson committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=a1a9f0e5c4f7d7331072854250a7eb9046e4f111

fdo#81264 Accessiblitiy focus not tracked for cells in Calc

It will be available in 4.4.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 25 Commit Notification 2014-11-07 11:40:46 UTC
Niklas Johansson committed a patch related to this issue.
It has been pushed to "libreoffice-4-3":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=162535ded77633a07871be917b719861bcaf9f43&h=libreoffice-4-3

fdo#81264 Accessiblitiy focus not tracked for cells in Calc

It will be available in 4.3.3.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 26 V Stuart Foote 2014-11-07 12:38:35 UTC
setting target to 4.3.4
Comment 27 V Stuart Foote 2014-11-15 02:37:54 UTC
Functioning well now on Windows in 4.3.4.1 release.

Good tracking in NVDA and in Accprobe.  Occasionally still have to toggle <F2> <Esc> to force focus into active table cell.
Comment 28 V Stuart Foote 2014-11-27 03:49:40 UTC
*** Bug 74410 has been marked as a duplicate of this bug. ***
Comment 29 Robinson Tryon (qubit) 2015-12-18 09:31:51 UTC Comment hidden (obsolete)