Bug 91739 - nvda can not read documents from top to bottom.
Summary: nvda can not read documents from top to bottom.
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: UI (show other bugs)
Version:
(earliest affected)
unspecified
Hardware: Other Windows (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: accessibility
Depends on:
Blocks: a11y-Windows
  Show dependency treegraph
 
Reported: 2015-05-30 07:17 UTC by zahra
Modified: 2017-10-11 19:10 UTC (History)
7 users (show)

See Also:
Crash report or crash signature:


Attachments
one text to confirm the problem. (38.62 KB, text/richtext)
2015-05-30 07:40 UTC, zahra
Details

Note You need to log in before you can comment on or make changes to this bug.
Description zahra 2015-05-30 07:17:50 UTC
nvda cant read documents from top to bottom by incert down. 
when we use microsoft office nvda can read entire document when we press incert down. 
but in libreoffice and open office when we press incert down it reads only many lines and when we want to continue to read we should press incert down again and nvda reads only many lines and again and again. its very difficult specially in large documents and reading bookks. 
and also when we navigate in document word by word using control left and right to read previous and next word after ending sentences after . when we press control left and right it reads blank. in this case we should read charactor by charactor to understand if its realy a blank line or not.
Comment 1 zahra 2015-05-30 07:40:49 UTC
Created attachment 116166 [details]
one text to confirm the problem.
Comment 2 V Stuart Foote 2015-05-30 13:00:16 UTC
Setting new, but believe this remains an issue with NVDAs support for parsing IAccessible2 based content.

This has an open issue on NVDA support: http://community.nvda-project.org/ticket/4119 or with Symphony's early IA2 based support NVDA in ticket 910. But issues with SayAll with ODF and OpenOffice existed back to the JavaAccessBridge/JRE Java Accessibility era--see NVDA tickets 422, 96

@Jamie, any thoughts on implementing SayAll for LibreOffice/Apache OpenOffice or, should I just close this not our bug?

Stuart
Comment 3 James Teh 2015-05-30 23:11:36 UTC
Say all for Symphony (NVDA ticket #910) was fixed years ago. Any remaining issues were due to bugs in Symphony.

The problem with say all in OO/LO is due to the fact that they only expose text that is currently on screen. As a result, once NVDA say all gets to the last paragraph, it asks for the next paragraph and is told that there isn't one, so say all can't go any further. There are hacks to get around this (e.g. send control+downArrow or scroll in some other way when LO reports no more paragraphs), but this is fairly inelegant and potentially unreliable. Whether this is a bug is debatable, but it's certainly a problem for us and is not what happens in other implementations. In Symphony, you could get at paragraphs that weren't yet on screen (barring weird bugs), which is why say all worked in Symphony.

Note that you don't have to generate accessibles for every paragraph in the document ahead of time. You could just generate them on demand when the flowsTo relation is required.
Comment 4 V Stuart Foote 2015-05-31 02:06:48 UTC
@Jamie, *,

This gets beyond my skill set pretty quickly...

But, looking at i27138 - we already have an XAccessibleContext providing flowsFrom and flowsTo in the internal UAA. And that should be providing the IAccessible2 relationships.

But, seems that  for our Windows implementation--UAccCOM--we may be defining just the parent and child relationships, and not actually defining IA2_RELATION_FLOWS_FROM and IA2_RELATION_FLOWS_TO (see refs).

Anyone have a better feel for the this?

@Jamie,  if the IA2_RELATION_FLOWS_FROM and IA2_RELATION_FLOWS_TO are fully instrumented, would that then facilitate the NVDA SayAll function?

=-refs-=
Core
http://opengrok.libreoffice.org/xref/core/accessibility/source/extended/textwindowaccessibility.cxx#1447
http://opengrok.libreoffice.org/xref/core/sw/source/core/access/accpara.hxx#265
http://opengrok.libreoffice.org/xref/core/sw/source/core/access/accpara.cxx#959
http://opengrok.libreoffice.org/xref/core/editeng/source/accessibility/AccessibleEditableTextPara.cxx#813

Windows IA2 UAccCOM
http://opengrok.libreoffice.org/xref/core/winaccessibility/source/UAccCOMIDL/UAccCOM.idl
http://opengrok.libreoffice.org/xref/core/winaccessibility/source/UAccCOM/MAccessible.cxx
http://opengrok.libreoffice.org/xref/core/winaccessibility/source/UAccCOMIDL/ia2_api_all.idl#354
Comment 5 James Teh 2015-05-31 23:44:46 UTC
LO *does* expose flowsFrom and flowsTo via IA2. The issue isn't that they aren't exposed. The issue is that they don't expose paragraphs that are currently off-screen in some cases.

So, steps to reproduce with the attached example document:
1. Open the document. The cursor will be at the first line.
2. Get the accessible for the first paragraph.
3. Call IAccessible::nRelations.
Result (correct): 1
4. Call IAccessible2::relations, passing 1 for maxRelations.
5. Get the first (and only) IAccessibleRelation returned.
6. On the relation from step 5, call IAccessibleRelation::relationType.
Result (correct): "flowsTo"
7. On the relation from step 5, call IAccessibleRelation::nTargets.
Result (correct): 1
8. On the relation from step 5, call IAccessibleRelation::target, passing 0 for targetIndex.
Expected: An IUnknown for the second paragraph.
Actual: E_FAIL
9. In the document, press control+downArrow to move to the second paragraph.
10. On the relation from step 5, call IAccessibleRelation::target, passing 0 for targetIndex.
Result (correct): An IUnknown for the second paragraph.

Interestingly, if you then press control+home to move to the top of the document and repeat step 10, you still get the second paragraph. So, it seems that if you've retrieved the paragraph once already, you can retrieve it again even if it goes off-screen.
Comment 6 Joanmarie Diggs 2015-06-11 04:47:10 UTC
This issue also impacts things like Orca's ability to do structural navigation (navigate amongst headings, for instance).

As for the flows-to relations, things get more interesting depending on what you are trying to accomplish. Consider something like:

Page 1 (which is visible)
* Paragraph 1
* Paragraph 2
Page 2 (not currently visible)
* Paragraph 3
* Paragraph 4

At least on my platform, if you get the last child of the accessible document, you get "Paragraph 2". If you then get the flows-to target of that child you get "Paragraph 3". So far so good. But if you get the parent of "Paragraph 3" you get the accessible document which contains "Paragraph 1" and "Paragraph 2". In other words, the parent of "Paragraph 3" does not have "Paragraph 3" as a child. That's a broken tree.

I understand the desire to not automatically expose the entire tree for all pages at once because that potentially would be a huge and non-performant solution. But in addition to a non-broken tree, I'd really like to have structural navigation for document content in Orca. And not everything has an immediate flows-to (tables, form fields). So what about something like:

For each page in the document, create an accessible (ATK_ROLE_PAGE on my platform; dunno about IA2). Regardless of the role, these page accessibles are all immediate children of the document and are all created upon document load. Also load the accessible children for a given page when that page is accessed. Lazy load the rest of the children of each page.
Comment 7 V Stuart Foote 2015-06-11 13:28:50 UTC
(In reply to Joanmarie Diggs from comment #6)

> I understand the desire to not automatically expose the entire tree for all
> pages at once because that potentially would be a huge and non-performant
> solution. But in addition to a non-broken tree, I'd really like to have
> structural navigation for document content in Orca. And not everything has
> an immediate flows-to (tables, form fields). So what about something like:
> 
> For each page in the document, create an accessible (ATK_ROLE_PAGE on my
> platform; dunno about IA2). Regardless of the role, these page accessibles
> are all immediate children of the document and are all created upon document
> load. Also load the accessible children for a given page when that page is
> accessed. Lazy load the rest of the children of each page.

IAccessible2 has the same... IA2_ROLE_PAGE, but it looks like NSAccessibility does not currently map it in any useful fashion, refs.

=-refs-=
IA2_ROLE_PAGE
http://opengrok.libreoffice.org/xref/core/winaccessibility/source/UAccCOMIDL/ia2_api_all.idl#901
http://opengrok.libreoffice.org/xref/core/winaccessibility/source/service/AccObject.cxx

NSAccessibility
http://opengrok.libreoffice.org/xref/core/vcl/osx/a11yrolehelper.mm#119
https://developer.apple.com/library/mac/documentation/AppKit/Reference/NSAccessibility_Protocol_Reference/index.html#//apple_ref/doc/constant_group/Roles
Comment 8 Jacobo Aragunde Pérez 2015-12-03 14:24:04 UTC
The accessible role PAGE seems to exist in LibreOffice internal FW [1], but I've done some grepping and I think it's never used.

[1] http://api.libreoffice.org/docs/idl/ref/namespacecom_1_1sun_1_1star_1_1accessibility_1_1AccessibleRole.html#a29bb572b50678c1b43fb73f8e0855d65
Comment 9 Jacobo Aragunde Pérez 2015-12-04 16:39:18 UTC
(In reply to Joanmarie Diggs from comment #6)
> 
> For each page in the document, create an accessible (ATK_ROLE_PAGE on my
> platform; dunno about IA2). Regardless of the role, these page accessibles
> are all immediate children of the document and are all created upon document
> load. Also load the accessible children for a given page when that page is
> accessed. Lazy load the rest of the children of each page.

Unfortunately, this approach seems to be difficult to implement. There is no concept of "pages" in the document model; pages are an artifact of the view that wraps the contents. There are contents that can be split between two pages, so we would still have to do some hacks to model the contents in a proper tree. Finally, the flow of paragraphs to/from the next/previous page when editions happen would have to trigger a rebuild of parts of the accessibility tree, this should be doable but it's definitely inconvenient and would probably generate a lot of event noise.


> Page 1 (which is visible)
> * Paragraph 1
> * Paragraph 2
> Page 2 (not currently visible)
> * Paragraph 3
> * Paragraph 4
> 
> At least on my platform, if you get the last child of the accessible
> document, you get "Paragraph 2". If you then get the flows-to target of that
> child you get "Paragraph 3". So far so good. But if you get the parent of
> "Paragraph 3" you get the accessible document which contains "Paragraph 1"
> and "Paragraph 2". In other words, the parent of "Paragraph 3" does not have
> "Paragraph 3" as a child. That's a broken tree.

When the paragraph 3 is got through the flows-to relation, is the parent component (the document) refreshed? Would that work to implement "say all"?

> This issue also impacts things like Orca's ability to do structural
> navigation (navigate amongst headings, for instance).

Regarding this topic, would it help if we implemented flows to/from relations among header objects?
Comment 10 Robinson Tryon (qubit) 2015-12-10 03:37:52 UTC Comment hidden (obsolete)
Comment 11 QA Administrators 2017-01-03 19:40:19 UTC Comment hidden (obsolete)
Comment 12 V Stuart Foote 2017-01-03 19:46:38 UTC
This is still an issue, no change supporting structural navigation by page top to bottom with current release or on master. NVDA screen reader exhausts buffer and stalls.
Comment 13 V Stuart Foote 2017-10-11 19:10:49 UTC
Here are links to some related historical reading from the Gnome ATK/AT-SPI side on providing support for the Accessibility::Collection IDL (AtkDocument/AtkCollection) as a possible solution to our limited window into a complex document that disrupts screen reading.

https://bugzilla.gnome.org/show_bug.cgi?id=652548
https://bugzilla.gnome.org/show_bug.cgi?id=345750