STR: 1. With NVDA running, start LibreOffice, either 4.2.0 or 4.3.0 nightly. 2. You'll hear NVDA repeatedly say "White Space", followed by the description of the "Option Panel" spoken initially. Exploring the object hierarchy, several direct children and grand children of the Option Pane are WhiteSpace accessibles. The information grabbed with NVDA+F1 for one of these looks as follows: INFO - globalCommands.GlobalCommands.script_navigatorObject_devInfo (17:31:59): Developer info for navigator object: name: u'' role: ROLE_WHITESPACE states: isFocusable: False hasFocus: False Python object: <NVDAObjects.IAccessible.IAccessible object at 0x05BFF2B0> Python class mro: (<class 'NVDAObjects.IAccessible.IAccessible'>, <class 'NVDAObjects.window.Window'>, <class 'NVDAObjects.NVDAObject'>, <class 'baseObject.ScriptableObject'>, <class 'baseObject.AutoPropertyObject'>, <type 'object'>) description: u'Click an icon to open a new document or a file dialog.' location: (129, 895, 164, 42) value: None appModule: <'soffice' (appName u'soffice', process ID 4648) at address 5bff490> appModule.productName: u'LibreOffice' appModule.productVersion: u'4.2.0.4' TextInfo: <class 'NVDAObjects.NVDAObjectTextInfo'> windowHandle: 2098126L windowClassName: u'SALFRAME' windowControlID: 0 windowStyle: 349110272 windowThreadID: 2112 windowText: u'LibreOffice' displayText: u'' IAccessibleObject: <POINTER(IAccessible2) ptr=0x594124 at 5c12670> IAccessibleChildID: 0 IAccessible event parameters: windowHandle=2098126L, objectID=-4, childID=-77 IAccessible accName: u'' IAccessible accRole: ROLE_SYSTEM_WHITESPACE IAccessible accState: (0) IAccessible accDescription: u'Click an icon to open a new document or a file dialog.' IAccessible accValue: None IAccessible2 windowHandle: 2098126 IAccessible2 uniqueID: -77 IAccessible2 role: ROLE_SYSTEM_WHITESPACE IAccessible2 states: (0) IAccessible2 attributes: exception: (-2147467259, 'Unspecified error', (None, None, None, 0, None))
@Marco, thanks for posting! I believe this nuisance issue results from the implementation of nested GtkBox or GtkGrid UI Widgets for layout of the StartCenter. Also noted in the Sidebar and it probably exists elsewhere where UI has been implemented. The JAB Java runtime based bridge announces the structures as 'filler', while IAccessible2 announces as 'white space'. Not sure if they are handled similarly in AX or ATK. Not sure if these GTK+ layout elements can be suppressed, otherwise maybe a better accessible role can be assigned. Is this something that could (or should?) be suppressed in the Assistive Technology? Ideas?
(In reply to comment #1) > I believe this nuisance issue results from the implementation of nested > GtkBox or GtkGrid UI Widgets for layout of the StartCenter. Also noted in > the Sidebar and it probably exists elsewhere where UI has been implemented. Interesting, since such purely presentational items should ideally not clutter the accessible tree at all. If they do not serve any really useful purpose, would it be an option to just not expose them, but make the buttons direct IAccessible children of the panel?
LibreOffice API specifies the Accessible role filler as: Filler role. An object that fills up space in a user interface. It is often used in interfaces to tweak the spacing between components, but serves no other purpose. So to me it seems that it should not be announced by screen readers. To my knowledge neither AX nor ATK announce objects with the accessible role filler.
Intersting - there should be no conceptual problem with the hierarchical helpers - surely; we use them for ATK without issues with orcus, but presumably getting the type right is important. Caolan was just doing an a11y talk so might have an idea =)
We probably should make NVDA ignore whitespace objects. That said: * Whitespace does seem like an inappropriate role here, as does filler. Whitespace/filler suggest that they fill up space *between* elements, not that they *contain* elements. I'd think a role of pane or similar would be better here. * Even if ATs do ignore them, exposing them if they really serve no purpose at all is not ideal in terms of performance, as ATs will hit each one if they have to walk up the hierarchy.
NVDA ticket: http://community.nvda-project.org/ticket/3839
I agree with jamie that these layout-containers should not be exposed. In Mozilla, we use the Whitespace role for <br /> elements. And only for those, see https://developer.mozilla.org/en-US/docs/Web/Accessibility/AT-APIs/Gecko/Roles/ROLE_WHITESPACE#content
Issue seems to arise from the way the GTK+ Glade UI widgets get assigned their accessible roles--I think with GAIL but I'm not sure of the plumbing to associate to UAA and from there out to JAB (for 4.2 and earlier) and IA2, AX and ATK. The nested boxes or grids are used only for layout, but as GTK+ objects they get parsed and assigned an accessibility role. I don't know if that can be adjusted to suppress when used solely for layout. But I beleive Caolán, Kendy and Cédric have the best handle on how the accessible roles for the GTK+ UI widgets are cobbled together and perhaps can comment on a solution. Meanwhile, I find I need a score card to keep track of the API's--the W3C HTML to Platform Accessibility APIs Implementation Guide fits the bill nicely. http://rawgithub.com/w3c/html-api-map/master/index.html Worth keeping handy...
Glade does use the role FILLER to layout objects, not only to add space in between objects. And so I believe that the role filler is correct. It seems that Mozilla and W3C maps filler to ROLE_SYSTEM_WHITESPACE so we should probably do the same. If I understand the code correctly on Mac LibreOffice ignore the role FILLER in the accessibility hierarchy, it is also mapped as AXUnknown http://opengrok.libreoffice.org/xref/core/vcl/osx/a11ywrapper.mm#712 Would it be possible/make sense to do something similar on Windows?
What happens when NDVA meets a "true" gtk application on windows ? e.g something like gimp (or maybe better, assuming it works, gladewin32 itself which presumably contains glade-previewer which might be suitable for testing simple standalone box/grid using testcases.
Caolán (In reply to comment #10) > What happens when NDVA meets a "true" gtk application on windows ? e.g > something like gimp (or maybe better, assuming it works, gladewin32 itself > which presumably contains glade-previewer which might be suitable for > testing simple standalone box/grid using testcases. Nothing =( It seems that the GIMP on Windows offers no Java Accessiblity API access via the JAB, nor UIA nor IAccessible2 interface--just minimal MSAA responses. So seems to be mute with NVDA. Anyone know if there is a version of GIMP for Windows built with support for Assistive Technology tools?
(In reply to comment #11) > Anyone know if there is a version of GIMP for Windows built with support for > Assistive Technology tools? As far as I know, GTK isn't accessible on Windows at all (unless of course the app has its own complete accessibility implementation).
It may well be worth looking at and/or screenshotting some Java apps under Windows to see what 'pane' type peers they use to represent their layout containers [ assuming that they do ] - there are plenty of GridBagLayoutOfDoom type things in Java =) I assume their MSAA bridge has some good tips there. Input appreciated ...
(In reply to comment #13) > It may well be worth looking at and/or screenshotting some Java apps under > Windows to see what 'pane' type peers they use to represent their layout > containers [ assuming that they do ] - there are plenty of > GridBagLayoutOfDoom type things in Java =) I assume their MSAA bridge has > some good tips there. Java doesn't expose anything via MSAA. The Java Access Bridge exposes a completely separate API in Windows which is entirely unrelated to MSAA, so it has its own roles, states, etc.
Mozilla has some great API documentation here: http://lxr.mozilla.org/seamonkey/source/accessible/accessible-docs.html Ominously their drawing of what gets a peer seems to show them exporting a different tree to their internal representation - which (I guess) militates against exposing the layout peers: even though they may well provide useful semantic information to ATs. I guess I didn't read what we're doing there yet; does the new VCL layout stuff have windows as real children of other windows ? or is the layout hierarchy tacked on the side ? ...
(In reply to comment #15) > Mozilla has some great API documentation here: > > http://lxr.mozilla.org/seamonkey/source/accessible/accessible-docs.html > > Ominously their drawing of what gets a peer seems to show them exporting a > different tree to their internal representation - which (I guess) militates > against exposing the layout peers: even though they may well provide useful > semantic information to ATs. Mozilla has only only one "real" window that gets exposed to assistive technologies, meaning one that has a window handle, control ID and Window Class. All the rest that is exposed is a hierarchy built from the XUL of the Chrome, and the HTML of the web content, if any. So the parent/child relationships are based on the DOM trees of both the browser chrome and content. The accessibility hierarchy is a subset of the DOM hierarchy. In XUL, vbox, hbox and other layout elements are left out, for example. They are for layout purposes only, and the important thing for ATs to know is the location and width and height of actual elements like labels, controls etc. How they got there, and how many boxes are around them, is nothing assistive tecnologies care about too much. :) > I guess I didn't read what we're doing there yet; does the new VCL layout > stuff have windows as real children of other windows ? or is the layout > hierarchy tacked on the side ? ... It looks very similar to what Mozilla is doing, exposing an accessibility element hierarchy to assistive technologies that resembles the hierarchy of VCL and document elements. And in here, these filler boxes that are used for layout are exposed as whitespace. In Mozilla, the whitespace accessibles are exclusively created for HTML:br elements to indicate to screen readers that particular text doesn't run together. Hope this helped!
hmm, so I assigned "VclBox" containers AccessibleRole::FILLER because that's what gtk (IIRC) assigned "GtkBox" (which I did find a bit odd). While "VclGrids" are assigned AccessibleRole::PANEL, again because that's what (IIRC) gtk did for their GtkGrids. The containers are "real windows" as opposed to sticking something on the side. I'm guessing that there isn't a problem with "VclGrids", so lets first experiment with assigning PANEL "A generic container that is often used to group objects" for VclBoxes under windows rather than eliding them from the a11y view, which might have somewhat complicated side-effects.
Caolan McNamara committed a patch related to this issue. It has been pushed to "master": http://cgit.freedesktop.org/libreoffice/core/commit/?id=685ec1899435037205d98a102a32ca8b6a4836d0 Related: fdo#74284 try AccessibleRole::PANEL for VclBoxes under windows The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
(In reply to comment #5) > We probably should make NVDA ignore whitespace objects. FWIW, I've already committed a change to NVDA to ignore whitespace accessibles as layout objects. It's currently incubating on our "next" branch. Barring any issues, it'll get merged to master in a couple of weeks. That said, I still think there's a more appropriate role. (In reply to comment #18) > Related: fdo#74284 try AccessibleRole::PANEL for VclBoxes under windows What does that role get mapped to in MSAA/IA2? (There's no panel role in either, but there are several pane roles.)
I can confirm that this build: Version: 4.3.0.0.alpha0+ Build ID: d20ffc370bc1b946366d4cb1073b815beda706a2 TinderBox: Win-x86@39, Branch:master, Time: 2014-02-18_03:05:33 eliminates the extraneous whitespace accessibles.
re "What does that role get mapped to in MSAA/IA2? (There's no panel role in either, but there are several pane roles.)" digging into winaccessibility/source/service/AccObject.cxx I get "IA2_ROLE_OPTION_PANE" (Under gtk's atk it maps to ATK_ROLE_PANEL).
Marco, James - thanks for the great support in NVDA ! that's great; if you're happy with the IA2_ROLE_OPTION_PANE change would it be good to back-port that to 4.2.<next> ?
Given Macro says that "I can confirm that ... d20ffc370bc1b946366d4cb1073b815beda706a2 ... eliminates the extraneous whitespace accessibles." and is the original reporter I'll call this fixed. I'll go ahead and submit the tweak for 4-2
(In reply to comment #21) > re "What does that role get mapped to in MSAA/IA2? (There's no panel role in > either, but there are several pane roles.)" digging into > winaccessibility/source/service/AccObject.cxx I get "IA2_ROLE_OPTION_PANE" > (Under gtk's atk it maps to ATK_ROLE_PANEL). That's interesting, because in the 4.3-dev build, I now don't see any option panes on that start screen any more. However, since bug 75151 might also play a part here, they may simply not be exposed at all any more since there is neither a name nor an acc description to expose. I am not using a build of NVDA that has the ignoring of whitespace accessibles in yet, so this is definitely something different.
(In reply to comment #22) > Marco, James - thanks for the great support in NVDA ! that's great; if > you're happy with the IA2_ROLE_OPTION_PANE change would it be good to > back-port that to 4.2.<next> ? Yup! :)
(In reply to comment #24) > That's interesting, because in the 4.3-dev build, I now don't see any option > panes on that start screen any more. However, since bug 75151 might also > play a part here, they may simply not be exposed at all any more since there > is neither a name nor an acc description to expose. I am not using a build > of NVDA that has the ignoring of whitespace accessibles in yet, so this is > definitely something different. With a build of master -- prior to Caolán's 2014-02-17 patch-- Version: 4.3.0.0.alpha0+ Build ID: 6364a3ec6f92fc1f4ac3c7a4e2bdce2a2140c736 TinderBox: Win-x86@39, Branch:master, Time: 2014-02-13_01:44:32 With NVDA 2013.2 active I was getting the multiple 'White Space' filler when passing between the GTK boxes. And as noted on bug 75151, as built the master builds don't receive the more annoying description translations that really stick out. With today's patched build the filler role has been suppressed. Version: 4.3.0.0.alpha0+ Build ID: d20ffc370bc1b946366d4cb1073b815beda706a2 TinderBox: Win-x86@39, Branch:master, Time: 2014-02-18_03:05:33
Thanks Stuart, your explanation from bug 75151 makes perfect sense, and NVDA may just suppress all of them now. I just took a look with accProbe, and am seeing them there, with no name and no description. All good! :)
Caolan McNamara committed a patch related to this issue. It has been pushed to "libreoffice-4-2": http://cgit.freedesktop.org/libreoffice/core/commit/?id=a9a2b6d63231b514754f16c029ca35bb7383e139&h=libreoffice-4-2 Resolves: fdo#74284 use AccessibleRole::PANEL for VclBoxes under windows It will be available in LibreOffice 4.2.3. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
(In reply to comment #24) > That's interesting, because in the 4.3-dev build, I now don't see any option > panes on that start screen any more. However, since bug 75151 might also > play a part here, they may simply not be exposed at all any more since there > is neither a name nor an acc description to expose. If neither name nor description is exposed on an option pane, NVDA will treat it as a layout object and thus ignore it. You'll be able to see it if you disable simple review and use object navigation. It's important to note that if these VclBoxes ever get names or descriptions, NVDA will stop ignoring them.
Caolan McNamara committed a patch related to this issue. It has been pushed to "libreoffice-4-2-1": http://cgit.freedesktop.org/libreoffice/core/commit/?id=47ff82608b065c3c0d98557583bb822b20fb4081&h=libreoffice-4-2-1 Resolves: fdo#74284 use AccessibleRole::PANEL for VclBoxes under windows It will be available already in LibreOffice 4.2.1. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Caolan McNamara committed a patch related to this issue. It has been pushed to "libreoffice-4-2-2": http://cgit.freedesktop.org/libreoffice/core/commit/?id=5754e66f54fbf0a3e2eb96fa7afdf963a230d670&h=libreoffice-4-2-2 Resolves: fdo#74284 use AccessibleRole::PANEL for VclBoxes under windows It will be available already in LibreOffice 4.2.2. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
correcting target, the script is confused...