Bug 149952 - 7.3.4.2 view jumps around when (linux) system zoom enabled
Summary: 7.3.4.2 view jumps around when (linux) system zoom enabled
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
7.3.4.2 release
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: Michael Weghorn
URL:
Whiteboard: target:7.5.0 target:7.3.6 target:7.4....
Keywords:
Depends on:
Blocks: a11y-Linux Wayland GTK3
  Show dependency treegraph
 
Reported: 2022-07-11 19:30 UTC by Heather Ellsworth
Modified: 2022-09-25 17:00 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:
Regression By:


Attachments
video showing viewability issue (982.39 KB, video/webm)
2022-07-11 19:30 UTC, Heather Ellsworth
Details
Screencast with https://gerrit.libreoffice.org/c/core/+/137027 (v2) in place (4.27 MB, video/webm)
2022-07-14 10:10 UTC, Michael Weghorn
Details
pyatspi script that prints selection events on paragraphs (401 bytes, text/x-python)
2022-07-15 11:29 UTC, Michael Weghorn
Details
Backtrace for unwanted paragraph selection event (6.43 KB, text/plain)
2022-07-15 11:29 UTC, Michael Weghorn
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Heather Ellsworth 2022-07-11 19:30:45 UTC
Created attachment 181231 [details]
video showing viewability issue

The viewing area jumps around when the system-wide zoom is in use on GNOME. This exists with the release 7.3.4.2 deb provided by TDF, on a fresh Debian testing VM, although it was first reported on Ubuntu 22.04 with 7.3.4-0ubuntu0.22.04.1.

To reproduce:
* have a GNOME desktop environment on Debian testing or Ubuntu 22.04
* Open the Accessibility panel of gnome-control-center
* Turn on Zoom with any scalability
* Open Libreoffice Writer/Impress/Calc (video only shows Writer although the problem is at least in these three applications)
* start typing such that text goes off of the viewing area
* see the viewing area jump back to the top left corner, out of view of the text being typed.
Comment 1 Heather Ellsworth 2022-07-11 23:35:04 UTC
This issue does not occur on Debian Testing with KDE Plasma, so it must be a GNOME/GTK issue. Please feel free to close this since it is not a LO issue and I'll find the proper project.

Thanks!
Comment 2 Sebastian Keller 2022-07-12 09:13:22 UTC
> so it must be a GNOME/GTK issue

It is a LibreOffice issue. The gnome-shell magnifier started requesting window relative (ATSPI_COORD_TYPE_WINDOW) character extents instead of screen relative (ATSPI_COORD_TYPE_SCREEN) ones, because on Wayland applications don't really know where their windows are placed on the screen:

https://gitlab.gnome.org/GNOME/gnome-shell/-/commit/cb0c4bd1c2c19645e4cd1be9733c22e8747b07d8

The coordinates provided by LibreOffice when requesting these window relative extents however seem wrong. When looking at the caret coordinates in Writer, the coordinates seem to be relative to the paragraph instead of the window. So it looks like when ATSPI_COORD_TYPE_WINDOW is requested LibreOffice is somehow assuming ATSPI_COORD_TYPE_PARENT.

Related:
https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/5642
https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/5509
Comment 3 Shinji Enoki 2022-07-12 12:22:06 UTC
Reproduced in:

Debian sid
Version: 7.5.0.0.alpha0+ / LibreOffice Community
Build ID: 4c8537589f2a73d1fbecaf4c1e9c64627b79e542
CPU threads: 8; OS: Linux 5.18; UI render: default; VCL: gtk3
Locale: ja-JP (ja_JP.UTF-8); UI: en-US
Calc: threaded
Comment 4 Zebull 2022-07-12 12:55:59 UTC
quote: "This issue does not occur on Debian Testing with KDE Plasma..."   

KDE / KDE plasma has (to my knowledge) never made the follow text-cursor/caret tracking to work at all with zoom enabled. Even if there is a tick-box.   

Therefore, it is unfortunately not possible to find out if LibreOffice has the same problem with KDE.
Comment 5 Michael Weghorn 2022-07-13 11:19:36 UTC
(In reply to Sebastian Keller from comment #2)
> It is a LibreOffice issue. The gnome-shell magnifier started requesting
> window relative (ATSPI_COORD_TYPE_WINDOW) character extents instead of
> screen relative (ATSPI_COORD_TYPE_SCREEN) ones, because on Wayland
> applications don't really know where their windows are placed on the screen:
> 
> https://gitlab.gnome.org/GNOME/gnome-shell/-/commit/
> cb0c4bd1c2c19645e4cd1be9733c22e8747b07d8
> 
> The coordinates provided by LibreOffice when requesting these window
> relative extents however seem wrong. When looking at the caret coordinates
> in Writer, the coordinates seem to be relative to the paragraph instead of
> the window. So it looks like when ATSPI_COORD_TYPE_WINDOW is requested
> LibreOffice is somehow assuming ATSPI_COORD_TYPE_PARENT.

Thanks, that's extremely helpful!

Indeed, that's exactly what I see when I check the positions with a Writer paragraph selected in Accerciser:

> In [82]: acc.queryComponent().getPosition(pyatspi.component.XY_SCREEN)
> Out[83]: (1989, 415)
> In [83]: acc.queryComponent().getPosition(pyatspi.component.XY_PARENT)
> Out[84]: (19, 113)
> In [84]: acc.queryComponent().getPosition(pyatspi.component.XY_WINDOW)
> Out[85]: (19, 113)
> In [85]: acc.queryText().getCharacterExtents(0, pyatspi.component.XY_SCREEN)
> Out[86]: (2065, 415, 12, 19)
> In [86]: acc.queryText().getCharacterExtents(0, pyatspi.component.XY_PARENT)
> Out[87]: (76, 0, 12, 19)
> In [87]: acc.queryText().getCharacterExtents(0, pyatspi.component.XY_WINDOW)
> Out[88]: (76, 0, 12, 19)

I'm working on a fix.
Comment 6 Michael Weghorn 2022-07-13 13:05:39 UTC
Potential fix: https://gerrit.libreoffice.org/c/core/+/137027

The values returned by Accerciser for the scenario described in comment 5 look somewhat plausible to me with that in place, but I haven't checked the actual scenario from this bug report in a GNOME session yet. (I'd have to set up a corresponding environment first).

If anybody wants to try that, that would be appreciated.
Comment 7 Michael Weghorn 2022-07-14 10:10:38 UTC
Created attachment 181260 [details]
Screencast with https://gerrit.libreoffice.org/c/core/+/137027 (v2) in place

(In reply to Michael Weghorn from comment #6)
> Potential fix: https://gerrit.libreoffice.org/c/core/+/137027
> 
> The values returned by Accerciser for the scenario described in comment 5
> look somewhat plausible to me with that in place, but I haven't checked the
> actual scenario from this bug report in a GNOME session yet. (I'd have to
> set up a corresponding environment first).

I have given that a try in a Debian testing VM, s. attached screencast.

Do I understand correctly that it works as expected now when typing actual text (roughly the first 12 seconds in the screencast).

However, the view jumps to the right of the doc when just pressing Enter (i.e. typing empty lines of text).
This looks like a potentially different issue (either in LO or in the GNOME magnifier) to me, but I haven't investigated any further so far. Was that behaving differently with previous GNOME versions?
Comment 8 Michael Weghorn 2022-07-14 13:25:29 UTC
I have done some further testing and have compared the behavior with the one on Debian bullseye (gnome 1:3.38+3, libreoffice-writer 1:7.0.4-4+deb11u1), which doesn't include the GNOME magnifier change and works fine.

(In reply to Michael Weghorn from comment #7)
> Do I understand correctly that it works as expected now when typing actual
> text (roughly the first 12 seconds in the screencast).

Behaves the same on Debian bullseye, so looks OK.

> However, the view jumps to the right of the doc when just pressing Enter
> (i.e. typing empty lines of text).
> This looks like a potentially different issue (either in LO or in the GNOME
> magnifier) to me, but I haven't investigated any further so far. Was that
> behaving differently with previous GNOME versions?

I don't see this problem in the Debian bullseye setup (which uses GNOME on Wayland).
I also don't see this in my Debian testing VM with the fix in place when in a GNOME X11 session, just in a Wayland one.

So, this actually looks like a different issue to me and it's unclear to me whether that's actually a LibreOffice issue or not.

@Sebastian: Do you possibly have any idea?

Unless there is any other feedback, I'll probably merge the pending change tomorrow and suggest to handle any remaining issues separately.
Comment 9 Sebastian Keller 2022-07-14 14:23:06 UTC
Thanks for the quick fix! I haven't tested it yet, because I first would have to figure out how to build LibreOffice. Or is there some easier way to test this?

> @Sebastian: Do you possibly have any idea?

Nothing specifically, but you could try this with a non-maximized window and move that around to see if that has any effect.

Also you could check what the 'object:text-caret-moved', 'object:state-changed:focused' or 'object:state-changed:selected' event is that seems to trigger this and whether it comes from LibreOffice or not and what the extents are.

> Unless there is any other feedback, I'll probably merge the pending change tomorrow and
> suggest to handle any remaining issues separately.

That seems like a good idea. Feel free to CC me in the new bug if there is anything I could help with.
Comment 10 Michael Weghorn 2022-07-14 15:31:38 UTC
(In reply to Sebastian Keller from comment #9)
> Thanks for the quick fix! I haven't tested it yet, because I first would
> have to figure out how to build LibreOffice. Or is there some easier way to
> test this?

I've temporarily uploaded my local debug build from Debian testing here, which should run on any Debian testing and potentially also other systems with similar/new enough library versions:
https://nextcloud.documentfoundation.org/s/XDY49kwiLpwccSt
(Just run soffice.bin from the instdir/program directory)

Once the change is submitted to master, the official daily builds can be used (which are more portable):
https://dev-builds.libreoffice.org/daily/master/current.html
https://wiki.documentfoundation.org/Installing_in_parallel/Linux has some more infos on how to use them without installing


> > @Sebastian: Do you possibly have any idea?
> 
> Nothing specifically, but you could try this with a non-maximized window and
> move that around to see if that has any effect.
> 
> Also you could check what the 'object:text-caret-moved',
> 'object:state-changed:focused' or 'object:state-changed:selected' event is
> that seems to trigger this and whether it comes from LibreOffice or not and
> what the extents are.

I'll try whether I can see anything suspicious from the LO side.

> Feel free to CC me in the new bug if there is
> anything I could help with.

Thanks! If there's anything you can see on the gnome-shell magnifier side, that would certainly be of great help.
Comment 11 Sebastian Keller 2022-07-14 16:31:50 UTC
> I've temporarily uploaded my local debug build ...

Thanks, it's working here on Fedora 36 and fixes the original issue.

Regarding the issue of jumping to the right, it looks like there is a 'object:state-changed:selected' for the paragraph when pressing enter multiple times. This causes the magnifier to center the entire paragraph, which has the width of the entire page according to the extents, so this will be further to the right compared to caret. Is this what you are observing or is it jumping even further to the right?
Comment 12 Commit Notification 2022-07-15 11:22:00 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/7b312771d7eb33f7410167e36efdaeca6f540b1c

tdf#149952 gtk3 a11y: Return pos relative to window when requested

It will be available in 7.5.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 13 Michael Weghorn 2022-07-15 11:29:11 UTC
Created attachment 181280 [details]
pyatspi script that prints selection events on paragraphs
Comment 14 Michael Weghorn 2022-07-15 11:29:45 UTC
Created attachment 181281 [details]
Backtrace for unwanted paragraph selection event
Comment 15 Michael Weghorn 2022-07-15 11:43:01 UTC
(In reply to Sebastian Keller from comment #11)
> Thanks, it's working here on Fedora 36 and fixes the original issue.

Great, I've merged that change to master, backports for 7.4 and 7.3 pending in Gerrit:
https://gerrit.libreoffice.org/c/core/+/137060
https://gerrit.libreoffice.org/c/core/+/137061

> Regarding the issue of jumping to the right, it looks like there is a
> 'object:state-changed:selected' for the paragraph when pressing enter
> multiple times. This causes the magnifier to center the entire paragraph,
> which has the width of the entire page according to the extents, so this
> will be further to the right compared to caret. Is this what you are
> observing or is it jumping even further to the right?

Thanks! That indeed seems to be the exact problem. The issue is no more reproducible with a local test change to no longer send this kind of events.

The pyatspi script (attachment 181280 [details]) shows that many more of these events are being sent, e.g. when just moving the cursor, not doing any actual selection.

The backtrace in attachment 181281 [details] seems to indicate that this is triggered by some IM (input method) handling. (I can't reproduce it on my usual Debian testing even in a GNOME Wayland session, just in my VM, so probably something is different in the IM-related setup there.)

The event seems to be triggered by SwEditWin::GetSurroundingText doing an "artificial"/internal text selection to retrieve the text around the cursor position, which to me looks like an an implementation detail that should not be exposed to the outside.

With the WIP/demo patch at https://gerrit.libreoffice.org/c/core/+/137104, this seems to work as expected, but that isn't ready to be merged as is yet.
Comment 16 Zebull 2022-07-15 21:30:49 UTC
I am heavily dependent on GNOME Zoom and LO (with working text-cursor).

Tried to download 7.5.0.0 daily build to test. Both DEB and 'make_libreoffice_appimage.sh' failed (Pop_os 22.04).

MY QUESTION:
(Before I wrestle with more installation attempts). Are the changes mentioned above included in version 7.5.0.0 daily build?
Comment 17 Michael Weghorn 2022-07-16 00:52:03 UTC
(In reply to Zebull from comment #16)
> (Before I wrestle with more installation attempts). Are the changes
> mentioned above included in version 7.5.0.0 daily build?

Right now, the daily build available at https://dev-builds.libreoffice.org/daily/master/current.html is from before the commit mentioned in comment 12 was merged, but a new build should become available later today that does include those changes (check the time stamp given on that download page/in the download path).

Upcoming daily builds should work fine with magnifier when typing actual text.
Changes that also fix the remaining issue about the focus moving to the right when typing empty lines are still work-in-progress, and therefore not included in the daily build for now.
Comment 18 Michael Weghorn 2022-07-16 00:54:37 UTC
(In reply to Michael Weghorn from comment #17)
> (In reply to Zebull from comment #16)
> > (Before I wrestle with more installation attempts). Are the changes
> > mentioned above included in version 7.5.0.0 daily build?
> 
> Right now, the daily build available at
> https://dev-builds.libreoffice.org/daily/master/current.html is from before
> the commit mentioned in comment 12 was merged, but a new build should become
> available later today that does include those changes (check the time stamp
> given on that download page/in the download path).
> 
> Upcoming daily builds should work fine with magnifier when typing actual
> text.
> Changes that also fix the remaining issue about the focus moving to the
> right when typing empty lines are still work-in-progress, and therefore not
> included in the daily build for now.

@Zebull: s. above (just realized you hadn't been subscribed to the bug yet)
Comment 19 Zebull 2022-07-16 13:11:33 UTC
I can happily confirm that LibreOfficeDev_7.5.0.0 (DEB) with timestamp 2022-07-16 08:40:30 works as expected (Pop_os 22.04, GNOME 42.2 X11). Zoomed focus now follows the text cursor as it should. Have tested Writer and Calc at the moment.   

I have so far not discovered the second problem with enter / empty row jumping to the right. Maybe it's a Wayland thing?   

This is the joy of open source. Talented people help with solving problems instead of keeping their solutions secret.
Comment 20 Michael Weghorn 2022-07-16 15:03:21 UTC
(In reply to Zebull from comment #19)
> I can happily confirm that LibreOfficeDev_7.5.0.0 (DEB) with timestamp
> 2022-07-16 08:40:30 works as expected (Pop_os 22.04, GNOME 42.2 X11). Zoomed
> focus now follows the text cursor as it should. Have tested Writer and Calc
> at the moment.

Great, thanks for confirming!
 
> I have so far not discovered the second problem with enter / empty row
> jumping to the right. Maybe it's a Wayland thing?

Yes, that's indeed Wayland only (and even there doesn't seem to appear in all setups, and could be worked around by setting GDK_BACKEND=x11 before starting LibreOffice to force it to use the x11 backend on Wayland, until a proper fix is in place).
Comment 21 Commit Notification 2022-07-17 13:48:40 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "libreoffice-7-3":

https://git.libreoffice.org/core/commit/0fba0df4f62b523e5315aea61393db0b40288e09

tdf#149952 gtk3 a11y: Return pos relative to window when requested

It will be available in 7.3.6.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 22 Commit Notification 2022-07-17 13:49:51 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "libreoffice-7-4":

https://git.libreoffice.org/core/commit/69c51d9f172da6490b676f6f4c30e9f173587582

tdf#149952 gtk3 a11y: Return pos relative to window when requested

It will be available in 7.4.0.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 23 Heather Ellsworth 2022-07-30 21:15:51 UTC
I tested the most recent daily build of 7.3.6.0 and confirm it fixes the issue. I also needed to export GDK_BACKEND=x11 to keep the screen from jumping to the right, as I am on Wayland and see the jumping without it.

Thank you so much for fixing this issue!!
Comment 24 Michael Weghorn 2022-08-01 07:40:47 UTC
(In reply to Heather Ellsworth from comment #23)
> I tested the most recent daily build of 7.3.6.0 and confirm it fixes the
> issue. I also needed to export GDK_BACKEND=x11 to keep the screen from
> jumping to the right, as I am on Wayland and see the jumping without it.

Thanks for confirming. I hope I'll find some time to continue working on https://gerrit.libreoffice.org/c/core/+/137104 sometime soon. Let's leave the ticket open until that's done.
Comment 25 Commit Notification 2022-09-23 05:50:15 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/782e4502898924eb94c4ff7f6c33d9ef36959f02

tdf#149952 sw a11y: Don't send a11y events for internal-only cursor

It will be available in 7.5.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 26 Michael Weghorn 2022-09-23 06:02:30 UTC
(In reply to Michael Weghorn from comment #24)
> I hope I'll find some time to continue working on
> https://gerrit.libreoffice.org/c/core/+/137104 sometime soon. Let's leave
> the ticket open until that's done.

That's done now in master. Backport for 7.4 pending in Gerrit:
https://gerrit.libreoffice.org/c/core/+/140466
Comment 27 Commit Notification 2022-09-23 08:18:22 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "libreoffice-7-4":

https://git.libreoffice.org/core/commit/57505216b119ede97ea4bfef154e7375bb7f0d91

tdf#149952 sw a11y: Don't send a11y events for internal-only cursor

It will be available in 7.4.3.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 28 Zebull 2022-09-25 17:00:08 UTC
I can happily state that this fix has now also reached backwards to LO 7.3.6.2.
This means that it is now included in the Ubuntu 22.04 repository.   
   
Nice work.