Bug 138068 - When there are a lot of pictures, typing the text is very slow (macOS/GTK3/GDI)
Summary: When there are a lot of pictures, typing the text is very slow (macOS/GTK3/GDI)
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
7.1.0.3 release
Hardware: All All
: highest critical
Assignee: Not Assigned
URL:
Whiteboard: target:7.2.0 target:7.0.5 target:7.1.1
Keywords: bibisected, bisected, perf, regression
: 134241 136265 137841 138125 138808 139755 139925 (view as bug list)
Depends on:
Blocks: Regressions-cairo-speedup
  Show dependency treegraph
 
Reported: 2020-11-08 10:40 UTC by GLAUDE
Modified: 2021-02-24 20:43 UTC (History)
21 users (show)

See Also:
Crash report or crash signature:


Attachments
fonts no correct (374.32 KB, image/png)
2020-11-08 10:46 UTC, GLAUDE
Details
Example file (141.72 KB, application/vnd.oasis.opendocument.text)
2020-11-09 18:39 UTC, Telesto
Details
Flamegraph (128.30 KB, application/x-bzip)
2021-02-05 16:59 UTC, Julien Nabet
Details
flamegraph (536.69 KB, image/png)
2021-02-05 19:22 UTC, Luboš Luňák
Details

Note You need to log in before you can comment on or make changes to this bug.
Description GLAUDE 2020-11-08 10:40:49 UTC
Description:
There is a bug in version 7 of Libre Office concerning memory management and fonts. When there are a lot of pictures, typing the text is very slow. In addition, all letters with accent shift and come before a syllable. Example: egeneration instead of generation.

Steps to Reproduce:
1.There is a bug in version 7 of Libre Office concerning memory management and fonts. When there are a lot of pictures, typing the text is very slow. In addition, all letters with accent shift and come before a syllable. Example: egeneration instead of generation.
2.
3.

Actual Results:
There is a bug in version 7 of Libre Office concerning memory management and fonts. When there are a lot of pictures, typing the text is very slow. In addition, all letters with accent shift and come before a syllable. Example: egeneration instead of generation.

Expected Results:
There is a bug in version 7 of Libre Office concerning memory management and fonts. When there are a lot of pictures, typing the text is very slow. In addition, all letters with accent shift and come before a syllable. Example: egeneration instead of generation.


Reproducible: Always


User Profile Reset: Yes



Additional Info:
There is a bug in version 7 of Libre Office concerning memory management and fonts. When there are a lot of pictures, typing the text is very slow. In addition, all letters with accent shift and come before a syllable. Example: egeneration instead of generation.
Comment 1 GLAUDE 2020-11-08 10:46:08 UTC
Created attachment 167093 [details]
fonts no correct
Comment 2 Telesto 2020-11-08 14:25:31 UTC Comment hidden (obsolete)
Comment 3 Telesto 2020-11-09 09:28:55 UTC
They attachment's I found in the bug report. I was talking about they document shown inside they screenshot (or similar document) which exhibits they issue: "When there are a lot of pictures, typing the text is very slow."
Comment 4 Telesto 2020-11-09 18:39:45 UTC
Created attachment 167158 [details]
Example file
Comment 5 Telesto 2020-11-09 18:40:21 UTC
Confirming slowness with GDI on Windows and also on MacOS
Version: 7.1.0.0.alpha1+ (x64)
Build ID: 5a96093f0ecee53432bdf35f247edd6deb501baf
CPU threads: 4; OS: Windows 6.3 Build 9600; UI render: default; VCL: win
Locale: nl-NL (nl_NL); UI: en-US
Calc: CL
Comment 6 Telesto 2020-11-09 21:22:24 UTC
Bibisected using bibisect-win64-7.0 in GDI mode to:
URL: https://cgit.freedesktop.org/libreoffice/core/commit/?id=828504974d70111e4a35b31d579cf42fe660a660
author: Armin Le Grand (Collabora) <armin.le.grand@me.com>
committer: Armin Le Grand <Armin.Le.Grand@me.com>
summary: tdf#130768 speedup huge pixel graphics Cairo

Adding CC: Armin Le Grand
Comment 7 Telesto 2020-11-12 08:56:38 UTC
FWIW: not sure what the proper approach would be. GDI will reach EOL. So more at the level of MacOS. There is some talk about moving to Skia; not sure how realistic actually is. But of course waste of time to look into this is Skia will be implemented anyhow

No clue if other (Linux) backends are affected too.
Comment 8 Telesto 2020-12-11 07:35:02 UTC
*** Bug 138808 has been marked as a duplicate of this bug. ***
Comment 9 Telesto 2020-12-19 16:15:08 UTC
*** Bug 138125 has been marked as a duplicate of this bug. ***
Comment 10 Telesto 2020-12-19 16:16:10 UTC
*** Bug 139068 has been marked as a duplicate of this bug. ***
Comment 11 Telesto 2020-12-19 16:26:04 UTC
@Xisco,
There is likely a pretty big perf flaw at Linux/MacOS level related to images (Windows Skia unaffected) . Would be lovely if you could double/triple check this and maybe bumping priority even more..

The complains starting to drop in.. and we are going in the direction of 7.0.5
Comment 12 Jean-François Fortin Tam 2020-12-25 17:29:26 UTC
Bug 139068 is not specific to the 7.0 performance regression, it goes back much further than that. I have thus reopened that bug report and attached a new heavier sample that triggers the problem even on 6.3.x. With some luck, it may have some sort of underlying relationship with 138068 (here) but possibly not, and I suspect that even if you revert the regression from 7.0 the other performance bug might remain.
Comment 13 Alberto Salvia Novella 2020-12-25 17:50:18 UTC
Unsubscribing. Feel free to add me back if you need more info from me.

My bug report was:
https://bugs.documentfoundation.org/show_bug.cgi?id=138808
Comment 14 Telesto 2021-01-02 13:16:23 UTC Comment hidden (obsolete)
Comment 15 Telesto 2021-01-05 10:04:49 UTC
Bibisected this again on Linux with GTK3.. same result as comment 6
Bugs about this a trickling.. (see also are often duplicates) FWIW: there is also bug 139068 for 7.1 which maybe entangled..

So taking the the step to bump priority to maximum. Feel free to correct me if being wrong doing that
Comment 16 Telesto 2021-01-16 08:57:47 UTC
FWIW: this was present under Windows in the past: bug 88678. Sample file there should show the issue pretty well.. Nowday's OK with Windows (after Skia buffering)
Comment 17 Telesto 2021-01-19 09:07:42 UTC
*** Bug 139755 has been marked as a duplicate of this bug. ***
Comment 18 Telesto 2021-01-30 07:01:15 UTC
*** Bug 139993 has been marked as a duplicate of this bug. ***
Comment 19 Yonghyun Kim 2021-02-04 14:46:19 UTC
Even if there is only one picture, keyboard input is very slow.
It seems that the lag has disappeared in Windows 10 from version 7.1.
However, in Linux, the input delay seems to be worse if there is still a picture.
Comment 20 Julien Nabet 2021-02-05 16:59:50 UTC
Created attachment 169504 [details]
Flamegraph

On pc Debian x86-64 with master sources updated today + gen rendering, I don't reproduce the slowliness when typing anything after an image.

I attached a Flamegraph but I don't think it'll help.
Comment 21 Luboš Luňák 2021-02-05 19:21:35 UTC
The document from comment #4 spends its time in DrawTransformedBitmap(). The problem is that if scaling is involved, it is left to Cairo to do it, and that is done by the CPU, thus relatively slow, and moreover each redraw needs that again and again. I don't know why the cache from Armin's change doesn't cache it, Skia code manages to cache it just fine. Either that cache needs to get fixed, or at least for the simple "at most translate+scale but no shear or rotate" case the scaling should be done manually using BitmapEx::Scale(), which should get cached. Skia's SkiaSalGraphicsImpl::drawTransformedBitmap() can be used as an inspiration for that case.
Comment 22 Luboš Luňák 2021-02-05 19:22:03 UTC
Created attachment 169513 [details]
flamegraph
Comment 23 sramme 2021-02-06 11:16:17 UTC
I confirm Comment 19: Even with only one image or picture of moderate size (about 600 kb) scrolling and typing gets very slow. I have tried it on Linux Mint 19.3 (Mate), Manjaro (Xfce, Virtual Machine), Ubuntu 20.4 (Gnome, Virtual Machine), as well as on Windows 10 (Virtual Machine). On Windows 10 the lagging appears not to be so profound as on Linux.
Scrolling and typing remains very slow when LibreOffice is started in "safe mode".
The type of the picture (foto, drawing, etc.) and the file-type (jpg, png, etc.) does make no difference in the behaviour.
The slow scrolling and typing is only the case, if the image is visible in the document. If you scroll up or down past the image, and the image is out of the view pane, scrolling and typing speed reverts to normal.

Linux Mint 19.3 Mate (normal install)
Version: 7.0.4.2
Build ID: 00(Build:2)
CPU threads: 8; OS: Linux 5.4; UI render: default; VCL: gtk3
Locale: de-DE (de_DE.UTF-8); UI: de-DE
Ubuntu package version: 1:7.0.4_rc2-0ubuntu0.18.04.2
Calc: threaded

Manjaro 20.2 Xfce (normal install, Virtual Machine)
Version: 7.0.4.2
Build ID: 00(Build:2)
CPU threads: 4; OS: Linux 5.4; UI render: default; VCL: gtk3
Locale: de-DE (de_DE.utf8); UI: de-DE
7.0.4-3
Calc: threaded

Ubuntu 20.4 Gnome (normal install, Virtual Machine)
Version: 7.0.4.2
Build ID: 1c5f81ee28659974774060c3fe084e73b3bd074b
CPU threads: 2; OS: Linux 5.4; UI render: default; VCL: gtk3
Locale: de-DE (de_DE.UTF-8); UI: de-DE
Calc: threaded

Windows 10 (normal install, Virtual Machine)
Version: 7.0.4.2 (x64)
Build ID: dcf040e67528d9187c66b2379df5ea4407429775
CPU threads: 3; OS: Windows 10.0 Build 19042; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: threaded

It also happened in the AppImage of the most recent release 7.1. The lagging in scrolling and typing appears to be even worse.

Linux Mint 19.3 Mate (AppImage)
Version: 7.1.0.3 / LibreOffice Community
Build ID: f6099ecf3d29644b5008cc8f48f42f4a40986e4c
CPU threads: 8; OS: Linux 5.4; UI render: default; VCL: gtk3
Locale: de-DE (de_DE.UTF-8); UI: de-DE
Calc: threaded


The only thing that will make working with such a file possible is to show place holders for the graphics (Option: 'View Images and Charts'; Deutsch: 'Bilder und Diagramme').

----------

There are *no* issues (even with several pictures inserted) in the latest 6.x Version of LibreOffice (tested as an AppImage).

Linux Mint 19.3 Mate (AppImage)
Version: 6.4.7.2
Build ID: 639b8ac485750d5696d7590a72ef1b496725cfb5
CPU threads: 8; OS: Linux 5.4; UI render: default; VCL: gtk3; 
Locale: de-DE (de_DE.UTF-8); UI-Language: en-US
Calc: threaded

----------

In Windows 10 and the most recent LibreOffice 7.1 the Problem seems to be solved (tested with the portable version).

Windows 10 (portable version of LibreOffice)
Version: 7.1.0.3 (x86) / LibreOffice Community
Build ID: f6099ecf3d29644b5008cc8f48f42f4a40986e4c
CPU threads: 3; OS: Windows 10.0 Build 19042; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: threaded
Comment 24 Michael Meeks 2021-02-08 13:37:06 UTC
Perhaps we could revert:

https://cgit.freedesktop.org/libreoffice/core/commit/?id=828504974d70111e4a35b31d579cf42fe660a660

while we try to work out how to avoid DrawTransformedBitmap continuing to be an expensive operation nearly everywhere. It is sad to have to have caches in the rendering backends  cairo, skia etc. to accelerate this when my understanding is that a device resolution cached version of the output really belongs higher up in the drawing-layer.

Anyhow the patch is not needed by us to solve any performance issue we see I think.
Comment 25 Luboš Luňák 2021-02-08 16:05:36 UTC
I have locally reverted the change (plus two related) and I can confirm that doing so noticeably improves the performance in this case. I don't know what other positive or negative effects it may have.

Since doing the revert was non-trivial, I have posted it at https://gerrit.libreoffice.org/c/core/+/110588 , in case reverting is the decision (but I don't want to be the one making it).
Comment 26 Telesto 2021-02-08 16:48:56 UTC
(In reply to Luboš Luňák from comment #25)
> Since doing the revert was non-trivial, I have posted it at
> https://gerrit.libreoffice.org/c/core/+/110588 , in case reverting is the
> decision (but I don't want to be the one making it).

Lets divert responsibility to some kind of collective one, where everyone can point into circles. I did it because I'm told. I did it because agreed.. :P

From me a go ahead; can't become worse, IMHO. Which means revert in Master.. Revert the revert is still an option, so.. Backporting has to wait +/- 8 days
Hope that will be enough for some basic testing (especially checking for regressions as Skia). Hope it can make it into 7.0.6 (ideally around 21 February)
Comment 27 Noel Grandin 2021-02-08 17:00:53 UTC
Possibly the low-level backends really are the best place to cache generated bitmaps, maybe with a little cooperation across the API, e.g. how we cache glyph renders for text.

The low-level backends are the place that finally has all the relevant information available.

I'm fairly sure Caolan implemented a cache for rendering pictures in writer, so I'm surprised another one (from Armin) was even necessary.
Comment 28 Luboš Luňák 2021-02-08 19:17:37 UTC
I tend to agree that this belongs rather in the lower levels. I'm admittedly not very knowledgeable about the higher levels, but I wonder how they would know how to actually do the caching. I mean, how can it know what the actual result will be, unless every place calling the lowlevel functions basically duplicates some of the logic?

As for the cache in Writer, if these is one, then it apparently does not work. The only reason why the Skia backend is not hit by this is because it has its own implementation of a cache.

And the Skia library provides SkImage::uniqueID() that changes on every content change, so it's pretty easy to do caching there. Possibly the only case when it wouldn't work and a cache might make sense on a higher level would be if there are places of code that modify the to-be-drawn bitmap on every invocation e.g. in order to apply transparency or some effect (i.e. the pixel data would be the same, but conceptually it would be a different bitmap). But even in that (IMO broken from a performance point of view) case I doubt higher levels would have an easier time caching that.
Comment 29 Luboš Luňák 2021-02-09 12:01:34 UTC
> As for the cache in Writer, if these is one, then it apparently does not
> work. The only reason why the Skia backend is not hit by this is because it
> has its own implementation of a cache.

Actually I take that back. Apparently Armin's patch broke the Writer cache, reverting the commit makes the performance good even if I disable the Skia cache.
Comment 30 Armin Le Grand 2021-02-10 13:06:04 UTC
@Luboš Luňák: Please do not do it in this raw form. It will re-introduce bugs (on quick ref I just saw tdf#130768 in vcl/source/outdev/bitmap.cxx, there may be more. But also primitive buffering for writer bitmaps, VOC mechanism and SysDep BitmapBuffering - all that is needed and will be needed in future changes would be gone.
Main problem seems to be that I preferred DrawTransformBitmapExDirect in  vcl/source/outdev/bitmap.cxx cmpared to ther versions, that is the only thing that can influence stuff besides cairo. Please give me time to have a look to find a solution *without* taking back quite some stuff already done for a better future...
Comment 31 Noel Grandin 2021-02-10 13:47:58 UTC
For reference, the Caolan caching I was talking about was commit 9f3926df5c2828ad3e8cfce2b4444b1c84352cf4, tdf#123165, which is pretty much the same bug as this one
Comment 32 Luboš Luňák 2021-02-10 16:33:49 UTC
(In reply to Armin Le Grand from comment #30)
> Main problem seems to be that I preferred DrawTransformBitmapExDirect in 
> vcl/source/outdev/bitmap.cxx cmpared to ther versions, that is the only
> thing that can influence stuff besides cairo.

I confirm, it's really just this part that causes the performance problem. So https://gerrit.libreoffice.org/c/core/+/110716 should be enough for this bugreport.

(In reply to Noel Grandin from comment #31)
> For reference, the Caolan caching I was talking about was commit
> 9f3926df5c2828ad3e8cfce2b4444b1c84352cf4, tdf#123165, which is pretty much
> the same bug as this one

That's not a Writer cache though, that's a cache for the best quality BitmapEx::Scale() algorithm. So I assume what's happened was that Armin's patch made DrawTransformBitmapExDirect() preferred to DrawBitmapEx() that does BitmapEx::Scale() internally, but all backend implementations for DrawTransformBitmapExDirect() except the Skia one use uncached and possibly slow platform API for the scaling, hence the slowdown.
Comment 33 Armin Le Grand 2021-02-10 16:52:46 UTC
@Luboš Luňák: Thanks, that is much better. But you mention the reason behind it: We have too many methods at LowLevel stuff to draw Bitmaps - of course of historical reasons. We need DrawTransformBitmapEx anyways, so all other cases *could* and *should* be elliminated. The low-level impls may locally switch to old versions (e.g. when not reanny rotated or scaled), but the API shpuld be greatly simplified - what was the plan from the beginning - but - sigh...
Same for Polygon paint, BTW. Same for all methods that still have integer API - sigh...
And of course this also means that something like this can happen at all - of course DrawTransformBitmapEx should do the *same* buggering if possible that a simple DrawBitmapEx probably currently does. When we would only have DrawTransformBitmapEx it could not be added on a specialized place, ignoring other cases.
In practise this means if anyone should *dare* to update the API usage to paint char bitmaps in SW to use DrawTransformBitmapEx (which some systems can directy supposrt in HW, so we need it to be able to use *that*) we will be back with this error...
I had hoped that when doing a new backend like skia it would be a good point in time to reduce the API in that sense - would've made impl easier, too. Or -as suggested - get rid of all of this - and just have some GetProcessor2D from backend - call which you could then feed *any* kind of primitives - just dreaming, this is the mid-term (far?) future...
Comment 34 Commit Notification 2021-02-12 10:18:34 UTC
Luboš Luňák committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/9d89d98d3349502b56da4bdd6ea287ac4cde9ce5

always optimize bitmap transform to translate+scale if possible (tdf#138068)

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 35 Commit Notification 2021-02-12 20:25:34 UTC
Luboš Luňák committed a patch related to this issue.
It has been pushed to "libreoffice-7-0":

https://git.libreoffice.org/core/commit/27a4aea50a9efa5c839b0ae2de1f9f14a7782f11

always optimize bitmap transform to translate+scale if possible (tdf#138068)

It will be available in 7.0.5.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 36 Xisco Faulí 2021-02-15 17:01:39 UTC
I do confirm the issue is fixed in

Version: 7.2.0.0.alpha0+ / LibreOffice Community
Build ID: cbcec4425e04e3614a2025b49fdc221216ac51d3
CPU threads: 4; OS: Linux 5.7; UI render: default; VCL: gtk3
Locale: en-US (en_US.UTF-8); UI: en-US
Calc: threaded

@Luboš Luňák, thanks for fixing this issue. Should it be closed as RESOLVED FIXED ?
Comment 37 Xisco Faulí 2021-02-15 17:22:12 UTC
*** Bug 136265 has been marked as a duplicate of this bug. ***
Comment 38 Xisco Faulí 2021-02-15 17:36:31 UTC
*** Bug 134241 has been marked as a duplicate of this bug. ***
Comment 39 Xisco Faulí 2021-02-16 08:29:43 UTC
*** Bug 137841 has been marked as a duplicate of this bug. ***
Comment 40 Xisco Faulí 2021-02-16 08:45:14 UTC
*** Bug 139925 has been marked as a duplicate of this bug. ***
Comment 41 Commit Notification 2021-02-16 08:47:57 UTC
Luboš Luňák committed a patch related to this issue.
It has been pushed to "libreoffice-7-1":

https://git.libreoffice.org/core/commit/b6d67d3d18d46daaabb3532d176759c234196da7

always optimize bitmap transform to translate+scale if possible (tdf#138068)

It will be available in 7.1.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 42 Luboš Luňák 2021-02-16 10:13:11 UTC
Yes, I think we can consider this fixed.
Comment 43 Telesto 2021-02-20 16:17:15 UTC
*** Bug 138808 has been marked as a duplicate of this bug. ***
Comment 44 Commit Notification 2021-02-24 14:25:16 UTC
Luboš Luňák committed a patch related to this issue.
It has been pushed to "libreoffice-7-1-1":

https://git.libreoffice.org/core/commit/7a0b7ef6d490e7ed7c35874b3feca75c79effdb7

always optimize bitmap transform to translate+scale if possible (tdf#138068)

It will be available in 7.1.1.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.