Bug 61558 - High CPU load when scrolling a document with many comments
Summary: High CPU load when scrolling a document with many comments
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
3.5.5.3 release
Hardware: All All
: high major
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: haveBacktrace, perf
: 92011 119485 122969 122971 (view as bug list)
Depends on:
Blocks: Writer-Comments Memory
  Show dependency treegraph
 
Reported: 2013-02-27 15:25 UTC by GiorgioMigliaccio
Modified: 2019-06-06 07:32 UTC (History)
14 users (show)

See Also:
Crash report or crash signature:


Attachments
Sample document (92.04 KB, application/vnd.oasis.opendocument.text)
2013-02-27 15:25 UTC, GiorgioMigliaccio
Details
proposal document edited with tracked changes (2.46 MB, application/msword)
2016-11-25 19:14 UTC, konsultor
Details
Example file (49.70 KB, application/vnd.oasis.opendocument.text)
2019-06-05 13:18 UTC, Telesto
Details

Note You need to log in before you can comment on or make changes to this bug.
Description GiorgioMigliaccio 2013-02-27 15:25:05 UTC
Created attachment 75642 [details]
Sample document

When you open attached document in LibreOffice, just using the page scrollbars constantly, or using page-up page-down keys make the CPU usage go to 80-100% and also the memory keeps on increasing about 1 MB/sec. This never stops increasing as long as you use the page scrollbar.
In our embedded situation we have linked the sections to real sub documents. In this case, when just open, LibreOffice uses 300 MB. When using the page scrollbar, memory goes up to 1.8 GB and then LibreOffice just crashes.
Comment 1 GiorgioMigliaccio 2013-02-27 15:26:23 UTC
We have the same problem on a LO 3.5.5 installation.
Comment 2 Jorendc 2013-02-27 15:43:28 UTC
@Giorgio: what's the difference using with Bug 60418?
Comment 3 Jorendc 2013-02-27 15:44:18 UTC Comment hidden (obsolete)
Comment 4 GiorgioMigliaccio 2013-02-27 16:03:58 UTC
The other Bug 60418 talks about huge memory consumption and CPU usage WHILE opening a complex document. This was even due to the presence of 'form:form' tags in the content.xml.
This bug speaks about huge performance peaks while EDITING the document, after it has been opened just fine.
Comment 5 Jorendc 2013-02-27 16:38:52 UTC
Thanks for your clear reply.

(In reply to comment #4)
> This bug speaks about huge performance peaks while EDITING the document,
> after it has been opened just fine.

I can confirm that behavior using Linux Mint 14 x64 and LibreOffice Version 4.1.0.0.alpha0+ (Build ID: 5fcb553c862d407aadb0320925723d3c2f70bfe).

Kind regards,
Joren
Comment 6 GiorgioMigliaccio 2013-02-28 09:37:54 UTC Comment hidden (obsolete)
Comment 7 GiorgioMigliaccio 2013-03-18 12:44:12 UTC Comment hidden (obsolete)
Comment 8 Jorendc 2013-03-19 11:35:18 UTC
(In reply to comment #7)
> Anyone?

Hi,

I'm sorry, I didn't saw your question. I think these 2 emails will be interesting to read: http://lists.freedesktop.org/archives/libreoffice/2013-February/045455.html and http://lists.freedesktop.org/archives/libreoffice/2013-February/046372.html

As mentioned there, you can contact existing consulting company like:

* Lanedo (http://www.lanedo.com/libreoffice.html)
* Credativ (http://www.credativ.com/)
* Codethink (http://www.codethink.co.uk/)
* Collabora (https://www.collabora.com/)

and individual certified developers (they are working on a list).

I hope this is an answer to your question?

Kind regards,
Joren
Comment 9 Michael Meeks 2013-03-19 11:41:32 UTC
And of course SUSE and others provide support & consultancy :-)
Comment 10 A. Fakhouri 2013-04-17 20:23:13 UTC
I can confirm this issue in LibreOffice 4.0.2.1 on Fedora i686.
Comment 11 Jorendc 2013-04-17 20:25:16 UTC Comment hidden (obsolete)
Comment 12 David 2013-05-04 01:30:15 UTC
I have been experiencing this on some of my documents also and I have discovered something that may help in locating the problem.  I tried it with the sample document attached to this bug and it worked for it also.  Go to the View menu and toggle the "Hidden Paragraphs" option.  Evidently this causes it to halt whatever loop it is stuck on.  This is not a permanent solution as my documents started to respond slowly again after a time and then I needed to toggle the option again.  To force it to exhibit the problem all one needs to do is to go to the Tools menu and select Update - Update all.  An extreme lag when doing any typing will be immediately noticed which can then be fixed by toggling the "Hidden Paragraphs" option again.  It doesn't matter if the option is initially enabled or disabled.  The lag will occur with it either way and the document will start responding again when it's toggled.

I would assume this bug is related to bugs 59714 and 58029 which were fixed.  Bug 56963 may also be affected by this but I haven't been able to test it because the sample document seems to be no longer available.

I believe this bug is a duplicate of bug 60418 in which the "Update All" and the "Hidden Paragraphs" commands exhibit the same behavior as they do for this bug.
Comment 13 ign_christian 2013-07-09 14:24:23 UTC
Using attached file, I can scrolling up & down fast enough (though there is little lagging). No halt when toggle off Hidden Paragraph, & no crash at all. 

Using 'only' 1.3GB dual Pentium processor with 2GB ram (LO 4.0.4.2 Win7 32bit).

@Giorgio/Have you tried latest stable release & resetting user profile?
Comment 14 GiorgioMigliaccio 2013-07-09 14:55:22 UTC
@ign_christian : no, haven't tried yet with latest stable release.
One specific behaviour though was, apart from the small lag, the memory consumption of soffice.bin kept growing and growing just by using the scrollbar. And if you keep doing this long enough (several minutes), the memory consumption became so high that eventually LO crashed (going over 1.x GB (32-bit)).
Can you confirm the ever-increasing memory consumption by just using scrollbars?
Comment 15 ign_christian 2013-07-09 15:04:56 UTC
Yes increasing slightly, I think it's not significant. 
No huge increasing like Bug 62768.

If using Windows, I suggest to do regularly defragmenting together with some registry & file cleaning. I believe it gives significant result :)
Comment 16 GiorgioMigliaccio 2013-07-09 15:10:19 UTC
Increasing 'slightly' a second, BUT constantly by small amounts just by using the scrollbar, until, after having worked for hours on the document and having used the scrollbar a lot, it reaches more than one GIGABYTE and finally an APPCRASH occurs. I don't see how this is insignificant....
Comment 17 Michael Meeks 2013-07-09 16:10:59 UTC
It is an unusual problem; the document has no large images in it (that I can see) which would be the obvious causes of leaks.

If someone has a build with symbols they can reproduce this in, then running under valgrind like this:

cd /path/to/program
export OOO_DISABLE_RECOVERY=1
valgrind --leak-check=full --show-reachable=yes --leak-resolution=high soffice.bin -writer >& /tmp/val-log

please be aware this will run 80x slower or so; when it's done (do a bit of scrolling etc.) then gzip and attach the /tmp/val-log

Hopefully that shows us where the issue is quite easily :-) NB. you need a build with symbols [ on a distro just installing the debuginfo prolly does it ].

Thanks !
Comment 18 David 2013-07-11 02:03:56 UTC
And remember also that lags which cause increased cpu usage can be triggered by going to Tools | Update | Update All when View | Comments is enabled.  The extreme lag will be immediately noticed when typing anything into the document.  The lag can then be caused to disappear by toggling the View | Hidden Paragraphs option either on or off.
Comment 19 QA Administrators 2014-05-17 00:34:36 UTC Comment hidden (obsolete)
Comment 20 QA Administrators 2014-06-01 20:30:32 UTC Comment hidden (obsolete)
Comment 21 retired 2014-06-02 10:24:12 UTC
Hi all,

tested and I can confirm the described behavior: Opened test file in LO 4.3b1, grab scrollbar and move it up and down constantly. Activity monitor on OSX shows CPu usage going to 100%. If I let go of the scrollbar, CPU usage normalizes.

This is a document with lots of comment. So it's for Devs to decide wether this CPU usage is expected and "normal" or if this needs fixing.

From QA side at least, I can set this to NEW.
Comment 22 David 2014-06-02 21:51:01 UTC
With version 4.2.4 under linux I see nothing abnormal anymore when trying to scroll.  But there still is an EXTREME lag when trying to type anything into the document as exhibited when perfoming the actions described in an earlier post.
Comment 23 tonighx 2014-06-24 07:02:43 UTC
I have the same issue, especially with documents containing (not so) large immages. 
Affects .odt as well as .doc and .docx
The lag is far more evident when the incriminated immage is visible.
Not all immages cause the problem.

LO 4.2.3 and 4.2.2 on Ubuntu 12.04 on a HP 255 laptop
Comment 24 caralu1974 2014-11-25 20:41:44 UTC
With LO Versión: 4.3.4.1 Id. de compilación: 430m0(Build:1) I`m editing a file of 224 pages and lots of tables and it freezes constantly, it keep using 100% of one cpu core, and quit of responding even after several minutes. So I must kill LO and reopen the document with the correspondent recuperation and lost of work.
Comment 25 QA Administrators 2015-12-20 16:21:48 UTC Comment hidden (obsolete)
Comment 26 Paul 2016-03-25 02:17:06 UTC
Suffering same problem with LO 5.1.3 on Linux Lite 2.8 (ubuntu derivative) x64, system fully updated.

Any file of any larger size will freeze on normal operations such as find/replace all paragraph endings. CPU typically goes to 49%, which doesn't sound too bad, but the program freezes. This is happening on a simple .odt file of 1.7MB size.

Have had this problem since Day One on this system, using several versions of LO.

Machine is 2.7 GHz Core2Duo with 4gb ram and an SSD. Should not be a problem.
Comment 27 Ash 2016-04-04 13:09:18 UTC Comment hidden (me-too)
Comment 28 Ash 2016-04-04 13:13:13 UTC
This has been a problem since staroffice and possible. solved by minimizing the number of graphic elements loaded at startup of a document. For pages that are not plus minus 10 pages from the image on the screen.

This is a problem on all platforms including Windows 7, 8, 10.
Comment 29 Armin Le Grand 2016-04-12 08:16:44 UTC
Took a look. What seems very expensive are the comment visualizationms at the right. Switching these off makes all much better/responsive. I could not reproduce the ever-growing mem consumtion and not the crash, may be that a mem leak we had is already fixed.
Comment 30 Cor Nouws 2016-04-12 11:59:16 UTC
Testing in Version: 5.2.0.0.alpha0+
Build ID: 5a4b01f63d3f2a7d7d6fa8cf9ca6a328c5da7a6a
CPU Threads: 2; OS Version: Linux 4.2; UI Render: default; 
TinderBox: Linux-rpm_deb-x86@71-TDF, Branch:master, Time: 2016-04-08_03:26:11
Locale: nl-NL (nl_NL.UTF-8)

- after some scrolling ~constantly uses full cpu (one core)
- after killing, turning off Tools > Options > Writer > View .. Comments
  I can do with the file what I want.

@GiorgioMigliaccio all the comments.. looks as a paste from html?
Or what about all the locked text frames.. Anyway: special document :)
Comment 31 Armin Le Grand 2016-04-13 10:31:03 UTC
Yes, special document, but the comments - rendering/creation/handling seems too expensive. Would be worth a look.
Comment 32 Armin Le Grand 2016-04-18 09:00:44 UTC
Did some evaluation and indeed the comments are quite expensive currently, they seem unnecessarily often to be created/layouted and positioned, most probably without even being visible. Would require some work, but could definitely be enhanced...
Comment 33 Thorsten Behrens (CIB) 2016-06-10 13:50:44 UTC
This seems it was not possible to tackle by volunteer developers. So to update on Joren's (and others) comments, there's a thriving ecosystem of LibreOffice support companies now; a list of those with certified developers is linked here:

http://www.libreoffice.org/get-help/professional-support/
Comment 34 Thorsten Behrens (CIB) 2016-06-10 13:51:43 UTC
FWIW, ballpark number for fixing this is around one week of work ...
Comment 35 konsultor 2016-06-12 04:14:21 UTC Comment hidden (off-topic)
Comment 36 konsultor 2016-11-25 19:14:35 UTC Comment hidden (off-topic)
Comment 37 konsultor 2016-11-25 19:18:13 UTC Comment hidden (off-topic)
Comment 38 konsultor 2016-11-25 19:26:19 UTC Comment hidden (off-topic)
Comment 39 Heiko Tietze 2017-03-04 22:14:48 UTC
Dup of bug 97698?
Comment 40 Telesto 2018-10-22 17:53:05 UTC
*** Bug 120102 has been marked as a duplicate of this bug. ***
Comment 41 Buovjaga 2019-02-14 11:33:49 UTC
(In reply to Cor Nouws from comment #30)
> Testing in Version: 5.2.0.0.alpha0+
> Build ID: 5a4b01f63d3f2a7d7d6fa8cf9ca6a328c5da7a6a
> CPU Threads: 2; OS Version: Linux 4.2; UI Render: default; 
> TinderBox: Linux-rpm_deb-x86@71-TDF, Branch:master, Time: 2016-04-08_03:26:11
> Locale: nl-NL (nl_NL.UTF-8)
> 
> - after some scrolling ~constantly uses full cpu (one core)
> - after killing, turning off Tools > Options > Writer > View .. Comments
>   I can do with the file what I want.
> 
> @GiorgioMigliaccio all the comments.. looks as a paste from html?
> Or what about all the locked text frames.. Anyway: special document :)

Can everyone please try with a fresh build or 6.2? Michael Stahl has done some work in this area and I confirm that I cannot get LibO into the state described by Cor above.

We have various reports for general bad perf regarding comments: https://bugs.documentfoundation.org/buglist.cgi?keywords=perf%2C%20&keywords_type=allwords&list_id=911413&query_format=advanced&resolution=---&short_desc=comments&short_desc_type=allwordssubstr

Hopefully we can close this one?

Arch Linux 64-bit
Version: 6.3.0.0.alpha0+
Build ID: a2a762a9b6cad4ab0bb1b71b99aebc8c047c94d0
CPU threads: 8; OS: Linux 4.20; UI render: default; VCL: gtk3; 
Locale: fi-FI (fi_FI.UTF-8); UI-Language: en-US
Calc: threaded
Built on 14 February 2019
Comment 42 Dieter Praas 2019-02-14 11:48:45 UTC
(In reply to Buovjaga from comment #41)
> Can everyone please try with a fresh build or 6.2?

I opened sample document from initial bug report without any problems (3 or 4 seconds and CPU usage around 40% during that time). 

Version: 6.3.0.0.alpha0+ (x64)
Build ID: f42554a1886ebe49170c25096dc3281b2c7bb1f4
CPU threads: 4; OS: Windows 10.0; UI render: default; VCL: win; 
TinderBox: Win-x86_64@42, Branch:master, Time: 2019-02-08_22:37:30
Locale: en-US (de_DE); UI-Language: en-US
Calc: threaded

So I would say RESOLVED WORKSFORME.
Comment 43 Telesto 2019-02-14 13:33:43 UTC
This still still holds: "using the page scrollbars constantly, or using page-up page-down keys make the CPU usage go to 80-100%" (comment 0)

Same for comment 32 & comment 34. However, the title should be rephrased, because
1. The memory issue is long gone.. 
2. The performance of tracking changes has improved 

Comment 32 and 34 are still adequate (see also bug 60418  bug 119485, bug 122971, bug 122969.. which have all the same root-cause)

I personally would  rephrase the title leave this one open, and close the rest..
Comment 44 Buovjaga 2019-02-15 11:44:19 UTC
*** Bug 60418 has been marked as a duplicate of this bug. ***
Comment 45 Buovjaga 2019-02-15 11:44:27 UTC
*** Bug 119485 has been marked as a duplicate of this bug. ***
Comment 46 Buovjaga 2019-02-15 11:44:35 UTC
*** Bug 122971 has been marked as a duplicate of this bug. ***
Comment 47 Buovjaga 2019-02-15 11:44:43 UTC
*** Bug 122969 has been marked as a duplicate of this bug. ***
Comment 48 Buovjaga 2019-02-15 11:46:42 UTC
Callgrind traces in bug 60418 and bug 122969
Comment 49 Telesto 2019-02-15 15:15:53 UTC
*** Bug 92011 has been marked as a duplicate of this bug. ***
Comment 50 Telesto 2019-02-15 15:17:44 UTC
(In reply to Buovjaga from comment #48)
Callgrind traces in bug 60418 and bug 122969 & bug 92011 comment 7
Comment 51 Noel Grandin 2019-02-18 14:42:04 UTC
Some notes from my unsucessful attempt to fix this:

(1) I can fix the typing part by removing the WB_DIALOGCONTROL flag from the SwEditWin constructor. The time is being spent searching through tons of child windows on every key stroke. I don't think this flag is necessary because we're not doing dialog things in the main window.
But I'm probably wrong and there is some weird reason we need it.

(2) I can fix the paging/scrolling part by commenting out the 
     if ( pPostItMgr ) // #i88070#
     {
        pPostItMgr->Rescale();
        pPostItMgr->CalcRects();
        pPostItMgr->LayoutPostIts();
     }
code in void SwViewShell::VisPortChgd
As far as I can tell, the UI still works fine without those lines of code, because don't actually need to run the bulk of layout when the viewport changes.

According to the bug report in the comment
    https://bz.apache.org/ooo/show_bug.cgi?id=88070
they are there because accessibility stuff is being calculated during the layout loop. So one possible fix would be to pass some sort of flag down to CalcRects and LayoutPostIts to say "only do the accessibility stuff"
Comment 52 ludochat 2019-05-20 09:35:47 UTC Comment hidden (spam)
Comment 53 Telesto 2019-06-05 13:18:29 UTC
Created attachment 151939 [details]
Example file