Bug 148013 - Larger ODT with ~500 pages and comments an record changes very slow to load, worse than before
Summary: Larger ODT with ~500 pages and comments an record changes very slow to load, ...
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
7.3.0.0.alpha1+
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:7.4.0
Keywords: bibisected, bisected, perf, regression
: 144995 148003 (view as bug list)
Depends on:
Blocks: Track-Changes File-Opening
  Show dependency treegraph
 
Reported: 2022-03-15 16:39 UTC by Timur
Modified: 2022-06-30 10:05 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Timur 2022-03-15 16:39:52 UTC
Opening ODT attachment 167497 [details] is very slow. 
Not sure if having 487 pages with ~ 532 comments afffects that.

Here are the loading times: 

43allo	149,93s user 0,44s system 99% cpu 2:30,86 total
4.1m	 73,47s user 0,49s system 90% cpu 1:22,12 total
4.2m  	113,86s user 0,51s system 92% cpu 2:03,70 total
4.3m	135,74s user 0,29s system 99% cpu 2:16,47 total
5.2m	133,05s user 0,63s system 89% cpu 2:29,51 total
6.0m 	196,54s user 1,41s system 90% cpu 3:38,19 total
7.0m	195,36s user 1,12s system 92% cpu 3:31,70 total
7.2m	130,91s user 1,03s system 96% cpu 2:16,15 total
7.3m	185,54s user 0,99s system 91% cpu 3:24,52 total		
7.4+	185,31s user 0,82s system 99% cpu 3:07,16 total

It got significantly slower in 4.2 with commit of innocent name, but being also in other different bugs. I checked with the previous commit.  
source-hash-b8002169336b6b7597d32755e41fa3dc2688539e 
(prev source-hash-7fb73e6c30e66f028fe759376e3789456bf3ad33)
author	Michael Stahl <mstahl@redhat.com>	2013-11-06 	
remove INPATH and PROEXT

Next little slow down in 4.3 I couldn't bibisect because of frequent Error MsgBox, skipping didn't work, when I checked the result. 

Fresh slow down in 7.3 
source bebb1a3f2dc3e131f95a078f8a9c40d0491af7cd
(prev 88c85c8aa377ccc017582d8a08e5f73391b5a446)
author	Noel Grandin <noel.grandin@collabora.co.uk>	2021-07-2
pass SvXMLNamespaceMap around by value

There are other perf bugs but I put this one separately because it has times and bibisects.
CC Michael and Noel, please see.
Comment 1 Telesto 2022-03-15 17:02:09 UTC
*** Bug 148003 has been marked as a duplicate of this bug. ***
Comment 2 Noel Grandin 2022-03-16 12:09:07 UTC
The performance profile here is entirely dominated by scanning the redline table inside SwTextNode::Update, so we effectively have an O(n^2) issue here.
Comment 3 Noel Grandin 2022-03-16 13:55:53 UTC
Oh, this is a broken file (it has overlapping redlines), which means my optimised paths in DocumentRedlineManager::GetRedlinePos dont' trigger.

Which means I'll push one small improvement, and then I'm not interested anymore (I don't optimise for broken files)
Comment 4 Commit Notification 2022-03-16 16:39:42 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/3d349d84bd8fcbb38f7809e140bc8591056c6c29

tdf#148013 speed up loading large file with lots of redlines

It will be available in 7.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 5 Commit Notification 2022-03-17 08:02:40 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/2c8933594a4ea955eb4e785c7507d011c96c2aad

tdf#148013 speed up loading large file with lots of redlines

It will be available in 7.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 6 Timur 2022-03-17 09:43:13 UTC
(In reply to Noel Grandin from comment #3)
> Oh, this is a broken file (it has overlapping redlines)

There are more of overlapping redlines. IIUC those are not broken, it just isn't supported well in LO: I add bug 144995 to See Also. Maybe meta should be created.
Comment 7 Telesto 2022-03-17 19:26:26 UTC
*** Bug 144995 has been marked as a duplicate of this bug. ***
Comment 8 Telesto 2022-03-17 19:52:19 UTC
(In reply to Timur from comment #6)
> (In reply to Noel Grandin from comment #3)
> > Oh, this is a broken file (it has overlapping redlines)
> 
> There are more of overlapping redlines. IIUC those are not broken, it just
> isn't supported well in LO: I add bug 144995 to See Also. Maybe meta should
> be created.

Bug 144995 is same document or different version of the same file... so not useful 

But I would like to know:
a) how to detect the presence of overlapping redlines without a full perf trace
Could/should it throw SAL_INFO/WARN or something like that
b) Is it possible to check the bug tracker document pool for documents with overlapping redlines. So verify if this issue here being unique or representing a more common problem .. It's not that easy to spot with smaller less complex files (so might go unnoticed)

If the issue of overlapping redlines being more common, QA could investigate... to figure out how those files a generated.. there might be (still) some bug with LibO causing this to happen... 

The file here has likely been saved into DOCX and back to ODT. The DOCX is possibly edited with Word somewhere in the process