Bug 125665 - Big ODT documents take too long to load or save
Summary: Big ODT documents take too long to load or save
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
6.3.0.0.beta1+
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:6.4.0
Keywords: bibisectRequest, perf, regression
Depends on:
Blocks:
 
Reported: 2019-06-03 17:45 UTC by David García
Modified: 2019-07-03 14:04 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
ODT document to test how long it takes for LO to open or save a big file (2.34 MB, application/vnd.oasis.opendocument.text)
2019-06-03 17:47 UTC, David García
Details

Note You need to log in before you can comment on or make changes to this bug.
Description David García 2019-06-03 17:45:28 UTC
Description:
Hello,

Yesterday, I found a bug report about a document taking too long to load (https://bugs.documentfoundation.org/show_bug.cgi?id=63640 ) and, after being suggested to create a new bug report for my particular case, here I am.

For a long time, I’ve noticed that big ODT files (that is, ODT files containing a huge number of text across hundreds of pages) take too long to load on LO, and so, inspired by the aforementioned bug report, I decided to do some testing to compare versions 6.2.4 and 6.3.0 beta 1 in order to check if there have been improvements in this area recently.

Since the file that I always use for testing purposes belongs to my brother and I can’t upload it here, I decided to create an ODT document specially for the occasion (this new document is much smaller, but I believe it’s big enough). I created the document by opening a text file in LO 6.2.4 and then saving it as an ODT document.

My computer is a first-generation Intel Core i7 with 8 GB of DDR 3 RAM running Windows 10 1903, 18362.145. It’s quite old and slow by modern standards, and so I presume other people will get better results than me.

These were my results on 6.2.4:

- Time to open the file: 3m51s.
- Time to save the file (after replacing one character): 3m03s.

As opposed to the behaviour detected in 6.3.0 beta 1, a progress bar was displayed in 6.2.4, but I noticed two things:

- When opening the document, LO showed the progress bar and then got stuck for a good while until the document was displayed.

- When saving the document, it happened the other way round: LO got stuck for a while, and then it showed the progress bar.

These were my results on 6.3.0 beta 1:

- Time to open the file: 2m25s.
- Time to save the file (after replacing a character): 2m10s.

These are my conclusions:

- No progress bar was displayed when opening or saving the document.
- I noticed that both operations were faster compared to 6.2.4, but still quite slow (at least, on my computer, which, as I said, it’s quite old and slow).

I hope this bug report is useful, and I’m willing to provide more information if necessary.

Thank you!

Steps to Reproduce:
You open or save a big ODT document on LO and measure how long it takes for the operation to be completed.

Actual Results:
As explained on the description, it took a few minutes for my document to be loaded or saved.

Expected Results:
I would have expected that opening or saving an ODT file would take only a few seconds.


Reproducible: Always


User Profile Reset: No


OpenGL enabled: Yes

Additional Info:
Comment 1 David García 2019-06-03 17:47:45 UTC
Created attachment 151881 [details]
ODT document to test how long it takes for LO to open or save a big file

ODT document to test how long it takes for LO to open or save a big file (meaning a file containing hundreds of pages of text).
Comment 2 Durgapriyanka 2019-06-03 21:51:28 UTC
Thank you for reporting the bug.

Version: 6.3.0.0.alpha0+
Build ID: b6b28931435e44aca92b8c0e1659f701e3ed1a87
CPU threads: 2; OS: Windows 6.1; UI render: default; VCL: win; 
TinderBox: Win-x86@42, Branch:master, Time: 2019-01-30_06:57:04
Locale: en-US (en_US); UI-Language: en-US
Calc: threaded

Time taken to open: 1m 30secs, to save: 1m 47 secs

LibreOffice 3.3.0 
OOO330m19 (Build:6)
tag libreoffice-3.3.0.4

Time taken to open: 16 secs, to save: 54 secs

So version 3.3 opens and saves quicker than other versions.
Comment 3 Telesto 2019-06-04 07:15:23 UTC
With 4.4.7.2
File open: 10 seconds
File saving: 10 seconds

With 6.4.0.0
File open: 30 seconds
File saving 32 seconds

Version: 6.4.0.0.alpha0+ (x86)
Build ID: 93477d1a963e38e3319013e43835a8ffef200972
CPU threads: 4; OS: Windows 6.3; UI render: default; VCL: win; 
TinderBox: Win-x86@42, Branch:master, Time: 2019-06-02_10:16:52
Locale: it-IT (nl_NL); UI-Language: en-US
Calc: threaded
Comment 4 Xisco Faulí 2019-06-04 10:50:35 UTC
it takes

real	0m9,667s
user	0m9,335s
sys	0m0,266s

in

Version: 5.4.0.0.alpha1+
Build ID: 9feb7f7039a3b59974cbf266922177e961a52dd1
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); Calc: group

( oldest commit in bibisect-linux64-6.0 )

while in

Version: 6.0.0.0.alpha1+
Build ID: 6eeac3539ea4cac32d126c5e24141f262eb5a4d9
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); Calc: group threaded

( oldest commit in bibisect-linux64-6.1 )

it takes

real	0m55,007s
user	0m52,374s
sys	0m0,512s

in master it has improved, and it takes

real	0m34,569s
user	0m34,601s
sys	0m0,466s

in

Version: 6.3.0.0.beta1+
Build ID: 4abdaf4afb2245d404f6709124b3c627b07b8a3c
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded
Comment 5 Xisco Faulí 2019-06-04 10:54:52 UTC
in

Version: 6.2.0.0.alpha1+
Build ID: a20a2d7e0d28658f2d9089da076961a599833a28
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded

it takes

real	0m54,140s
user	0m53,973s
sys	0m0,690s

so it got improved recently, most likely by the performance improvements done by Noel
@Noel, I thought you might be interested in this issue...
Comment 6 David García 2019-06-04 11:18:52 UTC
Hi,

Thank you for all your feedback! It looks like my computer takes the longest to open and save the documents.

Also, I've noticed that there's a more technical way to provide both the computing time of an operation and the specs of your machine (and the operating system). Could anybody tell me how?

My other concern is that I can't find a way to edit my messages. Yesterday, in another thread, instead of editing my comments, I ended up publishing the same one again.

Thanks in advance!
Comment 7 Noel Grandin 2019-06-04 14:45:28 UTC
The load time is dominated by the RDF metadata stuff, not much I can do there
Comment 8 David García 2019-06-27 11:47:24 UTC
Hello,

I've tested my file on LO 6.3.0 beta 2 and here are my results:

- Time to open the file: 3m12s.
- Time to save the file: 3m04s.

Conclusions:

- On beta 2, it takes longer for my PC to open and save the file compared to beta 1 (47 and 54 seconds respectively).

- There are still no progress bars neither when opening nor when saving the file.

- As I explained in a previous message, there's a progress bar on 6.2.4, but it doesn't cover the whole process, which means that part of the time needed to open or save a file, LO is apparently stuck and there's no indication of the time we need to wait for the process to be completed.
Comment 9 Commit Notification 2019-06-28 10:48:34 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/47e04cf31c6165dd55dc20962ad9c72962b958bd%5E%21

tdf#125706 and tdf#125665 writer, speed up RDF

It will be available in 6.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 10 Xisco Faulí 2019-07-01 13:38:21 UTC
it takes

real	1m7,764s
user	1m6,705s
sys	0m2,400s

in

Version: 6.4.0.0.alpha0+
Build ID: 9fbedb7929936a45967ae49bc15b985f95e2ebd3
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded(In reply to Xisco Faulí from comment #5)
> in
> 
> Version: 6.2.0.0.alpha1+
> Build ID: a20a2d7e0d28658f2d9089da076961a599833a28
> CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
> Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
> Calc: threaded
> 
> it takes
> 
> real	0m54,140s
> user	0m53,973s
> sys	0m0,690s
> 
> so it got improved recently, most likely by the performance improvements
> done by Noel
> @Noel, I thought you might be interested in this issue...

I've just tried with the very same version and today it takes

real	4m29,163s
user	4m21,894s
sys	0m3,212s

in

Version: 6.4.0.0.alpha0+
Build ID: 9fbedb7929936a45967ae49bc15b985f95e2ebd3
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded

it takes

real	1m12,923s
user	1m9,893s
sys	0m2,580s

I don't really understand why the measurement is so different from one day to the other...
Comment 11 Xisco Faulí 2019-07-01 13:38:45 UTC
@David Garcia,
Could you please try to reproduce it with a master build from http://dev-builds.libreoffice.org/daily/master/ ?
You can install it alongside the standard version.
Comment 12 David García 2019-07-01 17:07:22 UTC
Hello, @Xisco Faulí,

I don’t have experience dealing with master builds and I wasn’t sure about the difference between folders <Win-x86_64@42/> and <Win-x86_64@62-TDF/>. Eventually, I downloaded the installer from the former since it contained the latest one.

In any case, I’m now on 6.4.0.0.alpha0+ (x64) and here are my results:

- Time to open the file: 1m36s (compared to 3m51s on 6.2.4).
- Time to save the file: 2m18 (compared to 3m03s on 6.2.4).

And here are my observations:

- Both operations are significantly faster on 6.4.0.0 than on current 6.2.4.
- For the first time since I began testing the issue, saving the document takes longer than opening it.
- There is still no progress bar to indicate how long the operation will take.
Comment 13 Commit Notification 2019-07-03 08:45:59 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/54481a2d6a6f9d415e0979a03c0c878195a10845%5E%21

Revert "tdf#125706 and tdf#125665 writer, speed up RDF"

It will be available in 6.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 14 Commit Notification 2019-07-03 08:46:16 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/f2c513e686536dc308609c56fa9354d4d10b072c%5E%21

tdf#125706 and tdf#125665 writer, speed up RDF, take2

It will be available in 6.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 15 Xisco Faulí 2019-07-03 14:04:18 UTC
(In reply to Xisco Faulí from comment #10)
> 
> Version: 6.4.0.0.alpha0+
> Build ID: 9fbedb7929936a45967ae49bc15b985f95e2ebd3
> CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
> Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
> Calc: threaded
> 
> it takes
> 
> real	1m12,923s
> user	1m9,893s
> sys	0m2,580s
> 
> I don't really understand why the measurement is so different from one day
> to the other...

it takes

real	0m54,424s
user	0m37,634s
sys	0m0,756s

in

Version: 6.4.0.0.alpha0+
Build ID: 3cdc1b35b9d86bcfa1277e3e94925ae7b18b8fde
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded

after the revert and the second try