Bug 113539 - FILEOPEN ODT: Slower opening of file containing a large table (25 rows 2000 columns)
Summary: FILEOPEN ODT: Slower opening of file containing a large table (25 rows 2000 c...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
4.1 all versions
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:6.3.0
Keywords: bibisected, haveBacktrace, perf, regression
Depends on:
Blocks: Writer-Tables
  Show dependency treegraph
 
Reported: 2017-10-30 22:12 UTC by Telesto
Modified: 2019-03-28 07:34 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
Example file (18.94 KB, application/vnd.oasis.opendocument.text)
2017-10-30 22:13 UTC, Telesto
Details
Very Sleepy time profile (49.90 KB, image/png)
2017-10-30 22:23 UTC, Telesto
Details
Callgrind output from master (5.29 MB, application/x-xz)
2017-11-22 10:01 UTC, Buovjaga
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Telesto 2017-10-30 22:12:54 UTC
Description:
Slow opening of file containing a large table (25 rows 2000 columns)

Steps to Reproduce:
1. Open the attached file an notice of the time required to open 

Actual Results:  
With LibreOffice 6 -> 14 seconds
With LibreOffice 4.4.8.2 -> 9 seconds

It has been worse, in the time between 5 and 6, but still not back to normal



Expected Results:
9 seconds (as measured with the Time command)


Reproducible: Always


User Profile Reset: No



Additional Info:
Found in
Version: 6.0.0.0.alpha1+
Build ID: d30522e46ca884e9bc74af21711d9537e8118859
CPU threads: 4; OS: Windows 6.3; UI render: default; 
TinderBox: Win-x86@42, Branch:master, Time: 2017-10-29_23:13:14
Locale: nl-NL (nl_NL); Calc: CL

and in 
5.1.0.3


User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0
Comment 1 Telesto 2017-10-30 22:13:16 UTC
Created attachment 137388 [details]
Example file
Comment 2 Telesto 2017-10-30 22:23:13 UTC
Time required did build up a bit while bibisecting:
$ time instdir/program/soffice "C:\Demo.odt"

real    0m8,290s
user    0m0,000s
sys     0m0,000s

Bit worse:
$ git bisect bad
Bisecting: 8 revisions left to test after this (roughly 3 steps)
[2970f29b0ee7e9988ffbb183f2e6bb156e374409] source sha:4020f9bbd92becd3662cdc3b24ad70b370307e5e

~/bibisect-win32-5.0
$ time instdir/program/soffice "C:\Demo.odt"

real    0m11,197s
user    0m0,000s
sys     0m0,015s

Worser
~/bibisect-win32-5.0
$ time instdir/program/soffice "C:\Demo.odt"

real    0m16,629s
user    0m0,000s
sys     0m0,000s


I ended up here when following the 16 second lead:
author	Bjoern Michaelsen <bjoern.michaelsen@canonical.com>	2015-05-22 14:42:23 (GMT)
committer	Bjoern Michaelsen <bjoern.michaelsen@canonical.com>	2015-05-25 22:51:11 (GMT)
commit	50356872b9421303b9c7b4a3135b3cdcc3a0af50 (patch)
tree	e87d19df1c26853d567516f37f08344f287fbe72
parent	96898cd49830333d752b9aa56fe91a8e21c9dca8 (diff)
new unocrsrs for SwXTextCursor
Change-Id: I5932f75ee4814ca42f16d349094c7fa8bbb2ee63
Comment 3 Telesto 2017-10-30 22:23:36 UTC
Created attachment 137389 [details]
Very Sleepy time profile
Comment 4 Xisco Faulí 2017-10-31 07:07:20 UTC
The import time seems similar (few seconds up or down ) between

Version 4.1.0.0.alpha0+ (Build ID: efca6f15609322f62a35619619a6d5fe5c9bd5a)

and

Version: 6.0.0.0.alpha1+
Build ID: d30522e46ca884e9bc74af21711d9537e8118859
CPU threads: 4; OS: Linux 4.10; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); Calc: group

in my case, around 

real	0m24,508s
user	0m16,352s
sys	0m0,416s

however, it's extremely fast in

LibreOffice 3.3.0 
OOO330m19 (Build:6)
tag libreoffice-3.3.0.4

real	0m5,255s
user	0m4,840s
sys	0m0,252s
Comment 5 Telesto 2017-10-31 08:46:13 UTC
(In reply to Xisco Faulí from comment #4)
There seems to be a difference between Linux & Windows. LibreOffice 3.3.0 is indeed the fastest. 

$ time /cygdrive/h/LibreOfficePortable/3.3.0/App/libreoffice/program/swriter.exe "C:\Demo.odt"

real    0m4,796s
user    0m0,000s
sys     0m0,000s

However, the import time on Linux always always was 14-16 seconds between 4.1 and 6.0. A contrary to Windows. The import time always was 5-8 seconds (since 3.3.0), until the change in LibO 5.0.0.
Comment 6 Telesto 2017-11-21 19:37:29 UTC
A callgrind trace could be helpful.. ;-)
Comment 7 Buovjaga 2017-11-22 10:01:29 UTC
Created attachment 137912 [details]
Callgrind output from master

Arch Linux 64-bit
Version: 6.0.0.0.alpha1+
Build ID: 1d9eed341db208f11de6f020538dfdb74a5c48dd
CPU threads: 8; OS: Linux 4.13; UI render: default; VCL: kde4; 
Locale: fi-FI (fi_FI.UTF-8); Calc: group threaded
Built on November 22nd 2017
Comment 8 Telesto 2017-12-06 17:49:59 UTC
This seems to more or less a dupe of bug 84635. Except for the part where it got worse in the 5.0 branch.
Comment 9 Buovjaga 2018-05-30 17:57:41 UTC
(In reply to Telesto from comment #2)
> I ended up here when following the 16 second lead:
> author	Bjoern Michaelsen <bjoern.michaelsen@canonical.com>	2015-05-22
> 14:42:23 (GMT)
> committer	Bjoern Michaelsen <bjoern.michaelsen@canonical.com>	2015-05-25
> 22:51:11 (GMT)
> commit	50356872b9421303b9c7b4a3135b3cdcc3a0af50 (patch)
> tree	e87d19df1c26853d567516f37f08344f287fbe72
> parent	96898cd49830333d752b9aa56fe91a8e21c9dca8 (diff)
> new unocrsrs for SwXTextCursor
> Change-Id: I5932f75ee4814ca42f16d349094c7fa8bbb2ee63

Björn: what do you think?
Comment 10 Björn Michaelsen 2018-05-31 21:19:28 UTC
Hi all,

so IIRC the old implementation was unsafe at any speed: It had shared ownership of the object an manually tracked it from multiple places (badly).

So whatever the solution to this is, its not reverting to that.

Both the before and after the path, the object was/is heap allocated. The shared_ptr timingwise just adds synchronization. The way forward would be one of:

- unfsck this design, so that in doesnt need shared ownership
- make whatever is using massive amounts of unocrsrs in these example files use something more lowlevel (and faster) instead

Clearly 1/ is the best long-term solution, but it might be quite tricky to unwind this hairball (though admittedly I havent look at trying that yet).
Comment 11 Telesto 2019-02-17 11:01:08 UTC

*** This bug has been marked as a duplicate of bug 84635 ***
Comment 12 Commit Notification 2019-03-28 06:43:55 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/9c5d33e3c9e4a680af61a9e7af8fa73d08b33834%5E%21

tdf#113539 Slow opening of ODT file containing a large table

It will be available in 6.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 13 Buovjaga 2019-03-28 07:34:55 UTC
Tests with OOO_EXIT_POST_STARTUP=1

attachment 137388 [details]

6.2.2:
real    0m5,055s
user    0m4,666s
sys     0m0,314s

Master with the latest commit:
real    0m4,738s
user    0m4,254s
sys     0m0,373s

attachment 107266 [details] from bug 84635

6.2.2:
real    0m9,534s
user    0m9,217s
sys     0m0,483s

Master with the latest commit:
real    0m10,906s
user    0m10,469s
sys     0m0,498s

As the results are different, let's undupe this.

Thanks for the work!

Arch Linux 64-bit
Version: 6.2.2.2
Build ID: 6.2.2-2
CPU threads: 8; OS: Linux 5.0; UI render: default; VCL: gtk3; 
Locale: fi-FI (fi_FI.UTF-8); UI-Language: en-US
Calc: threaded

Arch Linux 64-bit
Version: 6.3.0.0.alpha0+
Build ID: 9c5d33e3c9e4a680af61a9e7af8fa73d08b33834
CPU threads: 8; OS: Linux 5.0; UI render: default; VCL: gtk3; 
Locale: fi-FI (fi_FI.UTF-8); UI-Language: en-US
Calc: threaded
Built on 28 March 2019