Bug 113266 - LO (UI) slow on opening specific 6 MB XLS with packed 45 MB drawing
Summary: LO (UI) slow on opening specific 6 MB XLS with packed 45 MB drawing
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: graphics stack (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:6.3.0
Keywords: filter:xls, haveBacktrace, perf
Depends on:
Blocks: XLS
  Show dependency treegraph
 
Reported: 2017-10-19 11:55 UTC by Mike Kaganski
Modified: 2019-05-02 14:55 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
A file from https://www.boredpanda.com/73-year-old-excel-paintings-tatsuo-horiuchi/ (6.52 MB, application/vnd.ms-excel)
2017-10-19 11:55 UTC, Mike Kaganski
Details
bt with debug symbols (15.28 KB, text/plain)
2019-04-22 09:43 UTC, Julien Nabet
Details
perf flamegraph (2.11 MB, application/gzip)
2019-04-22 18:41 UTC, Julien Nabet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Mike Kaganski 2017-10-19 11:55:19 UTC
Created attachment 137113 [details]
A file from https://www.boredpanda.com/73-year-old-excel-paintings-tatsuo-horiuchi/

The file opens and shows its contents (huge line art), but UI is unresponsive. As FLHerne at #libreoffice pointed out, "a fun stress test".

Tested with Version: 5.4.2.2 (x64)
Build ID: 22b09f6418e8c2d508a9eaf86b2399209b0990f4
CPU threads: 4; OS: Windows 6.19; UI render: default; 
Locale: ru-RU (ru_RU); Calc: CL
Comment 1 Xisco Faulí 2017-10-22 22:12:25 UTC
Confirmed in

Version: 6.0.0.0.alpha1+
Build ID: 118a0a3734a3f794c67a9d7d4376d8ed78a96fee
CPU threads: 4; OS: Linux 4.10; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); Calc: group
Comment 2 Xisco Faulí 2017-10-22 22:13:41 UTC
Actually, it's responsive, but it needs some time ;-)
Comment 3 Xisco Faulí 2017-10-22 22:17:07 UTC
Also reproduced in

LibreOffice 3.3.0 
OOO330m19 (Build:6)
tag libreoffice-3.3.0.4
Comment 4 Francis Herne 2017-10-23 15:06:34 UTC
Reproduced for me with

Version: 5.4.2.2.0+
Build ID: 5.4.2-2
CPU threads: 4; OS: Linux 4.13; UI render: default; VCL: gtk3; 
Locale: en-GB (en_GB.UTF-8); Calc: group

It does eventually become responsive, but becomes immediately unresponsive when trying to scroll.
Comment 5 QA Administrators 2018-10-24 02:57:27 UTC Comment hidden (obsolete)
Comment 6 Francis Herne 2019-03-06 13:45:50 UTC
There is some improvement in 6.2.

The UI becomes unresponsive for about 15s after each interaction with the document; in LO 5.4 this took almost two minutes.

Version: 6.2.0.3
Build ID: 6.2.0-3
CPU threads: 4; OS: Linux 4.20; UI render: default; VCL: gtk3; 
Locale: en-GB (en_GB.UTF-8); UI-Language: en-GB
Calc: threaded
Comment 7 Julien Nabet 2019-04-22 09:43:49 UTC
Created attachment 150912 [details]
bt with debug symbols

On pc Debian x86-64 with master sources updated today, I could reproduce this.
Comment 8 Julien Nabet 2019-04-22 09:45:12 UTC
Noel: noticing your optimization on Calc, thought you might be interested in this one.
Of course, don't hesitate to uncc yourself if no time or focus only on conditional formatting part.
Comment 9 Xisco Faulí 2019-04-22 13:27:19 UTC
it takes

real	3m22,013s
user	2m32,302s
sys	0m1,874s

to open in

Version: 6.3.0.0.alpha0+
Build ID: e913727c7ee3af0bb4031c6829abfb3373306492
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded
Comment 10 Julien Nabet 2019-04-22 18:41:27 UTC
Created attachment 150927 [details]
perf flamegraph
Comment 11 Julien Nabet 2019-04-22 18:47:17 UTC
BTW, I waited several minutes but it didn't finish so I had to stop it.
Comment 12 Julien Nabet 2019-04-22 19:16:10 UTC
Also I noticed a lot of these on console:
warn:svl.items:25499:25499:svl/source/inc/poolio.hxx:68: make this item sortable to speed up managing this set
warn:sc:25499:25499:sc/source/filter/excel/xlstyle.cxx:159: XclDefaultPalette::GetDefColor - unknown default color index: 193 (there were also other numbers too: 16384, 8193, 459, 387...)
Comment 13 Noel Grandin 2019-04-23 09:13:54 UTC
Julien, are you generating a flame graph from a debug build - because that will produce misleading results....
Comment 14 Julien Nabet 2019-04-23 10:16:30 UTC
(In reply to Noel Grandin from comment #13)
> Julien, are you generating a flame graph from a debug build - because that
> will produce misleading results....

Yes, I use a debug build. So I should stop using Flamegraph?
Comment 15 Noel Grandin 2019-04-23 10:25:27 UTC
(In reply to Julien Nabet from comment #14)
> 
> Yes, I use a debug build. So I should stop using Flamegraph?

No, just use a --enable-symbols build when you need to do performance analysis :-)
Comment 16 Commit Notification 2019-04-23 12:11:37 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/10c934147d469965dba6abc78efd02759a010b8e%5E%21

tdf#113266 slow opening XLS with 45 MB drawing

It will be available in 6.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 17 Roman Kuznetsov 2019-04-23 13:18:17 UTC
Noel, is this bug fixed? Or will be else 5-6 parts of fixing? =)
Comment 18 Noel Grandin 2019-04-23 13:25:10 UTC
I won't be working on this bug anymore, I have no opinion over whether it is fixed or not, my change would only have shaved a few % off the rendering time
Comment 19 Xisco Faulí 2019-04-23 20:59:39 UTC
(In reply to Xisco Faulí from comment #9)
> it takes
> 
> real	3m22,013s
> user	2m32,302s
> sys	0m1,874s
> 
> to open in
> 
> Version: 6.3.0.0.alpha0+
> Build ID: e913727c7ee3af0bb4031c6829abfb3373306492
> CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
> Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
> Calc: threaded

it takes

real	1m20,541s
user	1m19,989s
sys	0m0,520s

in

Version: 6.3.0.0.alpha0+
Build ID: 0a04150b6eefb5feb7ecefaa5cd63dbac8c1574f
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded

quite a nice improvement... My take: RESOLVED FIXED. Up to Mike Kaganski as he's the reporter...
Comment 20 Xisco Faulí 2019-04-23 21:10:33 UTC
it would be nice if other could measure it before and after...
Comment 21 Aron Budea 2019-04-24 04:50:19 UTC
(In reply to Xisco Faulí from comment #19)
> quite a nice improvement... My take: RESOLVED FIXED. Up to Mike Kaganski as
> he's the reporter...
It opens in 1:18 for me.
In Excel 2013 it opens in 10s.

UI is still very unresponsive in Calc, I'd go as far to say that it hangs.
Excel isn't very responsive when clicking the image, but only takes ~10s, otherwise it's quite responsive.
Comment 22 Aron Budea 2019-04-24 04:54:37 UTC
(In reply to Aron Budea from comment #21)
> It opens in 1:18 for me.
I was checking with a 32-bit Windows daily build from today (2019-04-24, 951282a27a9dd4c64fc206fcbdd805b4cb602816).
Comment 23 Timur 2019-04-24 14:48:47 UTC
In my case, MSO opens in 58 seconds and LO in 64 seconds. So opening time is OK. 
But LO is not responsive after opening. 
Is it another issue, related to image handling like Bug 47148 or specific to this drawing type image?
Comment 24 Xisco Faulí 2019-04-27 14:11:30 UTC
I've just rechecked with the same commit I mentioned in commit 19 and this time I've got:

real	1m46,080s
user	1m24,239s
sys	0m1,146s

while in

Version: 6.3.0.0.alpha0+
Build ID: 3ab6d246cc44617af5ed416b5d49f2f35b61ceea
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded

it takes

real	1m8,876s
user	0m47,064s
sys	0m1,210s

@Timur, @Aron, could you please check again on your end ?
Comment 25 Timur 2019-04-29 09:14:04 UTC
Similar to Aron, I tested with Windows build. master~2019-04-24_00.11.09_LibreOfficeDev_6.3.0.0.alpha0_Win_x86.msi. 
Today I had machine loaded and opening takes 100 seconds in LO and 130 secs in MSO. 
With libo-master64~2019-03-26_23.06.31_LibreOfficeDev_6.3.0.0.alpha0_Win_x64.msi it's 120 seconds. Not sure how much the fix improved loading time, seems some.  
My point was that LO is faster to open than MSO and it's hard to expect more.
Comment 26 Alex Thurgood 2019-04-29 15:15:34 UTC Comment hidden (obsolete)
Comment 27 Xisco Faulí 2019-04-29 16:12:54 UTC
(In reply to Alex Thurgood from comment #26)
> On macOS10.14
> 
> Excel 16.16.9 (190412) : 35s
> 
> LO6222 : 1m52s
> 
> Tested with Version: 6.3.0.0.alpha0+ : 45s
> 
> Build ID: dfae42730911256dceb8369528ee9d9944a0fa3e
> CPU threads: 8; OS: Mac OS X 10.14.3; UI render: GL; VCL: osx; 
> Locale: fr-FR (fr_FR.UTF-8); UI-Language: en-US
> Calc: threaded

That commit doesn't exists -> https://cgit.freedesktop.org/libreoffice/core/commit/?id=dfae42730911256dceb8369528ee9d9944a0fa3e
Comment 28 Mike Kaganski 2019-04-29 16:25:39 UTC
The main problem with the file was, as described in the initial description, "The file opens and shows its contents (huge line art), but UI is unresponsive". Also, the bug title was "LO (UI) hangs on opening specific (XLS) document" on creation. That seems like a funny shift of focus, when later rename made the focus be on opening time ;-)

It's really nice to see the improvement of load time, but ... the original problem needs addressing, too :-)
Comment 29 Aron Budea 2019-04-29 20:45:47 UTC
(In reply to Xisco Faulí from comment #24)
> @Timur, @Aron, could you please check again on your end ?

In 6.3.0.0.alpha0+ (8f03bdee8225c619305ef210391dcc3b6c6fe284) (x86) / Windows 7, it takes ~52s with default rendering, and ~60s with OpenGL rendering.

Unfortunately, as I wrote it quite often hangs completely afterwards, when just leaving LO there, not doing anything.
Comment 30 Julien Nabet 2019-05-02 14:42:37 UTC
I noticed this log in a never ending loop:
warn:legacy.osl:4916:8100:basegfx/source/polygon/b2dtrapezoid.cxx:650: Trapezoid decomposer in illegal state (!)
Comment 31 Julien Nabet 2019-05-02 14:49:51 UTC Comment hidden (obsolete)
Comment 32 Mike Kaganski 2019-05-02 14:55:19 UTC
(In reply to Julien Nabet from comment #31)
> Taking a look at basegfx/source/polygon/b2dtrapezoid.cxx, I noticed this
> comment twice "consume both edges" (lines 714 and 872) but saw this:
> maTrDeEdgeEntries.pop_front();
> maTrDeEdgeEntries.pop_front();
> 
> whereas I would have expected:
> maTrDeEdgeEntries.pop_front();
> maTrDeEdgeEntries.pop_back();

No problem there; the code checks if (bSameStartPoint && bSameEndPoint); the two are comparison of aLeft and aRight; and the latter two are taken from iterating maTrDeEdgeEntries twice from the beginning.