Bug 129228 - FILEOPEN XLSX: file won't open then very slow (2 speedups in this bug)
Summary: FILEOPEN XLSX: file won't open then very slow (2 speedups in this bug)
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
6.2.8.2 release
Hardware: x86-64 (AMD64) All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:6.5.0 target:6.4.0.2
Keywords: filter:xlsx, perf
Depends on:
Blocks: XLSX
  Show dependency treegraph
 
Reported: 2019-12-06 10:06 UTC by sourcingp
Modified: 2023-03-31 21:45 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
The specific file, but also available from url specified in description (9.72 MB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2019-12-06 10:06 UTC, sourcingp
Details
perf flamegraph (19.59 KB, application/x-bzip)
2019-12-12 20:20 UTC, Julien Nabet
Details
Flamegraph (21.46 KB, application/x-bzip)
2019-12-20 20:05 UTC, Julien Nabet
Details
perf flamegraph (517.15 KB, application/x-bzip)
2021-11-26 12:17 UTC, Julien Nabet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description sourcingp 2019-12-06 10:06:55 UTC
Created attachment 156350 [details]
The specific file, but also available from url specified in description

Windows 10 Pro

I attempt to open this file
https://download.lenovo.com/pccbbs/options_iso/ocm_december_2019.xlsx

It's approximately 10MB.  The loading bar appears across the bottom of Calc and gets nearly to completion but hangs on the last 5% and then the application freezes.  I have 16GB RAM so I doubt that is the problem.

Eventually I have to force quit Calc. I have tried a few times and behavior repeats.

I would expect that the file would open.  If the file is somehow corrupt I would expect a suitable error message.
Comment 1 sourcingp 2019-12-06 10:10:32 UTC
LibreOffice_6.2.8_Win_x64.msi

Downloaded from https://www.libreoffice.org/donate/dl/win-x86_64/6.2.8/en-US/LibreOffice_6.2.8_Win_x64.msi
Comment 2 NISZ LibreOffice Team 2019-12-06 12:43:48 UTC
Freeze happens in 6.2 series, but not anymore in 6.3.0.
Opening performance still can not be considered an "experience" though:

Verzió: 6.5.0.0.alpha0+ (x64)
Build az.: 7e09d08807b5ba2fd8b9831557752a415bdad562
CPU szálak: 4; OS: Windows 6.3 Build 9600; Felületmegjelenítés: GL; VCL: win; 
Területi beállítások: en-US (hu_HU); Felület nyelve: hu-HU
Calc: CL

thinks during opening it for about 2:40.

Excel on the other hand opens it from recently used documents list in about 10 secs.
Comment 3 Roman Kuznetsov 2019-12-12 14:11:04 UTC Comment hidden (obsolete)
Comment 4 Julien Nabet 2019-12-12 14:24:17 UTC Comment hidden (obsolete)
Comment 5 Julien Nabet 2019-12-12 20:20:29 UTC
Created attachment 156525 [details]
perf flamegraph

Here's a Flamegraph on pc Debian x86-64 with master sources updated today.

Notice: I had this warning message at the end :
Warning loading document ...:
The data could not be loaded complete because the maximum number of columns per sheet was exceeded.
Comment 6 Xisco Faulí 2019-12-19 11:12:21 UTC
Don't see why it's a high severity issue
Comment 7 Xisco Faulí 2019-12-19 11:17:30 UTC
it takes

real	11m33,504s
user	11m45,776s
sys	0m10,721s

in

Version: 6.5.0.0.alpha0+
Build ID: fb1eac64df88baae9f211d052793773686c0e180
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded

until the warning message is displayed
Comment 8 Xisco Faulí 2019-12-19 11:18:26 UTC
(In reply to Julien Nabet from comment #5)
> Created attachment 156525 [details]
> perf flamegraph
> 
> Here's a Flamegraph on pc Debian x86-64 with master sources updated today.

@Noel, I thought you might be interested in this issue...
Comment 9 Commit Notification 2019-12-20 06:17:32 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/f2f5d74067705b7c64f8fb510952d7cece417627

tdf#129228 speedup load xls file with lots of comments

It will be available in 6.5.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 10 Xisco Faulí 2019-12-20 09:40:07 UTC
it takes

real	8m42,639s
user	8m55,738s
sys	0m10,707s

in

Version: 6.5.0.0.alpha0+
Build ID: dee81fb2e1df5091702b3c8b0e4a3f2b58e89291
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded

import time has been reduced but it's still far from ideal, putting it back to NEW
Comment 11 Timur 2019-12-20 10:30:36 UTC
(In reply to Xisco Faulí from comment #6)
> Don't see why it's a high severity issue

Because Prioritizing_Bugs_Flowchart says Major/High for any inability to open particular documents. And it makes sense because LO looks bad then.
Comment 12 Commit Notification 2019-12-20 14:16:21 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "libreoffice-6-4":

https://git.libreoffice.org/core/commit/543fd1b9f8e05988d4c45602f4247566c352ee81

tdf#129228 speedup load xls file with lots of comments

It will be available in 6.4.0.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 13 Julien Nabet 2019-12-20 20:05:42 UTC
Created attachment 156704 [details]
Flamegraph

Here's a new Flamegraph with master sources updated today.
commit d0b69965012a711c0b181bf359c8b27a782be2c3 (HEAD -> master, origin/master, origin/HEAD)
Author: Batuhan Görkem Benzer <batuhan1167@gmail.com>
Date:   Fri Dec 20 08:16:27 2019 -0500

    tdf#124176: Use pragma once instead of include guards
Comment 14 Commit Notification 2019-12-29 06:53:05 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/8f25472d3de4f43cf594fa41ebfba2a1b01d4c9d

tdf#129228 speedup opening of xlsx file with lots of comments

It will be available in 6.5.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 15 Xisco Faulí 2019-12-29 13:50:06 UTC
it takes

real	1m47,438s
user	1m50,278s
sys	0m2,469s

in

Version: 6.5.0.0.alpha0+
Build ID: b9e86836d79039a28661dfb6195df0a4dc1e7572
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded

Impressive improvement!
@Noel, thanks for fixing !
Comment 16 Commit Notification 2020-01-01 08:38:02 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/4992d61600536fe14b97b718dbb11f00e936c6a9

tdf#129228 speedup opening of xlsx file with lots of comments

It will be available in 6.5.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 17 Commit Notification 2020-01-01 08:40:38 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "libreoffice-6-4":

https://git.libreoffice.org/core/commit/3208e69e5ff7a4d151a108d7f2af134664a26bab

tdf#129228 speedup opening of xlsx file with lots of comments

It will be available in 6.4.0.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 18 Commit Notification 2020-01-07 16:48:06 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "libreoffice-6-4":

https://git.libreoffice.org/core/commit/caccac87b369988b63b952957d367f4d2ee6791c

tdf#129228 speedup opening of xlsx file with lots of comments

It will be available in 6.4.0.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 19 sourcingp 2020-03-29 16:46:00 UTC
On 6.4.2 can confirm the file now actually opens and is a lot faster than it was.  It now takes about 2m17s to open on my system.  I do get a warning message about "maximum number of columns per sheet was exceeded" but after clicking ok I am able to use the document.
Comment 20 Noel Grandin 2020-05-22 12:07:42 UTC
I had a good look at this, and there is no easy or even medium way to make it faster.
Comment 21 Julien Nabet 2021-11-26 12:17:53 UTC
Created attachment 176519 [details]
perf flamegraph

Just in case, I retrieved a new Flamegraph on pc Debian x86-64 with master sources updated today.(b18c2a0024c6d33cdf142ed2adf0d127483411e8)
Comment 22 m_a_riosv 2023-03-31 21:45:33 UTC
*** Bug 154525 has been marked as a duplicate of this bug. ***