Bug 53698 - FILEOPEN particular large .xlsm takes 200 times longer than with EXCEL Viewer
Summary: FILEOPEN particular large .xlsm takes 200 times longer than with EXCEL Viewer
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: All All
: high major
Assignee: Not Assigned
URL:
Whiteboard: target:5.3.0
Keywords: perf
Depends on:
Blocks: Calc-Images
  Show dependency treegraph
 
Reported: 2012-08-19 06:35 UTC by michael helm
Modified: 2021-09-03 13:49 UTC (History)
10 users (show)

See Also:
Crash report or crash signature:


Attachments
bts + console logs on master (8.52 KB, application/zip)
2012-10-06 05:24 UTC, Julien Nabet
Details
Test XLSM (8.39 MB, application/x-7z-compressed)
2017-06-16 10:25 UTC, Timur
Details

Note You need to log in before you can comment on or make changes to this bug.
Description michael helm 2012-08-19 06:35:54 UTC
Situation is a lot like bug #53653
Try to open large (7 MB) excel-formatted large spreadsheet, hangs for a
long time, maybe permanently.  Impossible to stop or kill
process - to get rid of the hanging spreadsheet, a reboot
is necessary.

This file causes this problem:
https://dl.dropbox.com/u/17907527/R1b-L21_Haplotypes.zip
Comment 1 Rainer Bielefeld Retired 2012-08-19 08:24:03 UTC
[Reproducible] with "LibreOffice 3.6.1.1  German UI/Locale [Build-ID:  4db6344] on German WIN7 Home Premium (64bit) 

It takes a minute or so until progress bar reaches 80%, then LibOO stops responding. That's some better than AOOo 3.4, what stops responding ag 20% porgress bar, but mich worse than MS Excel Viewer, what opens the document within 10 seconds.

Parallel installation of Master "LOdev  3.7.0.0.alpha0+   - WIN7 Home Premium (64bit) ENGLISH UI [Build ID: 6900781]" (tinderbox: 2008R2@20, pull time 2012-08-14 09:27:23) opens the document within several minutes.

I will still do some additional tests.

@michael helm:
Did you ever see that working better with LibO or OOo?
Comment 2 Rainer Bielefeld Retired 2012-08-19 09:15:31 UTC
Also 3.6.6.1 will open the document after 1/2 h or so. 

There is some excessive conditional formatting in the document what might be part of the problem? I did not test this.

I doubt that for this issue will be any action for 3.5, so Performance issue for 3.6

@Spreadsheet Team
Please set Status to ASSIGNED and add yourself to "Assigned To" if you accept this Bug (and remove others in team from CC)
Comment 3 michael helm 2012-08-24 03:27:14 UTC
oO (the newest one - 3.4.1) is at least as bad - as far as I can
tell it never managed to open the document completely & left
the spreadsheet application in a weird non functional state.
I probably should report that to Apache, too.
Comment 4 Julien Nabet 2012-10-06 05:24:13 UTC
Created attachment 68145 [details]
bts + console logs on master

On pc Debian x86-64 with master sources updated yesterday, I've got a freeze (or at least, i can't load the file after more than a minute perhaps 2).

I attached several bts, after the third I had a popup dialog about macro security.
I attached too console logs.
Comment 5 Timur 2013-01-15 16:58:29 UTC Comment hidden (obsolete)
Comment 6 QA Administrators 2013-09-24 01:54:48 UTC Comment hidden (obsolete)
Comment 7 Timur 2013-09-24 07:27:41 UTC Comment hidden (obsolete)
Comment 8 QA Administrators 2015-04-19 03:23:15 UTC Comment hidden (obsolete)
Comment 9 Timur 2015-04-20 07:19:52 UTC
The same problem remains with LO 4.4 both on Windows 7 and Ubuntu 64-bit.
Inherited from OO. According to https://wiki.documentfoundation.org/images/0/06/Prioritizing_Bugs_Flowchart.jpg marked as Major-High.
Comment 10 Robinson Tryon (qubit) 2015-12-09 18:08:03 UTC Comment hidden (obsolete)
Comment 11 Maarten Bosmans 2016-08-30 10:49:24 UTC
On git master, it takes 2 minutes to load this document on my desktop computer. Still way to long, but at least it finishes.

A significant part of the time is spend in getting locale details in order to do cell number formatting. The result of LocaleDataImpl::getAllFormats(lang) is supposed to be cached, but that seems not to happen.

I have some patches fixing this, reducing the load time to 1 minute. When that is posted, we'll see how to go from there.
Comment 12 Commit Notification 2016-09-22 07:16:26 UTC
Maarten Bosmans committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=110183572bfe9da0020b4c506d4b458bf69b1e85

tdf#53698: Add a NumberFormatMapper member to SvNumberformatScan

It will be available in 5.3.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 13 Commit Notification 2016-09-22 19:02:21 UTC
Maarten Bosmans committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=450a2fd5e2dafd1a0c08e73ef85db978f6b1a927

tdf#53698: Cache more than 1 item in NumberFormatCodeMapper

It will be available in 5.3.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 14 Xisco Faulí 2016-10-23 12:30:58 UTC Comment hidden (obsolete)
Comment 15 Xisco Faulí 2017-06-16 09:15:01 UTC Comment hidden (obsolete)
Comment 16 Timur 2017-06-16 10:25:13 UTC
Created attachment 134061 [details]
Test XLSM

Doesn't look good to me, please see if it's acceptable
Comment 17 Xisco Faulí 2017-06-16 11:17:36 UTC
in

Version: 5.5.0.0.alpha0+
Build ID: 6ab249ea6aecef5d3f35d624622a368061cad9c3
CPU Threads: 4; OS Version: Linux 4.8; UI Render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); Calc: group

it takes to open

real	1m58.074s
user	2m1.104s
sys	0m1.628s

while in

Version: 4.3.0.0.alpha1+
Build ID: c15927f20d4727c3b8de68497b6949e72f9e6e9e

I killed it after

real	12m11.521s
user	12m11.028s
sys	0m0.884s
Comment 18 Xisco Faulí 2017-07-13 10:18:25 UTC Comment hidden (obsolete)
Comment 19 josselin.stark 2017-11-11 09:46:55 UTC
I'm having a very similar issue. I am trying to open a 16MB large XLSM file (https://www.dropbox.com/s/m4m1nurerbg0dpf/DS-NKh-Sentence%20Maker.zip?dl=0).
The progress bar goes up to about 80% before the process hangs. Task manager shows me LibreOffice is using 25% of the CPU and 1.2GB of RAM, but it doesn't seem to be able to finish the task.
Comment 20 josselin.stark 2017-11-11 09:47:55 UTC
PS : Using LO 5.4.2.2
Comment 21 QA Administrators 2018-11-12 03:40:56 UTC Comment hidden (obsolete)
Comment 22 QA Administrators 2020-11-12 04:53:52 UTC Comment hidden (obsolete)
Comment 23 Roman Kuznetsov 2020-12-25 16:25:30 UTC
LO opened it about for 1 minute in

Version: 7.2.0.0.alpha0+ (x64)
Build ID: ad9e04321df25824d2288a2ef1f4275f070f1cf7
CPU threads: 4; OS: Windows 6.1 Service Pack 1 Build 7601; UI render: Skia/Raster; VCL: win
Locale: ru-RU (ru_RU); UI: ru-RU
Calc: CL

Xisco, can you retest it? I mean is it still a bug? Excel opens the file about for 20 sec for me
Comment 24 Xisco Faulí 2021-01-04 13:46:27 UTC
it takes

real	0m43,637s
user	0m44,475s
sys	0m2,125s

in

Version: 7.2.0.0.alpha0+
Build ID: 2c9708cbb870483a8a1c93d722085be5f789d234
CPU threads: 4; OS: Linux 5.7; UI render: default; VCL: gtk3
Locale: en-US (en_US.UTF-8); UI: en-US
Calc: threaded

and

Version: 7.0.5.0.0+
Build ID: 7893e24c82c97de9c066f6e09df0ac107a6d97dd
CPU threads: 4; OS: Linux 5.7; UI render: default; VCL: gtk3
Locale: en-US (en_US.UTF-8); UI: en-US
Calc: threaded

I guess we can close this as RESOLVED WORKSFORME
Comment 25 Timur 2021-09-02 11:38:10 UTC
OO 3.3
very long

LO 4.3 m
too long as in Comment 17 

5.4 m
real	4m45,608s
user	4m55,664s
sys	0m4,217s

6.2 m 
real	4m7,726s
user	4m11,673s
sys	0m6,010s

6.3 m
real	1m23,676s
user	1m30,713s
sys	0m3,867s

6.4 m
real	1m37,252s
user	1m43,380s
sys	0m4,039s

7.0 m
real	1m50,430s
user	1m36,137s
sys	0m2,961s

7.3 m
real	1m8,606s
user	1m11,307s
sys	0m1,667s

LO improved in LO 6.3.
Comment 26 Timur 2021-09-02 14:48:53 UTC
I wanted to find a commit, but my 6.2 master is NOK and 6.3 oldest is OK.
Comment 27 Timur 2021-09-03 12:28:40 UTC
(In reply to Timur from comment #26)
> I wanted to find a commit, but my 6.2 master is NOK and 6.3 oldest is OK.

Buovjaga, please see this.
Comment 28 Buovjaga 2021-09-03 13:49:00 UTC
Bibisected the fix to https://git.libreoffice.org/core/commit/0702541b3d4661846dccde26efa9c2fb5ed65f35
tdf#126663: optimize style list display in sidebar

Julien is in Cc here, so let's thank him of this 2019 fix :)