Bug 121094 - Performance: Opening xlxs with many rows takes several minutes
Summary: Performance: Opening xlxs with many rows takes several minutes
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.4.7.2 release
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:6.4.0 target:6.3.0.1
Keywords: filter:xlsx, perf
Depends on:
Blocks: XLSX File-Opening CPU-AT-100%
  Show dependency treegraph
 
Reported: 2018-11-01 14:29 UTC by Gabor Kelemen (allotropia)
Modified: 2019-07-04 18:48 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
Example file from Excel with 800k rows (1.17 MB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2018-11-01 14:29 UTC, Gabor Kelemen (allotropia)
Details
Opening the file in Calc is sloooow (116.31 KB, image/png)
2018-11-01 14:38 UTC, Gabor Kelemen (allotropia)
Details
Opening the xl\worksheets\sheet1.xml (145.48 KB, image/png)
2018-11-01 14:40 UTC, Gabor Kelemen (allotropia)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Gabor Kelemen (allotropia) 2018-11-01 14:29:09 UTC
Created attachment 146224 [details]
Example file from Excel with 800k rows

Attached user-made xlsx has a few rows with actual data then 800k empty rows that are actually saved. I have no idea how or why they made it so.

Opening this in Excel takes a few seconds, but in Calc it takes several minutes with 100% processor use. Memory use jumps inside a range of a few megabytes, so it does not run away.

It would be nice to improve performance in case of such "extreme" files.
Comment 1 Gabor Kelemen (allotropia) 2018-11-01 14:38:14 UTC
Created attachment 146225 [details]
Opening the file in Calc is sloooow

Version: 6.2.0.0.alpha1+
Build ID: b6b31bbb1a9e2272ac77de127825c4ee9f71effa
CPU threads: 4; OS: Windows 6.3; UI render: GL; VCL: win; 
Locale: hu-HU (hu_HU); Calc: CL
Comment 2 Gabor Kelemen (allotropia) 2018-11-01 14:40:49 UTC
Created attachment 146226 [details]
Opening the xl\worksheets\sheet1.xml

Opening 25 Mb of this can be slow:

<row r="812055" ht="15" customHeight="1" x14ac:dyDescent="0.25"/><row r="812057" ht="15" customHeight="1" x14ac:dyDescent="0.25"/><row r="812059" ht="15" customHeight="1" x14ac:dyDescent="0.25"/>
Comment 3 Telesto 2018-11-01 16:24:16 UTC
Repro with
Version: 6.2.0.0.alpha1+
Build ID: 4fa9e6f7f891b335ae1b432e0848c1e46c8fe3ef
CPU threads: 4; OS: Windows 6.3; UI render: default; VCL: win; 
TinderBox: Win-x86@42, Branch:master, Time: 2018-10-30_22:44:48
Locale: nl-NL (nl_NL); Calc: CL

and with
Versie: 4.4.7.2 
Build ID: f3153a8b245191196a4b6b9abd1d0da16eead600
Locale: nl_NL
Comment 4 Commit Notification 2019-06-04 07:23:48 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/79927a934b8a27bd4dee7a263171981e6ca22f8a%5E%21

tdf#121094 Opening xlxs with many rows takes several minutes

It will be available in 6.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 5 Commit Notification 2019-06-04 11:16:08 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "libreoffice-6-3":

https://git.libreoffice.org/core/+/cabaadf278ba099c53ed2b7a32f1e11bc632ad3a%5E%21

tdf#121094 Opening xlxs with many rows takes several minutes

It will be available in 6.3.0.1.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 6 Xisco Faulí 2019-06-04 12:26:54 UTC
It opens instantly in

Version: 6.3.0.0.beta1+
Build ID: cabaadf278ba099c53ed2b7a32f1e11bc632ad3a
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded

@Noel, thanks for fixing this issue!
Comment 7 Justin L 2019-07-04 18:48:22 UTC
(In reply to Gabor Kelemen from comment #0)
> I have no idea how or why they made it so.
Probably from LO 5.2.0-5.2.5 bug 105840 - FILESSAVE XLS: file size increases to 21mb after re-saving a particular Calc document