Bug 95346 - XLSX file crashes LibreOffice calc when save or very slow and eats 600 MB to 2 GB of RAM
Summary: XLSX file crashes LibreOffice calc when save or very slow and eats 600 MB to ...
Status: RESOLVED WONTFIX
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
3.4.0 release
Hardware: All All
: high major
Assignee: Not Assigned
URL:
Whiteboard: target:7.4.0
Keywords: haveBacktrace, perf
Depends on:
Blocks: XLSX Memory
  Show dependency treegraph
 
Reported: 2015-10-26 23:05 UTC by mcjarod
Modified: 2022-03-02 14:32 UTC (History)
6 users (show)

See Also:
Crash report or crash signature:


Attachments
confidencial finantial information (418.39 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2015-10-26 23:05 UTC, mcjarod
Details
Callgrind output from master (2.70 MB, application/x-xz)
2018-09-24 15:51 UTC, Buovjaga
Details
Perf flamegraph of saving (646.46 KB, image/svg+xml)
2019-04-19 11:42 UTC, Buovjaga
Details
doc saved as ods in LO 6.4.1.2 (357.48 KB, application/vnd.oasis.opendocument.spreadsheet)
2020-03-10 22:18 UTC, paulystefan
Details
perf flamegraph (99.08 KB, application/x-bzip)
2020-03-11 20:32 UTC, Julien Nabet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description mcjarod 2015-10-26 23:05:26 UTC
Created attachment 119985 [details]
confidencial finantial information

I have problems with the attachment file, with this file normally work (but with ms office), now I trying to make a change in LibreOffice and save it, but then LibreOffice crashes.
Comment 1 raal 2015-10-27 08:05:38 UTC
I can confirm with LO 5.0.2.2 and LO Version 4.0.0.3 (Build ID: 7545bee9c2a0782548772a21bc84a9dcc583b89); win7
I can confirm with LO Version: 5.1.0.0.alpha1+
Build ID: 233b9b0ec95069b5ce98aab942304459ca7344a8
Lo quickly eats 1,6GB of memory and doesn't respond.
Comment 2 Timur 2015-10-29 15:03:15 UTC Comment hidden (obsolete)
Comment 3 Timur 2017-08-08 10:31:40 UTC
Fix from bug 66668 didn't fix this one.
Comment 4 bdy 2017-08-10 14:36:13 UTC
Version: 5.4.0.3 (x64), Build-ID: 7556cbc6811c9d992f4064ab9287069087d7f62c, Win10:

Does not crash, but is busy during save as „Microsoft Excel 2007-2013 XML“ for a while (1-2 minutes?)

Eats up more than 2.2 GBytes(!) of RAM while saving, then returns to normal behaviour an frees the RAM.

Obviously something need to be optimized.
Comment 5 Timur 2017-08-11 14:53:31 UTC
This is a circuclar reference file, but that's not a cause. XLSX without circ. fer. is also slow. Valgrind needed, I guess.
Comment 6 Buovjaga 2018-09-24 15:51:28 UTC
Created attachment 145140 [details]
Callgrind output from master

From saving

Arch Linux 64-bit
Version: 6.2.0.0.alpha0+
Build ID: 8b1501d80dc9d3f42c351c6e026fa737e116cae5
CPU threads: 8; OS: Linux 4.18; UI render: default; VCL: gtk3_kde5; 
Locale: fi-FI (fi_FI.UTF-8); Calc: threaded
Built on 23 September 2018
Comment 7 Buovjaga 2019-04-19 11:42:58 UTC
Created attachment 150884 [details]
Perf flamegraph of saving

Pretty nice plateaus in there

Arch Linux 64-bit
Version: 6.3.0.0.alpha0+
Build ID: 1fee3f1da6291bfbcd75455512726215d41b3e83
CPU threads: 8; OS: Linux 5.0; UI render: default; VCL: gtk3; 
Locale: fi-FI (fi_FI.UTF-8); UI-Language: en-US
Calc: threaded
Built on 18 April 2019
Comment 8 paulystefan 2020-03-10 22:18:56 UTC
Created attachment 158580 [details]
doc saved as ods in LO 6.4.1.2

doc saved as ods in LO 6.4.1.2 x64 win 10-64

no problem here
Comment 9 Buovjaga 2020-03-11 05:44:04 UTC
Tested saving to XLSX (the original problem) and it only takes like 10 secs, so WFM is correct status.

Arch Linux 64-bit
Version: 7.0.0.0.alpha0+
Build ID: 08334285ec9c7d5356f4b89192a5fba6e6733328
CPU threads: 8; OS: Linux 5.5; UI render: default; VCL: kf5; 
Locale: fi-FI (fi_FI.UTF-8); UI-Language: en-US
Calc: threaded
Built on 8 March 2020
Comment 10 Timur 2020-03-11 08:48:06 UTC
I have a slow Linux test system and it takes 50 secs with master a11c10a83f6fceae6cfb519725d06f8eaf1013fb of yesterday.
And oldest 7.0+ was 60 secs, just slightly better.
Saving from XLSX to XLSX.
I'll set New until retested.
Comment 11 Timur 2020-03-11 09:40:43 UTC
master 7.0+: 
real	0m54,719s
user	1m56,268s
sys	0m5,638s

master 7.0+ with file overwrite: 
real	0m57,447s
user	1m54,335s
sys	0m5,673s

oldest 7.0+ with file overwrite:
real	0m54,229s
user	1m48,155s
sys	0m5,854s

I guess those numbers confirm what I wrote ,please correct if not.
Comment 12 Julien Nabet 2020-03-11 20:32:47 UTC
Created attachment 158624 [details]
perf flamegraph

Here's a Flamegraph retrieved on pc Debian x86-64 with master sources updated today.
Comment 13 b. 2020-06-09 05:52:27 UTC
save as xlsx has a slow point of about 10 sec. with below ver, but no mem problem or crash, 

wouldn't have commented but if the document contains real world data - didn't test but looks like - id recommend to pull it back, in many states spreading of such private information is a verdict ... 

Version: 7.0.0.0.alpha1
Build ID: 6a03b2a54143a9bc0c6d4c7f1...
CPU threads: 8; OS: Linux 4.19; UI render: default; VCL: gtk3; 
Locale: de-DE (en_US.UTF-8); UI: en-US
Calc: threaded
Comment 14 paulystefan 2020-06-20 20:16:02 UTC
seems ok with also 10 sec

in 7.0.0 beta2 x64 in win 10 x64
Comment 15 Luboš Luňák 2022-03-01 14:09:59 UTC

*** This bug has been marked as a duplicate of bug 140893 ***
Comment 16 Commit Notification 2022-03-02 08:27:54 UTC
Luboš Luňák committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/f8edf0a94e9608d7e7d8ed5fb46036b2ceb71693

try to avoid using map when searching most used item (tdf#95346)

It will be available in 7.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 17 Commit Notification 2022-03-02 08:28:03 UTC
Luboš Luňák committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/4e279e8c4986ca3de79bea2216c5783cb113a5bb

reuse a vector instead of repeatedly creating it (tdf#95346)

It will be available in 7.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 18 Timur 2022-03-02 10:29:13 UTC
Luboš, please explain why it's marked a duplicate of bug 140893 when it's not resolved with that, still 2,5 GB or RAM in saving. 
Also, you had other patches, so I guess Fixed would be a better status.
Comment 19 Luboš Luňák 2022-03-02 14:32:16 UTC
Several sheets contain a value in the last row. The export filter needs to process all rows because of that and the internal representation needs that memory. I don't see this pathological case worth the effort of doing something about it.