Bug 69733 - FILEOPEN: Huge memory usage with 6000+ conditional formatting entries
Summary: FILEOPEN: Huge memory usage with 6000+ conditional formatting entries
Status: RESOLVED WORKSFORME
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.0.5.2 release
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: BSA
Keywords: regression
Depends on:
Blocks:
 
Reported: 2013-09-23 20:23 UTC by ari100krat3dorder
Modified: 2015-01-26 05:03 UTC (History)
6 users (show)

See Also:
Crash report or crash signature:


Attachments
This file is journal that I make for my students. (313.34 KB, application/vnd.oasis.opendocument.spreadsheet)
2013-09-23 20:23 UTC, ari100krat3dorder
Details

Note You need to log in before you can comment on or make changes to this bug.
Description ari100krat3dorder 2013-09-23 20:23:44 UTC
Created attachment 86418 [details]
This file is journal that I make for my students.

Problem description: 

Steps to reproduce:
1. Open file. LO crash.

OpenOffice correct open this file. Google drive correct (for that service) convert and open.

All other file ods open correctly.

Advanced:
1. LibreOffice 4.0.5 on windows 7x64 don't open this file.
2. LibreOffice 4.0.5 on linux x32 (openSuse) open this file.
3. LibreOffice 4.1.x on linux x32 (openSuse) don't open this file.
Operating System: Windows 7
Version: 4.1.1.2 release
Comment 1 Maxim Monastirsky 2013-09-23 20:52:41 UTC
Hi,

I can open your file without any problem using LO 4.1.1.2 from PPA under Ubuntu 13.04 (64-bit).

Does LO really 'crashing', or it just some sort of error ('General Error' or 'Input\Output Error' or Corrupted file or something else)? If it's a crash, would be great if you could provide a backtrace (See https://wiki.documentfoundation.org/QA/BugReport/Debug_Information#GNU.2FLinux:_How_to_get_a_backtrace). thanks.

(I've changed 'Platform' to 'All' since you have this problem on Linux too.)
Comment 2 tommy27 2013-09-24 05:29:49 UTC
file doesn't open in 4.0.5, 4.1.0 and 4.1.1 final releases under Win7 64bit. 
I receive "read error" message and loading aborts.

file opens with no read error in 4.0.4 and earlier releases
set status to NEW. changed version field. added regression keyword. 
CC'ing Calc expert.
Comment 3 Eike Rathke 2013-10-04 11:27:08 UTC
I could load the document in all versions, it doesn't crash per se, but eats a large amount of memory (~4GB) which on some systems may lead to a crash if the memory is not available, i.e. 32-bit environments (the Windows executable is 32-bit!) or insufficient swap-space.

@Moggi:
This may be due to conditional formats that are used a lot in that document. Furthermore the memory is not released when closing the document.
Comment 4 Eike Rathke 2013-10-04 12:33:40 UTC
@Moggi:
Indeed, after having removed all conditional formats, saved document, closed app, started again and loaded the new document the memory consumption is gone.
Comment 5 Markus Mohrhard 2013-10-26 10:20:40 UTC
The document contains about 6000 conditional formats (one for nearly each cell) which are mostly the same.

Removing the conditional format and applying them in 4.0+ again will surely result in much better performance. Let me see if I find a way to merge these conditional formats during import and therefore bring the document to a decent performance.

All in all the file only uses about 350MB for me but takes quite long due to some nasty problem with CompileXML and ScConditionalFormat::SourceChanged

I can fix the second one but can't reproduce the memory problem in my dbgutil master build.

@Eike: Which build did you use for your test?
Comment 6 Eike Rathke 2013-10-31 18:00:23 UTC
@Moggi:
I don't recall anymore, but usually I check if an error occurs also in master because I have that as debug build. If it isn't reproducible there I use the latest on the release branch indicated in Version, which would be pre-4.0.6 in this case or also pre-4.1.3 as comment 2 said it would occur also in 4.1.1
Comment 7 raal 2014-10-17 18:19:52 UTC
With Version: 4.4.0.0.alpha0+
Build ID: 9aa36a1ad39e37c372cc833a44fba450b8cc30cd
TinderBox: Linux-rpm_deb-x86_64@46-TDF, Branch:master, Time: 2014-10-09_04:46:44
Version: 4.3.4.0.0+
Build ID: afd19a5ee99b1855bc2c2a48a29d2da16be883d1
TinderBox: Linux-rpm_deb-x86_64@46-TDF, Branch:libreoffice-4-3, Time: 2014-10-10_01:47:41
I can open file without problem. Soffice.bin uses ~350MB of memory.
Comment 8 Matthew Francis 2015-01-26 05:03:56 UTC
I can also confirm that on OSX 10.10 with LO 4.3.5.1 and 4.4.0.2, the file loads in a few seconds and doesn't use an unreasonable amount of memory

As a specific commit or commits which fix this haven't been identified, setting:
-> RESOLVED WORKSFORME