Bug 89110 - LibreOffice Calc crashes when soffice.bin reaches 1.600.000 K RAM in Taskmanager (out of memory)
Summary: LibreOffice Calc crashes when soffice.bin reaches 1.600.000 K RAM in Taskmana...
Status: RESOLVED DUPLICATE of bug 72918
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.3.5.2 release
Hardware: x86-64 (AMD64) Windows (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-02-04 08:58 UTC by Jonas Müller
Modified: 2016-04-26 16:38 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
cache memory settings (58.59 KB, image/png)
2015-02-05 13:30 UTC, raal
Details
File that crashes Calc when scrolling up to load in the new images (766.15 KB, application/vnd.oasis.opendocument.spreadsheet)
2015-02-05 16:47 UTC, Jonas Müller
Details
File that crashes Calc when scrolling up to load in the new images (804.87 KB, application/vnd.oasis.opendocument.spreadsheet)
2015-02-05 16:57 UTC, Jonas Müller
Details
Stack trace of crash and recover (35.70 KB, text/plain)
2015-03-06 16:35 UTC, V Stuart Foote
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jonas Müller 2015-02-04 08:58:09 UTC
I'm working on quite simple spreadsheet that does have about 300 highres .png images in a column.

When opening the file soffice.bin allocates about 1.060.000 K according to Taskmanager. The more I scroll through this file and the more images LibreOffice loads the more memory it allocates.

When it reaches ~1.600.000 K LibreOffice just crashes. It will either show the recovery dialogue or a message pointing towards an out of memory error appears.

In the settings there is an option for removing objects from the cache after a few minutes. If I set this to 1 Minute, the memory allocation will shrink down to 1.060.000 K after 1 minute but it will still reach 1.600.000 K when scrolling through the whole file.

This means I have to constantly monitor memory usage in the Taskmanager while working on the file.

Please make LibreOffice remove objects from the cache when it reaches ~1.600.000K to avoid crashes. 

(This limit around 1.6 GB seems to be quite normal for applications that are not large address aware)
Comment 1 raal 2015-02-05 13:30:12 UTC
Created attachment 113145 [details]
cache memory settings

Hello,
what is your graphics cache settings? Can you provide document for testing? (Please note that the attachment will be public, remove any sensitive information before attaching it. 
See https://wiki.documentfoundation.org/QA/FAQ#How_can_I_eliminate_confidential_data_from_a_sample_document.3F for help on how to do so.)
Comment 2 Jonas Müller 2015-02-05 16:46:13 UTC
I have uploaded an example file here: https://office.mailbox.org/publications/documents/2570616/d5964796d8ee29bd58e168cf82dfee49

Open the file, then watch memory usage in taskmanager rise as you slowly scroll up in the file.

The images in the file are a bit large so LibreOffice will crash before you can reach even 1.400.000 K but it should show the problem.

I tried it with default graphics cache settings and with reduced settings like

undo steps: 100
Cache: 2MB/1MB/1 Minute
number of objects: 3
Comment 3 Jonas Müller 2015-02-05 16:47:51 UTC
Created attachment 113148 [details]
File that crashes Calc when scrolling up to load in the new images
Comment 4 Jonas Müller 2015-02-05 16:56:03 UTC
Uploaded a better example here: https://office.mailbox.org/publications/documents/2570616/a25fdeda659b1d1e17cf6b90dee4f0c4
Comment 5 Jonas Müller 2015-02-05 16:57:09 UTC
Created attachment 113149 [details]
File that crashes Calc when scrolling up to load in the new images
Comment 6 Buovjaga 2015-02-07 20:26:30 UTC
(In reply to Jonas Müller from comment #5)
> Created attachment 113149 [details]
> File that crashes Calc when scrolling up to load in the new images

Doesn't crash for me, takes less than 1GB memory.
Can you test with 4.4 and report here?

Set to NEEDINFO.
Change back to UNCONFIRMED after you have provided the information.
Change to RESOLVED WORKSFORME, if the problem went away and you are happy using 4.4.

Win 7 Pro 64-bit, LibO Version: 4.4.0.3
Build ID: de093506bcdc5fafd9023ee680b8c60e3e0645d7
Locale: fi_FI
Comment 7 Jonas Müller 2015-02-09 17:47:36 UTC
Okay, LibreOffice 4.4.0 seems to have improved memory usage and I couldn't create a file that shows this bug.

It seems like LibreOffice 4.4.0 allocates more memory right from the start but then consumes the memory better as the files that crashed previously don't crash for me anymore (memory allocation is still at a very high 1.450.000K though).

However LibreOffice 4.4. still behaves weird when running >1.350.000 K memory. For example it refuses to paste a 3000x3000 image from the clipboard in this case with a generic error message: https://office.mailbox.org/publications/documents/2570616/565179bac530799d26a9f75149610ec4

(The error message reads: "The content of the clipboard could not be inserted")

Then I waited a bit for LibreOffice to delete cached graphics and continued inserting some random screenshots from the clipboard and it crashed at about 1.540.000 K: https://office.mailbox.org/publications/documents/2570616/44c425d717c2045e623a6f4ea32cc143

(This screenshot was taken a few seconds later when LibreOffice was down to 1.340.000 K again so there was still some stuff that LibreOffice could have done to avoid that crash!)

This all may sound ridiculous and very artificial however I experienced these out of memory crashes during my actual work on a huge price list just a week ago.

(Also why does LibreOffice paste a 3000x3000 (1:1) image copied from Paint.net as a 62,51 cm wide + 59,61 cm high image (NOT 1:1)?)

(I would love to test LibreOffice 4.4.X at work with the actual file I work on however 4.4.0 has just too many bugs right now to use it at work.)

I will try 4.4.1 at work and check if the bug appears in the price list again however with the high memory consumption of 4.4.0 right from the start I'm not sure I can even open the price list anymore...
Comment 8 Jonas Müller 2015-02-11 16:07:20 UTC
I tried it with a fresh installation of LibreOffice 4.4.0.3 portable at work and while working on the price list the memory quickly fills up to 1.673.XXX K however it stays that way even when scrolling trough the whole file!

That's a huge improvement.

However I then decided to close this file while memory consumption was still at 1.673.XXX K. LibreOffice asked me whether I wanted to save the changes, I clicked no and LibreOffice crashed with the fatal error "osl::Thread::create failed". 

Screenshot: https://office.mailbox.org/publications/documents/2570616/f2ddb2f8ecaa2666cea3372a310363a6

Tried it again at home with the normal Version LibreOffice 4.4.0.3 on the same file and here soffice.bin reaches its maximum at about 1.570.000 K however when I double click a cell to edit it, LibreOffice also crashes with the Thread create failed error.

If I do the same when memory consumption is just a bit below this mark (e.g. 1.555.000 K) then it doesn't crash.

It seems like there is some kind of memory limit that LibreOffice exceeds when you start using a new "feature" while memory allocation is at its max.
Comment 9 Buovjaga 2015-02-23 10:38:45 UTC
Jonas: this would be an interesting case to try with the 64-bit Windows build.

http://dev-builds.libreoffice.org/daily/master/Win-x86_64@42/
https://wiki.documentfoundation.org/Development/msvc-x86_64
Comment 10 Jonas Müller 2015-02-23 16:35:41 UTC
Downloaded libo-master64~2015-02-23_01.55.33_LibreOfficeDev_4.5.0.0.alpha0_Win_x64 and tried it again.

This version reaches about 2.115.000 K memory and does NOT crash when editing a cell or closing the file while the memory allocation is at this level! So far I couldn't get it to crash.

(Also this version runs quite a bit smoother while scrolling trough all the images and it responds very quickly. The x86 4.4.0.3 version is dropping some frames and lagging a bit while skimming through the file while the x86-64 version is silky-smooth at all times on my 120 hz screen)
Comment 11 Buovjaga 2015-02-23 17:14:08 UTC
Very nice! I guess if you are satisfied with the performance after some days of testing, you can close this as RESOLVED WORKSFORME.
Comment 12 Jonas Müller 2015-02-23 19:11:45 UTC
(In reply to Beluga from comment #11)
> Very nice! I guess if you are satisfied with the performance after some days
> of testing, you can close this as RESOLVED WORKSFORME.

Well, I'm satisfied with the improved performance and increased stability for now however as there seem to be no plans to release a stable 64-bit version it isn't really resolved for me as I can't use an experimental nightly alpha 0 version at work.

Also LibreOffice x86 still crashes because it runs out of memory which could be avoided with more aggressive memory management as ~50% of the memory is used for unneeded cached objects.
Comment 13 Buovjaga 2015-02-23 19:20:19 UTC
Ok, let's set to UNCONFIRMED, then, as no one else has been able to reproduce those crashes you are having.
Comment 14 Jonas Müller 2015-02-24 16:18:27 UTC
The better memory management in 4.4.0 and newer makes it impossible to create an example file. This sounds good but in my real life spreadsheet the bug still happens.

I can't simply upload this file here in public however I can upload it and send the link to some of you to take a look at it. What is the best way to do this?
Comment 15 Buovjaga 2015-02-24 16:51:54 UTC
(In reply to Jonas Müller from comment #14)
> I can't simply upload this file here in public however I can upload it and
> send the link to some of you to take a look at it. What is the best way to
> do this?

Best to attach here or if it's too big, upload it somewhere else.
If it has confidential information, you can anonymize it: https://wiki.documentfoundation.org/Anonymizing_a_document
Comment 16 Jonas Müller 2015-02-26 16:43:35 UTC
Okay here is the actual real world file: http://www.file-upload.net/download-10350364/price-listLibO.7z.html

(make sure you download the actual file from the file-upload.net server!)

Password is: LibO

Please don't upload/share this file anywhere else!

LibreOffice 4.3 series: Just scroll through the file. When soffice.bin uses more than >1.500.000 K memory it should crash.

---------

LibreOffice 4.4 series: Open this file, scroll through the file with page down to get the memory allocation of soffice.bin as high as possible and then try to edit a random cell by double clicking it.

For me it crashes if I do this and memory allocation is >1.500.000 K, however it may be higher for you.

Also there are some other random glitches at this high memory allocation:

- Sometimes when you try to open the "save as" dialogue the "file" dropdownlist simply closes without loading the "save as" dialogue!
- Sometimes when you can open the "save as" dialogue and you try to save the file it won't react to clicking on "save" or the return key.
- Sometimes when you try to overwrite an existing file the "!" symbol is missing from the warning message.

These glitches disappear when the memory allocation goes down again.

---------

It would be great if multiple people with LibreOffice x86 could verify this as an earlier example file used far less memory on Beluga' setup than on mine.
Comment 17 raal 2015-02-26 18:26:54 UTC
I'm not able able to reproduce, win7 2GB RAM. Max RAM ~1,4 GB. No crash.
Comment 18 V Stuart Foote 2015-03-06 16:35:34 UTC
Created attachment 113934 [details]
Stack trace of crash and recover

(In reply to Jonas Müller from comment #16)
> 
> It would be great if multiple people with LibreOffice x86 could verify this
> as an earlier example file used far less memory on Beluga' setup than on
> mine.

On Windows 7 sp1, 64-bit en-US with
Version: 4.3.6.2
Build ID: d50a87b2e514536ed401c18000dad4660b6a169e

Does not abort the soffice.bin process. Rather on document crash it cycles into document recovery. Stack trace attached seems to show that while scrolling it was happily generating bitmaps (salbmp.cxx) and then experiences a kernel exception.  This is 4.3.6.2, so memory management of bug 84729 had already been patched.

I didn't catch the exact value when crash/recovery kicked in but looked to be about the 1.7GB point--process falls back to idle at 1.14GB memory use.
Comment 19 Jonas Müller 2015-03-07 09:42:41 UTC
Thanks :)

Can you try it again with 4.4 series as mentioned in Comment #16?
Comment 20 Jonas Müller 2015-06-05 09:36:42 UTC
We're currently working on a new price list on a completely new pc and the crashes described in #16 still happen with LibreOffice 4.4.3 Win x86.

Another problem is that now there are no x86_64 nightlies for Windows anymore so it is really difficult to work on this file.

In general LibreOffice seems to use far too much memory when compared to MS Excel 2010 however even then it shouldn't crash.

I'm no programmer but is it possible to add a check that makes sure LibreOffice deletes cached objects and tries other things to reduce memory before it actually runs out of memory?

Currently there seems to be no such check and LibreOffice will simply crash resulting in data loss.




I have reuploaded the sample file here (Password LibO): 

(make sure you download the actual file from the file-upload.net server!)

Please don't upload/share this file anywhere else!

Try scrolling trough the whole file quickly and then do another action like copy&pasting a line or edit a cell
Comment 21 Jonas Müller 2015-06-05 09:38:25 UTC
Sorry, forgot the download link:

http://www.file-upload.net/download-10667463/price-listLibO.7z.html
Comment 22 Julien Nabet 2015-12-05 00:06:19 UTC
Jonas: For the test, could you give a try to last stable LO x86-64 version 5.0.3?
Comment 23 Buovjaga 2016-04-26 16:38:05 UTC

*** This bug has been marked as a duplicate of bug 72918 ***