Bug 108637 - Memory usage increases considerably when deleting rows in a document with a 1000 empty sheets (see comment 3)
Summary: Memory usage increases considerably when deleting rows in a document with a 1...
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.1.0.4 release
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: bibisected, perf, regression
Depends on:
Blocks: Cell-Add-Delete Memory
  Show dependency treegraph
 
Reported: 2017-06-19 18:50 UTC by Telesto
Modified: 2023-01-12 03:20 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments
Example file (4.57 MB, application/vnd.oasis.opendocument.spreadsheet)
2017-07-08 15:20 UTC, Telesto
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Telesto 2017-06-19 18:50:48 UTC
Description:
Memory usage has increased for creating a document with lots of empty sheets.
With LibO 6.0.0.0 the total memory usage for 1000 sheets is around 390 MB.  LibO4.0.0.3 requires only 231 MB. There is a steady increase visible.



Steps to Reproduce:
1. Open Calc
2. Monitor memory usage (Process explorer)
3. Menu Sheet - Insert Sheet -> 1000

Additional steps for a related issue
4. Delete one sheet and monitor memory increase. 

Actual Results:  
Total memory requirement (Calc + sheets)
4.0.0.3 = 231 MB
4.1.0.4 = 345 MB
4.4.6.3 = 303 MB
5.2.2.1 = 452 MB
6.0.0.0 = 390 MB

Memory increase after deleting one sheet
4.0.0.3 = 70 MB 
4.1.04 = 232 MB 
6.0.0.0 = 208

Expected Results:
Something similar to 4.0.0.3


Reproducible: Always

User Profile Reset: No

Additional Info:
Found in
Version: 6.0.0.0.alpha0+
Build ID: cbf371e07fd5dea1ea08a1f299360d1273961ebd
CPU threads: 4; OS: Windows 6.19; UI render: default; 
TinderBox: Win-x86@42, Branch:master, Time: 2017-06-14_23:13:57
Locale: en-US (nl_NL); Calc: CL

and in
4.1.0.4

but not in
Versie 4.0.0.3 (Bouw-id: 7545bee9c2a0782548772a21bc84a9dcc583b89)


User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0
Comment 1 m_a_riosv 2017-06-19 23:06:07 UTC
I think undo usage of memory has somerhinf to do with that, a lot of undo issues have been solved along the time.
Comment 2 Eike Rathke 2017-06-26 18:29:23 UTC
So what? The (Calc + sheets) number is irrelevant as it doesn't say anything about the memory use per sheet because it includes the memory consumed by the one Calc application instance plus the one document instance as well. If at all you'll have to measure the difference between one document with one sheet and adding 1000 sheets. And of course deleting the first sheet consumes memory as an Undo document is created and an Undo sheet added.
Comment 3 Telesto 2017-06-26 20:07:08 UTC
(In reply to Eike Rathke from comment #2)
> So what? The (Calc + sheets) number is irrelevant as it doesn't say anything
> about the memory use per sheet because it includes the memory consumed by
> the one Calc application instance plus the one document instance as well. If
> at all you'll have to measure the difference between one document with one
> sheet and adding 1000 sheets. And of course deleting the first sheet
> consumes memory as an Undo document is created and an Undo sheet added.

So I adjust my initial report
1. Open Calc
2. Insert 1000 sheets
3. Select row 1 until 45 in a random sheet and delete the empty rows
4. Repeat step 3 two times with a x86 build -> Crash hitting the 1.3 GB memory limit. In LibO 4.0.0.3 i can repeat the steps 12 times before I reach 1.2 GB (and still no crash) and undo/redo seems to work fine

A side note: it's possible to increase memory limit from 1,3 GB to 4 GB on x86 windows system: http://www.cnblogs.com/zjoch/p/6322252.html
Comment 4 Buovjaga 2017-06-27 10:36:37 UTC
Ok, so let's move the goalposts.

Starting with 1 sheet: 179M RES
After 3 deletions: 184M RES

New document:
After adding 1000 sheets: 731M RES
After 3 deletions: 2900M RES

Arch Linux 64-bit, KDE Plasma 5
Version: 6.0.0.0.alpha0+
Build ID: f808c50c6eece87d515df3b84b1c774395b5d9bc
CPU threads: 8; OS: Linux 4.11; UI render: default; VCL: kde4; 
Locale: fi-FI (fi_FI.UTF-8); Calc: group
Built on June 26th 2017
Comment 5 Telesto 2017-06-28 08:08:15 UTC
(In reply to Buovjaga from comment #4)
Same conclusion as with you description in bug 108617#c2. It's definitely related to the undo. The default value for "Steps" is 100. Set it to 0 and the memory growth is hitting a wall.

You can test it by opening Tools
- Options - LibO - Advanced - Expert config: org.openoffice.Office.Common/Undo

Seems the undo data is created for all sheets, when making a change to one sheet.
Comment 6 Telesto 2017-07-08 15:20:33 UTC
Created attachment 134550 [details]
Example file

Another example for the memory bump, without the interference of undo data
1. Open Calc
2. Set Undo steps to zero (0) ->  Options - LibO - Advanced - Expert config: org.openoffice.Office.Common/Undo
3. Open the attached file
4. Open the task manager and to monitor for memory usage 
5. Right click a sheet at the bottom and select "Select All"
6. Deselect 1 sheet with CTRL + click (on a sheet)
7. Select delete the selected sheets - Menu -> Sheet -> Delete sheet
8. The memory usage will increase from 483 mb with 250 mb to 733 mb at the start of the deletion process. Dropping slowly while deleting. After deleting all sheets except one around 250 mb is used. 

It's still using 200 mb after closing the calc document (returning to Start Center) where I would expect 110 mb or so (which is used after fresh start of Calc).
Comment 7 Buovjaga 2018-07-05 12:16:39 UTC
Still repro, will bibisect later.

Arch Linux 64-bit
Version: 6.2.0.0.alpha0+
Build ID: ea39c41fdf63191579d25f327db81db14862251c
CPU threads: 8; OS: Linux 4.17; UI render: default; VCL: gtk3; 
Locale: fi-FI (fi_FI.UTF-8); Calc: group threaded
Built on July 4th 2018
Comment 8 Telesto 2018-07-05 12:22:13 UTC
(In reply to Buovjaga from comment #7)
> Still repro, will bibisect later.
> 
Thanks for all the bibisecting lately :-). Especially for the hard & time consuming  ones.
Comment 9 Buovjaga 2018-07-05 18:35:57 UTC
Bibisected on Linux with 43all to range https://cgit.freedesktop.org/libreoffice/core/log/?qt=range&q=34847f1cf7538c333e9b8700eb4012ae358644a6...d4779887636fa9ab5b477f3436bcd3728a3e30ba

I noticed there were levels of worsening (I marked all as bad).
Comment 10 QA Administrators 2021-01-11 03:54:53 UTC Comment hidden (obsolete)
Comment 11 QA Administrators 2023-01-12 03:20:38 UTC
Dear Telesto,

To make sure we're focusing on the bugs that affect our users today, LibreOffice QA is asking bug reporters and confirmers to retest open, confirmed bugs which have not been touched for over a year.

There have been thousands of bug fixes and commits since anyone checked on this bug report. During that time, it's possible that the bug has been fixed, or the details of the problem have changed. We'd really appreciate your help in getting confirmation that the bug is still present.

If you have time, please do the following:

Test to see if the bug is still present with the latest version of LibreOffice from https://www.libreoffice.org/download/

If the bug is present, please leave a comment that includes the information from Help - About LibreOffice.
 
If the bug is NOT present, please set the bug's Status field to RESOLVED-WORKSFORME and leave a comment that includes the information from Help - About LibreOffice.

Please DO NOT

Update the version field
Reply via email (please reply directly on the bug tracker)
Set the bug's Status field to RESOLVED - FIXED (this status has a particular meaning that is not 
appropriate in this case)


If you want to do more to help you can test to see if your issue is a REGRESSION. To do so:
1. Download and install oldest version of LibreOffice (usually 3.3 unless your bug pertains to a feature added after 3.3) from https://downloadarchive.documentfoundation.org/libreoffice/old/

2. Test your bug
3. Leave a comment with your results.
4a. If the bug was present with 3.3 - set version to 'inherited from OOo';
4b. If the bug was not present in 3.3 - add 'regression' to keyword


Feel free to come ask questions or to say hello in our QA chat: https://web.libera.chat/?settings=#libreoffice-qa

Thank you for helping us make LibreOffice even better for everyone!

Warm Regards,
QA Team

MassPing-UntouchedBug