Bug 161900 - Calc row manipulation slows down dramatically after saving document
Summary: Calc row manipulation slows down dramatically after saving document
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
7.4.0.3 release
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: bibisected, haveBacktrace, perf
Depends on:
Blocks:
 
Reported: 2024-07-05 07:58 UTC by KosulinLeonidas
Modified: 2024-11-01 08:32 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
Demo file (77.80 KB, application/vnd.oasis.opendocument.spreadsheet)
2024-07-05 07:59 UTC, KosulinLeonidas
Details
Perf flamegraph of DeletionTest run before saving (191.62 KB, image/svg+xml)
2024-10-31 15:40 UTC, Buovjaga
Details
Perf flamegraph of DeletionTest run after saving (209.84 KB, image/svg+xml)
2024-10-31 15:40 UTC, Buovjaga
Details

Note You need to log in before you can comment on or make changes to this bug.
Description KosulinLeonidas 2024-07-05 07:58:34 UTC
Description:
If a document is on the larger side and takes a while to save, then saving dramatically slows down the insertion and deletion of rows (we are talking about a hundred times slower on my main doc, and I couldn't find a way to speed it up again without restarting the program)

In this example, I opened a new calc doc, wrote random data in the first 26x26 range using a macro, put background colors and borders on random cells and copied that sheet 19 times for a total of 20 sheets.

The deletion and insertion remained quick, however, if I save the document (even without changing any data at all) both the deletion and insertion becomes awfully slow. 

Steps to Reproduce:
1. Open Document
2. Run the deletion or insertion functions
3. Note the speed (should be fast)
4. Save Document (you can even undo the changes before saving)
5. Run the deletion or insertion functions again
6. Note the difference in speed

Actual Results:
Both Insertion and Deletion of rows becomes slow

Expected Results:
Insertion and Deletion should have kept their original speed


Reproducible: Always


User Profile Reset: Yes

Additional Info:
Version: 24.2.4.2 (X86_64) / LibreOffice Community
Build ID: 51a6219feb6075d9a4c46691dcfe0cd9c4fff3c2
CPU threads: 16; OS: Windows 10.0 Build 19045; UI render: Skia/Vulkan; VCL: win
Locale: de-DE (de_DE); UI: en-US
Calc: CL threaded
Comment 1 KosulinLeonidas 2024-07-05 07:59:25 UTC
Created attachment 195114 [details]
Demo file
Comment 2 Telesto 2024-07-06 01:55:02 UTC Comment hidden (obsolete)
Comment 3 ady 2024-07-06 02:53:56 UTC Comment hidden (obsolete)
Comment 4 Telesto 2024-07-06 06:52:54 UTC Comment hidden (obsolete)
Comment 5 Buovjaga 2024-08-23 05:19:08 UTC
No problem for me. Any change if you deactivate Tools - Options - LibreOffice - View - Use Skia for all rendering?

Version: 24.8.0.3 (X86_64) / LibreOffice Community
Build ID: 0bdf1299c94fe897b119f97f3c613e9dca6be583
CPU threads: 2; OS: Windows 11 X86_64 (10.0 build 22621); UI render: Skia/Raster; VCL: win
Locale: en-US (en_US); UI: en-US
Calc: threaded
Comment 6 KosulinLeonidas 2024-08-23 07:25:21 UTC
(In reply to Buovjaga from comment #5)
> No problem for me. Any change if you deactivate Tools - Options -
> LibreOffice - View - Use Skia for all rendering?
> 
> Version: 24.8.0.3 (X86_64) / LibreOffice Community
> Build ID: 0bdf1299c94fe897b119f97f3c613e9dca6be583
> CPU threads: 2; OS: Windows 11 X86_64 (10.0 build 22621); UI render:
> Skia/Raster; VCL: win
> Locale: en-US (en_US); UI: en-US
> Calc: threaded

I deactivated Skia rendering and the bug persisted. I also tested it on a different windows pc and was able to reproduce it.

How many ticks did it take for you to run the subroutines before and after? For me, it takes around 10 ticks pre-saving and around 200 after.
Comment 7 Buovjaga 2024-08-23 07:34:23 UTC
(In reply to KosulinLeonidas from comment #6)
> (In reply to Buovjaga from comment #5)
> > No problem for me. Any change if you deactivate Tools - Options -
> > LibreOffice - View - Use Skia for all rendering?
> > 
> > Version: 24.8.0.3 (X86_64) / LibreOffice Community
> > Build ID: 0bdf1299c94fe897b119f97f3c613e9dca6be583
> > CPU threads: 2; OS: Windows 11 X86_64 (10.0 build 22621); UI render:
> > Skia/Raster; VCL: win
> > Locale: en-US (en_US); UI: en-US
> > Calc: threaded
> 
> I deactivated Skia rendering and the bug persisted. I also tested it on a
> different windows pc and was able to reproduce it.
> 
> How many ticks did it take for you to run the subroutines before and after?
> For me, it takes around 10 ticks pre-saving and around 200 after.

What do you mean by "ticks" and "subroutines"?
Comment 8 KosulinLeonidas 2024-08-23 07:40:03 UTC
(In reply to Buovjaga from comment #7)
> (In reply to KosulinLeonidas from comment #6)
> > (In reply to Buovjaga from comment #5)
> > > No problem for me. Any change if you deactivate Tools - Options -
> > > LibreOffice - View - Use Skia for all rendering?
> > > 
> > > Version: 24.8.0.3 (X86_64) / LibreOffice Community
> > > Build ID: 0bdf1299c94fe897b119f97f3c613e9dca6be583
> > > CPU threads: 2; OS: Windows 11 X86_64 (10.0 build 22621); UI render:
> > > Skia/Raster; VCL: win
> > > Locale: en-US (en_US); UI: en-US
> > > Calc: threaded
> > 
> > I deactivated Skia rendering and the bug persisted. I also tested it on a
> > different windows pc and was able to reproduce it.
> > 
> > How many ticks did it take for you to run the subroutines before and after?
> > For me, it takes around 10 ticks pre-saving and around 200 after.
> 
> What do you mean by "ticks" and "subroutines"?

1. Open the document
2. Go to tools-> Macros-> Edit Macros
3. Select either the "DeletionTest" or the "InsertionTest" Subroutines (DeletionTimeDemo.ods -> Standard -> Module1)
4. Run it
5. Note the "before" Result (A msgbox is supposed to pop up that tells you how many ticks it took for the subroutine/function to finish running)

6.save the document (Ctrl+S)
7. Run the Subroutines again
8. Note the "after" Result
Comment 9 Buovjaga 2024-08-23 09:15:48 UTC
Thanks, with the clearer steps I was able to reproduce. I had thought manual insertion and deletion would be enough.

Bibisected changes with linux-64-7.4. There was about a 10x increase in the ticks with 4c5f8ccf0a2320432b8fe91add1dcadf54d9fd58, which changed the default number of Calc columns to 16384

The current state was introduced with 2e86718626a07e1656661df3ad69a64848bf4614
don't allocate unnecessary columns when inserting a row

I will not call this a regression as the change is related to the bigger number of supported columns.
Comment 10 Noel Grandin 2024-10-30 14:37:30 UTC
What does "slow" mean here? 
On current master, these tests are reporting around 30ms for me.
(on my admittedly fairly fast machine.)
Comment 11 KosulinLeonidas 2024-10-30 15:14:37 UTC
(In reply to Noel Grandin from comment #10)
> What does "slow" mean here? 
> On current master, these tests are reporting around 30ms for me.
> (on my admittedly fairly fast machine.)


On my machine it is approx 8ms before saving and 225ms after

Can you try to open the document and run the macro before saving?
Then save, and run it again.
I assume your 30ms is the "after" value, so when running the test without saving, it will probably be at approx 3ms.

You have to resist the urge to press Ctrl+S after opening the document, and only do so after you ran the function once for the faster time. Saving it, even once seems to break something in the document, and the only way to fix it is to close it and open it up again.
Comment 12 Noel Grandin 2024-10-31 06:42:18 UTC
I think something must have improved recently on master, because I am getting very similar times before and after saving.
Comment 13 Buovjaga 2024-10-31 08:47:50 UTC
For DeletionTest I now get 6 ms before and 93 ms after.
For InsertionTest 8 ms before and 80 ms after.

In 7.4, DeletionTest takes 9 ms before and 110 ms after. The numbers do fluctuate, might be even 23 ms before. So it doesn't look like there was any change from 7.4

Arch Linux 64-bit
Version: 25.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 0e05add86cfb996fe6a11c088c36b3fa010e2445
CPU threads: 8; OS: Linux 6.10; UI render: default; VCL: kf6 (cairo+wayland)
Locale: fi-FI (fi_FI.UTF-8); UI: en-US
Calc: CL threaded
Built on 31 October 2024
Comment 14 Noel Grandin 2024-10-31 11:26:14 UTC
(In reply to Buovjaga from comment #13)
> For DeletionTest I now get 6 ms before and 93 ms after.


Very weird. I get 37ms before and 59ms after
Comment 15 Buovjaga 2024-10-31 15:40:22 UTC
Created attachment 197318 [details]
Perf flamegraph of DeletionTest run before saving
Comment 16 Buovjaga 2024-10-31 15:40:51 UTC
Created attachment 197319 [details]
Perf flamegraph of DeletionTest run after saving
Comment 17 Telesto 2024-11-01 08:32:53 UTC
Quite similar too bug 125619, I think. Except for the bibisected point of introduction at comment 9