Download it now!
Bug 119650 - FILESAVE: It takes >2:40 to save a small spreadsheet with comments and more than 400Mb in extra memory usage (because laying out the text for each caption 3 times)
Summary: FILESAVE: It takes >2:40 to save a small spreadsheet with comments and more t...
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.1 all versions
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:6.3.0
Keywords: haveBacktrace, perf
: 131675 (view as bug list)
Depends on:
Blocks: Calc-Comments Memory
  Show dependency treegraph
 
Reported: 2018-09-02 22:11 UTC by john.kirk
Modified: 2020-04-26 21:56 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
very long saving (614.14 KB, application/vnd.oasis.opendocument.spreadsheet)
2018-09-02 22:11 UTC, john.kirk
Details
Perf flamegraph (5.33 MB, image/svg+xml)
2019-04-18 19:41 UTC, Buovjaga
Details

Note You need to log in before you can comment on or make changes to this bug.
Description john.kirk 2018-09-02 22:11:08 UTC
Created attachment 144617 [details]
very long saving

Hello,

It takes >2:40 to save a small spreadsheet with comments and >400Mb in extra memory usage after saving on linux with GTK2, GTK3 VCL.

Tested on Debian 8-9, Ubuntu 18.04, Linux Mint 19, Fedora 28.   

Best wishes,
John
Comment 1 john.kirk 2018-09-02 22:13:36 UTC
Version: 6.1.0.3
Build ID: efb621ed25068d70781dc026f7e9c5187a4decd1
CPU threads: 16; OS: Linux 4.15; UI render: default; VCL: gtk2; 
Locale: en-US (en_US.UTF-8); Calc: group threaded 

Best wishes, 
John
Comment 2 Xisco Faulí 2018-09-03 09:46:52 UTC Comment hidden (obsolete)
Comment 3 Xisco Faulí 2018-09-03 09:57:52 UTC
In

Version: 5.2.0.0.alpha0+
Build ID: 3ca42d8d51174010d5e8a32b96e9b4c0b3730a53
Threads 4; Ver: 4.10; Render: default; 

it takes

real	7m15.914s
user	7m13.094s
sys	0m2.573s

using 'time instdir/program/soffice --headless --convert-to ods /home/xisco/Baixades/bug\(1\).ods --outdir /home/xisco/Escriptori/'
Comment 4 Xisco Faulí 2018-09-03 10:03:46 UTC
(In reply to Xisco Faulí from comment #2)
> Reproduced in
> 
> Version: 6.2.0.0.alpha0+
> Build ID: 4b5fcd417587cfb9e6d8b61ecb037ab165eeb5b9
> CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
> Locale: ca-ES (ca_ES.UTF-8); Calc: threaded

In this version it takes

real	5m33.235s
user	5m31.907s
sys	0m1.567s
Comment 5 Xisco Faulí 2018-09-03 10:12:19 UTC
it takes

real	7m10.271s
user	7m8.012s
sys	0m1.191s


in

Version 4.1.0.0.alpha0+ (Build ID: efca6f15609322f62a35619619a6d5fe5c9bd5a)
Comment 6 john.kirk 2018-10-15 13:53:38 UTC
This bug is also related to the bug 119075 (the same attachment).

Best wishes,
John
Comment 7 Buovjaga 2019-04-18 19:41:52 UTC
Created attachment 150861 [details]
Perf flamegraph

Arch Linux 64-bit
Version: 6.3.0.0.alpha0+
Build ID: 1fee3f1da6291bfbcd75455512726215d41b3e83
CPU threads: 8; OS: Linux 5.0; UI render: default; VCL: gtk3; 
Locale: fi-FI (fi_FI.UTF-8); UI-Language: en-US
Calc: threaded
Built on 18 April 2019
Comment 8 Xisco Faulí 2019-04-23 21:24:48 UTC
it takes

real	4m21,024s
user	4m20,009s
sys	0m1,322s

in

Version: 6.3.0.0.alpha0+
Build ID: 0a04150b6eefb5feb7ecefaa5cd63dbac8c1574f
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded

vcl backend is irrelevant here. it can be reproduced in headless mode

@Noel, I thought you might be interested in this issue...
Comment 9 Commit Notification 2019-04-26 13:09:53 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/f4ea84ff370d33a02a8fb1d6405b9d964491258e%5E%21

tdf#119650 slow saving spreadsheet with comments, part 1

It will be available in 6.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 10 Commit Notification 2019-04-26 13:12:34 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/b4a1b89cc84086dfd6f471d7f23fecf0ec8f3331%5E%21

tdf#119650 slow saving spreadsheet with comments, part 2

It will be available in 6.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 11 Noel Grandin 2019-04-26 13:14:17 UTC
I have done as much as I can here, right now. 

There are two paths forward, for anyone else who wants to work on this:

(A) I can see that we are still doing at least 3 times as much work as we should be, constructing EditEngine stuff - we are laying out the text for each caption 3 times.

(B) The better approach would be to stop the code from constructing and laying out every single caption, which is not necessary, but instead use the ScCaptionInitData for exporting. However, there is some weird interaction that I could not find the source off, where someone constructing the ScCaption internal data registers the caption with something, so that if I disable the "construct every single caption on export" logic, we end up losing all the captions.
Comment 12 Xisco Faulí 2019-04-27 14:15:58 UTC
it takes

real	2m6,394s
user	2m5,190s
sys	0m1,572s

in

Version: 6.3.0.0.alpha0+
Build ID: 3ab6d246cc44617af5ed416b5d49f2f35b61ceea
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded

which is half the time my previous measurement. Nice!!
Comment 13 john.kirk 2019-08-08 15:35:45 UTC
Hello, 

It takes ~1:30 to save and ~4 sec to change sheets after saving.

Version: 6.3.0.4
Build ID: 057fc023c990d676a43019934386b85b21a9ee99
CPU threads: 16; OS: Linux 4.19; UI render: default; VCL: gtk3; 
Locale: en-US (en_US.UTF-8); UI-Language: en-US
Calc: threaded

Best wishes,
John
Comment 14 Telesto 2020-01-12 17:30:17 UTC
@Xisco
Should we mark this a fixed and move Comment 11 into a new bug
Comment 15 Timur 2020-01-13 16:15:13 UTC
My view is that by doing so we just get more bugs and phony stats, but loose all the info from this bug.
Comment 16 Telesto 2020-04-26 21:56:22 UTC
*** Bug 131675 has been marked as a duplicate of this bug. ***