Bug 128873 - Calc - long time to open file in 6.4 beta
Summary: Calc - long time to open file in 6.4 beta
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
6.4.0.0.beta1+
Hardware: All All
: high major
Assignee: Serge Krot (CIB)
URL:
Whiteboard: target:7.0.0 target:6.4.2
Keywords: bibisected, bisected, perf, regression
Depends on:
Blocks: File-Opening regressions-printable-page-borders
  Show dependency treegraph
 
Reported: 2019-11-18 12:36 UTC by kabilo
Modified: 2020-02-13 19:35 UTC (History)
10 users (show)

See Also:
Crash report or crash signature:


Attachments
file attached (9.19 MB, application/vnd.oasis.opendocument.spreadsheet)
2019-11-18 12:37 UTC, kabilo
Details
perf flamegraph (237.07 KB, application/x-bzip)
2019-11-21 07:43 UTC, Julien Nabet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description kabilo 2019-11-18 12:36:49 UTC
Description:
long time to open file in 6.4 beta vs 6.3.3.2

Actual Results:
the time to open (render) the file is too long (10x vs 6.3.3.2 in a document with 600000 lines)

Expected Results:
optimize


Reproducible: Always


User Profile Reset: No



Additional Info:
Comment 1 kabilo 2019-11-18 12:37:55 UTC
Created attachment 155916 [details]
file attached
Comment 2 Xavier Van Wijmeersch 2019-11-18 12:53:12 UTC
It take's 4 min with 325000 rows and 10 column's to open

Version: 6.5.0.0.alpha0+
Build ID: 335852ffcf72e2ef6c4b02ebe645492bb7bc36d3
CPU threads: 8; OS: Linux 4.19; UI render: default; VCL: gtk3; 
Locale: nl-BE (en_US.UTF-8); UI-Language: en-US
Calc: threaded
Comment 3 BogdanB 2019-11-18 16:34:07 UTC
I have made some tests:

13 seconds with Version: 6.2.7.1
Build ID: 23edc44b61b830b7d749943e020e96f5a7df63bf
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ro-RO (ro_RO.UTF-8); UI-Language: en-US
Calc: threaded

15 seconds with Version: 6.3.3.2
Build ID: a64200df03143b798afd1ec74a12ab50359878ed
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ro-RO (ro_RO.UTF-8); UI-Language: en-US
Calc: threaded

28 seconds with Version: 6.4.0.0.beta1
Build ID: 4d7e5b0c40ed843384704eca3ce21981d4e98920
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ro-RO (ro_RO.UTF-8); UI-Language: en-US
Calc: threaded

In the last version (6.4) there was a mesage in the status bar below "Adapt row height" and this last for 8-10 seconds... Maybe this could be a clue...
Comment 4 Xisco Faulí 2019-11-20 17:24:23 UTC
it takes

real	1m5,716s
user	1m16,223s
sys	0m2,307s

in

Version: 6.5.0.0.alpha0+
Build ID: d468958331f36310d11265ba55d7c27366ab58ab
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded

while in

Version: 6.3.0.0.alpha1+
Build ID: c98b1f1cd43b3e109bcaf6324ef2d1f449b34099
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded

it takes

real	0m32,762s
user	0m43,440s
sys	0m2,784s

so it takes double the time, but 'long time' as the description claims is very inaccurate
Comment 5 Xisco Faulí 2019-11-20 22:42:34 UTC
Regression introduced by:

https://cgit.freedesktop.org/libreoffice/core/commit/?id=caeb7b141280a65e60525f11a7e6514b76e12e11

author	Ilhan Yesil <ilhanyesil@gmx.de>	2019-07-10 15:41:27 +0200
committer	Eike Rathke <erack@redhat.com>	2019-10-01 22:13:09 +0200
commit caeb7b141280a65e60525f11a7e6514b76e12e11 (patch)
tree 4f7b1f0db28fc7229b89a7f198eb87596135c4d8
parent 32f28dfa4c1de2b92664a5c0c3eca4fffecc0b28 (diff)
tdf#124983 In calc make printable page borders also initially visible

Bisected with: bibisect-linux64-6.4

Adding Cc: to Ilhan Yesil

for me it goes from 23 seconds to 65 seconds more or less
Comment 6 Xisco Faulí 2019-11-20 22:43:51 UTC
@Julien, would it be possible to have a perf chart for this issue ?
Comment 7 Julien Nabet 2019-11-21 07:43:17 UTC
Created attachment 155993 [details]
perf flamegraph

Here's a Flamegraph on pc Debian x86-64 with master sources updated today + enable-symbols.
Comment 8 Xisco Faulí 2019-11-21 12:00:05 UTC
@Noel, I thought you might be interested in this performance issue...
Comment 9 kabilo 2019-11-21 15:34:07 UTC
Hi,
opening the attached file takes:
35 sec in LO 6.3.3.1 (x64) OS: Windows 6.3  ID : f41f4c7f9507aeca13cb9df51f34d80e8ba30a99
95 sec in LO 6.4.0.0.beta1(x64) OS: Windows 6.3  ID : 4d7e5b0c40ed843384704eca3ce21981d4e98920

but openning file with 700000 rows takes:
98 sec in 6.3.3.1 (x64)
350 sec in 6.4.0.0.beta1 (x64)

Opening the same file with 700000 lines resaved in xlsx format takes:
only 30 sec in 6.3.3.1 (x64) (it is 3x faster than ods format)
350 sec in 6.4.0.0.beta1 (x64)

It is 'long time'
Comment 10 BogdanB 2019-11-21 16:09:15 UTC
If this bug comes from showing the page breaks... Isn't there a way to do this just for the visible part of the document, not for all document at once?

Or maybe for 1000 rows or something, and just after some move down in the cells to generate the page breaks for some more...
Comment 11 Xisco Faulí 2020-01-07 12:50:27 UTC Comment hidden (obsolete)
Comment 12 Xisco Faulí 2020-01-07 14:21:58 UTC
forget my previous comment. The import time is similar to comment 4, Something was wrong with the bisect repository. Got fixed after removing the user profile
Comment 13 Xisco Faulí 2020-01-07 14:23:47 UTC
@Miklos, Since you recently fixed bug 129552 which was introduced by the same commit as this issue, I thought you might be interested in this bug...
Comment 14 Miklos Vajna 2020-01-09 13:29:32 UTC
Sorry, no idea about this one.
Comment 15 Michael Weghorn 2020-01-13 10:46:43 UTC
Note: The discussion in the Gerrit change of the commit mentioned in comment 5 already talks about potential performance issues.
Comment 16 kabilo 2020-01-13 12:39:13 UTC
Hi all,
this bug can be closed. Calculating page breaks when opening a file (default for 6.4) is really a cause for the long time. Thanks to all.
Comment 17 Xisco Faulí 2020-01-13 12:41:45 UTC
(In reply to kabilo from comment #16)
> Hi all,
> this bug can be closed. Calculating page breaks when opening a file (default
> for 6.4) is really a cause for the long time. Thanks to all.

Not a reason for closing it either
Comment 18 Serge Krot (CIB) 2020-02-07 17:30:49 UTC
added patch to speed up switching in page layout in calc:
https://gerrit.libreoffice.org/c/core/+/88231
Comment 19 Commit Notification 2020-02-08 17:10:29 UTC
Serge Krot committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/153c4c7e6ab066c6b1c06704e08e5be815cfc024

tdf#128873 speed up switching into page layout

It will be available in 7.0.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 20 Xisco Faulí 2020-02-10 11:46:04 UTC
in

Version: 7.0.0.0.alpha0+
Build ID: 4188c7e2132ec3f1c3e879c179e92ff75c86d24f
CPU threads: 4; OS: Linux 4.19; UI render: default; VCL: gtk3; 
Locale: en-US (en_US.UTF-8); UI-Language: en-US
Calc: threaded

it takes

real	1m50,043s
user	2m31,119s
sys	0m10,331s

while before 153c4c7e6ab066c6b1c06704e08e5be815cfc024, it takes

real	4m20,368s
user	4m59,149s
sys	0m10,449s

measured using my poor old computer
Comment 21 Roman Kuznetsov 2020-02-10 13:19:07 UTC
(In reply to Xisco Faulí from comment #20)
> in
> 
> Version: 7.0.0.0.alpha0+
> Build ID: 4188c7e2132ec3f1c3e879c179e92ff75c86d24f
> CPU threads: 4; OS: Linux 4.19; UI render: default; VCL: gtk3; 
> Locale: en-US (en_US.UTF-8); UI-Language: en-US
> Calc: threaded
> 
> it takes
> 
> real	1m50,043s
> user	2m31,119s
> sys	0m10,331s
> 
> while before 153c4c7e6ab066c6b1c06704e08e5be815cfc024, it takes
> 
> real	4m20,368s
> user	4m59,149s
> sys	0m10,449s
> 
> measured using my poor old computer

It takes 1 minute for file from attach in

Версия: 7.0.0.0.alpha0+ (x64)
ID сборки: 41177730421f9be9ad955766a5a19667d8003b11
Потоков ЦП: 4; ОС: Windows 10.0 Build 18362; Отрисовка ИП: по умолчанию; VCL: win; 
Локаль: ru-RU (ru_RU); Язык интерфейса: ru-RU
Calc: threaded

build from 10.02.2020

CPU Intel Core i3-3120M (2 core with 2,5GHz), it isn't a newest CPU also ;-)
Comment 22 Commit Notification 2020-02-13 11:33:21 UTC
Serge Krot committed a patch related to this issue.
It has been pushed to "libreoffice-6-4":

https://git.libreoffice.org/core/commit/66c3b56a2a635aa2ae3779e8575db83400c119c4

tdf#128873 speed up switching into page layout

It will be available in 6.4.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 23 Xisco Faulí 2020-02-13 19:35:48 UTC
I believe we can close this issue as RESOLVED FIXED now. Besides, the problematic commit has been reverted in https://git.libreoffice.org/core/commit/c4281cb41e6b76cabd5fe42fc707877e864dfb82
Closing as RESOLVED FIXED