Bug 131595 - Very slow switching between sheets in the attached xlsx document
Summary: Very slow switching between sheets in the attached xlsx document
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
6.3.5.2 release
Hardware: All Linux (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:25.8.0
Keywords: perf
Depends on:
Blocks: Performance
  Show dependency treegraph
 
Reported: 2020-03-26 10:18 UTC by Kevin Suo
Modified: 2025-03-17 09:21 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Test xlsx file (480.36 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2020-03-26 10:18 UTC, Kevin Suo
Details
perf flamegraph (when loading the file) (254.55 KB, application/x-bzip)
2020-03-28 11:51 UTC, Julien Nabet
Details
Flamegraph (when switching sheet) (211.33 KB, application/x-bzip)
2020-03-28 12:45 UTC, Julien Nabet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Kevin Suo 2020-03-26 10:18:17 UTC
Created attachment 159039 [details]
Test xlsx file

Steps to reproduce:

1. Open the attached xlsx document.

2. Try to click to second sheet tab (to switch to that tab).
--> Very slow switching to the 2nd tab. After successful switch, the soffice.bin process eats 3.6 GB of my RAM, while my system RAM is 8 GB.

3. After successfully switched to the second tab, try to click some cells.
--> Very slow navigating between between cells.

Reproduced in
Version: 7.0.0.0.alpha0+
Build ID: f6f37e3341f2a3ad292af1a923dbc33d81e439c9
CPU threads: 4; OS: Linux 5.5; UI render: default; VCL: gtk3; 
Locale: zh-CN (zh_CN.UTF-8); UI-Language: en-US
Calc: threaded

And in 6.3.5.2
Fedora 31 64.

Step 2 and 3 is fast in MSO running in my VirtalBox WinXP guest machine with only 1 GB total RAM.
Comment 1 Kevin Suo 2020-03-26 10:22:24 UTC
This issue was originally reported on the LibreOffice Chinese discussion forum:
https://bbs.libreofficechina.org/forum.php?mod=redirect&tid=2421
Comment 2 m_a_riosv 2020-03-26 20:17:57 UTC
Opening the file there is a message that it exceeds the columns number limits.

And file crash LibreOffice when I try to go to the second sheet.
Comment 3 Julien Nabet 2020-03-28 10:56:52 UTC
Trying loading the document is already quite slow, more than (for a 480KB file on pc with Ryzen 2600 + 32GB)
real	0m16,036s
user	0m15,769s
sys	0m1,313s

+ I got this warning:
Warning loading document ...:
The data could not be loaded completely because the maximum number of columns per sheet was exceeded.
Comment 4 Julien Nabet 2020-03-28 11:51:31 UTC
Created attachment 159095 [details]
perf flamegraph (when loading the file)

Here's a Flamegraph retrieved on pc Debian x86-64 with master sources updated today.
It corresponds to loading only.

I'll attach another Flamegraph about the switching part.
Comment 5 Julien Nabet 2020-03-28 12:45:49 UTC
Created attachment 159096 [details]
Flamegraph (when switching sheet)

I must say that I use gtk3 so with all accessibility mechanism which seems to slow down things.
Comment 6 Buovjaga 2021-05-03 12:21:59 UTC
Tab switching is not very slow for me and memory use does not explode.

Kevin: please re-test

Version: 7.2.0.0.alpha0+ (x64) / LibreOffice Community
Build ID: 9c930c4f3109d123c0831d0fcecf9c8b32e5bbc7
CPU threads: 2; OS: Windows 10.0 Build 19042; UI render: default; VCL: win
Locale: fi-FI (fi_FI); UI: en-US
Calc: threaded
Comment 7 Kevin Suo 2022-06-28 12:06:07 UTC
(In reply to Buovjaga from comment #6)
Switching to the 2nd tab is still slow for me in
Version: 7.3.4.2 / LibreOffice Community
Build ID: 30(Build:2)
CPU threads: 8; OS: Linux 5.15; UI render: default; VCL: gtk3
Locale: zh-CN (zh_CN.UTF-8); UI: zh-CN
Ubuntu package version: 1:7.3.4-0ubuntu0.22.04.1
Calc: threaded

and

Version: 7.4.0.0.beta1+ / LibreOffice Community
Build ID: 6ea86593851c731e2e2b969e458255e4159c80d9
CPU threads: 8; OS: Linux 5.15; UI render: default; VCL: gtk3
Locale: zh-CN (zh_CN.UTF-8); UI: zh-CN
Calc: threaded
Comment 8 QA Administrators 2024-12-29 03:13:37 UTC Comment hidden (obsolete)
Comment 9 Commit Notification 2025-03-14 11:27:31 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/ab4cf371eda16668b875dbd99950034b09d0dc4c

use mutex before touching maBuffered2DDecomposition (tdf#131595 related)

It will be available in 25.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 10 Commit Notification 2025-03-14 11:47:35 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/303e5631f8119ef69c8ff515e7e47d4cf6ab74c6

use mutex before touching maBuffered2DDecomposition (tdf#131595 related)

It will be available in 25.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 11 Commit Notification 2025-03-14 13:04:48 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/e8e1c9cbd58c123cf37681ccff1c417e537a9272

fix deadlock in SystemDependentDataBuffer::flushAll (tdf#131595 related)

It will be available in 25.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Commit Notification 2025-03-17 06:16:12 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/8a17b7f0a679ebf21bcfb425186b205d996d129b

tdf#131595 Improve drawinglayer flushing mechanism.

It will be available in 25.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 13 Buovjaga 2025-03-17 09:21:51 UTC
There's a slight delay when the switching involves the second tab, but I guess that's in line with commit 8a17b7f0a679ebf21bcfb425186b205d996d129b

Quoting a comment:

This is a "garbage collection" approach to flushing.

We store entries in a set. Every 2 seconds, we scan the set for entries that have not
been used for 10 seconds or more, and if so, we flush the buffer primitives in those entries.

This mechanism is __deliberately__ not perfect.
Sometimes things will be flushed a little too soon, sometimes things will wait a little too long,
since we only have a granularity of 2 seconds.
But what is gains from not being perfect, is scalability.

It is very simple, scales to lots and lots of primitives without needing lots of timers, and performs
very little work in the common case.