Bug 100894 - FILEOPEN FILESAVE IMPORT Conditional formatting: xls file with a lot of Conditional formatting freeze Calc when opening Styles sidebar
Summary: FILEOPEN FILESAVE IMPORT Conditional formatting: xls file with a lot of Condi...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
3.5.0 release
Hardware: All Linux (All)
: high major
Assignee: Not Assigned
URL:
Whiteboard: interoperability target:6.2.0 target:...
Keywords: filter:xls, haveBacktrace, notBibisectable, perf
Depends on:
Blocks: XLS-Conditional-Formatting
  Show dependency treegraph
 
Reported: 2016-07-13 12:43 UTC by Maxim Britov
Modified: 2024-10-17 09:48 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Example with a lot of conditional formatting (611.64 KB, application/zip)
2016-07-13 12:44 UTC, Maxim Britov
Details
Same example. Exported from Office 365 for Android. OpenXML. (644.51 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2018-06-18 14:51 UTC, Maxim Britov
Details
strace creenshot (132.28 KB, image/jpeg)
2024-10-07 12:52 UTC, Maxim Britov
Details
strace -ftT scalc qqq6.xls (5.38 MB, text/plain)
2024-10-07 15:52 UTC, Maxim Britov
Details
Perf flamegraph of file opening with GTK3 (1.29 MB, image/svg+xml)
2024-10-08 09:21 UTC, Buovjaga
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Maxim Britov 2016-07-13 12:43:40 UTC
There is file from our user created in Excel with a lot of Conditional formatting.

I can open it in LibreOffice 5.1.4 on Windows.
I can open it in Excel 2007 / 2010 / 2016.
I can't open it in LibreOffice 5.1.4 on my Linux x64 (freeze / 100% cpu)
I can't open it in LibreOffice 5.2RC2 on my Linux x64 (freeze / 100% cpu)

On Windows, where I can open it in LibreOffice I have freeze on: save as xls, save as ods, on F11 key.

File with be attached. I removed most data from file, but I can't track down to some minimal level for reproducing. Sorry.
Comment 1 Maxim Britov 2016-07-13 12:44:51 UTC
Created attachment 126197 [details]
Example with a lot of conditional formatting
Comment 2 Buovjaga 2016-07-18 04:07:46 UTC
I could open it.

64-bit, KDE Plasma 5
Build ID: 5.1.4.2 Arch Linux build-1
CPU Threads: 8; OS Version: Linux 4.6; UI Render: default; 
Locale: fi-FI (fi_FI.UTF-8)

Arch Linux 64-bit, KDE Plasma 5
Version: 5.3.0.0.alpha0+
Build ID: ab1b351840160655a9f0caedbb35e9fdf203c5a0
CPU Threads: 8; OS Version: Linux 4.6; UI Render: default; 
Locale: fi-FI (fi_FI.UTF-8); Calc: group
Built on July 16th 2016
Comment 3 Maxim Britov 2016-07-18 08:42:51 UTC
(In reply to Buovjaga from comment #2)
> I could open it.

How about F11 or File - Save?

I can open file too, on Gentoo Build.
But RPMs from DF - not.
Comment 4 Buovjaga 2016-07-18 10:32:41 UTC
Oh yeah, sorry didn't test F11 etc.
Now I can make it freeze with F11 on Windows.

Win 7 Pro 64-bit Version: 5.3.0.0.alpha0+
Build ID: 28ac6fdc11559b58ac62089300aa99530b0b822d
CPU Threads: 4; OS Version: Windows 6.1; UI Render: default; 
TinderBox: Win-x86@42, Branch:master, Time: 2016-07-18_02:54:20
Locale: fi-FI (fi_FI); Calc: CL
Comment 5 Justin L 2016-12-26 14:09:10 UTC
There is something fundamentally wrong with qqq6.xls.  It is 17MB in size.

I gave up trying to open it in bibisect43all oldest.  The progress bar moved slower and slower as it kept loading.

F11 pegged my CPU at 100% for 5 minutes before I killed LO5.4alpha on Ubuntu x64
Comment 6 QA Administrators 2018-03-25 02:29:29 UTC Comment hidden (obsolete)
Comment 7 Xavier Van Wijmeersch 2018-03-25 07:26:26 UTC
Can open file with master, but the conditional formatting in every sheet is a disaster. The hole thing is messed up, even deleting 6 sheets wont help.

Version: 6.1.0.0.alpha0+
Build ID: 1fbe46cf08f525e78016feef83f4c38b79b337ba
CPU threads: 8; OS: Linux 4.14; UI render: default; VCL: kde4; 
TinderBox: Linux-rpm_deb-x86_64@70-TDF, Branch:master, Time: 2018-03-24_23:04:38
Locale: nl-BE (en_US.UTF-8); Calc: group
Comment 8 Timur 2018-05-21 15:23:22 UTC
Could we make it a duplicate of Bug 61484?
Comment 9 Buovjaga 2018-05-21 17:00:22 UTC
(In reply to Timur from comment #8)
> Could we make it a duplicate of Bug 61484?

Dunno, Maxim's file opens pretty fast for me in master, but F11 still makes it freeze.

Arch Linux 64-bit
Version: 6.1.0.0.alpha1+
Build ID: 21b11273ae91f0cf7fd5f3f9fd2168e4349852c4
CPU threads: 8; OS: Linux 4.16; UI render: default; VCL: kde4; 
Locale: fi-FI (fi_FI.UTF-8); Calc: group
Built on May 11th 2018
Comment 10 Buovjaga 2018-05-21 17:10:06 UTC
(In reply to Timur from comment #8)
> Could we make it a duplicate of Bug 61484?

Just to confirm, this is different from bug 61484, because the xls in bug 61484 takes a long time for me to open on Linux.
Comment 11 Commit Notification 2018-06-14 07:13:13 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=d70f53a8269be31f0412926afa5ba826faed6633

tdf#100894 freeze when editing calc file with bazillions of cond formatting

It will be available in 6.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 How can I remove my account? 2018-06-14 10:06:27 UTC
See ea55492a6e55290d92a59324b3cb31ed958981ab which dedicates conditional formats in .ods import. Something similar needs to be done for .xls import, too, I guess.
Comment 13 How can I remove my account? 2018-06-14 10:12:24 UTC
s/dedicates/deduplicates. thanks, autocorrect.
Comment 14 Luke 2018-06-16 01:44:11 UTC
My 2 cents: This file is also broken under both Excel 2013 and 2016. It seems like it was possibly created programmatically or by someone intentionally trying to break Excel. This file uses overlapping conditional formats, a feature we don't properly support. See Bug 91654.

Wouldn't it make more sense to focus on improving interoperability with real world files, rather than optimizing a feature we don't even support properly?
Comment 15 Maxim Britov 2018-06-18 14:48:13 UTC
(In reply to Luke from comment #14)
> My 2 cents: This file is also broken under both Excel 2013 and 2016. It
> seems like it was possibly created programmatically or by someone
> intentionally trying to break Excel. This file uses overlapping conditional
> formats, a feature we don't properly support. See Bug 91654.
> 
> Wouldn't it make more sense to focus on improving interoperability with real
> world files, rather than optimizing a feature we don't even support properly?

Please read comments. Initial comment for example. File from real user, personal data dropped from tables. Excel 2003.
Converted and saved from Office 365 from my Android. I will attach it.
Comment 16 Maxim Britov 2018-06-18 14:51:52 UTC
Created attachment 142868 [details]
Same example. Exported from Office 365 for Android. OpenXML.
Comment 17 QA Administrators 2019-06-19 02:48:58 UTC Comment hidden (obsolete)
Comment 18 QA Administrators 2021-06-19 03:54:42 UTC Comment hidden (obsolete)
Comment 19 Katka 2021-07-26 14:57:52 UTC
Still repro with:
Version: 7.3.0.0.alpha0+ (x64) / LibreOffice Community
Build ID: cb2827f5f65324f309fa0e3c30d0b19ad237410e
CPU threads: 16; OS: Windows 10.0 Build 19043; UI render: Skia/Vulkan; VCL: win

Files could not open / not responding.
Comment 20 QA Administrators 2023-07-27 03:15:28 UTC Comment hidden (obsolete)
Comment 21 Roman Kuznetsov 2023-08-22 13:03:24 UTC
https://gerrit.libreoffice.org/c/core/+/155951
Comment 22 Commit Notification 2023-08-23 16:25:16 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/6fd755fb36472938757b2581cbe99f5e5fe1ae40

tdf#100894 lots of Conditional formatting freeze calc Styles sidebar

It will be available in 24.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 23 Roman Kuznetsov 2023-08-24 08:27:45 UTC
No freeze (but there is some delay) in

Version: 24.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: b5aaf194866c5e416167cb54d37f9f04dabc5375
CPU threads: 8; OS: Windows 10.0 Build 19043; UI render: Skia/Vulkan; VCL: win
Locale: ru-RU (ru_RU); UI: ru-RU
Calc: threaded

Thank you, Noel
Comment 24 Commit Notification 2023-09-01 10:33:44 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/ff66803ec9ec26c918ef652a1478ff9d7da0c97b

Revert "tdf#100894 lots of Conditional formatting freeze calc Styles sidebar"

It will be available in 24.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 25 Jiří Nováček 2024-09-29 20:00:37 UTC
It takes some time but it does not freeze
Version: 25.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 4cc532c0393914a743c11e806bd25dfa3d28fb5c
CPU threads: 4; OS: Windows 10 X86_64 (10.0 build 19045); UI render: Skia/Raster; VCL: win
Locale: cs-CZ (cs_CZ); UI: en-US
Calc: threaded

Also tested on MacOS with the same result
Comment 26 ady 2024-09-30 17:01:07 UTC
@Maxim, Ilmari, Katka,

WFM (with F11 too)?
Comment 27 Buovjaga 2024-09-30 17:11:35 UTC
(In reply to ady from comment #26)
> @Maxim, Ilmari, Katka,
> 
> WFM (with F11 too)?

Opening Styles panel takes a few secs, so matches what Roman said in comment 23 before the revert.

Version: 25.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 3d0058306385d0322c18d397fce1cd6e94daa1a3
CPU threads: 8; OS: Linux 6.10; UI render: default; VCL: kf6 (cairo+wayland)
Locale: fi-FI (fi_FI.UTF-8); UI: en-US
Calc: CL threaded
Comment 28 Maxim Britov 2024-10-07 12:51:32 UTC
For me, it still freezes to start.
On Windows it works.
On Linux it freezes.

1. Open Calc. Open side panel, open Styles.
2. Restart Calc. Check Sidepanel/Syles auto open
3. /path-lo-LO/program/scale qqq6.xls
3.1. I'm waiting from 2-3min (i5-7500) to 6-7min (i5-5300U).
3.2. At this moment I can edit data, but Menu still not working and 1 core CPU 100% load.
3.3. 2-4min and I can use Menu. For example, I can:  File - Exit.

Tested 24.2.6 / 28.4.2
My desktop i5-7500, my notebook i5-5300U.
Tested 24.2 on clean install Ubuntu 24.04 i5-4650
Comment 29 Maxim Britov 2024-10-07 12:52:14 UTC
Created attachment 196948 [details]
strace creenshot
Comment 30 Buovjaga 2024-10-07 13:02:50 UTC
(In reply to Maxim Britov from comment #28)
> For me, it still freezes to start.
> On Windows it works.
> On Linux it freezes.
> 
> 1. Open Calc. Open side panel, open Styles.
> 2. Restart Calc. Check Sidepanel/Syles auto open
> 3. /path-lo-LO/program/scale qqq6.xls
> 3.1. I'm waiting from 2-3min (i5-7500) to 6-7min (i5-5300U).
> 3.2. At this moment I can edit data, but Menu still not working and 1 core
> CPU 100% load.
> 3.3. 2-4min and I can use Menu. For example, I can:  File - Exit.
> 
> Tested 24.2.6 / 28.4.2
> My desktop i5-7500, my notebook i5-5300U.
> Tested 24.2 on clean install Ubuntu 24.04 i5-4650

But can you test with 25.2, please?
Comment 31 Maxim Britov 2024-10-07 15:51:42 UTC
(In reply to Buovjaga from comment #30)
> (In reply to Maxim Britov from comment #28)
> But can you test with 25.2, please?

Ubuntu 24.04 + LO 25.2a0
Still freezes. Look at strace -fTt log
Comment 32 Maxim Britov 2024-10-07 15:52:48 UTC
Created attachment 196951 [details]
strace -ftT scalc qqq6.xls
Comment 33 Buovjaga 2024-10-07 18:49:52 UTC
Tested again and it looks like the slowness is now only seen with gtk3 UI.

Maxim: if you want to work around this, you can launch from the command line with this for the moment:

SAL_USE_VCLPLUGIN=gen libreoffice
Comment 34 Buovjaga 2024-10-08 09:21:15 UTC
Created attachment 196959 [details]
Perf flamegraph of file opening with GTK3

I let qqq6.xls open for a few minutes and then canceled perf. It shows a gtk-specific thing taking a long time, gtk_tree_store_insert_with_values

Arch Linux 64-bit
Version: 25.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 40beeb144a00c9725cde4239c251f67c658d31a8
CPU threads: 8; OS: Linux 6.10; UI render: default; VCL: gtk3
Locale: fi-FI (fi_FI.UTF-8); UI: en-US
Calc: CL threaded
Built on 6 October 2024
Comment 35 Commit Notification 2024-10-14 19:52:06 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/8af3358203e71564b69ae4a687989a57a2a8605e

tdf#100894 reduce time spent iterating styles

It will be available in 25.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 36 Commit Notification 2024-10-16 05:39:46 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/7f42f031ed1b9f374f3fcba9c814dc5862492026

tdf#100894 speed up style tree creation

It will be available in 25.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.