Bug 102263 - Spreadsheet grinds to halt with 150*100 matrix & simple formulas
Summary: Spreadsheet grinds to halt with 150*100 matrix & simple formulas
Status: RESOLVED WORKSFORME
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
5.0.5.2 release
Hardware: Other All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: filter:xls, haveBacktrace, perf
Depends on:
Blocks: Conditional-Formatting XLS
  Show dependency tree / graph
 
Reported: 2016-09-18 20:00 UTC by Raymond Peterson
Modified: 2022-06-19 21:58 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:

Attachments
slow big file (8.14 MB, application/vnd.ms-excel)
2016-09-18 23:18 UTC, Raymond Peterson 		Details
nice fast file (2.67 MB, application/vnd.ms-excel)
2016-09-18 23:19 UTC, Raymond Peterson 		Details
Callgrind output from master (5.85 MB, application/x-xz)
2018-09-24 13:10 UTC, Buovjaga 		Details
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Raymond Peterson 2016-09-18 20:00:27 UTC 
User-Agent:       Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36
Build Identifier:  5.1.5.2

1. Last week made 5 1-sheet spreadsheets. Variations on theme. Data ~ 200 rows x 100 columns.

2. On Friday & Saturday built out 3 of these into 6-sheet spreadsheets. Each sheet has about same 200 rows x 100 columns using simple formulas. Zip, zip. Copy, paste, formulas, formatting. LibreOffice process instantaneously.

3. Sunday AM start last 2 spreadsheets. Same formulas, operations, volume of data, etc. Files balloon from 2MB to 10MB. 300MB memory. Operations take 10 mins to complete.

4. First 3 spreadsheets still fine. Operations go zip zip.

4. Could be Windows 10 x64 update? Update from LibreOffice 5.1 to 5.2?

5. Totally baffling. Cost me 8 hours to do 2 hours work.

I can provide file

Reproducible: Always







Reset User Profile?No
Comment 1 Aron Budea 2016-09-18 20:42:47 UTC 
Yes, if you can share the file(s) publicly, attaching it would be great.
Which exact version of LibreOffice are you using?
Comment 2 Raymond Peterson 2016-09-18 23:18:17 UTC 
Created attachment 127419 [details]
slow big file
Comment 3 Raymond Peterson 2016-09-18 23:19:51 UTC 
Created attachment 127420 [details]
nice fast file
Comment 4 Aron Budea 2016-09-19 05:58:34 UTC 
Confirmed, "slow big file" loads ~45s in v5.2.1.2  / Windows 7. "nice fast file" loads in 2s.

It must be something with Styles / Conditional Formatting. When I click select View -> Styles and Formatting, LibreOffice hangs hogging a core.

Raymond, can you check what the difference is between the two files that is related to styles and formatting?


I made a 100s limited performance measurement with the slow file, two functions stand out:
sc\source\core\data\patattr.cxx:EqualPatternSets(...) 49% / 45%
sc\source\core\data\table1.cxx:AddCondFormat(...) 14% / 14%

There's been a performance fix in AOO for EqualPatternSets(...), but the import of that change was rejected:
https://cgit.freedesktop.org/libreoffice/core/commit/?h=aoo/trunk&id=b39f169896005a4ab3f2a0cfdb116de947dda207
(it claims ~10% improvement for their test file, anyway, not very significant)
Comment 5 Raymond Peterson 2016-09-19 11:07:22 UTC 
Aron,

Yup. That was it.

Had created 'Bold Red Font' & applied conditionally to 'matrix' sheet. No recalculation on this sheet.

For fast file, created new downstream matrix sheets. Did not create conditional. These matrices get recalculated.

For slow file, copied matrix sheet. Which brought along conditional formatting.

'Clear direct formatting' of all matrices solved speed & file size issue. Slow files now fast. And size ~2.5MB. Same as others.

Thank you!

Regards,

Raymond
Comment 6 Xisco Faulí 2017-09-29 08:48:59 UTC Comment hidden (obsolete)
Comment 7 Aron Budea 2018-07-04 02:57:24 UTC 
Takes about 25s in a current 6.2 master build / Windows 7.
Not quite there yet, but slowly improving.
Comment 8 Buovjaga 2018-09-24 13:10:02 UTC 
Created attachment 145138 [details]
Callgrind output from master

Opening attachment 127419 [details]

Arch Linux 64-bit
Version: 6.2.0.0.alpha0+
Build ID: 8b1501d80dc9d3f42c351c6e026fa737e116cae5
CPU threads: 8; OS: Linux 4.18; UI render: default; VCL: gtk3_kde5; 
Locale: fi-FI (fi_FI.UTF-8); Calc: threaded
Built on 23 September 2018
Comment 9 QA Administrators 2019-09-25 03:00:38 UTC Comment hidden (obsolete)
Comment 10 Buovjaga 2021-05-03 10:14:24 UTC 
attachment 127419 [details] loads in 19 secs

Version: 7.2.0.0.alpha0+ (x64) / LibreOffice Community
Build ID: 9c930c4f3109d123c0831d0fcecf9c8b32e5bbc7
CPU threads: 2; OS: Windows 10.0 Build 19042; UI render: default; VCL: win
Locale: fi-FI (fi_FI); UI: en-US
Calc: threaded
Comment 11 Roman Kuznetsov 2021-12-05 21:20:16 UTC 
LO opens Slow big file for 38 sec in

Version: 7.4.0.0.alpha0+ (x64) / LibreOffice Community
Build ID: 2934472ab888ebfe64a153984af2902fac63a7a0
CPU threads: 4; OS: Windows 6.1 Service Pack 1 Build 7601; UI render: default; VCL: win
Locale: ru-RU (ru_RU); UI: en-US
Calc: CL
Comment 12 Roman Kuznetsov 2022-06-19 21:58:40 UTC 
LO opens the "slow big file" for 4 sec in

Version: 7.5.0.0.alpha0+ / LibreOffice Community
Build ID: e4d23c27288b99c3ed3cfa332ff308b31c01f97d
CPU threads: 4; OS: Linux 5.14; UI render: default; VCL: gtk3
Locale: ru-RU (ru_RU.UTF-8); UI: en-US
Calc: threaded Jumbo

Let's close this as WORKSFORME. Of course you feel free to reopen it if you'll get worse result