Bug 102263 - Spreadsheet grinds to halt with 150*100 matrix & simple formulas
Summary: Spreadsheet grinds to halt with 150*100 matrix & simple formulas
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
5.0.5.2 release
Hardware: Other All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: filter:xls, haveBacktrace, perf
Depends on:
Blocks: Conditional-Formatting XLS
  Show dependency treegraph
 
Reported: 2016-09-18 20:00 UTC by Raymond Peterson
Modified: 2018-09-24 13:10 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments
slow big file (8.14 MB, application/vnd.ms-excel)
2016-09-18 23:18 UTC, Raymond Peterson
Details
nice fast file (2.67 MB, application/vnd.ms-excel)
2016-09-18 23:19 UTC, Raymond Peterson
Details
Callgrind output from master (5.85 MB, application/x-xz)
2018-09-24 13:10 UTC, Buovjaga
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Raymond Peterson 2016-09-18 20:00:27 UTC
User-Agent:       Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36
Build Identifier:  5.1.5.2

1. Last week made 5 1-sheet spreadsheets. Variations on theme. Data ~ 200 rows x 100 columns.

2. On Friday & Saturday built out 3 of these into 6-sheet spreadsheets. Each sheet has about same 200 rows x 100 columns using simple formulas. Zip, zip. Copy, paste, formulas, formatting. LibreOffice process instantaneously.

3. Sunday AM start last 2 spreadsheets. Same formulas, operations, volume of data, etc. Files balloon from 2MB to 10MB. 300MB memory. Operations take 10 mins to complete.

4. First 3 spreadsheets still fine. Operations go zip zip.

4. Could be Windows 10 x64 update? Update from LibreOffice 5.1 to 5.2?

5. Totally baffling. Cost me 8 hours to do 2 hours work.

I can provide file

Reproducible: Always







Reset User Profile?No
Comment 1 Aron Budea 2016-09-18 20:42:47 UTC
Yes, if you can share the file(s) publicly, attaching it would be great.
Which exact version of LibreOffice are you using?
Comment 2 Raymond Peterson 2016-09-18 23:18:17 UTC
Created attachment 127419 [details]
slow big file
Comment 3 Raymond Peterson 2016-09-18 23:19:51 UTC
Created attachment 127420 [details]
nice fast file
Comment 4 Aron Budea 2016-09-19 05:58:34 UTC
Confirmed, "slow big file" loads ~45s in v5.2.1.2  / Windows 7. "nice fast file" loads in 2s.

It must be something with Styles / Conditional Formatting. When I click select View -> Styles and Formatting, LibreOffice hangs hogging a core.

Raymond, can you check what the difference is between the two files that is related to styles and formatting?


I made a 100s limited performance measurement with the slow file, two functions stand out:
sc\source\core\data\patattr.cxx:EqualPatternSets(...) 49% / 45%
sc\source\core\data\table1.cxx:AddCondFormat(...) 14% / 14%

There's been a performance fix in AOO for EqualPatternSets(...), but the import of that change was rejected:
https://cgit.freedesktop.org/libreoffice/core/commit/?h=aoo/trunk&id=b39f169896005a4ab3f2a0cfdb116de947dda207
(it claims ~10% improvement for their test file, anyway, not very significant)
Comment 5 Raymond Peterson 2016-09-19 11:07:22 UTC
Aron,

Yup. That was it.

Had created 'Bold Red Font' & applied conditionally to 'matrix' sheet. No recalculation on this sheet.

For fast file, created new downstream matrix sheets. Did not create conditional. These matrices get recalculated.

For slow file, copied matrix sheet. Which brought along conditional formatting.

'Clear direct formatting' of all matrices solved speed & file size issue. Slow files now fast. And size ~2.5MB. Same as others.

Thank you!

Regards,

Raymond
Comment 6 Xisco Faulí 2017-09-29 08:48:59 UTC Comment hidden (obsolete)
Comment 7 Aron Budea 2018-07-04 02:57:24 UTC
Takes about 25s in a current 6.2 master build / Windows 7.
Not quite there yet, but slowly improving.
Comment 8 Buovjaga 2018-09-24 13:10:02 UTC
Created attachment 145138 [details]
Callgrind output from master

Opening attachment 127419 [details]

Arch Linux 64-bit
Version: 6.2.0.0.alpha0+
Build ID: 8b1501d80dc9d3f42c351c6e026fa737e116cae5
CPU threads: 8; OS: Linux 4.18; UI render: default; VCL: gtk3_kde5; 
Locale: fi-FI (fi_FI.UTF-8); Calc: threaded
Built on 23 September 2018