Bug 90129 - FILEOPEN Large (xlsx) file with pivot tables takes ages to open
Summary: FILEOPEN Large (xlsx) file with pivot tables takes ages to open
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
(earliest affected) release
Hardware: Other All
: high major
Assignee: Not Assigned
Keywords: perf
Depends on:
Reported: 2015-03-20 16:43 UTC by Katarina Behrens
Modified: 2016-10-21 10:59 UTC (History)
7 users (show)

See Also:
Crash report or crash signature:
Regression By:

same sample, but without external links (2.93 MB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2015-12-24 12:33 UTC, Timur
callgrind.out.11282 (2.74 MB, text/plain)
2015-12-25 13:10 UTC, Timur

Note You need to log in before you can comment on or make changes to this bug.
Description Katarina Behrens 2015-03-20 16:43:37 UTC
Attachment 113629 [details] from bug 89597 (which is a large xlsx file with some 40 sheets and lots of pivot tables) takes some 20 minutes to open on my not-so-slow laptop (SSD, 8 cores). Not sure if it is OOXML import issue or Calc core issue, anyway Calc seem to be very busy while outputting the following line:

warn:legacy.osl:12946:1:sc/source/core/data/dpsave.cxx:1192: ScDPSaveData::WriteToSource

To reproduce:
0. Get some daily build that contains my fix for bug 89597 ( http://cgit.freedesktop.org/libreoffice/core/commit/?id=529e9b61171f3155a76fe68e3019f5f3eb23bc4e ) without it the attachment can't be opened (OOXML import aborts)
1. Open attachment 113629 [details] 
2. Go get yourself some tea while waiting for the document to open
Comment 1 Buovjaga 2015-03-21 19:40:21 UTC
Yep, took ages, but finally opened.

Ubuntu 14.10 64-bit 
Build ID: 45c949c5a34cb73cdb08f85b2f33ae498c7c3c5c
TinderBox: Linux-rpm_deb-x86_64@46-TDF-dbg, Branch:master, Time: 2015-03-20_23:52:13
Locale: en_US
Comment 2 Timur 2015-12-24 12:33:29 UTC
Created attachment 121533 [details]
same sample, but without external links

The same sample, but without external links, also loads slowly.
Comment 3 Michael Meeks 2015-12-24 14:35:59 UTC
in order to tackle this we will need a cachegrind trace; please get a Linux build with debugging symbols (but not dbgutil)

unset G_SLICE


valgrind --tool=callgrind --simulate-cache=yes --dump-instr=yes ./soffice.bin --splash-pipe=0 /path/to/filename.txt

and attach the larger callgrind.1235.txt file =)

Comment 4 Timur 2015-12-25 13:10:43 UTC
Created attachment 121542 [details]

It's so slow that I don't know when it's suppose to be done. File is not open yet. Still, I'll attach callgrind. Please comment. 

Here are console messages:
warn:legacy.osl:11197:1:oox/source/docprop/docprophandler.cxx:315: For now unexpected tags are ignored!
warn:legacy.osl:11197:1:oox/source/helper/graphichelper.cxx:117: GraphicHelper::GraphicHelper - cannot get target frame
warn:legacy.osl:11197:17:oox/source/helper/progressbar.cxx:66: ProgressBar::setPosition - invalid position
warn:oox:11197:1:oox/source/drawingml/shapecontext.cxx:124: ShapeContext::onCreateContext: unhandled element: 2488
Comment 5 Timur 2015-12-25 16:12:44 UTC
Well, I stopped it to early, here's what I got so far: http://www75.zippyshare.com/v/2Zz1tFWC/file.html
Comment 6 Julien Nabet 2016-01-03 08:38:26 UTC
Eike: Timur's Callgrind trace (from comment 5) is quite interesting.
If I don't misunderstand kCachegrind graphic, it seems a lot of time is spent in ScColumn::GetOptimalHeight function because this one is called a lot of times by GetOptimalHeightsInColumn.
Comment 7 Yousuf Philips (jay) (retired) 2016-10-21 10:39:47 UTC
Loaded in less than a minute on my slow core 2 duo.

Build ID: e64ea98801d20e5024da900a0ac8faaf565f4bf3
CPU Threads: 2; OS Version: Linux 3.19; UI Render: default; 
TinderBox: Linux-rpm_deb-x86_64@70-TDF, Branch:master, Time: 2016-10-18_04:29:35
Locale: en-US (en_US.UTF-8); Calc: group

This may have been fixed with bug 102694.
Comment 8 Timur 2016-10-21 10:59:45 UTC
Yes, acceptable now. But seems like fixed earlier, fine even with 5.2.2 64-bit.