Bug 104875 - FILEOPEN: Opening xlsx file 15x times slower then in last version, performance speed regression
Summary: FILEOPEN: Opening xlsx file 15x times slower then in last version, performanc...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
5.2.4.2 release
Hardware: All All
: high major
Assignee: Eike Rathke
QA Contact:
URL:
Whiteboard: target:6.0.0 target:5.4.0.1 target:5.3.5
Keywords: bibisected, bisected, perf, regression
Depends on:
Blocks:
 
Reported: 2016-12-23 09:22 UTC by Mikeyy - L10n HR
Modified: 2017-06-20 17:51 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
New test file (84.39 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2017-01-14 14:54 UTC, Mikeyy - L10n HR
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Mikeyy - L10n HR 2016-12-23 09:22:44 UTC
Description:
Comparing releases 5.2.3 and 5.2.4 somewhere in fixes between those versions big perfomance regression was introduced when opening certain xlsx files (created by excel).

I cannot include file here since it's business related, but I will send it to QA email when requested.

Steps to Reproduce:
Just open file, from windows explorer or LibreOffice start menu, it doesn't matter.

Actual Results:  
LO 5.2.4. file opening:
1st opening: 1:39:82
2nd opening: 0:15:39
3rd opening: 0:15:14

1st opening of another similliar file, without restarting computer, after all test above: 0:15:84
If I restart computer and try to open this file first: 1:39:21

Expected Results:
LO 5.2.3. file opening:
1st opening: 0:06:55
2nd opening: 0:04:93
3rd opening: 0:05:18

1st opening of another similliar file, without restarting computer, after all test above: 0:06:16


Reproducible: Always

User Profile Reset: No

Additional Info:
Conclusion:
First opening of file take extremly long, 15x times longer then with 5.2.3. release. But after file was opened first time, until reboot, you can open same file and all similliar files much faster, but still 3x times slower then with 5.2.3. release.


User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:50.0) Gecko/20100101 Firefox/50.0
Comment 1 Mikeyy - L10n HR 2016-12-23 09:37:57 UTC
Searching for fixed which could have introduced it and without bibisect we will not know for sure, but bug 79892 feels like a good candidate.
Comment 2 Aron Budea 2016-12-23 21:40:20 UTC
Hm, that's interesting... If you're fine with e-mailing the file to me, I can take a look. Otherwise I'd suggest sanitizing the data.
What's the system locale?
Comment 3 Mikeyy - L10n HR 2016-12-23 22:04:07 UTC
Link sent on your email.
Windows 8.1, system locale HR (croatian), LO UI croatian.
LO 5.2.3. testing was done with portable US locale version.
Comment 4 Aron Budea 2016-12-24 05:03:15 UTC Comment hidden (bibisection)
Comment 5 Aron Budea 2016-12-24 05:06:19 UTC
I can confirm the regression. From start to the time when I get a dialog about updating external files the times are these (OS: Windows 7):
-5.2.3.3: 6s,
-5.2.4.2: 29s, then 24s next time.

Bug could be bibisected, and the bug is caused by the commit below.
Adding Cc: to Eike Rathke. Please take a look. It's up to Mikeyy to share the bugdoc.

https://cgit.freedesktop.org/libreoffice/core/commit/?id=8464ea6961b9cc54af9c11cce1b80ed7e0cc77e2
"Resolves: tdf#79442 in OOXML import add external files to LinkManager

Now that we store formula results without recalculating, the implicit
logic that adds files of external references to the LinkManager is not
triggered, explicitly force it during import."
Comment 6 Aron Budea 2016-12-24 05:56:28 UTC
The file contains 11 external references, which point to network resources.
For those this line (and basically the underlying system routine) takes up a long time:

if (osl::DirectoryItem::get(GetName(), item) == osl::FileBase::E_None) {
http://opengrok.libreoffice.org/xref/core/sfx2/source/doc/docfile.cxx#2608
Comment 7 Mikeyy - L10n HR 2017-01-14 14:54:01 UTC
Created attachment 130427 [details]
New test file

Managed to find small file which I could clean up in code.
It probably has less connections then first test file so it takes "only" 48 seconds to open this file in LO 5.2.4, while it takes 3 seconds to open it in LO 3.3
Comment 8 Eike Rathke 2017-06-14 17:28:18 UTC
External links point to Samba shares, setting up the link tries to access the file content to detect the filter to use, which for each file has to wait for the timeout if the share is not mounted.
Comment 9 Commit Notification 2017-06-15 09:15:25 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=416752b9e4bc4605c479d3eff7797be9f0ef2a38

Resolves: tdf#104875 defer filter detection to first load/update of external

It will be available in 6.0.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 10 Eike Rathke 2017-06-15 09:28:08 UTC
Pending review
https://gerrit.libreoffice.org/38819 for 5-4
https://gerrit.libreoffice.org/38820 for 5-3
Comment 11 Commit Notification 2017-06-15 11:56:53 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-5-4":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=3d6797cde0663f3a558568a26ad19c7b5c98efa7&h=libreoffice-5-4

Resolves: tdf#104875 defer filter detection to first load/update of external

It will be available in 5.4.0.1.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Commit Notification 2017-06-20 17:51:17 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-5-3":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=4175045ebe4c895ff4617fbcdaacbbb058751597&h=libreoffice-5-3

Resolves: tdf#104875 defer filter detection to first load/update of external

It will be available in 5.3.5.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.