Bug 68016 - Excel file causes crash
Summary: Excel file causes crash
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
3.5.0 release
Hardware: All All
: high major
Assignee: Tobias Lippert
URL:
Whiteboard: target:5.1.0
Keywords:
Depends on:
Blocks:
 
Reported: 2013-08-12 13:02 UTC by Silviu C.
Modified: 2016-10-10 12:01 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Silviu C. 2013-08-12 13:02:15 UTC
Trying to open this file pegs the CPU and then potentially crashes. Excel 2007 opens the file very fast and does no have any issues.
Comment 1 Silviu C. 2013-08-12 13:05:57 UTC
Link to the big excel file:

https://dl.dropboxusercontent.com/u/3371527/Temperatures%20June.xlsx
Comment 2 Julien Nabet 2013-08-14 23:08:55 UTC
On pc Debian x86-64 with master sources updated today, I reproduced the hanging. I didn't wait for the crash but saw the memory consumption increase more and more.
Comment 3 Silviu C. 2013-08-25 14:01:06 UTC
I've adjusted the Importance settings according to this flowchart:

https://wiki.documentfoundation.org/images/0/06/Prioritizing_Bugs_Flowchart.jpg
Comment 4 Silviu C. 2013-08-30 10:23:34 UTC
Still an issue in 4.1.1.2
Comment 5 Silviu C. 2013-11-05 07:16:20 UTC
The reported problem can still be reproduce on version 4.1.3.
Comment 6 Julien Nabet 2013-11-05 21:52:07 UTC
Kohei: perhaps you might be interesting in this one since you're doing some refactoring/optimizing (at least, it seemed to me as a beotian :-) )
Comment 7 Silviu C. 2014-02-20 15:34:34 UTC
Hi,

This is still a problem in version 4.2.1
Comment 8 Joel Madero 2015-05-02 15:43:28 UTC Comment hidden (obsolete)
Comment 9 Silviu C. 2015-05-02 22:18:28 UTC
Yes. this bug still occurs on my machine as originally described. One CPU core is pegged at 100% and the application freezes.
 


silviu@trusty:~$ apt-cache policy libreoffice
libreoffice:
  Installed: 1:4.4.2~rc2-0ubuntu1~trusty1
  Candidate: 1:4.4.2~rc2-0ubuntu1~trusty1
  Version table:
 *** 1:4.4.2~rc2-0ubuntu1~trusty1 0
        500 http://ppa.launchpad.net/libreoffice/ppa/ubuntu/ trusty/main amd64 Packages
        100 /var/lib/dpkg/status
     1:4.2.8-0ubuntu2 0
        500 http://archive.ubuntu.com/ubuntu/ trusty-security/universe amd64 Packages
        500 http://archive.ubuntu.com/ubuntu/ trusty-updates/universe amd64 Packages
     1:4.2.3~rc3-0ubuntu2 0
        500 http://archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages

OS is Ubuntu 14.04.2 64bit
Comment 10 Tobias Lippert 2015-05-27 20:43:58 UTC
Hello,
I can reproduce the issue on 4.4.3.2 for Fedora 22.
I will look into it.
Tobias
Comment 11 Tobias Lippert 2015-06-01 18:05:53 UTC
Hello Silviu,

thank you for providing the test-data, and for your patience :-) While the fixes will probably not help with your problem, other users will profit!

I have identified the performance bottlenecks and will work on resolving them.

Just for the record: The bottlenecks I have found so far are:

chart2uno.cxx:
ScChart2DataSequence::getNumberFormatKeyByIndex()
Here, the check for hidden rows eats a lot of time. Caching them should help.

listener.cxx:
SvtListener::EndListeningAll()
cellsuno.cxx:
ScLinkListener::EndListeningAll()
Could be resolvable by offering a method to end listening of a bulk set of listeners

ExplicitCategoriesProvider.cxx:
lcl_fillDateCategories()
The check for the date format takes a lot of time. A caching mechanism should resolve this too

Tobias
Comment 12 Silviu C. 2015-06-01 18:22:32 UTC
All I can say is, thank you very much for looking into this :)
Comment 13 Tobias Lippert 2015-06-18 08:25:36 UTC
I have fixed the bug. I can open the file on my machine. The code is currently waiting for review. (https://gerrit.libreoffice.org/#/c/16320/5)
The fix will be available in the next version.
Comment 14 Silviu C. 2015-06-18 08:33:29 UTC
Oh my! This is great news!
 
When you say "next version" do you mean during the 4.x cycle or 5.x?
Comment 15 Julien Nabet 2015-06-18 08:40:11 UTC
Silviu: for the moment the patch must be approved in master branch (= future 5.1.0).
If ok, I suppose Tobias will cherry-pick it on 5.0 branch and perhaps on 4.4 branch.
Comment 16 Tobias Lippert 2015-06-18 08:42:13 UTC
@Julien - Sure. Let me know if you need help.
Comment 17 Tobias Lippert 2015-06-18 08:54:34 UTC
(In reply to Tobias Lippert from comment #16)
> @Julien - Sure. Let me know if you need help.

Oh, I read that wrong. I though you wanted to try to port the fixes. :-)

I will port them after the fix has been reviewed and verified and write here how it went.
Comment 18 Timur 2015-06-18 17:44:55 UTC
LO could open it finally up to 3.4.4. Since it was always slow, I wouldn't call it a regression.
Comment 19 Tobias Lippert 2015-06-24 17:12:12 UTC
Could somebody please have a look at the gerrit commits? They can be found here.
https://gerrit.libreoffice.org/#/c/16315/4
Comment 20 Commit Notification 2015-06-29 15:56:49 UTC
Tobias Lippert committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=b014150e64cdc23dfd999061bc210f0ad701f0a2

tdf#68016 Write fixture for current behaviour of SfxListener

It will be available in 5.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 21 Tobias Lippert 2015-06-30 17:30:37 UTC
There is still one patch missing. I will post an update as soon as it passes code review.
Comment 22 Commit Notification 2015-08-04 06:44:00 UTC
Tobias Lippert committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=431957118d7bbf96bf78e142df9decbcaff6475b

fdo#68016 Speed up ScChartListenerCollection::EndListeningHiddenRange

It will be available in 5.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 23 Commit Notification 2015-08-04 06:45:12 UTC
Tobias Lippert committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=e355661fe6352c93305406330349b630735fd4cd

fdo#68016 Add unittests for ScChartListenerCollection

It will be available in 5.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 24 Tobias Lippert 2015-08-04 08:12:30 UTC
There is still one commit waiting for review before the fix can be tested.

https://gerrit.libreoffice.org/#/c/16485/
Comment 25 Commit Notification 2015-08-25 13:31:05 UTC
Tobias Lippert committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=1979c48bf96bc1fd8f3558b6c9970935ddd79723

tdf#68016 Speed up ScChart2DataSequence by caching addresses

It will be available in 5.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 26 Tobias Lippert 2015-08-31 12:57:14 UTC
This bug should be fixed on the master branch. If someone could validate this, that would be great.
Comment 27 Silviu C. 2015-08-31 14:25:24 UTC
Hi Tobias,

I'm testing Build ID: edf9fa4a37261202196e8873169151f376cfd33b
 
Wow, it actually manages to open the file in reasonable time. The window still goes black (like the application froze). But this being said, it's just a *huge* improvement over any of the previous stable versions of LO.
 
Switching between sheets with graphs on them still takes around 6-8 seconds the first time I switch to them. The good part is that subsequent switching between those sheets is actually very fast and responsive.
 
I have uploaded a video here:
https://www.youtube.com/watch?v=UQSOiOreLxw

Thank you very much!
Comment 28 Silviu C. 2015-08-31 14:27:49 UTC
I forgot to ask. Do I need to open a separate bug report to address the performance?
Comment 29 Julien Nabet 2015-08-31 18:59:00 UTC
(In reply to Silviu C. from comment #28)
> I forgot to ask. Do I need to open a separate bug report to address the
> performance?

I think so, crash is different that perf problem.
Comment 30 Tobias Lippert 2015-08-31 19:00:02 UTC
(In reply to Silviu C. from comment #28)
> I forgot to ask. Do I need to open a separate bug report to address the
> performance?

Hello Silviu,

Great - thanks for the video :-) It's good to see that it is actually working on a machine other than mine. I have seen the strangest differences between computers.

If it is not too much work, it would be good to create a new bug. The file opens (albeit slowly), and optimizing for performance is another task.

Tobias
Comment 31 Silviu C. 2015-08-31 19:15:17 UTC
Ok, I created a new bug report here:

https://bugs.documentfoundation.org/show_bug.cgi?id=93819

Thank you again for all the work you've done.
Comment 32 tommy27 2015-09-01 05:18:06 UTC
crash fixed according to comment 27
so RESOLVED FIXED

the performance issue will be discussed in Bug 93819