Bug 72929 - FILEOPEN: loading .xlsx freezes LibO
Summary: FILEOPEN: loading .xlsx freezes LibO
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.0.4.2 release
Hardware: x86-64 (AMD64) All
: medium major
Assignee: Eike Rathke
URL:
Whiteboard: target:4.3.0 target:4.2.0
Keywords: regression
Depends on:
Blocks:
 
Reported: 2013-12-20 18:49 UTC by Islam Sharabash
Modified: 2014-01-13 13:51 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
gdb log for freeze (162.62 KB, text/plain)
2013-12-20 18:49 UTC, Islam Sharabash
Details
File to reproduce bug (28.58 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2013-12-27 01:02 UTC, Islam Sharabash
Details
Array Formulas (9.55 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2013-12-29 13:47 UTC, Jacques Guilleron
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Islam Sharabash 2013-12-20 18:49:06 UTC
Created attachment 91055 [details]
gdb log for freeze

Hi,

I have a xlsx spreadsheet with ~30 sheets that freezes upon opening with libreoffice (cpu jumps to 100% and then it hangs).

I can't give access to the original file unfortunately, but I do have a gdb.log that I'm attaching. This freeze happens on both OSX and ubuntu.

Please let me know how I can help debug this, or what I should look for.
Comment 1 Tim Lloyd 2013-12-22 05:42:13 UTC
Hi Islam,

could you save the doc as xls in excel and see if the same problem occurs in libreoffice? It would be interesting to know whether this problem affects both xlsx and xls

Cheers
Comment 2 retired 2013-12-22 12:16:59 UTC
Islam: Did you try if the issue persists with LO 4.2.0.1? http://www.libreoffice.org/download/pre-releases/

Can you please provide a test document so this can be tested against and subsequently be confirmed. If your document contains sensitive data, please clear that or replace it with random information.

A step-by-step description of how to reproduce the issue is most helpful and will help to speed up the processing of this problem a lot.

Setting to NEEDINFO until more detail is provided.

After providing the requested info, please reset this bug to UNCONFIRMED. Thanks :)
Comment 3 Islam Sharabash 2013-12-25 18:19:01 UTC
(In reply to comment #1)
> Hi Islam,
> 
> could you save the doc as xls in excel and see if the same problem occurs in
> libreoffice? It would be interesting to know whether this problem affects
> both xlsx and xls
> 
> Cheers

Just tried saving it as xls and it seems to work fine! -- What gave you that thought?
Comment 4 Islam Sharabash 2013-12-25 18:36:41 UTC
(In reply to comment #2)
> Islam: Did you try if the issue persists with LO 4.2.0.1?
> http://www.libreoffice.org/download/pre-releases/
> 
> Can you please provide a test document so this can be tested against and
> subsequently be confirmed. If your document contains sensitive data, please
> clear that or replace it with random information.
> 
> A step-by-step description of how to reproduce the issue is most helpful and
> will help to speed up the processing of this problem a lot.
> 
> Setting to NEEDINFO until more detail is provided.
> 
> After providing the requested info, please reset this bug to UNCONFIRMED.
> Thanks :)



Hi Foss, It still fails with 4.2.0.1
Do you know of a way to quickly randomize all data in an excel file?
Clearing it out also gets rid of the bug.
Comment 5 retired 2013-12-26 16:23:47 UTC
That should already be helpful info to narrow the bug down. Do I understand correctly XLS works while XLSX fails with 4.2.0.1?

There's not really any alternative to manually going over the document and clearing sensitive information. But if you keep trying to reproduce the bug that should again help to narrow the place in the doc where the bug is deriving from. Would be great if you coudl go through the doc little by little and see how long the bug is reproducible.
Comment 6 Islam Sharabash 2013-12-27 01:01:36 UTC
(In reply to comment #5)
> That should already be helpful info to narrow the bug down. Do I understand
> correctly XLS works while XLSX fails with 4.2.0.1?
> 
> There's not really any alternative to manually going over the document and
> clearing sensitive information. But if you keep trying to reproduce the bug
> that should again help to narrow the place in the doc where the bug is
> deriving from. Would be great if you coudl go through the doc little by
> little and see how long the bug is reproducible.

Foss, yep you had it correct, fails with 4.2.0.1 but the xls succeeds.
Spent some time picking through the file and tracked it down to a certain sheet, then a few formula. I was able to make a replica workbook that reproduces the bug (attached).

Also noting that the formula in question needed to be an array formula (http://office.microsoft.com/en-us/excel-help/introducing-array-formulas-in-excel-HA001087290.aspx).
Comment 7 Islam Sharabash 2013-12-27 01:02:01 UTC
Created attachment 91204 [details]
File to reproduce bug
Comment 8 tommy27 2013-12-28 23:29:13 UTC
issue confirmed in 4.1.4.2 under Win7 64bit and 4.0.4.2

the file can be loaded in few seconds using 3.6.7 so I suspect a regression during 4.0.x development

set status to NEW. edited version field and summary notes.
added Calc expert to CC list
Comment 9 Jacques Guilleron 2013-12-29 13:24:12 UTC
It seems to me that those functionalities (Array and Table) are being implemented. See:
Bug 66377 - FILEOPEN: MS .xlsx Fails to import properly. Rows not colored (1/6)
So, before an official announce, we cannot considerer this report as a bug, but rather as a lack.
However, trying the provided link, I sa
Comment 10 Jacques Guilleron 2013-12-29 13:44:17 UTC
Hi,

I don't want anticipate the answer of a developper but, it seems to me that those functionalities (Array and Table) are being implemented. See:
Bug 66377 - FILEOPEN: MS .xlsx Fails to import properly. Rows not colored (1/6)
So, before an official announce, we cannot considerer this report as a bug, but rather as a lack.
However, trying the provided link, I saw that we can import such a file in Calc.
And writing array formulas is already available and works with LO 4.1.3.2 but not recommended.

Kind regards,

Jacques
Comment 11 Jacques Guilleron 2013-12-29 13:47:47 UTC
Created attachment 91289 [details]
Array Formulas
Comment 12 Islam Sharabash 2013-12-30 16:40:17 UTC
(In reply to comment #11)
> Created attachment 91289 [details]
> Array Formulas

Try the file I uploaded in the latest calc -- it should crash it. The difference between the one I uploaded and yours is that my array formula was cross-sheet.
Comment 13 Eike Rathke 2014-01-07 17:07:41 UTC
Effectively it doesn't freeze, it just would take a day or two to complete ;-)

@Kohei:
Problem is a large multi_type_vector result matrix (entire column, 1048576 rows) of a jump matrix. The result matrix is initialized to error values (needed in case interpret ends prematurely due to errors encountered) i.e. double values. Subsequent string results each lead to the initial block being shrunk by one and a string appended to a (first time inserted new) block. All together a significant overhead.

Called from sc/source/core/tool/interpr1.cxx ScInterpreter::JumpMatrix() line 570 via pResMat->PutString(aStr, nC, nR) on master in this case.
Comment 14 Eike Rathke 2014-01-07 17:29:26 UTC
Let me give this a try..
Comment 15 Eike Rathke 2014-01-08 01:02:25 UTC
Have this working but it needs a libmdds update for non-internal mdds.
Comment 16 Eike Rathke 2014-01-08 16:23:40 UTC
Prerequisite libmdds-0.10.1 introduced with http://cgit.freedesktop.org/libreoffice/core/commit/?id=12890dd7adfc70bc562bb2760084515cc1269a38
Comment 17 Commit Notification 2014-01-08 16:29:22 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=ae4dc8e0ea3b78d804d1022905501a56030073c0

resolved fdo#72929 buffer jump matrix' result matrix blocks



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 18 Eike Rathke 2014-01-08 17:56:44 UTC
Pending review for 4-2 at https://gerrit.libreoffice.org/7320
Comment 19 Commit Notification 2014-01-08 18:46:38 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-4-2":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=e1023ab62a62bb76b73841c02e44eacaba699d0e&h=libreoffice-4-2

resolved fdo#72929 buffer jump matrix' result matrix blocks


It will be available in LibreOffice 4.2.1.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 20 Eike Rathke 2014-01-08 18:49:02 UTC
Pending review for 4-2-0 at https://gerrit.libreoffice.org/7321
Comment 21 Jacques Guilleron 2014-01-10 08:32:00 UTC
Hello Eike,

This file now opens correctly. Verified with LO 4.3.0.0.alpha0+
Build ID: 03e0781145bceebca84e5d891ed0df72c75389da
TinderBox: Win-x86@39, Branch:master, Time: 2014-01-09_12:14:17

Thank you very much for the fix,

Kind regards,

Jacques
Comment 22 Islam Sharabash 2014-01-10 17:01:26 UTC
Confirming it was resolved for the original file as well, thanks so much Eric, you rock!
I was wondering whether this fix would be applied to 4.1.X release or not?
Comment 23 Eike Rathke 2014-01-10 19:37:47 UTC
Who is Eric? ;-)

4.1.x still has mdds 0.8.1, I'd rather not introduce a release 2 versions greater to a stable branch and additionally force distributions that use system mdds to upgrade that as well.
Comment 24 Islam Sharabash 2014-01-10 20:43:29 UTC
(In reply to comment #23)
> Who is Eric? ;-)
> 
> 4.1.x still has mdds 0.8.1, I'd rather not introduce a release 2 versions
> greater to a stable branch and additionally force distributions that use
> system mdds to upgrade that as well.

Sorry Eike, and roger that! Since it's against the 4.2.0 branch though that means it will get into the Jan 27 release? Thanks for clarifying this too.
Comment 25 Commit Notification 2014-01-11 16:03:42 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-4-2-0":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=b7adbad0176da2528229d2ea166b5ae58ef18649&h=libreoffice-4-2-0

resolved fdo#72929 buffer jump matrix' result matrix blocks


It will be available already in LibreOffice 4.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 26 Eike Rathke 2014-01-13 13:51:33 UTC
(In reply to comment #24)
> Since it's against the 4.2.0 branch though that
> means it will get into the Jan 27 release?

With the commit on libreoffice-4-2-0 yes (see comment #25).