Bug 56394 - FILEOPEN - FILESAVE: very slow opening of .xlsx and freeze when saving
Summary: FILEOPEN - FILESAVE: very slow opening of .xlsx and freeze when saving
Status: RESOLVED INVALID
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.0.1.2 release
Hardware: All All
: highest normal
Assignee: Not Assigned
URL:
Whiteboard: BSA
Keywords: bibisectRequest, perf
: 70867 (view as bug list)
Depends on:
Blocks: mab4.3
  Show dependency treegraph
 
Reported: 2012-10-25 13:48 UTC by Callegar
Modified: 2016-08-07 01:23 UTC (History)
6 users (show)

See Also:
Crash report or crash signature:


Attachments
Test file (1.00 MB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2015-09-30 06:41 UTC, jmgigandet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Callegar 2012-10-25 13:48:46 UTC
Consider the file at http://www.info.sciverse.com/documents/files/scopus-training/resourcelibrary/xls/title_list.xlsx 

It cannot be opened. LibO seems to hang forever. LibO freezes. Other opened documents freeze.

The same happens with Apache Openoffice

Platform (if different from the browser): 
              
Browser: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:16.0) Gecko/20100101 Firefox/16.0
Comment 1 Callegar 2012-10-25 14:54:21 UTC
A little update.

Calligra seems able to open it, but not to save it back in ods.

MS office 2007 can save it to ods and read back the ods. However, LibO cannot open the resulting ods.
Comment 2 Callegar 2012-10-25 15:29:55 UTC
Another update. LibO in the end seems to be able to open the file.  Only it takes 2 hours.  This on a rather robust xeon machine. Calligra takes 1 minute. MS Office takes less than 30 seconds.

Requiring a long time to import in some sense could even be acceptable.

However, issues that should be fixed IMHO include:

1) Complete lack of visual feedback while performing this painful import.
2) Once the document is imported, its management should become normal.  Conversely, trying to save the document as ods again takes hours.  We are talking of about 2-3 orders of magnitude more than MSOffice.
Comment 3 Callegar 2012-10-29 15:28:24 UTC
I am changing the title and rising the priority, reflecting the fact that:

1) LibO can actually open the xlsx file provided as a link in the first comment.
2) You can work on the file when it is loaded in memory and edit it.

But... you cannot save the file...

Asking LibO to save *in its own ods format* makes LibO hang apparently forever.

Note: the xlsx file takes about 15 minutes to open on a fast machine. After that, I tried to save the file as ods and the same machine has now been at 100% cpu for about 3 hours, without being able to save.

Now:

1) the bug is "new", rather than "unconfirmed" since I have tested on multiple machines and the behavior is consistently reproduced. Furthermore, there is a test file that anyone can download and use for debugging.

2) the priority is "major".  As a matter of fact IMHO it should be a "blocker" given that a spreadsheet that lets you open (even if painfully) a document and then lets you edit it, but makes it impossible for you to save your edited document is a quite serious problem.
Comment 4 Callegar 2012-10-29 16:03:21 UTC
Weird... copying the tables (with select-all, copy and paste) into a new spreadsheet leads to a document that can be saved.

Hence, apparently, it is the xlsx importer that creates in memory some data structures (formatting stuff? style stuff? who knows) that is lost in copy/paste and that prevents saving.
Comment 5 ringe 2013-03-07 21:33:15 UTC
Today, I saw this behavior with LibreOffice 4.0.1.2 running on 64-bit Windows Server 2008 through Citrix XenApp.

The xlsx file was really simple, just 11 rows. The user then added a bunch of rows and saw a freeze when trying to save it.

I'm afraid I can't publish the testcase, but I can provide it to someone who want to look into this.
Comment 6 ringe 2013-03-07 21:35:33 UTC
I can confirm this issue is valid for the latest release, as I tried the testcase provided on the latest version of LibreOffice on Ubuntu x64.
Comment 7 ringe 2013-03-08 11:44:09 UTC
Changing to blocker as this bug stops adoption in our business.
Comment 8 ringe 2013-03-08 19:13:48 UTC
I wonder if these two bugs are the same: #30770 and #43544
Comment 9 ringe 2013-03-08 19:22:44 UTC

*** This bug has been marked as a duplicate of bug 30770 ***
Comment 10 Markus Mohrhard 2013-03-11 12:06:42 UTC
Please stop marking performance bugs as duplicates. This should only be done by developers after profiling.

This case here has nothing to do with the other bug report.
Comment 11 tommy27 2014-05-01 08:37:31 UTC
@sergio.callegari

would you please tell us exactly how much time is needed to load that file on your machine? 2 hours like in comment 2 or 15 minutes like in comment 3.

moreover, the saving issue is only about trying to save in .ods format or even if you keep the .xlsx format?

I propose to change summary notes to:

FILEOPEN - FILESAVE: very slow opening of .xlsx and freeze when saving as .ods

do you agree with it?
has the situation changed in your system using LibO 4.2.3.3?
Comment 12 tommy27 2014-05-05 21:26:23 UTC
moved to mab4.2
killed LibO after 5 minutes freeze while trying to load xlsx file with 4.2.3.3 under Win7x64
Comment 13 tommy27 2014-10-05 18:25:45 UTC
(In reply to sergio.callegari from comment #3)
>.... 
> Note: the xlsx file takes about 15 minutes to open on a fast machine. After
> that, I tried to save the file as ods and the same machine has now been at
> 100% cpu for about 3 hours, without being able to save. 
>....

changed summary notes to:
FILEOPEN - FILESAVE: very slow opening of .xlsx and freeze when saving as .ods

retested under Win7x64 with an 5 years old laptop

FILEOPEN TIME
4.2.6.2  --> 3 minutes and 30 seconds
4.3.2.2  --> 1 minute and 30 seconds
4.4.0.0* --> 20 seconds

FILESAVE TO ODS
possible with 4.4.0.0+ (it takes almost 20-30 seconds)
freeze with 4.3.2.2 and impossible to save
did not try with 4.2.6.2

so basically Calc development in 4.4.x is showing promising results on both sides 



* Build ID: 268b9c10c9ff27c74678ace99762f28d58d33012
TinderBox: Win-x86@42, Branch:master, Time: 2014-10-02_23:35:24
Comment 14 tommy27 2014-11-06 18:25:15 UTC
*** Bug 70867 has been marked as a duplicate of this bug. ***
Comment 15 tommy27 2014-11-21 06:42:08 UTC
retested with 4.4.0.0 beta1 (*) under Win7x64

again another improvement

15 seconds to load test .xlsx
50 seconds to save it as .ods with transient freeze

(*) Build ID: 52b7e6bfc848a467baf02c320c29e7f77b92cc93
TinderBox: Win-x86@51-TDF, Branch:libreoffice-4-4, Time: 2014-11-20_23:58:07
Locale: it_IT

moving this from mab4.2 list to mab4.3 list since 4.2.x reached EOL
Comment 16 Björn Michaelsen 2014-11-28 09:45:26 UTC Comment hidden (obsolete)
Comment 17 Deepak KS 2015-01-06 09:46:17 UTC
Hi Dear,

Can i know what is the latest update on this bug?
can we know by when we can expect a fix for this?

Thanks,
Deepak.
Comment 18 tommy27 2015-01-06 10:15:54 UTC
(In reply to tommy27 from comment #15)
> retested with 4.4.0.0 beta1 (*) under Win7x64
> 
> again another improvement
> 
> 15 seconds to load test .xlsx
> 50 seconds to save it as .ods with transient freeze
> 
> (*) Build ID: 52b7e6bfc848a467baf02c320c29e7f77b92cc93
> TinderBox: Win-x86@51-TDF, Branch:libreoffice-4-4, Time: 2014-11-20_23:58:07
> Locale: it_IT
> 


retested with  4.5.0.0.alpha0+ under Win8.1 x64
Build ID: 2a20bf5105181d51aab40bdd4ce8c09615a0e599
TinderBox: Win-x86@42, Branch:master, Time: 2014-12-25_07:55:29

same loading time as in 4.4.0.0 beta1 (15 seconds)
slower .ods saving time (1 minute and 20 seconds vs. 50 seconds)
Comment 19 Joel Madero 2015-01-06 23:01:17 UTC
(In reply to Deepak KS from comment #17)
> Hi Dear,
> 
> Can i know what is the latest update on this bug?
> can we know by when we can expect a fix for this?
> 
> Thanks,
> Deepak.

Hi Deepak,

We do not give time estimates ever. This is a volunteer community, a volunteer will have to decide on their own that they want to fix the bug. The alternative is that you can submit a patch, find someone to submit a patch, or pay for a fix from a third party. Thanks for your patience and support of LibreOffice.
Comment 20 Matúš Kukan 2015-03-19 13:11:30 UTC
Load and save is something completely different. Should be two bugs. And load is now fixed in recent versions, right? So, it's about save as ods?

Anyhow, for performance problems, getting profile is really helpful. As described in https://wiki.documentfoundation.org/QA/BugReport/Debug_Information#GNU.2FLinux:_How_to_get_a_cachegrind_trace
Comment 21 tommy27 2015-06-15 21:10:25 UTC
I agree with Matus.
we should split this into 2 different reports.

having said that, I retested using LibO 4.4.3.2 and 5.1.0.0 alpha (*)
my results are:

4.4.3.2
FILEOPEN -> 18 seconds
FILESAVE -> 2 minutes and 38 seconds

5.1.0.0 alpha
FILEOPEN -> 58 seconds
FILESAVE -> 5 minutes

so performance got much worse in current master in respect to 4.4.x

(*) Build ID: bb9dad2ef23829b2500c9f99154bca6a8ba7d49a
TinderBox: Win-x86@39, Branch:master, Time: 2015-06-11_23:59:18
Locale: en-US (it_IT)
Comment 22 tommy27 2015-08-15 13:41:40 UTC
just retested the FILEOPEN thing

4.4.5.1 and 5.0.0.4 -> 17 seconds

5.1.0.0.alpha1+ (*) -> 65 seconds


so I confirm the severe performance loss in 5.1.x in respect to 4.4.x and 5.0.x
this needs bibisecting or profiling


(*) Build ID: 7d3fa6bae9f7a755eb2d0ca24bf1afd5f3646bb7
TinderBox: Win-x86@39, Branch:master, Time: 2015-08-09_08:38:08
Locale: it-IT (it_IT)
Comment 23 jmgigandet 2015-09-30 06:41:38 UTC
Created attachment 119134 [details]
Test file

Slow loading, and even sometimes crashes with a general I/O failure.
Comment 24 jmgigandet 2015-09-30 06:48:28 UTC
I confirm there's a problem with LO 5.
My attached file is loaded in less than 2 seconds with Excel 2003.
The same file opened with LO 5.0.1 or 5.0.2, fails to open with a general I/O error, or opens in more than 2mn15s !
When general I/O error is shown, answering "retry" allows LO to finish loading.
Comment 25 Markus Mohrhard 2015-10-15 18:27:33 UTC
To be honest this bug is a mess. IMHO it should be closed and each of the issues should be reported independently.

@QA team: I doubt that this deserves to be a MAB at all.
Comment 26 Joel Madero 2015-10-15 18:33:25 UTC
Closing as INVALID per expert's advise. Please report separate clean bugs. Also let's keep it off of MAB list.
Comment 27 Robinson Tryon (qubit) 2015-12-15 11:15:39 UTC Comment hidden (obsolete)
Comment 28 Gerd 2016-08-07 01:23:17 UTC
I have a similar problem. I am using noew for years LibO for years but since more than a month I connot open my excel or the coressponding LibO Spread sheets. I am at a loss as this are important calculations for me.
The LibO program starts and it seems it tries to open the file but nothing happens. This is also the case with some Writer documents.
I had hopes that it is a bug that is solved with the next update but I have Version: 5.1.4.2 now and the problem persists. What can I do?