Bug 43152 - Collection of all problem documents during calc import
Summary: Collection of all problem documents during calc import
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
Master old -3.6
Hardware: Other All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-11-21 18:50 UTC by Markus Mohrhard
Modified: 2011-11-28 14:51 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments
memcheck log for xlsx import (12.14 KB, application/zip)
2011-11-21 18:52 UTC, Markus Mohrhard
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Markus Mohrhard 2011-11-21 18:50:16 UTC
All documents that created problems with filters-test

fdo39260.ods
fdo40304.xls
fdo42584.xlsx
ooo89935.ods
ooo97680.ods
ooo107004.xlsx
ooo109737.xls
ooo109800.xls
ooo112565.xls
ooo114415.ods
ooo115255.xls
ooo117239.ods (fixed by Kohei)
ooo118218.ods (performance problem related to matrix code)

Fixed are already two bugs in ods documents related to dociter.cxx and the pivot table problem by Kohei.
Comment 1 Markus Mohrhard 2011-11-21 18:52:58 UTC
Created attachment 53759 [details]
memcheck log for xlsx import

fixed is already the problem in range name's uno implementation
Comment 2 Markus Mohrhard 2011-11-21 18:55:28 UTC
ah sry the attached document is a zip and not a plain text file

Some more compents to the documents:

I think I did not see any crashs in xls but calc froze during import.
Comment 3 Kohei Yoshida 2011-11-21 21:15:28 UTC
(In reply to comment #0)

> ooo118218.ods (performance problem related to matrix code)

This turned out to be a duplicate of Bug 40990, which I just fixed.  Now that document opens very quickly.
Comment 4 Kohei Yoshida 2011-11-21 21:20:16 UTC
What remains to be fixed:

fdo39260.ods
fdo40304.xls
fdo42584.xlsx
ooo89935.ods
ooo97680.ods
ooo107004.xlsx
ooo109737.xls
ooo109800.xls
ooo112565.xls
ooo114415.ods
ooo115255.xls
Comment 5 Kohei Yoshida 2011-11-21 22:08:22 UTC
ooo115255.xls is now fixed.

http://cgit.freedesktop.org/libreoffice/core/commit/?id=f2972242673cc9608960e9ca70e82766be5275e3

What remains:

fdo39260.ods
fdo40304.xls
fdo42584.xlsx
ooo89935.ods
ooo97680.ods
ooo107004.xlsx
ooo109737.xls
ooo109800.xls
ooo112565.xls
ooo114415.ods
Comment 6 Markus Mohrhard 2011-11-21 22:24:53 UTC
fdo#39260 is fixed by http://cgit.freedesktop.org/libreoffice/core/commit/?id=84de459738d5c16b3df8719ff219aaccc3532b8c

fdo#40304: creates more than 50GB of dbgutil output during import

ooo#97680: crashs LibO

more updates will follow

remaining docs:

fdo40304.xls
fdo42584.xlsx
ooo89935.ods
ooo97680.ods
ooo107004.xlsx
ooo109737.xls
ooo109800.xls
ooo112565.xls
ooo114415.ods
Comment 7 Kohei Yoshida 2011-11-22 09:17:41 UTC
Just fixed Bug 40304.

What remains:

fdo42584.xlsx
ooo89935.ods
ooo97680.ods
ooo107004.xlsx
ooo109737.xls
ooo109800.xls
ooo112565.xls
ooo114415.ods
Comment 8 Kohei Yoshida 2011-11-23 21:26:12 UTC
Well, ooo97680.ods crashes because it contains sheets with invalid sheet names i.e. name with slashes like "/Users/benson/data/bbn-ne-test/wsj23d.qa".  The amazing thing is, the old version of LibO/OOo allow these names to be loaded, but you can't rename them afterward.

In our new code, we check the name, sees it invalid, and won't create a new sheet.  The importer still tries to put cell values in, then poof, crash.
Comment 9 Kohei Yoshida 2011-11-23 23:20:59 UTC
Fixed ooo97680.ods, by using default sheet name in case the supplied sheet name is invalid.

What remains:

fdo42584.xlsx
ooo89935.ods
ooo107004.xlsx
ooo109737.xls
ooo109800.xls
ooo112565.xls
ooo114415.ods
Comment 10 Eike Rathke 2011-11-24 04:41:16 UTC
Taking a stab at ooo89935.ods

Remaining:
fdo42584.xlsx
ooo107004.xlsx
ooo109737.xls
ooo109800.xls
ooo112565.xls
ooo114415.ods
Comment 11 Eike Rathke 2011-11-24 06:34:08 UTC
ooo89935.ods seems to have been already fixed, taking a stab at ooo114415.ods

Remaining:
fdo42584.xlsx
ooo107004.xlsx
ooo109737.xls
ooo109800.xls
ooo112565.xls
Comment 13 Markus Mohrhard 2011-11-24 09:47:49 UTC
@kohei: fdo42584.xlsx is a crash in pivot table uno implementation
Comment 14 Eike Rathke 2011-11-25 12:34:55 UTC
ooo112565.xls doesn't hang anymore.

Remaining:
fdo42584.xlsx
ooo107004.xlsx
ooo109737.xls
ooo109800.xls
Comment 15 Eike Rathke 2011-11-25 12:38:50 UTC
ooo109800.xls doesn't hang anymore.

Remaining:
fdo42584.xlsx
ooo107004.xlsx
ooo109737.xls
Comment 16 Eike Rathke 2011-11-25 13:37:50 UTC
ooo109737.xls doesn't hang, it just takes looooooong to load, ~40 minutes in dbgutil with debugger.. probably because it looks so pretty ;-)
Something to be tackled with later.

Remaining:
fdo42584.xlsx
ooo107004.xlsx
Comment 17 Eike Rathke 2011-11-26 06:53:53 UTC
Also ooo107004.xlsx doesn't hang, it takes even longer to load.. I don't know how long, let it ran over night, at the end it takes 1430MB memory. Notably are ~24000 Conditional_* cell styles (which apparently aren't even applied to any cell, didn't check thoroughly though), already inserting them during load apparently takes its time, opening the Stylist feels like freezing. Certainly to profile.

In the document are conditionalFormatting elements of which of each set a bunch applies to the same cells/areas with different priorities, but equal conditions.

How does Excel cope with this? And why does it generate such idiotic document?

Remaining:
fdo42584.xlsx
Comment 18 Kohei Yoshida 2011-11-28 08:15:01 UTC
(In reply to comment #17)

> Remaining:
> fdo42584.xlsx

I'll take a stab at this since this is related pivot table.
Comment 19 Kohei Yoshida 2011-11-28 14:50:11 UTC
(In reply to comment #18)
> (In reply to comment #17)
> 
> > Remaining:
> > fdo42584.xlsx
> 
> I'll take a stab at this since this is related pivot table.

Done.  Now that 2nd attachment in Bug 42584 should open fine without the crash.  One caveat: the result looks different than in Excel due to the inconsistency between the pivot cache which Excel uses upon import, and the raw data which we use on import currently.
Comment 20 Kohei Yoshida 2011-11-28 14:51:00 UTC
I'll take the liberty of marking this bug resolved.  Thanks everyone. :-)