Bug 128244 - OD* files can't be opened if Zip64 was used in their creation
Summary: OD* files can't be opened if Zip64 was used in their creation
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
6.2.5.2 release
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: File-Opening XLSX-External-Generators
  Show dependency treegraph
 
Reported: 2019-10-18 22:39 UTC by Joshua
Modified: 2023-05-05 14:06 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
File that can be opened (3.38 KB, application/vnd.oasis.opendocument.spreadsheet)
2019-10-18 22:39 UTC, Joshua
Details
File that cannot be opened (3.47 KB, application/vnd.oasis.opendocument.spreadsheet)
2019-10-18 22:40 UTC, Joshua
Details
Zip64 test kit (14.39 KB, application/x-zip-compressed)
2019-10-19 12:52 UTC, Mike Kaganski
Details
Another pair of test files (25.09 KB, application/x-zip-compressed)
2019-10-22 15:50 UTC, Joshua
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Joshua 2019-10-18 22:39:13 UTC
Description:
OD* files can't be opened if Zip64 was used in their creation, despite Zip64 being part of Zip Version 6.2.0. This impacts streaming generation.

Discovered in Calc since it's the only one we mess with as of yet, but probably all formats affected.

Note the attached export32.ods is broken for other reasons but can be recovered; while the export64.ods is unrecoverable.

Steps to Reproduce:
Try opening export64.ods

Actual Results:
Error involving content length 0 and no content can be recovered.

Expected Results:
Like export32.ods


Reproducible: Always


User Profile Reset: No



Additional Info:
As I said, we have other generation problems because we used Excel as a baseline and Excel just doesn't even read META-INF.

Known to be broken in latest release; even online validator doesn't recognize Zip64 yet.
Comment 1 Joshua 2019-10-18 22:39:48 UTC
Created attachment 155136 [details]
File that can be opened
Comment 2 Joshua 2019-10-18 22:40:15 UTC
Created attachment 155137 [details]
File that cannot be opened
Comment 3 m_a_riosv 2019-10-19 02:32:31 UTC
Neither of files are ODF compliant, so if they are not generated by LibreOffice, I think there is nothing LibreOffice must do.

https://odfvalidator.org/
----------------------------------------------------------------------------------------
ODF Validator Result Page
Result for 15714198122809178.ods

The document is NOT conformant ODF 'Missing ODF version'!
Details:
15714198122809178.ods/META-INF/manifest.xml: Error: The ODF package '15714198122809178.ods' shall contain the 'META-INF/manifest.xml' file!
15714198122809178.ods/mimetype: Error: The file 'mimetype' shall not be compressed in the ODF package '15714198122809178.ods'!
15714198122809178.ods[-1,-1]: Error: The ODF package '15714198122809178.ods' contains a 'mimetype' file containing 'application/vnd.oasis.opendocument.spreadsheet', but no mediatype for its root document in its 'META-INF/manifest.xml'!
15714198122809178.ods/META-INF/manifest.xml: Warning: The file 'content.xml' shall be listed in the 'META-INF/manifest.xml' file as it exists in the ODF package '15714198122809178.ods'!
15714198122809178.ods/META-INF/manifest.xml: Warning: The file 'meta.xml' shall be listed in the 'META-INF/manifest.xml' file as it exists in the ODF package '15714198122809178.ods'!
15714198122809178.ods/META-INF/manifest.xml: Warning: The file 'styles.xml' shall be listed in the 'META-INF/manifest.xml' file as it exists in the ODF package '15714198122809178.ods'!
15714198122809178.ods: Error: At least 'content.xml' or 'styles.xml' have to be contained in the ODF XML Schema package '15714198122809178.ods'!
internal:/schema/odf1.1/OpenDocument-manifest-schema-v1.1.rng: Info: parsed.
15714198122809178.ods/mimetype: Info: 1 errors, no warnings
15714198122809178.ods: Info: Media Type:
internal:/schema/odf1.1/OpenDocument-schema-v1.1.rng: Info: parsed.
15714198122809178.ods/meta.xml: Info: Generator: Cedaron.FormBuilder.Presentation
15714198122809178.ods/meta.xml: Info: no errors, no warnings
15714198122809178.ods/styles.xml: Info: no errors, no warnings
15714198122809178.ods/content.xml[11,22]: Error: tag name "table:table-row" is not allowed. Possible tag names are: <dde-source>,<forms>,<scenario>,<shapes>,<table-column-group>,<table-column>,<table-columns>,<table-header-columns>,<table-source>
<table:table-row> ----^ 15714198122809178.ods/content.xml[81,81]: Error: attribute "office:boolean-value" has a bad value. Possible values are: false,true
"boolean" office:boolean-value="False"><text:p>False</text:p></table:table-cell ----^ 15714198122809178.ods/content.xml[85,81]: Error: attribute "office:boolean-value" has a bad value. Possible values are: false,true
"boolean" office:boolean-value="False"><text:p>False</text:p></table:table-cell ----^ 15714198122809178.ods/content.xml[91,81]: Error: attribute "office:boolean-value" has a bad value. Possible values are: false,true
"boolean" office:boolean-value="False"><text:p>False</text:p></table:table-cell ----^ 15714198122809178.ods/content.xml[98,107]: Error: attribute "office:time-value" has a bad value: "PT0H0M0" does not satisfy the "duration" type
PT0H0M0" table:style-name="Ctimestamp"><text:p>12:00 AM</text:p></table:table-c ----^ 15714198122809178.ods/content.xml[100,104]: Error: attribute "office:time-value" has a bad value: "PT12H34M0" does not satisfy the "duration" type
e="PT12H34M0" table:style-name="Ctime"><text:p>12:34 PM</text:p></table:table-c ----^ 15714198122809178.ods/content.xml[137,22]: Error: tag name "table:table-row" is not allowed. Possible tag names are: <dde-source>,<forms>,<scenario>,<shapes>,<table-column-group>,<table-column>,<table-columns>,<table-header-columns>,<table-source>
<table:table-row> ----^ 15714198122809178.ods/content.xml[145,22]: Error: tag name "table:table-row" is not allowed. Possible tag names are: <dde-source>,<forms>,<scenario>,<shapes>,<table-column-group>,<table-column>,<table-columns>,<table-header-columns>,<table-source>
<table:table-row> ----^ 15714198122809178.ods/content.xml[154,22]: Error: tag name "table:table-row" is not allowed. Possible tag names are: <dde-source>,<forms>,<scenario>,<shapes>,<table-column-group>,<table-column>,<table-columns>,<table-header-columns>,<table-source>
<table:table-row> ----^ 15714198122809178.ods/content.xml: Info: 9 errors, no warnings
15714198122809178.ods: Info: 13 errors, 3 warnings

----------------------------------------------------------------------------------------
ODF Validator Result Page
Result for 15714198492559842.ods

The document is NOT conformant ODF 'Missing ODF version'!
Details:
15714198492559842.ods: Fatal: ZIP entry 'content.xml': invalid entry size (expected 0 but got 23018 bytes)
Comment 4 Mike Kaganski 2019-10-19 09:44:14 UTC
To provide a compliant ODF using Zip64, one needs a method to generate such a package from a known-good fileset, like a lorem ipsum document created with LO, extracted and then re-compressed by a tool creating Zip64 (and making sure that mimetype is not compressed, and other files are either not compressed, or deflate-compressed). Then a different issue of specific document having non-compliant content would be eliminated.
Comment 5 Mike Kaganski 2019-10-19 12:52:07 UTC
Created attachment 155152 [details]
Zip64 test kit

I have created a simple lorem ipsum spreadsheet ('lorem ipsum.ods' in attachment) using LibreOffice; extracted its content into a 'loremipsum' directory; and used a script from [1] (modified to use ZIP_STORED compression level for all files; 'zip64.py' in the attachment) like `zip64.py /path/to/zip64.ods /path/to/loremipsum/mimetype /path/to/loremipsum/*` to create 'zip64.ods' with mimetype stored first (also in the attachment). So the two ODS files in the kit have identical contents, but zip64.ods has all files stored without compression, *and* it uses Zip64.

The file opens with "The file 'zip64.ods' is corrupt and therefore cannot be opened. LibreOffice can try to repair the file" warning; it *can* be repaired. It seems that the resulting file is conformant, and should be opened without warnings.
Comment 6 Mike Kaganski 2019-10-19 13:01:42 UTC
Forgot to mention the python script source in comment 5:

[1] https://gist.github.com/gumblex/5573ddb33c21fca4aecf
Comment 7 Xisco Faulí 2019-10-22 09:56:06 UTC
Moving to NEW based on Mike Kaganski's analysis
Comment 8 Joshua 2019-10-22 15:50:16 UTC
Created attachment 155237 [details]
Another pair of test files
Comment 9 Joshua 2019-10-22 15:54:40 UTC
I noted a difference in the repair effect between my sample and yours so I made yet another test pair. This time, the 32 bit one opens without error in LibreOffice but the 64 bit one still fails to recover at all.

At this point I think the online validator is having trouble and the content actually is valid.
Comment 10 Mike Kaganski 2019-10-23 07:30:50 UTC
The Zip64 document in attachment 155237 [details] differs from that in attachment 155152 [details] by including something that Info-Zip's zipinfo (i.e., `unzip -Z path/to/file.ods`) explains as "extended local header". Both include "extra fields" (that's natural, since [1] defines "ZIP64 extended information extra field"), but what is in the "extended local header" in the attachment 155237 [details] (both ODS there have it) is unclear.

In general, let's not throw in here more and more random files, and just focus this issue on supporting of Zip64 spec (i.e., not giving a warning mentioned in comment 5). The problem of possibly incorrect handling of "extended local header" looks independent, and likely needs a different issue.

[1] https://www.pkware.com/documents/APPNOTE/APPNOTE-6.2.0.txt
Comment 11 Justin L 2023-04-24 20:27:46 UTC
I had expected this to be fixed in LO 7.6, where work was done to enable zip64 import and export. However, export64.ods still didn't open - crashed on "do you want to repair this" prompt.
Comment 12 Joshua 2023-04-24 20:39:19 UTC
That's funny. The files are supposed to be differential testing between ICSharpCode.SharpZipLib in default streaming and force 32 bit streaming output.

As some others have pointed out they ended up being not totally valid files, but the 32 bit version was always valid enough to open. The difference between them was literally one boolean property changed whether to force 32 bit generation or not. So I'd expect once fixed the files would behave identically.
Comment 13 Xisco Faulí 2023-04-25 10:10:33 UTC
(In reply to Justin L from comment #11)
> I had expected this to be fixed in LO 7.6, where work was done to enable
> zip64 import and export. However, export64.ods still didn't open - crashed
> on "do you want to repair this" prompt.

it seems, after Atilla's work, that the file can be open, However, another commit introduced the crash, I'll report it in another ticket
Comment 14 Xisco Faulí 2023-04-25 10:13:42 UTC
(In reply to Xisco Faulí from comment #13)
> (In reply to Justin L from comment #11)
> > I had expected this to be fixed in LO 7.6, where work was done to enable
> > zip64 import and export. However, export64.ods still didn't open - crashed
> > on "do you want to repair this" prompt.
> 
> it seems, after Atilla's work, that the file can be open, However, another
> commit introduced the crash, I'll report it in another ticket

-> bug 155005
Comment 15 Justin L 2023-05-05 14:06:07 UTC
(In reply to Xisco Faulí from comment #13)
> it seems, after Atilla's work, that the file can be open, 
Agree. Fixed.