Bug 128244 - OD* files can't be opened if Zip64 was used in their creation
Summary: OD* files can't be opened if Zip64 was used in their creation
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
6.2.5.2 release
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: File-Opening XLSX-External-Generators
  Show dependency treegraph
 
Reported: 2019-10-18 22:39 UTC by Joshua
Modified: 2022-09-09 19:42 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:
Regression By:


Attachments
File that can be opened (3.38 KB, application/vnd.oasis.opendocument.spreadsheet)
2019-10-18 22:39 UTC, Joshua
Details
File that cannot be opened (3.47 KB, application/vnd.oasis.opendocument.spreadsheet)
2019-10-18 22:40 UTC, Joshua
Details
Zip64 test kit (14.39 KB, application/x-zip-compressed)
2019-10-19 12:52 UTC, Mike Kaganski
Details
Another pair of test files (25.09 KB, application/x-zip-compressed)
2019-10-22 15:50 UTC, Joshua
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Joshua 2019-10-18 22:39:13 UTC
Description:
OD* files can't be opened if Zip64 was used in their creation, despite Zip64 being part of Zip Version 6.2.0. This impacts streaming generation.

Discovered in Calc since it's the only one we mess with as of yet, but probably all formats affected.

Note the attached export32.ods is broken for other reasons but can be recovered; while the export64.ods is unrecoverable.

Steps to Reproduce:
Try opening export64.ods

Actual Results:
Error involving content length 0 and no content can be recovered.

Expected Results:
Like export32.ods


Reproducible: Always


User Profile Reset: No



Additional Info:
As I said, we have other generation problems because we used Excel as a baseline and Excel just doesn't even read META-INF.

Known to be broken in latest release; even online validator doesn't recognize Zip64 yet.
Comment 1 Joshua 2019-10-18 22:39:48 UTC
Created attachment 155136 [details]
File that can be opened
Comment 2 Joshua 2019-10-18 22:40:15 UTC
Created attachment 155137 [details]
File that cannot be opened
Comment 3 m.a.riosv 2019-10-19 02:32:31 UTC
Neither of files are ODF compliant, so if they are not generated by LibreOffice, I think there is nothing LibreOffice must do.

https://odfvalidator.org/
----------------------------------------------------------------------------------------
ODF Validator Result Page
Result for 15714198122809178.ods

The document is NOT conformant ODF 'Missing ODF version'!
Details:
15714198122809178.ods/META-INF/manifest.xml: Error: The ODF package '15714198122809178.ods' shall contain the 'META-INF/manifest.xml' file!
15714198122809178.ods/mimetype: Error: The file 'mimetype' shall not be compressed in the ODF package '15714198122809178.ods'!
15714198122809178.ods[-1,-1]: Error: The ODF package '15714198122809178.ods' contains a 'mimetype' file containing 'application/vnd.oasis.opendocument.spreadsheet', but no mediatype for its root document in its 'META-INF/manifest.xml'!
15714198122809178.ods/META-INF/manifest.xml: Warning: The file 'content.xml' shall be listed in the 'META-INF/manifest.xml' file as it exists in the ODF package '15714198122809178.ods'!
15714198122809178.ods/META-INF/manifest.xml: Warning: The file 'meta.xml' shall be listed in the 'META-INF/manifest.xml' file as it exists in the ODF package '15714198122809178.ods'!
15714198122809178.ods/META-INF/manifest.xml: Warning: The file 'styles.xml' shall be listed in the 'META-INF/manifest.xml' file as it exists in the ODF package '15714198122809178.ods'!
15714198122809178.ods: Error: At least 'content.xml' or 'styles.xml' have to be contained in the ODF XML Schema package '15714198122809178.ods'!
internal:/schema/odf1.1/OpenDocument-manifest-schema-v1.1.rng: Info: parsed.
15714198122809178.ods/mimetype: Info: 1 errors, no warnings
15714198122809178.ods: Info: Media Type:
internal:/schema/odf1.1/OpenDocument-schema-v1.1.rng: Info: parsed.
15714198122809178.ods/meta.xml: Info: Generator: Cedaron.FormBuilder.Presentation
15714198122809178.ods/meta.xml: Info: no errors, no warnings
15714198122809178.ods/styles.xml: Info: no errors, no warnings
15714198122809178.ods/content.xml[11,22]: Error: tag name "table:table-row" is not allowed. Possible tag names are: <dde-source>,<forms>,<scenario>,<shapes>,<table-column-group>,<table-column>,<table-columns>,<table-header-columns>,<table-source>
<table:table-row> ----^ 15714198122809178.ods/content.xml[81,81]: Error: attribute "office:boolean-value" has a bad value. Possible values are: false,true
"boolean" office:boolean-value="False"><text:p>False</text:p></table:table-cell ----^ 15714198122809178.ods/content.xml[85,81]: Error: attribute "office:boolean-value" has a bad value. Possible values are: false,true
"boolean" office:boolean-value="False"><text:p>False</text:p></table:table-cell ----^ 15714198122809178.ods/content.xml[91,81]: Error: attribute "office:boolean-value" has a bad value. Possible values are: false,true
"boolean" office:boolean-value="False"><text:p>False</text:p></table:table-cell ----^ 15714198122809178.ods/content.xml[98,107]: Error: attribute "office:time-value" has a bad value: "PT0H0M0" does not satisfy the "duration" type
PT0H0M0" table:style-name="Ctimestamp"><text:p>12:00 AM</text:p></table:table-c ----^ 15714198122809178.ods/content.xml[100,104]: Error: attribute "office:time-value" has a bad value: "PT12H34M0" does not satisfy the "duration" type
e="PT12H34M0" table:style-name="Ctime"><text:p>12:34 PM</text:p></table:table-c ----^ 15714198122809178.ods/content.xml[137,22]: Error: tag name "table:table-row" is not allowed. Possible tag names are: <dde-source>,<forms>,<scenario>,<shapes>,<table-column-group>,<table-column>,<table-columns>,<table-header-columns>,<table-source>
<table:table-row> ----^ 15714198122809178.ods/content.xml[145,22]: Error: tag name "table:table-row" is not allowed. Possible tag names are: <dde-source>,<forms>,<scenario>,<shapes>,<table-column-group>,<table-column>,<table-columns>,<table-header-columns>,<table-source>
<table:table-row> ----^ 15714198122809178.ods/content.xml[154,22]: Error: tag name "table:table-row" is not allowed. Possible tag names are: <dde-source>,<forms>,<scenario>,<shapes>,<table-column-group>,<table-column>,<table-columns>,<table-header-columns>,<table-source>
<table:table-row> ----^ 15714198122809178.ods/content.xml: Info: 9 errors, no warnings
15714198122809178.ods: Info: 13 errors, 3 warnings

----------------------------------------------------------------------------------------
ODF Validator Result Page
Result for 15714198492559842.ods

The document is NOT conformant ODF 'Missing ODF version'!
Details:
15714198492559842.ods: Fatal: ZIP entry 'content.xml': invalid entry size (expected 0 but got 23018 bytes)
Comment 4 Mike Kaganski 2019-10-19 09:44:14 UTC
To provide a compliant ODF using Zip64, one needs a method to generate such a package from a known-good fileset, like a lorem ipsum document created with LO, extracted and then re-compressed by a tool creating Zip64 (and making sure that mimetype is not compressed, and other files are either not compressed, or deflate-compressed). Then a different issue of specific document having non-compliant content would be eliminated.
Comment 5 Mike Kaganski 2019-10-19 12:52:07 UTC
Created attachment 155152 [details]
Zip64 test kit

I have created a simple lorem ipsum spreadsheet ('lorem ipsum.ods' in attachment) using LibreOffice; extracted its content into a 'loremipsum' directory; and used a script from [1] (modified to use ZIP_STORED compression level for all files; 'zip64.py' in the attachment) like `zip64.py /path/to/zip64.ods /path/to/loremipsum/mimetype /path/to/loremipsum/*` to create 'zip64.ods' with mimetype stored first (also in the attachment). So the two ODS files in the kit have identical contents, but zip64.ods has all files stored without compression, *and* it uses Zip64.

The file opens with "The file 'zip64.ods' is corrupt and therefore cannot be opened. LibreOffice can try to repair the file" warning; it *can* be repaired. It seems that the resulting file is conformant, and should be opened without warnings.
Comment 6 Mike Kaganski 2019-10-19 13:01:42 UTC
Forgot to mention the python script source in comment 5:

[1] https://gist.github.com/gumblex/5573ddb33c21fca4aecf
Comment 7 Xisco Faulí 2019-10-22 09:56:06 UTC
Moving to NEW based on Mike Kaganski's analysis
Comment 8 Joshua 2019-10-22 15:50:16 UTC
Created attachment 155237 [details]
Another pair of test files
Comment 9 Joshua 2019-10-22 15:54:40 UTC
I noted a difference in the repair effect between my sample and yours so I made yet another test pair. This time, the 32 bit one opens without error in LibreOffice but the 64 bit one still fails to recover at all.

At this point I think the online validator is having trouble and the content actually is valid.
Comment 10 Mike Kaganski 2019-10-23 07:30:50 UTC
The Zip64 document in attachment 155237 [details] differs from that in attachment 155152 [details] by including something that Info-Zip's zipinfo (i.e., `unzip -Z path/to/file.ods`) explains as "extended local header". Both include "extra fields" (that's natural, since [1] defines "ZIP64 extended information extra field"), but what is in the "extended local header" in the attachment 155237 [details] (both ODS there have it) is unclear.

In general, let's not throw in here more and more random files, and just focus this issue on supporting of Zip64 spec (i.e., not giving a warning mentioned in comment 5). The problem of possibly incorrect handling of "extended local header" looks independent, and likely needs a different issue.

[1] https://www.pkware.com/documents/APPNOTE/APPNOTE-6.2.0.txt