Bug 103144 - Switch to Zstandard for compression during save
Summary: Switch to Zstandard for compression during save
Status: RESOLVED WONTFIX
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: filters and storage (show other bugs)
Version:
(earliest affected)
5.2.0.4 release
Hardware: All All
: medium enhancement
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-10-12 06:50 UTC by Robert Pollak
Modified: 2016-10-12 15:33 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Pollak 2016-10-12 06:50:15 UTC
The following post introduced Facebook's new free "Zstandard" compression algorithm and gives impressive benchmarks:

https://code.facebook.com/posts/1658392934479273/smaller-and-faster-data-compression-with-zstandard/ .

My own tests with https://github.com/mcmilk/7-Zip-zstd confirm an incredible speedup.

So: Please use this algorithm instead of zipping!
Comment 1 Buovjaga 2016-10-12 12:30:09 UTC
And break backwards compatibility?
Comment 2 Robert Pollak 2016-10-12 13:43:46 UTC
So the first step would be to implement _accepting_ zstd-compressed documents, then wait for a decade? :-/

Or is there any chance this could get accepted into a future version of the OpenDocument standard? Who of LibreOffice is participating in the corresponding OASIS committees?

But probably Zstandard has to prove its advantages "in the wild" before. Could there e.g. be an additional output format like .ods.zst that behaves like an (uncompressed, see bug 71044) ods contained in a Zstandard archive?

An example why it could be worth it: I am currently working with an ods file of 34 MB that recompresses with Zstandard to only 9 MB in irrelevant time.
Comment 3 David Tardon 2016-10-12 15:33:48 UTC
First, Zip is not a compression algorithm. It's a container that can contain multiple files, each of whose can be stored either compressed or uncompressed. The compression algorithm is called Deflate and it is applied to each compressed file separately.

Second, the thing about standards is that one is supposed to do what the standard says. And it says the following (ODF 1.2, part 3, section 2.1.1):

"An OpenDocument Package shall meet the following requirements:
A) It shall be a Zip file, as defined by [ZIP]. All files contained in the Zip file shall be non compressed (STORED) or compressed using the “deflate” (DEFLATED) algorithm."
[more requirements follow]

Anything else IS NOT an ODF document. And we definitely don't want to extend the collection of formats for export by another one which only newest version(s) of libreoffice would be able to read.