Bug 158732 - deb packages don't use the maximum force for xz compression
Summary: deb packages don't use the maximum force for xz compression
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Installation (show other bugs)
Version:
(earliest affected)
24.8.0.0 alpha0+
Hardware: All Linux (All)
: medium enhancement
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: Installer-Linux
  Show dependency treegraph
 
Reported: 2023-12-16 13:46 UTC by Jérôme
Modified: 2024-07-15 03:51 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jérôme 2023-12-16 13:46:08 UTC
From the daily build language pack LibreOfficeDev_24.8.0.0.alpha0_Linux_x86-64_deb_langpack_de.tar.gz, 7z tells that the below deb package uses the xz compression method :
------------------------------
$ 7z l libreofficedev24.8-dict-de_24.8.0.0.alpha0-1_amd64.deb
[...]
Type = xz
Physical Size = 7713188
Method = LZMA2:23 CRC64
Streams = 1
Blocks = 4
------------------------------

Next, I extracted the content of this deb package with the below command :
7z x libreofficedev24.8-dict-de_24.8.0.0.alpha0-1_amd64.deb

This extracts the below file :
data.tar

Next I compressed this tar archive with xz using several compression forces :
------------------------------
$ xz -1 --threads=1 --stdout data.tar > d-1.xz
$ xz -4 --threads=1 --stdout data.tar > d-4.xz
$ xz -5 --threads=1 --stdout data.tar > d-5.xz
$ xz -9 --threads=1 --stdout data.tar > d-9.xz
------------------------------

Looking at the size of the files, it seems the deb archive doesn't use the maximum compression force of xz :
------------------------------
$ ls -lhS *
-rw-r--r-- 1 j j  82M déc.  16 03:40 data.tar
-rw-r--r-- 1 j j  16M déc.  16 14:21 d-1.xz
-rw-r--r-- 1 j j  11M déc.  16 14:22 d-4.xz
-rw-r--r-- 1 j j 7,4M déc.  16 03:40 libreofficedev24.8-dict-de_24.8.0.0.alpha0-1_amd64.deb
-rw-r--r-- 1 j j 6,7M déc.  16 14:24 d-5.xz
-rw-r--r-- 1 j j 3,8M déc.  16 14:25 d-9.xz
------------------------------

The memory requirement on GNU/Linux in order to use LibreOffice is at least 256MB but prefered 512MB :
https://www.libreoffice.org/get-help/system-requirements/#Linux

The man page of xz tells :
"decompressing a file created with xz -9 currently requires 65 MiB of memory".

So there is no limitation to use the maximum force ("-9") for xz compression for a LibreOffice deb package. 

Users would then be able to download the deb files faster and save storage. The servers would save storage and bandwidth. As this is a single compression for many downloads, the maximum compression force for xz would provide an overall benefit.
Comment 1 Stéphane Guillou (stragu) 2024-01-04 00:23:37 UTC
Thanks Jérôme.
Do you know what difference it makes in the time needed to decompress such packages, and overall install times? Could you test that too?
Comment 2 Jérôme 2024-01-04 14:44:25 UTC
On xz decompression speed, the man page of unxz tells :
"On  the same hardware, the decompression speed is approximately a constant number of bytes of compressed data per second. In other words, the better the compression, the faster the decompression will usually be."

My xz/unxz version :
-------------
$ unxz --version
xz (XZ Utils) 5.2.2
liblzma 5.2.2
$
-------------

If you want to know a part of the overall installation process performance, we can pipe the xz decompression process to file extraction process (tar in my test). I performed the below test with the core deb archive which is the largest. Of course, I ensured only one terminal ran :
-------------
$ mkdir t
$ dpkg-deb --extract LibreOfficeDev_24.8.0.0.alpha0_Linux_x86-64_deb/DEBS/lodevbasis24.8-core_24.8.0.0.alpha0-1_amd64.deb t
$ tar cf sys-tree.tar t
$ xz -9 --threads=1 --stdout sys-tree.tar > sys-tree-9.tar.xz
$ xz -1 --threads=1 --stdout sys-tree.tar > sys-tree-1.tar.xz
$ rm -rf t && mkdir t
$ time ( unxz --to-stdout sys-tree-1.tar.xz | tar xf - --directory t )

real	0m5,969s
user	0m5,992s
sys	0m0,528s
$ rm -rf t && mkdir t
$ time ( unxz --to-stdout sys-tree-9.tar.xz | tar xf - --directory t )

real	0m5,930s
user	0m5,360s
sys	0m0,588s
$
$ rm -rf t && mkdir t
$ time ( unxz --to-stdout sys-tree-1.tar.xz | tar xf - --directory t )

real	0m6,093s
user	0m6,004s
sys	0m0,560s
$ rm -rf t && mkdir t
$ time ( unxz --to-stdout sys-tree-9.tar.xz | tar xf - --directory t )

real	0m5,905s
user	0m5,368s
sys	0m0,624s
$ 
-------------

On my hardware, the core deb archive that has been compressed with the '-9' force parameter decompresses slightly faster than the archive that has been compressed with the '-1' force parameter.
Comment 3 Jamie Natali 2024-07-12 09:52:36 UTC Comment hidden (spam)
Comment 4 Stéphane Guillou (stragu) 2024-07-15 03:51:43 UTC
Thanks Jérôme. From what you said, I think it makes sense.
Cloph, is that an easy switch in packaging config?