Bug 99723 - Setting image Compression in PDF export does not result in smaller file size
Summary: Setting image Compression in PDF export does not result in smaller file size
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Printing and PDF export (show other bugs)
Version:
(earliest affected)
5.2.0.0.alpha1
Hardware: All All
: high major
Assignee: Jan Holesovsky
URL:
Whiteboard: target:5.3.0 target:5.2.4
Keywords: bibisected, bisected, regression
: 100890 101474 104401 (view as bug list)
Depends on:
Blocks:
 
Reported: 2016-05-07 21:14 UTC by Cor Nouws
Modified: 2017-02-17 03:49 UTC (History)
17 users (show)

See Also:
Crash report or crash signature:


Attachments
writer file with one image (310.36 KB, application/vnd.oasis.opendocument.text)
2016-05-07 21:15 UTC, Cor Nouws
Details
bibisect details in daily Linux dbgutil bibisect repository (1.95 KB, text/plain)
2016-05-08 20:37 UTC, Terrence Enger
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Cor Nouws 2016-05-07 21:14:26 UTC
Take a writer document with an image
Do export as PDF with various settings for Compression.
I did 50%, 70% and 90%.

File size in 5.0.6.3: 84,1 / 121,2 / 260,7 kB

File size in daily*: 294,5 / 294,5 / 294,5 kB

Will attach the test file.

Found Version: 5.2.0.0.alpha1+
Build ID: 52871b360c73efd59bfbc811b8b89a02b6375b29
CPU Threads: 2; OS Version: Linux 4.4; UI Render: default; 
TinderBox: Linux-rpm_deb-x86@71-TDF, Branch:master, Time: 2016-05-07_00:52:43
Locale: nl-NL (nl_NL.UTF-8)
Comment 1 Cor Nouws 2016-05-07 21:15:41 UTC
Created attachment 124900 [details]
writer file with one image
Comment 2 Aron Budea 2016-05-07 21:38:14 UTC
Confirmed on Windows. Works in 5.0.6.3, doesn't work in master build.
Comment 3 Terrence Enger 2016-05-08 20:37:19 UTC
Created attachment 124917 [details]
bibisect details in daily Linux dbgutil bibisect repository

Working in the daily Linux dbgutil bibisect repository, I have
narrowed introduction of the problem down ...

    what       commit   date        s-h
    ---------  -------  ----------  -------
    last good  3a67f57  2016-02-09  5381db9
    first bad  4805517  2016-02-09  76ec54e

    5381db9..76ec54e = 68 commits inclusive

The following jumps out at me ...

    commit 76ec54e8c9f3580450bca85236a4f5af0c328588
    Author: Michael Meeks <michael.meeks@collabora.com>
    Date:   Mon Feb 8 14:24:15 2016 +0000

        tdf#97662 - Try to preserve original compressed JPEGs harder.
    
        Avoiding de-compressing and re-compressing them saves lots of time too.
    
        Change-Id: Ie8eb68554627581b2f0584a55bbbdb43c9482bed
        Reviewed-on: https://gerrit.libreoffice.org/22219
        Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
        Tested-by: Michael Meeks <michael.meeks@collabora.com>

I am removing keyword bibisectRequest, adding bibisected.
Comment 4 Cor Nouws 2016-05-08 20:50:39 UTC
Thanks for testing, Terrence!

@Michael: can you please have a look?
Comment 5 Michael Meeks 2016-05-09 11:21:16 UTC
Interesting; so - of course, if the image is already highly compressed - then the intention is to avoid time bothering to re-compress it =)

Pictures/10000000000004B000000640E7179047B723756F.jpg JPEG 1200x1600 1200x1600+0+0 8-bit sRGB 293KB 0.060u 0:00.049

ImageMagic's identify says of it:

 Compression: JPEG
  Quality: 90

So - perhaps there is some issue with an inverting the meaning of quality setting - as in 90% quality vs. 10% compression or somesuch (?).
Comment 6 Cor Nouws 2016-05-09 11:52:43 UTC
(In reply to Michael Meeks from comment #5)

> So - perhaps there is some issue with an inverting the meaning of quality
> setting - as in 90% quality vs. 10% compression or somesuch (?).

Indeed, the wording "JPEG compression" doesn't sound convincing here.
But from user perspective, setting an image to 50% quality should result in a smaller file then when setting to 90%, I guess?
That is what it did anyway ;)
Comment 7 SiNONiMiTY 2016-05-11 08:47:04 UTC
Bumping this thread,

Same problem encountered in LibreOffice Draw 5.1.2.2 x64
Export to PDF Compression Parameters are not honored.

Inserted a PNG Image with a Compression Level of 9 - No changes in filesize
when exporting with different compression parameters.

However, when compressing the image via the context menu:
Right Click on Image >> Compress Image, the image is compressed
Comment 8 SiNONiMiTY 2016-05-11 09:09:24 UTC
UPDATE:

Inserted a PNG Image with a Compression Level of 0:
Lossless Compression - 4MB
JPEG Compression @ 50% - 4MB


This puzzles me, as using the same parameters above in compressing the image via the context menu produces a different result.
Comment 9 Cor Nouws 2016-07-13 08:53:48 UTC
*** Bug 100890 has been marked as a duplicate of this bug. ***
Comment 10 Cor Nouws 2016-07-29 13:25:44 UTC
(In reply to Cor Nouws from comment #6)
> (In reply to Michael Meeks from comment #5)
> 
> > So - perhaps there is some issue with an inverting the meaning of quality
> > setting - as in 90% quality vs. 10% compression or somesuch (?).
> 
> Indeed, the wording "JPEG compression" doesn't sound convincing here.

And not only to us ;)
Thanks @Caolán for https://cgit.freedesktop.org/libreoffice/core/commit/?id=3ce5a97f28780983cbaf57d65622df7766abb741
Comment 11 Cor Nouws 2016-07-29 13:30:30 UTC Comment hidden (obsolete)
Comment 12 Cor Nouws 2016-07-29 13:35:53 UTC Comment hidden (obsolete)
Comment 13 Aron Budea 2016-08-11 19:28:29 UTC
*** Bug 101458 has been marked as a duplicate of this bug. ***
Comment 14 Cor Nouws 2016-08-13 14:09:03 UTC
*** Bug 101474 has been marked as a duplicate of this bug. ***
Comment 15 Cor Nouws 2016-08-25 13:17:06 UTC
*** Bug 101563 has been marked as a duplicate of this bug. ***
Comment 16 Julien Nabet 2016-08-25 13:32:14 UTC
(In reply to Cor Nouws from comment #15)
> *** Bug 101563 has been marked as a duplicate of this bug. ***

Is it really a dup? tdf#101563 indicates "Export to PDF using lossless export". Anyway, I suppose both must be related and may concern same code area.
Comment 17 Paddy Landau 2016-08-25 17:34:06 UTC
@Julien, @Cor — This is not a duplicate of 101563. Please see my comment #7 on bug 101563 for reasoning.
Comment 18 Xisco Faulí 2016-09-15 08:54:05 UTC
Adding keyword bisected as the commit has been identified
Comment 19 Xisco Faulí 2016-09-26 15:29:10 UTC
Adding Cc: to Michael Meeks
Comment 20 Rico Tzschichholz 2016-10-21 13:04:02 UTC
This still problem is still present in 5.2.3.1
Comment 21 Cor Nouws 2016-10-21 14:11:40 UTC
bumping up priority for better visibility
Comment 22 Julien Nabet 2016-10-22 15:51:26 UTC
Michael: I noticed that if I add:
mbGroupIgnoreGDIMtfActions = false;
just after having called rOutDevData.HasAdequateCompression(...)
compression works.
(see http://opengrok.libreoffice.org/xref/core/vcl/source/gdi/pdfextoutdevdata.cxx#420)
Comment 23 Julien Nabet 2016-10-22 17:16:05 UTC
Here's gdb trace with the file attached + 70% compression required

nCurrentRatio=2621

loop for:
1) mnCompressionQuality=70 rRatio.mnQuality=100
=> nTargetRatio=400
2) mnCompressionQuality=70 rRatio.mnQuality=95
=> nTargetRatio=700
3) mnCompressionQuality=70 rRatio.mnQuality=90
=> nTargetRatio=1000
4) mnCompressionQuality=70 rRatio.mnQuality=85
=> nTargetRatio=1200
5) mnCompressionQuality=70 rRatio.mnQuality=80
=> nTargetRatio=1500
6) mnCompressionQuality=70 rRatio.mnQuality=75
=> nTargetRatio=1700
Comment 24 Julien Nabet 2016-10-22 18:13:32 UTC
I gave a try with this https://gerrit.libreoffice.org/#/c/30163/1, compress is working with it but not sure if it's the good way to fix this.

Don't hesitate to comment the patch.
Comment 25 Commit Notification 2016-10-25 11:30:30 UTC
Julien Nabet committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=88fb9d8f0aae0030fac75156f78818affae4298f

tdf#99723: target ratio must be reached

It will be available in 5.3.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 26 Julien Nabet 2016-10-25 11:54:52 UTC
Patch backported in 5.2 in review on Gerrit:
https://gerrit.libreoffice.org/#/c/30265/
Comment 27 Cor Nouws 2016-10-27 09:59:37 UTC
thanks Julian.

Works OK in Version: 5.3.0.0.alpha1+
Build ID: 9c34797cc98030614b384b6ea0f233d59ae82253
CPU Threads: 4; OS Version: Linux 4.8; UI Render: default; VCL: gtk2; 
TinderBox: Linux-rpm_deb-x86_64@70-TDF, Branch:master, Time: 2016-10-26_03:59:14
Locale: nl-NL (nl_NL.UTF-8); Calc: group
Comment 28 Michael Meeks 2016-10-27 11:03:16 UTC
Julien - did you see any instances where we don't do the (very expensive and extremely pointless) de-compress and re-compress with this patch ? I guess we really need a unit test to help make following this and stopping the optimization from regressing easier. Any chance of one ? =)

Thanks !
Comment 29 Commit Notification 2016-10-27 11:04:02 UTC
Julien Nabet committed a patch related to this issue.
It has been pushed to "libreoffice-5-2":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=873929d14aa4013eb3dac216db9b828ff01e8d8f&h=libreoffice-5-2

tdf#99723: target ratio must be reached

It will be available in 5.2.4.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 30 Julien Nabet 2016-10-27 11:29:47 UTC
(In reply to Michael Meeks from comment #28)
> Julien - did you see any instances where we don't do the (very expensive and
> extremely pointless) de-compress and re-compress with this patch ? I guess
> we really need a unit test to help make following this and stopping the
> optimization from regressing easier. Any chance of one ? =)
> 
> Thanks !

I haven't tested a case which wouldn't do the de-compress and re-compress.
About unit test, except a very simple test I had create for Basic part, I don't know how to do it.
Anyway, thank you for your review on the gerrit patch 5.2!
Comment 31 Terrence Enger 2016-10-27 15:53:30 UTC
My testing shows that the entered compression ratio started to change
the output file size between commit cec94a4 (2016-10-14) and commit
8165e30 (2016-10-25).  Some resulting file sizes ...

    terry@lynn-stretch:~$ lsla /tmp/bug_099723/
    total 2572
    -rw-r--r-- 1 terry terry  26976 Oct 26 22:43 master_8165e30_010.pdf
    -rw-r--r-- 1 terry terry  43026 Oct 26 22:42 master_8175e30_020.pdf
    -rw-r--r-- 1 terry terry  58265 Oct 26 22:42 master_8175e30_030.pdf
    -rw-r--r-- 1 terry terry  71677 Oct 26 22:41 master_8175e30_040.pdf
    -rw-r--r-- 1 terry terry  84041 Oct 26 22:31 master_8175e30_050.pdf
    -rw-r--r-- 1 terry terry 100771 Oct 26 22:39 master_8175e30_060.pdf
    -rw-r--r-- 1 terry terry 121144 Oct 26 22:38 master_8175e30_070.pdf
    -rw-r--r-- 1 terry terry 294411 Oct 26 22:31 master 8175e30_080.pdf
    -rw-r--r-- 1 terry terry 294411 Oct 26 22:30 master_8175e30_090.pdf
    -rw-r--r-- 1 terry terry 294411 Oct 26 22:29 master_8175e30_100.pdf
    -rw-r--r-- 1 terry terry 294411 Oct 25 20:46 master_cec94a4_050.pdf
    -rw-r--r-- 1 terry terry 294411 Oct 25 20:46 master_cec94a4_080.pdf
    -rw-r--r-- 1 terry terry 294411 Oct 25 20:45 master_cec94a4_090.pdf
    -rwxrwxrwx 1 terry terry 294411 Oct 25 20:43 master_cec94a4_100.pdf

I collected these numbers with local-built LibreOffice, configured ...
    CC=ccache /usr/bin/gcc
    CXX=ccache /usr/bin/g++
    --enable-option-checking=fatal
    --enable-dbgutil
    --enable-debug
    --without-system-postgresql
    --without-myspell-dicts
    --with-extra-buildid
    --without-doxygen
    --with-external-tar=/home/terry/lo_hacking/git/src
built and running on debian-stretch.
Comment 32 Michael Meeks 2016-10-27 16:02:11 UTC
Hi Terrence; if you have a large, and heavily compressed JPEG image - with an already low quality - then changing the quality setting down to near the quality that image is compressed with should have no impact on file-size, and the export should be noticably faster. If that's not the case then we just broke the optimization.
Comment 33 Jean-Baptiste Faure 2016-10-29 16:24:26 UTC
Verified fixed in LO 5.2.4.0+ built at home undere Ubuntu 16.04 x86-64.
Thank you very much.

Best regards. JBF
Comment 34 Xisco Faulí 2016-11-14 09:58:05 UTC
*** Bug 101563 has been marked as a duplicate of this bug. ***
Comment 35 Paddy Landau 2016-11-14 10:23:45 UTC
Removing bug 101563, which is for something different.
Comment 36 Danny 2016-11-28 18:51:21 UTC
Removed bug 101458 since I confirmed the patch did not fix the re-compression of pngs on 5.3beta1
Comment 37 Aron Budea 2016-12-07 01:31:10 UTC
*** Bug 104401 has been marked as a duplicate of this bug. ***
Comment 38 Paddy Landau 2016-12-11 15:03:52 UTC
*** Bug 104479 has been marked as a duplicate of this bug. ***
Comment 39 Steve Edmonds 2016-12-11 18:25:47 UTC
I have un-marked Bug 104479 as being marked as a duplicate of this bug for the reason Changing the compression in my instance does reduce PDF size.