Bug 44135 - Some PDF viewers are unable to render shaded objects from Libreoffice with transparency when exported as PDF (writer and draw verified)
Summary: Some PDF viewers are unable to render shaded objects from Libreoffice with tr...
Status: RESOLVED NOTOURBUG
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Printing and PDF export (show other bugs)
Version:
(earliest affected)
3.5.6.2 release
Hardware: x86 (IA32) Linux (All)
: medium major
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
: 40143 71104 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-12-24 21:02 UTC by Gabriel Diosan
Modified: 2014-05-28 19:37 UTC (History)
14 users (show)

See Also:
Crash report or crash signature:


Attachments
PDF showing transparency bug in Libreoffice Draw (4.15 KB, application/pdf)
2011-12-24 21:02 UTC, Gabriel Diosan
Details
transparency and colors lost in PDF export (13.82 KB, application/vnd.oasis.opendocument.presentation)
2012-01-19 12:20 UTC, markusN
Details
transparency and colors lost in PDF export (32.94 KB, text/pdf)
2012-01-19 12:21 UTC, markusN
Details
screenshot of transparency and colors lost in PDF export (183.51 KB, image/png)
2012-01-19 12:23 UTC, markusN
Details
File exported to PDF in OpenOffice 3.2.1, colors ok (18.20 KB, text/pdf)
2012-01-24 01:53 UTC, markusN
Details
transparency and colors lost in PDF export (LO 3.5.0) (32.94 KB, text/pdf)
2012-02-23 12:30 UTC, markusN
Details
Colours lost in PDF export (12.73 KB, application/vnd.oasis.opendocument.graphics)
2012-02-24 19:34 UTC, Gabriel Diosan
Details
Exported PDF file opened in three viewers (124.06 KB, image/png)
2012-02-27 10:11 UTC, Ivan Timofeev (retired)
Details
Simple ODT showing the grayscale output in place of colour (9.04 KB, application/vnd.oasis.opendocument.text)
2012-07-04 17:45 UTC, jwoithe
Details
Exported PDF mentioned in comment 20 (8.32 KB, application/pdf)
2012-07-04 17:47 UTC, jwoithe
Details
export2pdf_transparency_colors_PDFStreams.zip (87.98 KB, application/zip)
2012-12-10 17:26 UTC, stfhell
Details
Minimal ODT test case to illustrate this bug (7.95 KB, application/vnd.oasis.opendocument.text)
2013-02-18 03:52 UTC, jwoithe
Details
Minimal ODT test case to illustrate this bug (OO 3.2.1 PDF export) (1.51 KB, application/pdf)
2013-02-18 03:53 UTC, jwoithe
Details
Minimal ODT test case to illustrate this bug (LO 4.0.0 PDF export) (1.84 KB, application/pdf)
2013-02-18 03:55 UTC, jwoithe
Details
SVG produced from OO 3.2.1 PDF export (attachment 75024) (1.40 KB, image/svg+xml)
2013-05-11 08:10 UTC, jwoithe
Details
SVG produced from LO 4.0.0 PDF export (attachment 75026) (2.06 KB, image/svg+xml)
2013-05-11 08:11 UTC, jwoithe
Details
PDF object dump of OO 3.2.1 PDF export (attachement 75024) (1.75 KB, application/pdf)
2013-05-11 08:21 UTC, jwoithe
Details
PDF object dump of LO 4.0.0 PDF export (attachement 75026) (2.22 KB, application/pdf)
2013-05-11 08:22 UTC, jwoithe
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Gabriel Diosan 2011-12-24 21:02:50 UTC
Created attachment 54805 [details]
PDF showing transparency bug in Libreoffice Draw

Libreoffice Draw seems to have a bug in which if you draw an object (square, circle etc) and add a fill colour with transparency and then export to PDF, the colour will show up in gray scale. When trying the same thing but without transparency, the colour shows up correctly.

In addition, if a hatching is used with or without transparency the colour shows up correctly in the PDF for both. It seems as if the problem is only present when exporting to PDF with solid colour with transparency.

This is tested on Kubuntu 11.04. PDF was viewed with Okular 0.12.5 on top of KDE 4.6.5. Attached PDF shows the problem, on the left is no transparency, on the right is with transparency.
Comment 1 markusN 2012-01-19 12:20:46 UTC
Created attachment 55808 [details]
transparency and colors lost in PDF export
Comment 2 markusN 2012-01-19 12:21:36 UTC
Created attachment 55809 [details]
transparency and colors lost in PDF export
Comment 3 markusN 2012-01-19 12:23:28 UTC
Created attachment 55812 [details]
screenshot of transparency and colors lost in PDF export
Comment 4 markusN 2012-01-19 12:24:17 UTC
I have the same problem on Fedora 16. Screenshot plus test file attached.
Comment 5 markusN 2012-01-19 12:28:24 UTC
Perhaps related to bug 40143?
Comment 6 markusN 2012-01-24 01:53:21 UTC
Created attachment 56068 [details]
File exported to PDF in OpenOffice 3.2.1, colors ok
Comment 7 markusN 2012-01-24 01:54:36 UTC
I have exported the same ODP file to PDF in OpenOffice 3.2.1
(on Mandriva 2010.2, 64bit), colors are shown correctly in the
same Fedora Okular viewer. Hence it seems to be a Libreoffice
problem.

Note that the PDF created by OpenOffice 3.2.1 is way smaller.
Comment 8 markusN 2012-02-23 12:30:07 UTC
Created attachment 57553 [details]
transparency and colors lost in PDF export (LO 3.5.0)
Comment 9 Petr Mladek 2012-02-24 09:10:01 UTC
Could you please attach the original draw document that you use for testing?
Comment 10 markusN 2012-02-24 10:36:49 UTC
Comment on attachment 55808 [details]
transparency and colors lost in PDF export

Original document used to make the test
Comment 11 Gabriel Diosan 2012-02-24 19:34:00 UTC
I just tested this in the final release of Libreoffice 3.5.0, same problem.

I have added an .odg file to show the problem.
Comment 12 Gabriel Diosan 2012-02-24 19:34:55 UTC
Created attachment 57616 [details]
Colours lost in PDF export
Comment 13 Ivan Timofeev (retired) 2012-02-27 10:11:43 UTC
Created attachment 57725 [details]
Exported PDF file opened in three viewers

I have opened the PDF file in Evince, Adobe Reader and Okular, only the latter displays it wrong. So, I'd say it's not our bug.
Comment 14 Ivan Timofeev (retired) 2012-02-27 10:20:26 UTC
*** Bug 40143 has been marked as a duplicate of this bug. ***
Comment 15 Ivan Timofeev (retired) 2012-02-27 11:06:50 UTC
In addition to Comment 13 
Under Windows: Foxit Reader displays it right, but STDUViewer does not. And Google Docs does not, too. So, score is 3:3... Perhaps marking this as RESOLVED was a hasty decision, let's reopen it, maybe we can work around.
Comment 16 markusN 2012-02-27 11:19:43 UTC
If it works with openoffice/okular but no longer with libreoffice/okular
it is a showstopper for users migrating from OO to LO. And just keep
OO installed for the PDF export is unfortunate...
See related attachment "File exported to PDF in OpenOffice 3.2.1, colors ok"
(which refers to the file opened in okular).
Comment 17 jwoithe 2012-06-09 00:48:31 UTC
I think this issue affects more than just draw.  I can reproduce the same effect in writer 3.5.1, 3.5.3 and 3.5.4 (and there's a chance that 3.4.3 was similarly affected but I can't conveniently confirm that right now):

1) In a new text document, create a frame

2) Set the background of the frame to a "Color" and change the transparency to 50%

3) Attempt to print the document.  The frame background will be gray.

4) Export to PDF: when using xpdf the frame background will still be gray.

5) Further experimentation seems to indicate that the gray being printed is the alpha channel (50% transparency gives a 50% gray, 25% transparency gives 25% gray).

To summarise: the bug affects more than just draw, and printing is affected in the same way that "export to PDF" is.

This is running under Slackware 13.37 with CUPS 1.4.6, printing to an HP 2605DN postscript printer.  Since libreoffice printing now uses the PDF export engine (AFAIK) it's probably not surprising that printing and PDF files show the same problem.  If there are in fact PDF viewers which render the PDF correctly (as indicated in comment #15), then CUPS (version 1.4.6 at least) should be added to the list of PDF renderers which seemingly get things wrong.  Furthermore, xpdf also has a problem with the resulting PDF file and should be added to the "gets it wrong" list.
Comment 18 Michael Meeks 2012-06-14 02:50:44 UTC
Radek - this looks embarrassing (and perhaps simple to fix? ;-) are we really rendering our alpha channel as grey ?
Comment 19 Gabriel Diosan 2012-06-14 08:24:16 UTC
I have just tested a PDF export with Libreoffice 3.5.3 in Linux Mint 13 and Evince displays the colours correctly. I will try to retest on my Kubuntu 12.04 install and report back.
Comment 20 jwoithe 2012-07-04 17:45:47 UTC
Created attachment 63837 [details]
Simple ODT showing the grayscale output in place of colour

I have created a simple ODT document with Libreoffice 3.5.4 which
illustrates the problem on my system.  It consists of a single frame (insert-frame, keep default settings) with a colour assigned to the background with 50% transparency.  I put some red text to the left of this to confirm that colour is being processed.  I then did an export to PDF without changing default options.  These tests were done under Slackware 13.37.

When viewing this in xpdf 3.02, the frame is drawn with a 50% gray.  The red 
text is red.

When viewing this with epdfview 0.1.8 the frame has the correct colour, as does the red text.

When printing this via CUPS 1.4.6 on a colour HP postscript printer set for colour output the frame is printed with a 50% gray background while the red text is shown as red.

It seems there's something about the exported PDF which isn't being interpreted correctly by some PDF readers.  The showstopper for me is that CUPS seems to be one of those which is affected by the problem, so consequently printing is too.
Comment 21 jwoithe 2012-07-04 17:47:58 UTC
Created attachment 63838 [details]
Exported PDF mentioned in comment 20

This is the PDF mentioned in comment 20, corresponding to attachment 63837 [details].
Comment 22 jwoithe 2012-08-15 03:47:25 UTC
Today I re-tested this in libreoffice 3.6.0 and can confirm that the observations I made previously (as detailed in comment 20) still apply.
Comment 23 markusN 2012-08-30 09:08:40 UTC
Still broken in Fedora's LibreOffice 3.5.6.2 RPM.

I *really* hope that the new LO 3.6.1 will fix this bug which did not
exist in OpenOffice.
Comment 24 jwoithe 2012-08-31 12:36:57 UTC
I have re-tested with Libreoffice 3.6.1 (binary distribution from the document foundation) and have confirmed that sadly the problem still exists in this version.
Comment 25 sawsiba 2012-09-12 12:03:16 UTC
This problem is there in both LibreOffice as well as Openoffice for the version above 3.2 and seems that it is intentionally done in settings tor better printing compatibility.

Can any one suggest how to change this settings as in openoffice 3.2 this problem is not there.
Comment 26 jwoithe 2012-09-13 06:39:40 UTC
(In reply to comment #25)
> This problem is there in both LibreOffice as well as Openoffice for the version above 3.2

I can now also confirm that the problem exists in Libreoffice 3.4.3.  I don't have ready access to earlier versions to test those.  Evidently though it's a problem which has existed for a while.

> ... and seems that it is intentionally done in settings tor better printing compatibility.

That is ironic, given that the printing system (CUPS 1.4.6) is one of the programs affected by this bug (refer to my earlier posts for full details).

Is there any hope that this problem is going to be addressed?  I have had several users report this bug to me after they've hit it in their day-to-day work and I'm getting sick of having to repeatedly admit that "it's a bug but no one seems to be working on it yet".
Comment 27 jwoithe 2012-11-15 02:26:50 UTC
I tested this with Libreoffice 3.6.3 and the problem still exists.
Comment 28 Rainer Bielefeld Retired 2012-12-05 11:38:46 UTC
NOT [Reproducible] with "LibreOffice 3.6.4.3 rc" German UI/ German Locale [Build-ID: 2ef5aff] {pull date 2012-11-28} on German WIN7 Home Premium (64bit). I never saw such an export problem, and all attached PDF look fine for me with ARX.

I decided to close this Bug because we never will get a fix (because the problem still is absolutely unknown).

Here we have a horrible bunch of documents, nobody can expect what they are for and under what circumstances it might be to reproduce what bug ever. Currently it seems that we discuss a PDF viewer bug?

So INVALID

@reporter:
I see that the bug still exists for you, so please report a new bug. What we need is an idiot proof description how to reproduce the problem.
May be hints on <http://wiki.documentfoundation.org/BugReport> will help you to find out what information will be useful to reproduce your problem? 
Please:
- Write a meaningful Summary describing exactly what the problem is
- Attach a test kit with sample document (not only screenshot(or refer to an 
  existing sample document in an other Bug with a link), PDF Export result.
  For all attachment write useufl reports! Not simply "colors lost in PDF 
  export", but "This document created from source.odg with PDF export
  settings xxx, yyy, zzz opene with PDFViewer.exe version aaa in xxx mode with 
  yyy setting and .... show ....., see screenshot.ong in this testkit.zip"
- Attach screenshots with comments showing how your source looks, how your 
  result looks.  Best way is to insert your screenshots
  into a DRAW document and to add comments that explain what you want to show
  (attachment 68877 [details], attachment 68490 [details]).
- Contribute a document related step by step instruction containing every 
  key press and every mouse click how to reproduce your problem 
  (similar to example in Bug 43431)
– if possible contribute an instruction how to create a sample document 
  from the scratch
- add information 
  -- Concerning any relevant detail. For example all details of your PDF viewer
     areimportant, how the PDF looks in  DRAW, what the PDF viewer settings are,
     with what PDF settings you exported, ...
  -- concerning your PC (video card, ...)
  -- concerning your OS (Version, Distribution, Language)
  -- concerning your LibO version (with Build ID if it's not a public release)
     and localization (UI language, Locale setting)
  –- Libo settings that might be related to your problems 
    (video hardware acceleration, Experimental features enabled, ...)
  -- how you launch LibO and how you opened the sample document
  –- Whether your problem persists when you renamed your user profile 
     before you launch LibO (please see
     <https://wiki.documentfoundation.org/UserProfile#User_profile_location>)
  –- If you can contribute an AOOo Issue that might be useful
  -- everything else crossing your mind after you read linked texts
Comment 29 jwoithe 2012-12-05 13:31:51 UTC
(In reply to comment #28)
> NOT [Reproducible] with "LibreOffice 3.6.4.3 rc" German UI/ German Locale
> [Build-ID: 2ef5aff] {pull date 2012-11-28} on German WIN7 Home Premium
> (64bit). I never saw such an export problem, and all attached PDF look fine
> for me with ARX.

With respect, closing this as invalid is the height of arrogance.
 
> I decided to close this Bug because we never will get a fix (because the
> problem still is absolutely unknown).

I completely disagree.  During the time since this bug was first reported, a very good picture has been built up regarding the nature of the bug.  While the source of the bug is unknown (obviously, since it hasn't been fixed) the basic problem is very much known.

> Here we have a horrible bunch of documents, nobody can expect what they are
> for and under what circumstances it might be to reproduce what bug ever.

This is absolute rubbish.  If you care to read through and follow the comments (of where there aren't all that many) you will find that this "horrible bunch of documents" is in fact a rather succinct collection which illustrates the bug.

> Currently it seems that we discuss a PDF viewer bug?

No.  There is clearly a problem with the PDF output from Libreoffice which causes some PDF rasterisers to output an incorrect image.  If you care to glance at comments 7, 13, 15, 17 and 20 you will see that there's a 50/50 split between PDF rasterisers which work and those which don't.  To save you the hassle: xpdf, CUPS, Okular, STDUViewer and google docs get it wrong.  Adobe reader, Evince and Foxit get it right.  According to yourself, ARX (whatever that is) can be added to the "gets it right" camp.

> I see that the bug still exists for you, so please report a new bug. What we
> need is an idiot proof description how to reproduce the problem.

I fail to see why we need to open a new report for this.  You already have your "idiot proof description" of how to reproduce the problem: see comment 17.

> - Write a meaningful Summary describing exactly what the problem is

That has been done.  The initial description does a pretty good job.  Comment 17 also describes it in a fair amount of detail.  What more do you want?

> - Attach a test kit with sample document (not only screenshot(or refer to an 
>   existing sample document in an other Bug with a link), PDF Export result.

You've got that: attached as part of comment 20.

>   For all attachment write useufl reports! Not simply "colors lost in PDF 
>   export", but "This document created from source.odg with PDF export
>   settings xxx, yyy, zzz opene with PDFViewer.exe version aaa in xxx mode with 
>   yyy setting and .... show ....., see screenshot.ong in this testkit.zip"

In combination with the comments provided when attached, I don't see that any further descriptions are necessary beyond that which has been provided.

> - Attach screenshots with comments showing how your source looks, how your 
>   result looks.  Best way is to insert your screenshots
>   into a DRAW document and to add comments that explain what you want to show
>   (attachment 68877 [details], attachment 68490 [details]).

What part of "the colour renders as greyscale" is unclear?

> - Contribute a document related step by step instruction containing every 
>   key press and every mouse click how to reproduce your problem 

This was pretty much done in as much detail as necessary in comment 20.

> – if possible contribute an instruction how to create a sample document 
>   from the scratch

Comment 20 again.

> - add information 
>      with what PDF settings you exported, ...

It was already reported that they were the defaults.

>   -- concerning your PC (video card, ...)

It's got nothing to do with the video card.  It's been clearly demonstrated that non-video rasterisers are affected.

>   -- concerning your OS (Version, Distribution, Language)

This was clearly stated too: in my case it was Linux, Slackware distribution version 13.37.  Other reporters have included this information too.

>   -- concerning your LibO version ...

This has already been done where relevant.

In short, I am astounded that this ticket can be closed as invalid.  I have already spent a considerable amount of time testing various versions to confirm the presence of this bug (as have others).  I have documented the steps to reproduce the problem, I have provided an ODT file which, when exported to PDF, illustrates the problem (attachment 63837 [details]).  I have provided the PDF as exported so others can test it in their own PDF viewers (63838).  Honestly, what more can we possibly do?  Everything that you asked for which is relevant to this problem has been done.

I am surprised that this bug has remained unattended for over a year now.  However, for me it is a showstopper for reasons explained in the comments and I will continue testing and reporting until we see some positive movement towards solving the problem.  Others are finding the same problems.  It is creating problems for me when I try to encourage people to use libreoffice and I would dearly like the bug fixed.

The bug is real, it's affecting multiple people, and I am sure that none of us have the time to recreate yet another ticket which simply recreates the information that's already been entered in here.  I am going to reopen it because it is completely mindblowing that this existing ticket is deemed to be insufficient.  If it ends up being closed again citing comment 28 then the take-away message to me is that Libreoffice really doesn't care to resolve bugs like this even when users have taken considerable effort to describe the problem.  It will make it very difficult for me to maintain my enthusiasm for the program.  This is really a very poor way to treat bug reporters and is a very poor example of what FOSS is all about.  I have filed a considerable number of bug reports for various FOSS projects over a period spanning more than a decade and until now I have never encountered such arrogance towards bug reporters and the efforts they have gone to as is communicated in comment 28.
Comment 30 Leonardo Giordani 2012-12-05 13:38:43 UTC
I totally agree with the above comment 29. The bug is there, and closing our eyes doesn't make it disappear, neither makes the software better. I think ARX is Adobe Reader X, which is surely a good PDF reader but not the only one as stated by jwoithe. Please reconsider your decision.
Comment 31 markusN 2012-12-05 19:23:33 UTC
(In reply to comment #28)
> I decided to close this Bug because we never will get a fix (because the
> problem still is absolutely unknown).
> 
> Here we have a horrible bunch of documents, 

This is a nonsense. Please take the time to count:

3 Open Documents
5 PDF as proofs
1 PNG as proof

> nobody can expect what they are
> for and under what circumstances it might be to reproduce what bug ever.

Then just read the related comments attached to the
uploaded documents.

> Currently it seems that we discuss a PDF viewer bug?

Unfortunately not. Various viewers fail. The uploaded
test documents produce proper PDFs in OpenOffice as
stated above. Only LibreOffice fails to turn the test
document into a proper PDF file.

> So INVALID

Luckily reopened. I agree with other reporters here that this
level of arrogance is a bit disturbing.

> @reporter:
> I see that the bug still exists for you, so please report a new bug. What we
> need is an idiot proof description how to reproduce the problem.

Cough, see comments above.

This problem remains a showstopper for LibreOffice.
Comment 32 Michael Meeks 2012-12-07 14:41:38 UTC
> > So INVALID
>
> Luckily reopened. I agree with other reporters here that this
> level of arrogance is a bit disturbing.

Ho hum - the complaints:useful-work ratio is a bit disturbing here.

It seems the problem is that modern & capable PDF renderers can render the generated PDF acceptably; older and/or more broken ones have problems - AFAICs.

This may well not be a bug in LibreOffice - the question is: can we do much about it - can we detect and generate PDF differently ? What is the difference in the PDF.

Anyone wanting to complain would do better to dig through the PDF spec. and find a good way to pretty-print that PDF, and then work out which commands are different in it.

I had a go using a recent xpdf-poplar; interestingly the raw X backend of this fails to render the PDF, but the cairo / SVG output renders it fine: as does the evince viewer.

Using the pdf2svg utility - on before + after and diffing them, it seems like we have:

 <clipPath id="clip1">
-  <path d="M 0 0 L 793.800781 0 L 793.800781 595 L 0 595 Z "/>
-</clipPath>
-<clipPath id="clip2">
-  <path d="M 230.398438 208.699219 L 540 208.699219 L 540 403.300781 L 230.398438 403.300781 Z "/>
-</clipPath>
-<clipPath id="clip3">
-  <path d="M 129.601562 72 L 266.398438 72 L 266.398438 151.199219 L 129.601562 151.199219 Z "/>
+  <path d="M 0 0 L 793.601562 0 L 793.601562 595 L 0 595 Z "/>
 </clipPath>
+<filter id="alpha" filterUnits="objectBoundingBox" x="0%" y="0%" width="100%" height="100%">
+  <feColorMatrix type="matrix" in="SourceGraphic" values="0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0"/>
+</filter>
+<mask id="mask0">
+  <g filter="url(#alpha)">
+<rect x="0" y="0" width="794" height="595" style="fill:rgb(0%,0%,0%);fill-opacity:0.7;stroke:none;"/>
+  </g>
+</mask>
+<image id="image6" width="311" height="196" xlink:href="data:image/png;base64,... <snip huge base64 image >
/>
+<mask id="mask1">
+  <g filter="url(#alpha)">
+<rect x="0" y="0" width="794" height="595" style="fill:rgb(0%,0%,0%);fill-opacity:0.5;stroke:none;"/>
+  </g>
+</mask>
+<image id="image10" width="138" height="81" xlink:href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAIoAAABRCAYAAAANfj6IAAAABmJLR0QA/wD/AP+gvaeTAAAA+0lEQVR4nO3SsQ3CUBTAwBdETcWMZCYWpIINkMIK7n6Q7iZw4RkIttcxj5l5rg7h1PbL6gL+g1FIjEJiFBKjkBiFxCgkRiExColRSIxCYhQSo5AYhcQoJEYhMQqJUUiMQmIUEqOQGIXEKCRGITEKiVFIjEJiFBKjkBiFxCgkRiExColRSIxCYhQSo5AYhcQoJEYhMQqJUUiMQmIUEqOQGIXEKCRGITEKiVFIjEJiFBKjkBiFxCgkRiExColRSIxCYhQSo5AYhcQoJEYhMQqJUUiMQmIUEqOQGIXEKCRGITEKiVFIjEJiFBKjkBiFZPscc/vO3FeHcF7XmfcPhSYH5NAXBk0AAAAASUVORK5CYII="/>
...
-<g style="fill:rgb(0%,0%,0%);fill-opacity:1;">
+<g style="fill:rgb(12.155151%,10.978699%,10.586548%);fill-opacity:1;">
   <use xlink:href="#glyph0-1" x="39.7" y="554.8"/>
 </g>
-<g style="fill:rgb(0%,0%,0%);fill-opacity:1;">
+<g style="fill:rgb(12.155151%,10.978699%,10.586548%);fill-opacity:1;">
   <use xlink:href="#glyph0-1" x="395.5" y="554.8"/>
 </g>
 <path style="fill-rule:evenodd;fill:rgb(81.175232%,90.586853%,89.802551%);fill-opacity:1;stroke-width:0.12;stroke-linecap:butt;stroke-linejoin:round;stroke:rgb(50.195312%,50.195312%,50.195312%);stroke-opacity:1;stroke-miterlimit:10;" d="M 367.199219 292.601562 L 165.601562 292.601562 L 165.601562 559 L 568.800781 559 L 568.800781 292.601562 Z " transform="matrix(1,0,0,-1,0,595)"/>
-<g style="fill:rgb(0%,0%,0%);fill-opacity:1;">
-  <use xlink:href="#glyph1-1" x="328.1" y="175.5"/>
-  <use xlink:href="#glyph1-2" x="338.108" y="175.5"/>
-  <use xlink:href="#glyph1-3" x="342.104" y="175.5"/>
-  <use xlink:href="#glyph1-4" x="352.112" y="175.5"/>
-</g>
-<g style="fill:rgb(0%,0%,0%);fill-opacity:1;">
-  <use xlink:href="#glyph1-5" x="362.192" y="175.5"/>
-</g>
-<g style="fill:rgb(0%,0%,0%);fill-opacity:1;">
-  <use xlink:href="#glyph1-6" x="367.268" y="175.5"/>
-  <use xlink:href="#glyph1-7" x="372.254" y="175.5"/>
-  <use xlink:href="#glyph1-8" x="382.262" y="175.5"/>
+<g style="fill:rgb(12.155151%,10.978699%,10.586548%);fill-opacity:1;">
+  <use xlink:href="#glyph1-1" x="328" y="175.5"/>
+  <use xlink:href="#glyph1-2" x="338.008" y="175.5"/>
+</g>
+<g style="fill:rgb(12.155151%,10.978699%,10.586548%);fill-opacity:1;">
+  <use xlink:href="#glyph1-3" x="342.094" y="175.5"/>
 </g>
etc.

which (I hope) maps reasonably well to the actual PDF generated - certainly the later PDF is larger which presumably reflects this alpha-mask / image.

So - anyhow - this is a problem that an intelligent user can solve with only a few hours of research I expect: and here is the question:

What is the difference in the PDF - and what are the human-readable PDF elements that changed between the versions ?
Comment 33 Michael Meeks 2012-12-07 14:42:51 UTC
I'd also love an accurate list of PDF viewers and their versions that are and are not affected on whatever platforms. Thanks.
Comment 34 jwoithe 2012-12-08 01:09:16 UTC
(In reply to comment #33)
> I'd also love an accurate list of PDF viewers and their versions that are
> and are not affected on whatever platforms. Thanks.

This will require a collaborative effort from the reporters still listening on on this ticket since I do not have easy access to all the viewers.  Here's a summary of what I can tell you at this time.  The platform in all cases is Linux.  "Fail" means the semi-transparent object is shown as gray, "ok" means correct rendering.

CUPS 1.4.6, slackware 13.37: fail
CUPS 1.4.3, slackware 13.1: fail

xpdf 3.02pl6, slackware 13.37 (not linked to poppler): fail
xpdf 3.02pl4, slackware 13.1 (not linked to poppler): fail

epdfview 0.1.8, slackware 13.37 (linked to poppler 0.16.4): ok

gv 3.7.1 with ghostscript 9.00, slackware 13.37: error - no result
 - Error: /rangecheck in --.discardtransparencygroup
 - this would be a separate issue I think

The result from CUPS was determined from the result when printed to a colour printer (two different colour printers were tested).  Objects without transparency printed in colour; only the one with transparency was gray.

Hopefully others can add details of the other viewers reported in previous comments.
Comment 35 markusN 2012-12-08 11:15:04 UTC
FWIW, also Slideshare.net is unable to get it right:

http://www.slideshare.net/markusN/grass-gis-e-sextante

-> first slide, generated with LibreOffice


This older presentation is right, generated with OpenOffice:
http://www.slideshare.net/markusN/from-a-niche-to-a-global-user-community-open-source-gis-and-osgeo
Comment 36 stfhell 2012-12-10 17:26:57 UTC
Created attachment 71283 [details]
export2pdf_transparency_colors_PDFStreams.zip

I'm attaching a ZIP with a readable form of the PDF streams plus screenshots from Acrobat X PDF preflights. Acrobat preflight considers both PDFs as correct.

The attached PDF stream files just omit the binary encoded streams. The stream for object 2 is added in decoded form, object streams 4 and 6 are identical in both files.

The difference in size of the PDFs will be due to the fonts that are embedded (which are different). But I think it's interesting that the "new-style" PDF export creates separate transparency groups for objects 4 and 6 (the yellow and red forms):
  /Group<</S/Transparency/CS/DeviceRGB/K true>>
That is also shown in the Acrobat screenshots.
The "old-style" PDF-export put everything in a single transparency group in object 1 and didn't define groups for objects 4 and 6.
Note that Okular only renders these 2 objects as greyscale (not the blue one).

One way to check where the problem lies (the PDF export filter or the PDF viewers) would be to check if the "greyscale PDF viewers" also have problems with multiple transparency group PDFs created by other PDF writers than LO or OO.
Comment 37 Joel Madero 2013-02-13 17:23:07 UTC
LibreOffice 3.5 has come to the EOL and is no longer being developed. Because of this we're closing the 3.5 MAB meta tracker. 

This bug is in NEEDINFO status so we are requesting more information from the reporter so removing this from MAB. 

@Gabriel - if you could provide requested info (I think comment 33) and then mark the bug as NEW again.

I don't think that this belongs on MAB. In order for a Draw bug to get on MAB it really should be almost a show stopper as Draw is one of our least used components and MAB should be affecting a lot of users. This would only impact people using very specific pdf viewers and wanting to use Draw with transparency.

If a QA member of dev disagrees, once the bug is marked as NEW again, please feel free to move this to 3.6 MAB :-D

@Gabriel - thanks for your patience and helping us track down the root of this issue
Comment 38 jwoithe 2013-02-13 22:25:32 UTC
Comment #37 raises a number of issues.  I'll try to deal with each in turn.

Firstly, this issue affects all versions of libreoffice at least since 3.4.x 
(see comment #27 for example, covering 3.6.x).  It would be wrong to close   
this just because the version it was first observed on has now gone EOL.  To 
keep it alive I will update the version number and re-mark as NEW; to me    
this distorts the history of the ticket, but if having a version number of  
3.5.x causes the bug to be assumed dead then I guess this must be done.

Regarding the NEEDINFO status, I'm really at a loss as to what other        
information I can personally provide.  I do not have the internal knowledge of PDF which seems necessary to dive into the PDF dumps provided in comment #32 and I am unsure of where to get them (the perenial issue of a lack of spare time is another problem).

Regarding the information requested in comment #33, comment #34 provides a partial answer which includes as much as I can give.  Information about other readers mentioned in earlier comments require input from those who have those readers (particularly people on platforms other than Linux).

Another problem is that despite the original title (which reflects the observation made by the original poster) it turns out the problem is seen in all components (I have personally confirmed it in swriter).  Therefore it would be wrong to make triage decisions based on the idea that the problem is unique to the draw component.  I will change this to the "printing and pdf export" component since I think it's clear that this is where the problem lies.
Comment 39 Joel Madero 2013-02-13 22:48:25 UTC
Version is the oldest version that you can see the issue, not the latest it's been tested on
Comment 40 jwoithe 2013-02-18 03:43:56 UTC
Thanks for the clarification in comment #39.

I have tested Libreoffice 4.0.0 x86 Linux en-US and can confirm that PDF files created with this version still suffer from the problem.
Comment 41 jwoithe 2013-02-18 03:52:17 UTC
Created attachment 75023 [details]
Minimal ODT test case to illustrate this bug

This is a minimal ODT file which illustrates this bug.  It was created under OpenOffice 3.2.1 (Linux, x86, en-US) and consists of one small frame with a background colour set to 50% transparency.  The frame was created using Insert|Frame, with default settings accepted.

Using this file I created two PDFs which will be attached subsequently: one using OpenOffice 3.2.1 and the other using Libreoffice 4.0.0 (Linux, x86, en-US).  The problematic PDF viewers display the PDF created from OpenOffice 3.2.1 correctly, while the one from Libreoffice 4.0.0 shows greyscale where there should be a semi-transparent red-ish rectangle.

It would be good to run these two PDFs through the previously mentioned analysers to see how they differ.  They are simpler than the files previously looked at and perhaps the important differences may be more apparent.
Comment 42 jwoithe 2013-02-18 03:53:53 UTC
Created attachment 75024 [details]
Minimal ODT test case to illustrate this bug (OO 3.2.1 PDF export)

Result of exporting attachment 75023 [details] to PDF via OpenOffice 3.2.1.  When viewed in the problematic PDF viewers, this PDF renders correctly.
Comment 43 jwoithe 2013-02-18 03:55:43 UTC
Created attachment 75026 [details]
Minimal ODT test case to illustrate this bug (LO 4.0.0 PDF export)

This is the result of exporting attachment 75023 [details] to PDF using Libreoffice 4.0.0.  This PDF file renders the 50% transparent red area as greyscale in the problematic PDF viewers.
Comment 44 jwoithe 2013-05-11 08:07:32 UTC
(In reply to comment #41)
> This is a minimal ODT file which illustrates this bug. ...

I have compiled pdf2svg from http://www.cityinthesky.co.uk/opensource/pdf2svg and used it on the two minimal test case PDFs (attachment 75024 [details] and attachment 75026 [details]). I shall attach these following this message.

A "diff -u" between these two SVGs gave the following.

diff -u transparency_test-oo-3.2.1.svg transparency_test-lo-4.0.0.svg
--- transparency_test-oo-3.2.1.svg      2013-05-11 10:20:38.404892277 +0930
+++ transparency_test-lo-4.0.0.svg      2013-05-11 10:20:47.860891890 +0930
@@ -1,17 +1,31 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="612pt" height="792pt" viewBox="0 0 612 792" version="1.1">
 <defs>
+<filter id="alpha" filterUnits="objectBoundingBox" x="0%" y="0%" width="100%" height="100%">
+  <feColorMatrix type="matrix" in="SourceGraphic" values="0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 1 0"/>
+</filter>
+<mask id="mask0">
+  <g filter="url(#alpha)">
+<rect x="0" y="0" width="612" height="792" style="fill:rgb(0%,0%,0%);fill-opacity:0.5;stroke:none;"/>
+  </g>
+</mask>
+<clipPath id="clip2">
+  <path d="M 0.601562 22.800781 L 57.5 22.800781 L 57.5 0.300781 L 0.601562 0.300781 Z M 0.601562 22.800781 "/>
+</clipPath>
 <clipPath id="clip1">
-  <path d="M 277.601562 84.800781 L 334.5 84.800781 L 334.5 62.300781 L 277.601562 62.300781 Z M 277.601562 84.800781 "/>
+  <rect width="58" height="23"/>
 </clipPath>
+<g id="surface6" clip-path="url(#clip1)">
+<g clip-path="url(#clip2)" clip-rule="nonzero">
+<path style=" stroke:none;fill-rule:evenodd;fill:rgb(72.155762%,27.842712%,0%);fill-opacity:1;" d="M 0.601562 0.398438 L 57.398438 0.398438 L 57.398438 22.699219 L 0.601562 22.699219 Z M 0.601562 0.398438 "/>
+</g>
+</g>
 </defs>
 <g id="surface1">
-<g clip-path="url(#clip1)" clip-rule="nonzero">
-<path style=" stroke:none;fill-rule:evenodd;fill:rgb(72.155762%,27.842712%,0%);fill-opacity:0.5;" d="M 277.601562 62.398438 L 334.398438 62.398438 L 334.398438 84.699219 L 277.601562 84.699219 Z M 277.601562 62.398438 "/>
-</g>
-<path style=" stroke:none;fill-rule:evenodd;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 277.601562 84.800781 L 277.699219 84.800781 L 277.699219 62.398438 L 277.601562 62.398438 Z M 277.601562 84.800781 "/>
-<path style=" stroke:none;fill-rule:evenodd;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 334.398438 84.800781 L 334.5 84.800781 L 334.5 62.398438 L 334.398438 62.398438 Z M 334.398438 84.800781 "/>
-<path style=" stroke:none;fill-rule:evenodd;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 277.601562 62.5 L 334.5 62.5 L 334.5 62.398438 L 277.601562 62.398438 Z M 277.601562 62.5 "/>
-<path style=" stroke:none;fill-rule:evenodd;fill:rgb(0%,0%,0%);fill-opacity:1;" d="M 277.601562 84.800781 L 334.5 84.800781 L 334.5 84.699219 L 277.601562 84.699219 Z M 277.601562 84.800781 "/>
+<use xlink:href="#surface6" transform="matrix(1,0,0,1,277,62)" mask="url(#mask0)"/>
+<path style="fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:1;" d="M 277.699219 729.5 L 277.699219 707.300781 " transform="matrix(1,0,0,-1,0,792)"/>
+<path style="fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:1;" d="M 334.398438 729.5 L 334.398438 707.300781 " transform="matrix(1,0,0,-1,0,792)"/>
+<path style="fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:1;" d="M 277.699219 729.5 L 334.398438 729.5 " transform="matrix(1,0,0,-1,0,792)"/>
+<path style="fill:none;stroke-width:1;stroke-linecap:butt;stroke-linejoin:miter;stroke:rgb(0%,0%,0%);stroke-opacity:1;stroke-miterlimit:1;" d="M 277.699219 707.300781 L 334.398438 707.300781 " transform="matrix(1,0,0,-1,0,792)"/>
 </g>
 </svg>

Obviously this document is much simpler than the one used for the comparison in comment 32 since it contains only the single semi-transparent frame.  Does this make it any easier to work out what's going on?

I am completely unfamiliar with both PDF and SVG formats.  To my inexperienced eyes, it seems the older file relies on the use of a path with an explicit fill opacity setting of 0.5, while the newer one defines an alpha mask / filter pair to produce the same effect.  If this is indeed the fundamental issue, the question becomes whether the use of the alpha mask is essential given that it is evidently not handled correctly by some important rasterisers like CUPS (see comment 34 for a a survey of different programs and their behaviour).
Comment 45 jwoithe 2013-05-11 08:10:54 UTC
Created attachment 79131 [details]
SVG produced from OO 3.2.1 PDF export (attachment 75024 [details])

SVG produced from attachement 75024 using svg2pdf from http://www.cityinthesky.co.uk/opensource/pdf2svg, discussed in comment 44.
Comment 46 jwoithe 2013-05-11 08:11:36 UTC
Created attachment 79132 [details]
SVG produced from LO 4.0.0 PDF export (attachment 75026 [details])

SVG produced from attachement 75026 using svg2pdf from
http://www.cityinthesky.co.uk/opensource/pdf2svg, discussed in comment 44.
Comment 47 jwoithe 2013-05-11 08:20:35 UTC
(In reply to comment #44)

I have found a PDF parser called pdf-parser.py at http://blog.didierstevens.com/programs/pdf-tools/.  This produces a human readable description of objects found in a PDF file.  I shall attach the resulting dumps from attachment 75024 [details] and attachment 75026 [details] in case it is useful for someone.  To me it seems to confirm the conclusion reached in comment 44 but I'm ready to stand corrected by someone with more PDF format experience than I.
Comment 48 jwoithe 2013-05-11 08:21:24 UTC
Created attachment 79133 [details]
PDF object dump of OO 3.2.1 PDF export (attachement 75024)

Objects found in OO 3.2.1 PDF export (attachment 75024 [details]) as reported by
pdf-parser.py (http://blog.didierstevens.com/programs/pdf-tools/) and
discussed in comment 47.
Comment 49 jwoithe 2013-05-11 08:22:16 UTC
Created attachment 79134 [details]
PDF object dump of LO 4.0.0 PDF export (attachement 75026)

Objects found in LO 4.0.0 PDF export (attachment 75026 [details]) as reported by
pdf-parser.py (http://blog.didierstevens.com/programs/pdf-tools/) and
discussed in comment 47.
Comment 50 Michael Meeks 2013-05-11 09:19:56 UTC
jwoithe: thanks for your excellent work here ! comment#44 looks extremely plausible - I believe we changed how this rendering works (though I'm no expert) to use the drawinglayer; but ...

So - to some degree it's a "not our bug" - but on the other hand, producing PDF that CUPS cannot print, particularly when the advantage of PDF over PS is ultimately transparency related ;-) seems a bit sad.

Perhaps Thorsten has an idea here ... might also be worth a duplicate hunt here, I believe there were several similar issues out there, and this one should be the leader :-)
Comment 51 jwoithe 2013-06-29 11:53:01 UTC
FYI, cups version 1.5.4 (as supplied in Slackware 14.0) is still affected.  That is, when printing to a colour printer from Libreoffice on a system running cups 1.5.4, the transparent rectangle in attachment 75023 [details] is still grey.  Tested with Libreoffice 4.0.2.
Comment 52 jwoithe 2013-06-29 12:18:36 UTC
Regarding the suggestion of a duplicate search in comment 50, I've done a few quick searches of the bug tracker.  Searching for "transparency" gives around 64 matches, most of which seem to be related to gradients in some way.  The closest I've seen to the symptoms discussed here are bug 33591, bug 36766 (a meta bug summarising a number of transparency-related issues mostly related to presentation mode), bug 54924 and bug 55698.  However, after reading the detail from these I'm not convinced that any could be classified as duplicates of this one.  To my inexperienced eyes, while they may be conceptually similar (in that transparency is in some way related) the symptoms really are not a good match for the ones being dealt with here.
Comment 53 jwoithe 2013-12-20 11:27:37 UTC
Further to comment 51, it seems that Libreoffice 4.1.3 can successfully print the "Minimal ODT test case to illustrate this bug" (attachment 75023 [details]) on CUPS 1.5.4 under Slackware 14.0.  As per comment 51, this same CUPS version failed when used with Libreoffice 4.0.2.  It would appear that Libreoffice 4.1.x changed something which happens to make this version of CUPS happier.

However, the issue is not resolved yet because I note that the PDF exported by Libreoffice 4.1.3 still fails to render correctly under xpdf 3.02.  Whether the correct output from CUPS is due to a targeted change in Libreoffice or is a coincidental effect of other work I cannot say.
Comment 54 tmacalp 2014-05-21 03:10:26 UTC
*** Bug 71104 has been marked as a duplicate of this bug. ***
Comment 55 jwoithe 2014-05-26 02:49:22 UTC
I can confirm that the "Minimal ODT test case to illustrate this bug (LO 4.0.0 PDF export)" (PDF file attachment 75026 [details]) renders correctly under Slackware 13.37 when running xpdf 3.03.  If xpdf 3.02 is used the rectangle renders as greyscale, as before.  Given this observation and those in comment 51 and comment 53, it seems to me that a couple of things have happened which make this ticket less severe now.  Firstly, some change to the LO PDF output occurred in LO 4.1.x which appears to have made CUPS capable of rendering the transparent objects.  Secondly, unlike xpdf 3.02, xpdf 3.03 can now correctly render colour transparency in files produced by LO.

In light of these observations I am not sure that it is worth pursuing the underlying cause of the difficulties identified in this ticket.  On the one hand the amount of software adversely affected by the issue is shrinking.  However, there is always an outside chance that the problems seen by xpdf <= 3.02 and other software referenced in this ticket are indicative of an issue in LO's PDF output which happens by chance to still render correctly in some viewers but should still be identified and addressed lest they create further problems in the future.
Comment 56 Michael Meeks 2014-05-26 11:10:51 UTC
jwoith - thanks so much for your analysis and hard work on this ticket; sorry it didn't lead to a workaround in LibreOffice, but glad that the situation is improving for most users.

Would Resolved->NotOurBug work for this ? [ of course, quite possibly it is at least partially our bug of so not entirely satisfying ], if so, any chance of closing it ? At least the details are archived here for future reference.

Thanks !
Comment 57 crxssi 2014-05-26 12:46:46 UTC
(In reply to comment #55)
> On the one hand the amount of software adversely affected by the issue is shrinking. 

While that is true, some of us are "stuck" with what is [right now] current... and RHEL 6 is still what is current in business and it is likely to never be "fixed" for this issue (because it isn't going to use a very recent CUPS).  Some people can't just "upgrade" the OS on whim because they want to.  And for them (like me), this issue has been a pain for years and might remain a pain for more years to come.

LO created the problem, it would be nice if it were fixed instead of just waiting yet more years for everyone to just be on newer PDF renderers.  Let us not forget, the problem is not just with displaying PDF's generated by LO, it can also affect just regular printing in LO on affected systems ( https://bugs.freedesktop.org/show_bug.cgi?id=71104 )

(In reply to comment #56 by  Michael Meeks]
> Would Resolved->NotOurBug work for this ?

It would work to ignore a bug that isn't even majorly the fault of some other project...
Comment 58 Michael Meeks 2014-05-26 16:25:24 UTC
crxssi:

> While that is true, some of us are "stuck" with what is [right now] 
> current... and RHEL 6 is still what is current in business and it is
> likely to never be "fixed" for this issue (because it isn't going to
> use a very recent CUPS).

RedHat do a great job of supporting their product and have an active and engaged development team. I guess if you file with your RHEL contact this will get fixed / worked around wherever the bug truly is =) either in libreoffice or most likely cups.

Any other objections ?
Comment 59 jwoithe 2014-05-27 00:28:43 UTC
In reply to Comment 56:
> Would Resolved->NotOurBug work for this ? [ of course, quite possibly it is at least partially our bug of so not entirely satisfying ], if so, any chance of closing it ? At least the details are archived here for future reference.

"NotOurBug" is probably not telling the whole story, although I'm not sure what else to suggest since the resolution descriptions available are by design fairly generic.  It is clear that a change in LO's PDF generation code broke important applications like CUPS through LO versions 3.5, 3.6 and 4.0 (and possibly earlier).  LO 4.1 changed something which made at least some versions of CUPS render what was expected (see comment 53), but older versions of xpdf (3.02 and earlier) - and perhaps other renderers too - still could not get the render right.  Whether the 4.1 change was just a happy accident also remains to be seen.  A noted in comment 51, CUPS 1.5.4 could not render output from LO 4.0 correctly while (as per comment 53) the same CUPS version on the same machine could deal with the output from LO 4.1.  At the same time however, xpdf 3.02 could not render either output correctly.  Whatever change was made between LO 4.0 and LO 4.1, it would appear to have been subtle enough to make CUPS happy while still creating issues for things like xpdf 3.02.  Furthermore, while we don't know exactly what change made CUPS happy it is unclear whether the altered behaviour is due to a targetted code change or is merely a side effect of some other modification and therefore might break again later.

Personally I am a little uncomfortable with marking this as "NotOurBug" for these and similar reasons to those outlined in comment 57.  While crxssi could potentially pursue this though their RHEL support contract as suggested in comment 58, I am sure there are many other users subject to locked down systems from sources other than RHEL who would not have this option.  What makes the situation somewhat tricky is that all affected PDF renderers used to work; it was a change in LO which broke them.  At the same time, if LO's revised PDF output is technically correct then perhaps the behaviour does point to bugs (or missing feature implementations) in the affected renderers.  Checking the changelog for xpdf 3.03 for example (http://www.foolabs.com/xpdf/CHANGES) it is clear that a number of changes were made to transparency handling between 3.02 and 3.03 which explain why 3.03 may now have a better chance of getting it right.

Having said all that, I have probably personally taken this as far as I have time and knowledge to do.  I am not familiar with the LO codebase, nor have I in-depth knowledge of the PDF format and its various implementations.  For all machines which I manage (personally and at work) upgrading software is an option and for us this seems sufficient to address the problem at least for now.  As a result of all these things, spending more time on this is not currently feasible for me.  If there is no one else to pick up the investigation of the outstanding issues, then perhaps concluding it as "NotOurBug" makes sense if we were willing to blame the problem solely on deficiencies in other software.  However, looking back at the history this is not overly satisfying as mentioned by Michael in comment 56.
Comment 60 James Cloos 2014-05-27 05:32:00 UTC
I’m coming at this late, but some comments:

CUPS is not a renderrer.  I can, for some jobs, use a renderer.

One of the comments, noted that lo sent a pdf to cups, and that he was printing to a postscript printer.  That implies that cups converted from pdf to ps.  For that conversion, depending on the distribution, it will use one of ghostscript, poppler or xpdf.

Because postscript does not support transparency, each of those will render any pages which include transparency to a pixmap encased in postscript.

Current versions of ghostscript and poppler get it right.  (Note that poppler is based on xpdf; its pdftops on xpdf’s pdftops, and its splash display engine – used, eg, by occular – on xpdf.)  The bug probably was specific to xpdf.

Affected systems may be able to avoid that bug by configuring cups’ pdftops filter to call ghostscript rather than pdftops(1).  (It’s a compile-time options, IIRC.)

Printing to non-postscript printers probably calls cups’ gstoraster filter to have ghostscript render the pdf directly to a raster file, which is then converted to the printer’s raster format (pcl, esc/p, et cetera).

Someone on an affected system should test whether their local ghostscript can render the pdf to an image file (png256 gives a good test).  If that works well, then configuring cups to use gs rather than xpdf/poppler for conversions to postscript should fix printing there.

Beyond that, mupdf is a better viewer than xpdf; it is at least as small, fast and efficient.  And it gets this right.  There also is gv, which uses gs to do the rendering.  Gv should be available on any distribution in use, as like ghostscript it is older than linux.
Comment 61 jwoithe 2014-05-27 06:22:26 UTC
In reply to comment 60:
> CUPS is not a renderrer.

Yes, true.  I was using the term loosely.  "Processor" might have been a better term.  The point being that the issue affected more than just "viewers" which is why I was using a more generic term.

> One of the comments, noted that lo sent a pdf to cups, and that he was printing to a 
> postscript printer.  That implies that cups converted from pdf to ps.  For that conversion,
> depending on the distribution, it will use one of ghostscript, poppler or xpdf.

That was probably in response to my comment (comment 34).  I haven't looked that the data flow that closely.  The observation was simply that doing "File|Print" to these two colour printers (which were postscript printers) resulted in a greyscale rectangle.  It was only later that I came to be aware that LO was sending PDF to the printer system now, which provided a connection between PDF viewers and printer output.

For reference, on Slackware 13.x CUPS's pdftops is a binary which calls /usr/bin/pdftops, which is provided by poppler 0.16.4 on this distribution.

> Because postscript does not support transparency, each of those will render any pages which include 
> transparency to a pixmap encased in postscript.

Ok, I didn't know that.  However, the fact remains that regardless of what does this conversion, it used to work.  Something changed in LO 3.5 (or perhaps earlier) which broke it.

> Current versions of ghostscript and poppler get it right.  (Note that poppler is based on xpdf; its 
> pdftops on xpdf’s pdftops, and its splash display engine – used, eg, by occular – on xpdf.)  The bug 
> probably was specific to xpdf.

Note that moving to LO 4.1 without changing anything else on the system made CUPS work correctly while xpdf was still broken - see comment 53.  Also, I'm not entirely clear on the relationship but my understanding is that poppler is effectively a fork of xpdf's processing engine and that the code bases are diverging.  I think that most people consider xpdf and poppler to be distinct projects now with separate codebases.  It is entirely possible that the issue is common to both due to the common ancestry, but addressing it in one will not address it in the other.

> Affected systems may be able to avoid that bug by configuring cups’ pdftops filter to call
> ghostscript rather than pdftops(1) ...

Since it's a compile-time option this is a good suggestion for people to try if they have the resources and ability to recompile CUPS.  Most users would not fall into this category.

> Someone on an affected system should test whether their local ghostscript can render the pdf to an 
> image file (png256 gives a good test).

On Slackware 13.37, "gs -q -sDEVICE=png256 -SOutputFile=- -dNOPAUSE -dBATCH foo.pdf" where "foo.pdf" was attachment 75026 [details] seemed to work (which makes gv's failure as per comment 34 on the same system all the more interesting).

> Beyond that, mupdf is a better viewer than xpdf; it is at least as small, fast and efficient.

Agreed - mupdf looks promising.  Unfortunately to date I have not been able to find an easy to use GUI wrapper for it under Linux (the Android viewer is more complete in this regard).  Their default Linux viewer is functional and is something I could use, but it falls short of what I can reasonably expect general users to use.

> There also is gv, which uses gs to do the rendering.

As detailed in comment 34, gv 3.7.1 on Slackware 13.37 with gs 9.02 crashes when fed the test PDF.  It may work on other systems, but it wasn't a feasible workaround for me.
Comment 62 James Cloos 2014-05-27 10:17:54 UTC
> Something changed in LO 3.5 (or perhaps earlier) which broke it.

If that is when LO started sending pdf instead of postscript, that
probably is the change which triggered this bug.

If it sends postscript, it has to flatten the transparency itself.
Whether it was able still to send vector art, or just sent a pixmap,
by avoiding pdftops it avoided the bug.

> my understanding is that poppler is effectively a fork of xpdf's
> processing engine and that the code bases arediverging

Yes.  And the engine xpdf uses for display is different that the one it
uses to generate postscript.  Poppler’s splash backend is based on the
former, and its pdftops (as you’d expect) on the latter.

> Since it's a compile-time option this is a good suggestion for people
> to try if they have the resources and ability to recompile CUPS.  Most
> users would not fall into this category.

If the filter calls /usr/bin/pdftops (as it normally does), one could
replace /usr/bin/pdftops with a script which instead calls ghostscript.
Something based on pdf2ps, but accepting pdftops’s args.

> Note that moving to LO 4.1 without changing anything else on the
> system made CUPS work correctly while xpdf was still broken

If the printer was a ps printer, that implies that older pdftops and
xpdf had/have different bugs with transparency.

> On Slackware 13.37, "gs -q -sDEVICE=png256 -SOutputFile=- -dNOPAUSE
> -dBATCH foo.pdf" where "foo.pdf" was attachment 75026 [details] seemed to work
> (which makes gv's failure as per comment 34 on the same system all the
> more interesting).

Gv probably used gs’s x11alpha device; there have been improvements to
that device since 9.02 (at least a couple were mine).

> As detailed in comment 34, gv 3.7.1 on Slackware 13.37 with gs 9.02
> crashes when fed the test PDF.

I hadn’t gotten that far reading the comments....  Long bz!

Looking at pdf generated by 4.2.1.1, I don’t see anything in it which
should be controversial.

The best LO could do is offer an option on export and printing to
flatten transparency.  That involves breaking apart overlapping non-
opaque objects into separate objects for each final colour, and
specifying those colors in the exported file.  That will have the side
effect of converting any affected text into outlines.  But purely opaque
objects would export unchanged.

That, whether pdf or ps, should pass w/o harm through the affected consumers.

It needs to be an option because, for some exporting the transparency to
the pdf is important.  Such as when the pdf is to be included in some
other document.

There are open feature requests for poppler and ghostscript to do that
type of flattening when converting to postscript, but it should be an
easier enhancement for lo, given that lo works with higher-level objects.
Comment 63 Michael Meeks 2014-05-27 10:59:53 UTC
Hi James; great to have a CUPS / ghostscript / PDF hacker commenting at last :-) Thanks for the great input

> Looking at pdf generated by 4.2.1.1, I don’t see anything in it which
> should be controversial.

Glad you don't see anything objectionable in our PDF stream =) As such I will close this as NOTOURBUG - agreed - not the perfect resolution, but this points to it.

> The best LO could do is offer an option on export and printing to ...

This would be a great feature request to file; I'm not sure that people will love to file it, but perhaps there is indeed code to identify which pieces of the page to bitmap-ify, though there really is a reason why we prefer a meta-file format that handles transparency well ;-)

Either, way, please do file a new print dialog option / enhancement feature bug for that and link it to this with SeeAlso etc. It'd be nice to attach the single, most minimal test document from here to it as well - this bug got rather horribly un-readably unwieldy by now =)

Thanks for everyone's input.
Comment 64 jwoithe 2014-05-27 12:04:22 UTC
Thanks James for the additional information (comment 62).  It appears that using ghostscript to do the conversion may be a feasible workaround for installations such as those mentioned by crxssi in comment 57.

> And the engine xpdf uses for display is different that the one it uses to
> generate postscript.

Having different engines involved at various stages throughout the system undoubtedly added to the confusion surrounding this issue.

Hopefully a transparency flattening option as outlined in comment 62 can be made workable in time for those users who are unable to upgrade their base system frequently.  Of course whether this happens before such systems are updated remains to be seen.

Thanks to all who helped to clarify the source of this issue even though it is not possible to completely resolve it for all users at this time.
Comment 65 James Cloos 2014-05-27 21:38:12 UTC
>> The best LO could do is offer an option on export and printing to ...

> This would be a great feature request to file; I'm not sure that people will
> love to file it, but perhaps there is indeed code to identify which pieces of
> the page to bitmap-ify, though there really is a reason why we prefer a
> meta-file format that handles transparency well ;-)

PDF is indeed better, and we are getting closer to a world where it can
be directly rasterized on the printers.  IPP-Everywhere is a significant
step in that direction.  My PS printer, for example, while nearly
depreciated still has -- at the rate I print colour -- years of CMY
toner in its current cartiges.  And, at that rate, decades of imaging
unit life. ☺

The flattening does not need to rasterize.  It just needs to flatten
fill and stroke colors to the visible results of the tranparency.

When transparent objects completely overlap that is easy; the objects
are painted as opaque with the result colours.  Even text can be left
as text in this case.

The hard part is when they only partially overlap.  The code needs to
trace the outlines of the overlapping portions and generate new opaque
objects to paint over those areas.

When the partially overlapping areas have polygonal outlines, that is
again easy.  And one probably only needs centipoint to millipoint
precision to do it well enough to avode anomolies.

The hard part is dealing with the cubic outlines.  One needs to create a
spline extending from an arbitrary point along the orginal spline to
another arbitrary point, tracing along the original.  Doable, with
arbitrary tolerance.

And if the partially overlapping objects include a block of text, that
whole block needs to be converted to outlines before doing the above.

That could take significant processing, especially for a portable.
Comment 66 tmacalp 2014-05-28 02:47:11 UTC
(In reply to comment #65)

> The hard part is when they only partially overlap.  The code needs to
> trace the outlines of the overlapping portions and generate new opaque
> objects to paint over those areas.

I might be reading this wrong, but it sounds like you're describing a behavior similar to bug 61306 and bug 42442.
Comment 67 James Cloos 2014-05-28 19:37:16 UTC
>> The hard part is when they only partially overlap.  The code needs to
>> trace the outlines of the overlapping portions and generate new opaque
>> objects to paint over those areas.

> I might be reading this wrong, but it sounds like you're describing a
> behavior similar to bug 61306 and bug 42442.

The anomalies reported in those BZs appear to be due to using too-little
precision in the coordinates in the exported pdf.

Eg, the businesscards.pdf attached to bug 42442 uses only decipoints
in vector art streams.  That evidently is imprecise enough for the
edges to misalign at even 600 dpi.

Rounding instead to centipoints might be enough to avoid those even when
directly exposing offset plates at 150 dots/mm.  Or it might require
millpoints.

Higher precision would mean larger, slower files.

Finding the optimal tradeoff for a large, varied user base is at best
difficult.

I expect that is why some products offer multiple quality targets,
such as Draft, Web, Office, Litho, et cetera.

It would be interesting, though, to see whether centipoints are enough.

The vector-transparency-flattening algorithm I described in the previous
comment relates to those two bugs only in that one risks similaranomalies
unless one uses sufficient precision, with the same size and speed
tradeoffs I mention above.