Bug Hunting Session
Bug 95209 - Broken font rendering of some characters in Opus Standard font (was ok in LO4.4.2)
Summary: Broken font rendering of some characters in Opus Standard font (was ok in LO4...
Status: RESOLVED WORKSFORME
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: graphics stack (show other bugs)
Version:
(earliest affected)
5.0.2.2 release
Hardware: x86 (IA32) Windows (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: bibisectRequest, regression
Depends on:
Blocks: Font-Rendering
  Show dependency treegraph
 
Reported: 2015-10-20 23:09 UTC by Kai Struck
Modified: 2017-01-02 14:00 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
odt, svg, pdf (195.60 KB, application/zip)
2015-10-20 23:09 UTC, Kai Struck
Details
issues rendering document and SVG using Opus music font (280.96 KB, image/png)
2015-11-25 17:07 UTC, V Stuart Foote
Details
Untilted.otf: a dummy OpenType Type 1 font (1.59 KB, application/x-font-otf)
2015-11-26 11:50 UTC, Tor Lillqvist
Details
Test document for it (9.18 KB, application/vnd.oasis.opendocument.text)
2015-11-26 11:51 UTC, Tor Lillqvist
Details
OpusSpecialStd font and original svg-export from Sibelius (44.05 KB, application/zip)
2015-11-26 21:01 UTC, Kai Struck
Details
clip of Glyph info for untitled mapped to 0153 (10.47 KB, image/png)
2015-11-27 22:12 UTC, V Stuart Foote
Details
clip of info for Opus Std glyph #52 (11.10 KB, image/png)
2015-11-27 22:26 UTC, V Stuart Foote
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Kai Struck 2015-10-20 23:09:22 UTC
Created attachment 119811 [details]
odt, svg, pdf

Font rendering of some characters in music font "Opus Standard" is broken. It worked ok in LibreOffice 4.4.2 and seems broken in 5.0.2.2

Characters like w, h, q, œ should show up as noteheads.
œ is now showing as œ.

Attached is a zip file with 
1. the used font,
2. an .odt-file showing the problem as text line and in svg image
3. the svg image
4. original pdf of the svg image.
Comment 1 V Stuart Foote 2015-10-21 05:23:49 UTC
On Windows 10 Pro 64bit en-US with Opus Std font installed.
The SVG when inserted into Draw, and ODT paragraphs are rendered "correctly"
 
Confirming glyphs are as user expected in

Version: 4.4.5.2
Build ID: a22f674fd25a3b6f45bdebf25400ed2adff0ff99
Locale: en_US

And into the 5.0 release, start being handled "incorrectly" between

Version: 5.0.1.2 (2015-08-24)
Build ID: 81898c9f5c0d43f3473ba111d7b351050be20261
Locale: en-US (en_US)

and

Version: 5.0.2.1 (2015-09-08)
Build ID: 9a18d52abbdfbdc2ac9acebec2b92e7859eb73b7
Locale: en-US (en_US)

@Tor, is this another gotcha from the "Drop SimpleWinLayout" pushed at 5.0.2? If so, it seems it might be a bit more troublesome. But then it seems the Opus Std font has some rather odd code page mappings--definitely not Unicode friendly.

=-ref-=
http://cgit.freedesktop.org/libreoffice/core/commit/?id=1927a743755d8b3b55189b02a031b8cf90ba25ec&h=libreoffice-5-0-2
Comment 2 Tor Lillqvist 2015-11-25 10:50:17 UTC
FWIW, Word 2013 does not display that .odt file correctly either. It does not show the last character (which LO shows as 'œ') at all.
Comment 3 V Stuart Foote 2015-11-25 17:07:39 UTC
Created attachment 120794 [details]
issues rendering document and SVG using Opus music font

There is some font strangeness going on, on Windows 10 Pr with the Opus Standard font installed for testing. See attached.

Comparing to the PDF from the test kit:

Imagemagick's IMDisplay (usually a good SVG parser) is choking on the Opus mapping as well.

Inkscape does the same, mishandling use of Opus Std font.

Strange in that FireFox does well, only hicup on composing the Left Repeat (shows as a pair of ™ glyphs). That may be pulled from the Opus Special Std font that is embedded in the PDF but not otherwise available.

As for LibreOffice: builds through 4.4 compose with svgio using the Opus Std font, same issue with the Left Repeat as pair of ™ glyphs.  But at 5.0.3.2 we choke just as bad as Imagemagick and Inkscape with out svgio filter insert of the SVG.

Checking older builds and filter/source/svg has never handled this well when opening the SVG into Draw.

I'll recheck it all on a Linux VM, but the "regression" on Windows at 5.0 could be because we're doing things more natively with Uniscribe as noted.
Comment 4 V Stuart Foote 2015-11-25 17:30:35 UTC
@Xisco, Armin, *

Not that we have to save the world on this, but any thoughts on why the filter/source/svg misses all font elements of the SVG import?  

Suppose it is just another round of bug 32248
Comment 5 Tor Lillqvist 2015-11-26 10:00:39 UTC
Opus Std is a PostScript Type 1 font (embedded in an OpenType container, .otf file). Do we know if such fonts work in general in LibreOffice?
Comment 6 Tor Lillqvist 2015-11-26 10:29:11 UTC
And with "work" I mean work at least for Latin characters including non-ASCII ones, which is what the text here uses, "w, h, q, œ". (That the glyphs for these characters look like musical notation notes in that particular font is fairly irrelevant.)
Comment 7 Tor Lillqvist 2015-11-26 11:49:20 UTC
I created a dummy font 'Untilted' in FontForge, with just two glyphs, for characters 'A' and 'œ'. LibreOffice has no problem in using those two characters. Go figure.
Comment 8 Tor Lillqvist 2015-11-26 11:50:38 UTC
Created attachment 120812 [details]
Untilted.otf: a dummy OpenType Type 1 font
Comment 9 Tor Lillqvist 2015-11-26 11:51:04 UTC
Created attachment 120813 [details]
Test document for it
Comment 10 Kai Struck 2015-11-26 21:01:55 UTC
Created attachment 120829 [details]
OpusSpecialStd font and original svg-export from Sibelius

The svg in earlier attachment https://bugs.documentfoundation.org/attachment.cgi?id=119811 was created by Inkscape, which shows all correct here. Win7, Inkscape 0.91

uploaded new zip with the other used font OpusSpecialStd and an svg-export of the same music score directly from the Notation Software. In this svg the quarter note is character  as opposed to the Inkscape export, where it is œ.   ???
Comment 11 V Stuart Foote 2015-11-27 08:03:15 UTC
(In reply to Kai Struck from comment #10)
> Created attachment 120829 [details]
> ...
> uploaded new zip with the other used font OpusSpecialStd and an svg-export
> of the same music score directly from the Notation Software. In this svg the
> quarter note is character  as opposed to the Inkscape export, where it is
> œ. ???

So, the issue for you started with the SVG exported from Inkscape. So guess this could be resolved--not our bug--as you do get serviceable SVG when exporting from your notation software.

=-=-=

What remains troubling--admittedly a corner case of handling non-Unicode or PostScript 1 fonts--is that we have for MS Windows builds at least changed our handling of text strings, or SVG xml, containing Opus Std. font.  Still needs to be checked on Linux and OSX.

This regression is weird because some glyphs belonging to Opus Std. are correctly rendered, the "Ampersand" key input paints the Opus Std. Treble clef, "w" paints the whole note. But others map out to a different font and so place the wrong glyph. 

As noted in comment 1, something was changed. Bit of a WAG here bug suspect some of the Opus Std. glyps are being treated as "Windows 1252" code page mapped characters--they seem to be being mapped out to Unicode as in this reference:

http://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1252.TXT
 
Since 5.0.2.1 release, at least these 5 glyps from Opus Std. are rendered in wrong font in LibreOffice

0x89 --> gets Unicode 2030 (per mille sign) [numpad enter alt-0137]
0x99 --> gets Unicode 2122 (trademark sign) [numpad enter alt-0153]

0x8c --> gets Unicode 0152 (latin capital ligature oe) [numpad enter alt-0140]
0x9C --> gets Unicode 0153 (latin small ligature oe) [numpad enter alt-0156]
0x9F --> gets Unicode 0178 (latin capital letter Y with diaeresis) [numpad enter alt-0159]

Windows 7 maps them to Arial (I think)

Windows 8, 8.1 & 10 maps them to Segoe.

Simple Steps to Reproduce

1. Opus Std. font installed
2. open Writer prior to  5.0.2.1,  new document.
3. set paragraph character to Opus Std.
4. type in "&" "w" "h" "q" "alt-0137" "alt-0152" "alt-0153" "alt-0159"
5. will see entry rendered in Opus Std. glyphs

6. open Writer 5.0.2.1 or newer build, new document
7. set paragraph character to Opus Std.
8. type in same string, "&" "w" "h" "q" "alt-0137" "alt-0152" "alt-0153" "alt-0159" as before.  The "&""w" "h" "q" render in Opus Std. glyphs, the others are mapped out to Unicode glyph and rendered from a font other than Opus Std.
Comment 12 Tor Lillqvist 2015-11-27 09:01:48 UTC
So what is the difference between the Opus Std font and the dummy Untilted font from comment #8? The œ is found in the latter, not in the former.

Please refrain from guesses without investigating what APIs LibreOffice actually uses. I am fairly sure no codepages are involved, just Unicode.
Comment 13 Tor Lillqvist 2015-11-27 09:03:50 UTC
Also, problems in SVG export might be interesting, but surely we should concentrate first on fixing the problem that is immediately visible already in the Writer GUI?

Also, please don't attach .zip archives containing just a couple of files. It is much easier to handle files attached individually. (But good that at least it is .zip and not some horror like .7z.)
Comment 14 V Stuart Foote 2015-11-27 22:12:39 UTC
Created attachment 120844 [details]
clip of Glyph info for untitled mapped to 0153
Comment 15 V Stuart Foote 2015-11-27 22:26:58 UTC
Created attachment 120845 [details]
clip of info for Opus Std glyph #52

(In reply to Tor Lillqvist from comment #12)
> So what is the difference between the Opus Std font and the dummy Untilted
> font from comment #8? The œ is found in the latter, not in the former.
> 

This thread made me curious, and I found a good tool to view the glyphs in context--cr8software.net's "Type light 3".  This handy tool confirms the Opus Std to be PostScript, but also shows the Glyphs by their index as well as their mapping(s) to Unicode.

As expected, from its metadata the #52 glyph is mapped to Unicode 0153 oe (Latin Extende-A), but Opus Std. also assigns a PUA address of: F0CF uniFOCF (Private Use Area) to the glyph.  

That F0CF responds to decimal 0207 is valid for CLI character entry, but when entered that way, the 0153 Unicode value is not in the string--CF U+0207 is.

The recent additon of the Alt+x Unicode toggle displays this.
Comment 16 V Stuart Foote 2015-12-13 17:35:02 UTC
adding to the bug 71732 Meta tracker
Comment 17 Xisco Faulí 2016-09-13 10:30:13 UTC
Since we have a bibisect repository for windows covering the branch where this regression was introduced, adding keyword 'bibisectRequest'.
More info: https://wiki.documentfoundation.org/QA/Bibisect/Windows
Comment 18 Khaled Hosny 2016-11-11 23:59:13 UTC
Just for the record, that OpusStd.otf is an OpenType font with PostScript outlines and it should generally work in LibreOffice just fine.
Comment 19 Kai Struck 2017-01-02 11:50:35 UTC
Many thanks to the development team!

Seems like this issue is now working correctly. In LO 5.2.3.3 Portable 32bit Win7 the problematic SVG from 
https://bugs.documentfoundation.org/attachment.cgi?id=120794
as well as the original SVG from
https://bugs.documentfoundation.org/attachment.cgi?id=120829

are shown correctly in LO as well as in an exported PDF.
Comment 20 Xisco Faulí 2017-01-02 11:57:26 UTC
Closing this as RESOLVED WORKSFORME as per comment 19
Comment 21 V Stuart Foote 2017-01-02 14:00:59 UTC
Yes confirming correct handling of this symbol font -- w/5.2.4.1 with both default and OpenGL rendering.

At 5.3.0 the Special Character dialog handling is corrected, and good Harfbuzz layout (default and OpenGL) as well with current 5.4.0 master (default and OpenGL).

=-ref-=
Tested on Windows 10 Pro 64-bit (1607) en-US with

Version: 5.2.4.1 (x64)
Build ID: 9b50003582f07ac674d6451e411e9b77cccd2b22
CPU Threads: 8; OS Version: Windows 6.19; UI Render: GL; 
Locale: en-US (en_US); Calc: group

Version: 5.3.0.0.beta2 (x64)
Build ID: a7e30712ad6d8bc9286007b37aa581983e0caba3
CPU Threads: 8; OS Version: Windows 6.19; UI Render: GL; Layout Engine: new; 
Locale: en-US (en_US); Calc: CL

Version: 5.4.0.0.alpha0+
Build ID: 1cb1cc30b5eb75483b6aaeec0192f7ee1d833461
CPU Threads: 8; OS Version: Windows 6.19; UI Render: GL; 
TinderBox: Win-x86@42, Branch:master, Time: 2017-01-01_23:25:36
Locale: en-US (en_US); Calc: CL