Bug 98349 - Handling of PUA mapped "symbol fonts" like Chess fonts, some characters not mapping back into keyboard when used in LO
Summary: Handling of PUA mapped "symbol fonts" like Chess fonts, some characters not m...
Status: RESOLVED WORKSFORME
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: graphics stack (show other bugs)
Version:
(earliest affected)
5.1.0.3 release
Hardware: x86 (IA32) Windows (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: bibisectRequest, regression
: 98719 (view as bug list)
Depends on:
Blocks: Font-Rendering
  Show dependency treegraph
 
Reported: 2016-03-02 12:57 UTC by Robert Sadler
Modified: 2017-07-20 16:06 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
Incorrect rendering of Chess Leipzig font. (97.93 KB, application/pdf)
2016-03-02 12:57 UTC, Robert Sadler
Details
Chess Leipzig in ODT. (10.78 KB, application/vnd.oasis.opendocument.text)
2016-03-03 07:39 UTC, Robert Sadler
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Sadler 2016-03-02 12:57:05 UTC
Created attachment 123147 [details]
Incorrect rendering of Chess Leipzig font.

The "Chess Leipzig" font is not rendered correctly anymore. See attached PDF. The TTF file is not corrupt, as it renders correctly in other applications.
Comment 1 V Stuart Foote 2016-03-03 04:01:05 UTC
Hi Robert,

Thanks for filing.

Any improvement if you turn off OpenGL support. 

Uncheck the Tools -> Options -> View: Use OpenGL for all rendering (restart required)?

Also, please provide graphics card and OS details

1. run msinfo32.exe and copy text from the Components -> Display panel

2. post the opengl_device.log from your LibreOffice user cache found in

%APPDATA%\Roaming\LibreOffice\4\cache

Finally, we can see the corruption in the PDF page, but would be helpful to have an actual page or two in the ODF Writer document.

Post that all up and set back to Unconfirmed.

Thanks!
Comment 2 Robert Sadler 2016-03-03 07:37:12 UTC
OpenGL is set to OFF.

msinfo32.exe:

[Display]

Item	Value	
Name	ATI Mobility Radeon X1600	
PNP Device ID	PCI\VEN_1002&DEV_71C5&SUBSYS_FF101179&REV_00\4&31B7BFB9&0&0008	
Adapter Type	ATI Mobility Radeon X1600 (0x71C5), ATI Technologies Inc. compatible	
Adapter Description	ATI Mobility Radeon X1600	
Adapter RAM	512.00 MB (536,870,912 bytes)	
Installed Drivers	ati2dvag.dll	
Driver Version	6.14.10.6635	
INF File	oem1.inf (ati2mtag_M56 section)	
Color Planes	1	
Color Table Entries	4294967296	
Resolution	1280 x 800 x 60 hertz	
Bits/Pixel	32	
Memory Address	0xC0000000-0xCFFFFFFF	
I/O Port	0x00002000-0x00002FFF	
Memory Address	0xDC000000-0xDC0FFFFF	
IRQ Channel	IRQ 16	
I/O Port	0x000003B0-0x000003BB	
I/O Port	0x000003C0-0x000003DF	
Memory Address	0xA0000-0xBFFFF	
Driver	c:\windows\system32\drivers\ati2mtag.sys (6.14.10.6635, 1.64 MB (1,723,904 bytes), 06/04/2006 01:04)


opengl_device.log:

DriverVersion: 8.292.0.0
DriverDate: 8-29-2006
DeviceID: PCI\VEN_1002&DEV_71C5&SUBSYS_FF101179&REV_00
AdapterVendorID: 0x1002
AdapterDeviceID: 0x71c5
AdapterSubsysID: 0xff101179
DeviceKey: System\CurrentControlSet\Control\Video\{2DBAFFC1-D90C-4459-AD6D-E6CD90278FEF}\0000
DeviceString: ATI Mobility Radeon X1600


ODF document attached.

--------

The "Chess Leipzig" font rendered correctly in LibreOffice 4.
Comment 3 Robert Sadler 2016-03-03 07:39:29 UTC
Created attachment 123180 [details]
Chess Leipzig in ODT.

First block is the characters in "normal" font; second block is the characters in "Chess Leipzig" font.
Comment 4 Robert Sadler 2016-03-04 11:30:47 UTC
I tested this a bit more ... it seems that LibreOffice 5 is moving the symbols to the "private use area" of the font table, e.g. what should be U+71 is moved to U+F071. So for example: if I make the font 'Liberation Serif' and type a '2' (U+32) and then change the font to 'Chess Leipzig', I still have a two. However, if using 'Liberation Serif' I type character U+F032 I get a '2', but then when I change the font to 'Chess Leipzig' I get the correct chess character.
Comment 5 V Stuart Foote 2016-03-04 16:28:44 UTC
Sorry, can not confirm.

Obtained and installed a set of Chess Leipzig from http://www.enpassant.dk/chess/fonteng.htm

Reviewed in BableMap and TypeLight and the font is treated as a "Symbol Font" with Unicode points in the Private Use Area U+F020-F07F block. The font defines mapping of most to corresponding U+0020-007F for Basic Latin ASCII keyboard entry.

Checked LO 4.4 -> 5.1 and we handle the Chess Leipzig --"Symbol Font"-- composition correctly. Mapping of the Private Use Area Unicode to appropriate default Basic Latin ASCII keys behaves correctly and can not reproduce incorrect font substitution.  

On Windows 10 Pro 64-bit en-US with current
Version: 5.1.1.2 (x64)
Build ID: fe4d9e69c82c6ee6db3c27cd5e2d47558afa80ac
CPU Threads: 8; OS Version: Windows 6.19; UI Render: GL; 
Locale: en-US (en_US)

nor with

Version: 5.2.0.0.alpha0+
Build ID: 817f6bd75e98a321c238b5906bfcb031908914b2
CPU Threads: 8; OS Version: Windows 6.19; UI Render: GL; 
TinderBox: Win-x86@39, Branch:master, Time: 2016-03-01_01:14:57
Locale: en-US (en_US)

Entry of the board row numbers and column letters responds correctly to U+<CodePoint> conversion with <Alt>+X, as do the additional white pieces (font glyphs 90 - 95).

Printing to PDF, and Export to PDF handle the "Symbol Font" mappings correctly.

Sorry but this seems to all be behaving correctly.  Do things improve if you repair your user profile?  

https://wiki.documentfoundation.org/UserProfile
Comment 6 Robert Sadler 2016-03-05 14:45:44 UTC
Thanks.

A work-around for this problem is to insert the characters directly, as LO5 then displays it correctly.

I have tested it again on a Windows XP machine that still had LO4 installed and the remappings work correctly. Updating to LO5 (to get rid of that annoying "memory error" error) now also doesn't display the font correctly.

So let's just blame it on Windows XP that isn't fully supported anymore.
Comment 7 Robert Sadler 2016-03-05 15:26:46 UTC
Ah! The source of the problem, according to me, is when one starts the board off using single lines, i.e. the characters 1-9.

I have also noticed that those 1-9's are rendered differently when changing the font to something else, like "Liberation Serif".
Comment 8 V Stuart Foote 2016-03-05 16:29:56 UTC
(In reply to Robert Sadler from comment #7)
> Ah! The source of the problem, according to me, is when one starts the board
> off using single lines, i.e. the characters 1-9.
> 
> I have also noticed that those 1-9's are rendered differently when changing
> the font to something else, like "Liberation Serif".

You can enter the row values as well in 5.1 via the OS assigned Private Use Area (PUA) mappings using the Unicode <alt>+X toggle. 

So in the Chess Lipzig font the Single border 1-8 would receive PUA mappings of:

1 U+F0C0
2 U+F0C1
3 U+F0C2
4 U+F0C3
5 U+F0C4
6 U+F0C5
7 U+F0C6
8 U+F0C7

While the double border 1-8 receive PUA mappings of:

1 U+F0E0
2 U+F0E1
3 U+F0E2
4 U+F0E3
5 U+F0E4
6 U+F0E5
7 U+F0E6
8 U+F0E7

Of course these PUA mappings would have different mapping, or likely not be mapped, when other fonts are assigned to the paragraph.

A font design tool, like Type Light 3 ( http://www.cr8software.net/typelight.html ), would be helpful to collect all the PUA code mappings being assigned.
Comment 9 Robert Sadler 2016-03-06 08:56:42 UTC
Let me explain more fully:

If I directly insert the "Private Use Area" characters for the single board lines, then they insert correctly. This means that U+F031 to U+F039 inserts the correct character.

Also, if I insert the single board lines using the PUA directly, then the "letters" for the pieces are rendered correctly; i.e. ponmbvrtqwkl*PONMBVRTQWKL+.

However, if I try to create the single board lines using 1-9, i.e. U+31 to U+39, then LO5 insert an actual 1-9 (as the case may be).

This has three results:

(1) The subsequent board pieces are not rendered correctly. Even though "pair kerning" is off, LO5 still appears to attempt kerning the symbols. Deleting the "4" and "5" from the ends immediately causes LO5 to "redraw" the board pieces at the correct positions.

(2) Highlighting such an "incorrect" board and changing the font to something else, e.g. Arial, leaves the directly inserted PUA symbols in place. Also, clicking on Omega (to insert special characters) shows that "Chess Leipzig" now is available in the PUA of, in this example, Arial.

(3) The 1-9 is displayed in the "OpenSymbol" font.

So, conclusion for me is that LO5 on Windows XP is having difficulties with (at least) one font, as all my other applications still render the Chess Leipzig font correctly. And seeing as you cannot reproduce the problem using LO5 on a newer version of Windows, then it must be an incompatibility between LO5 and Windows XP that is causing this particular problem.
Comment 10 Robert Sadler 2016-03-06 11:36:34 UTC
I uninstalled LibreOffice 5.1.0.3 and installed LibreOffice 4.4.7.2; now Chess Leipzig is rendered correctly again. So the problem is definitely version 5 on XP.
Comment 11 V Stuart Foote 2016-03-08 23:52:47 UTC
adding to the bug 71732 meta for tracking font issues.

This looks to be same issue in handling Latin 1252 mapped "Symbol fonts" as affected bug 95209 - "Broken font rendering of some characters in Opus Standard font"
Comment 12 V Stuart Foote 2016-03-17 07:12:03 UTC
*** Bug 98719 has been marked as a duplicate of this bug. ***
Comment 13 Xisco Faulí 2016-09-13 10:25:17 UTC
Since we have a bibisect repository for windows covering the branch where this regression was introduced, adding keyword 'bibisectRequest'.
More info: https://wiki.documentfoundation.org/QA/Bibisect/Windows
Comment 14 Robert Sadler 2016-10-06 09:14:10 UTC
What's the progress on this issue?
Comment 15 V Stuart Foote 2017-07-20 16:06:01 UTC
Symbol font handling for sample ODT in attachment 123180 [details] is correct when the symbol font is installed to system.

Windows 8.1 Ent 64-bit en-US with
Version: 5.4.0.1 (x64)
Build ID: 962a9c4e2f56d1dbdd354b1becda28edd471f4f2
CPU threads: 8; OS: Windows 6.29; UI render: default; 
Locale: en-US (en_US); Calc: CL

and
Version: 5.3.4.2 (x64)
Build ID: f82d347ccc0be322489bf7da61d7e4ad13fe2ff3
CPU Threads: 8; OS Version: Windows 6.29; UI Render: GL; Layout Engine: new; 
Locale: en-US (en_US); Calc: group