Bug 107487 - Unicode ExtB+ chars do not use fallback font correctly in Writer
Summary: Unicode ExtB+ chars do not use fallback font correctly in Writer
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
5.3.2.2 release
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:6.4.0 target:6.3.0.1 target:6.2.5
Keywords:
Depends on:
Blocks: CJK Font-Substitution
  Show dependency treegraph
 
Reported: 2017-04-27 15:43 UTC by Danny Lin
Modified: 2019-06-05 12:36 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments
figure 1 (49.64 KB, image/png)
2017-04-27 15:46 UTC, Danny Lin
Details
figure 2 (75.18 KB, image/png)
2017-04-27 15:47 UTC, Danny Lin
Details
figure 3 (53.80 KB, image/png)
2017-04-27 15:47 UTC, Danny Lin
Details
figure 4 (56.21 KB, image/png)
2017-04-27 15:48 UTC, Danny Lin
Details
HanaMinA.reg (342 bytes, text/x-ms-regedit)
2017-05-20 11:03 UTC, Volga
Details
CJK Ideographs testcase (12.31 KB, application/vnd.oasis.opendocument.text)
2017-05-20 11:04 UTC, Volga
Details
HanaMinA.reg (342 bytes, text/x-ms-regedit)
2017-05-20 11:08 UTC, Volga
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Danny Lin 2017-04-27 15:43:14 UTC
Description:
Some fonts are 

Steps to Reproduce:
1.Open a new .odt file in Writer. Make sure the font is set to "新細明體". (figure 1)
2.Paste in "大黃䗪蟲丸狐𧌒䒜𧀬" using plain text format. (figure 2)

Actual Results:  
The chars are displayed in a gibberish way. (figure 3)

Expected Results:
The chars should be displayed correctly.


Reproducible: Always

User Profile Reset: No

Additional Info:
Font "新細明體" has code points for "大黃䗪蟲丸狐"、"䒜", and font "新細明體-ExtB" has code points for "𧌒" and "𧀬".

In Microsoft Word, if the font is set to "新細明體" and "大黃䗪蟲丸狐𧌒䒜𧀬" is pasted, "𧌒" and "𧀬" will be rendered using font "新細明體-ExtB" while other chars are rendered using font "新細明體". If the font is set to "新細明體-ExtB" and "大黃䗪蟲丸狐𧌒䒜𧀬" is pasted, "𧌒" and "𧀬" will be rendered using font "新細明體-ExtB" while other chars are rendered using font "新細明體".

In Writer, if the font is set to "新細明體" and "大黃䗪蟲丸狐𧌒䒜𧀬" is pasted, "𧌒" and "𧀬" are tried to be rendered using font "新細明體", causing a poor rendering. If the font is set to "新細明體-ExtB" and "大黃䗪蟲丸狐𧌒䒜𧀬" is pasted, "𧌒" and "𧀬" will be rendered using font "新細明體-ExtB" while other chars are rendered using the default font (on Windows 7 it's usually "微軟正黑體"), causing an inconsistent rendering (figure 4).


Related issues see:
https://bugs.documentfoundation.org/show_bug.cgi?id=96457


User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:53.0) Gecko/20100101 Firefox/53.0
Comment 1 Danny Lin 2017-04-27 15:46:03 UTC
Created attachment 132907 [details]
figure 1
Comment 2 Danny Lin 2017-04-27 15:47:24 UTC
Created attachment 132908 [details]
figure 2
Comment 3 Danny Lin 2017-04-27 15:47:41 UTC
Created attachment 132909 [details]
figure 3
Comment 4 Danny Lin 2017-04-27 15:48:01 UTC
Created attachment 132910 [details]
figure 4
Comment 5 Danny Lin 2017-04-27 15:51:55 UTC
The same result happened when the user types "大黃䗪蟲丸狐𧌒䒜𧀬" one by one instead of pasting the sentence at once.
Comment 6 Volga 2017-04-28 03:44:32 UTC
Windows Registry have the following registry paths for Unicode Plane 2 fallback fonts that you can see them in regedit.exe:

\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\LanguagePack\SurrogateFallback\MingLiU
\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\LanguagePack\SurrogateFallback\MingLiU_HKSCS
\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\LanguagePack\SurrogateFallback\PMingLiU
\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\LanguagePack\SurrogateFallback\SimSun

All these paths have an item named "Plane2" to indicate which fallback fonts used to render the text respectively. According to their items, this relationship can be list as below:

Plane 1       | Plane 2 fallback
---------------------------------
MingLiU       | MingLiU-ExtB
MingLiU_HKSCS | MingLiU_HKSCS-ExtB
PMingLiU      | PMingLiU-ExtB
SimSun        | SimSun-ExtB
Comment 7 Volga 2017-04-28 03:56:34 UTC
64-bit version of Windows has additional resistry paths with the same purpose:

\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Windows NT\CurrentVersion\LanguagePack\SurrogateFallback\MingLiU
\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Windows NT\CurrentVersion\LanguagePack\SurrogateFallback\MingLiU-HKSCS
\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Windows NT\CurrentVersion\LanguagePack\SurrogateFallback\PMingLiU
\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Windows NT\CurrentVersion\LanguagePack\SurrogateFallback\SimSun
Comment 8 Volga 2017-04-28 04:01:01 UTC
In regedit.exe I created a new item path under the following paths, remaned "HanaMinA":

\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\LanguagePack\SurrogateFallback\
\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Windows NT\CurrentVersion\LanguagePack\SurrogateFallback\

Then I add a item "Plane2", the value is HanaMinB. I try them on LO Writer, they seems works with HanaMinA font, but characters are still wrongly placed.
Comment 9 Buovjaga 2017-04-29 17:43:07 UTC
NEW per Volga's testing.
Comment 10 Volga 2017-05-20 11:03:22 UTC Comment hidden (obsolete)
Comment 11 Volga 2017-05-20 11:04:55 UTC Comment hidden (obsolete)
Comment 12 Volga 2017-05-20 11:08:29 UTC Comment hidden (obsolete)
Comment 13 Volga 2017-08-26 09:18:56 UTC
Still reproduce in LO 5.4.1 RC1.

Version: 5.4.1.1 (x64)
Build ID:a5be49f0c45fe24a575c7f41559aa8fc79a781a2
CPU 线程:4; 操作系统:Windows 6.19; UI 渲染:默认; 
Locale: zh-CN (zh_CN); Calc: group
Comment 14 Commit Notification 2019-06-03 02:52:03 UTC
Mark Hung committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/16faad3657fad114aa984f92a14a5f212a2f8ff5%5E%21

tdf#107487 fix font fallback for non-BMP characters.

It will be available in 6.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 15 Commit Notification 2019-06-05 09:07:05 UTC
Mark Hung committed a patch related to this issue.
It has been pushed to "libreoffice-6-3":

https://git.libreoffice.org/core/+/76f0cf39fd06ec1b523e9bdf8fbcc98a55a541b4%5E%21

tdf#107487 fix font fallback for non-BMP characters.

It will be available in 6.3.0.1.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 16 Commit Notification 2019-06-05 12:36:28 UTC
Mark Hung committed a patch related to this issue.
It has been pushed to "libreoffice-6-2":

https://git.libreoffice.org/core/+/ff8b31743305635b968b0a0f21c2778a29c0ce03%5E%21

tdf#107487 fix font fallback for non-BMP characters.

It will be available in 6.2.5.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.