Bug 92505 - Unicode character rendering artifacts when substituting missing glyphs
Summary: Unicode character rendering artifacts when substituting missing glyphs
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
4.2.8.2 release
Hardware: All Linux (All)
: medium normal
Assignee: Miklos Vajna
URL:
Whiteboard: target:5.1.0 target:5.0.3 target:4.4.6
Keywords: bibisected, bisected, regression
Depends on:
Blocks:
 
Reported: 2015-07-02 15:51 UTC by Glutanimate
Modified: 2016-10-25 19:19 UTC (History)
6 users (show)

See Also:
Crash report or crash signature:


Attachments
sample document demonstrating unicode rendering under different fonts (19.60 KB, application/vnd.oasis.opendocument.text)
2015-07-02 15:51 UTC, Glutanimate
Details
Sample document rendered under LibreOffice 4.1.6.2 (48.88 KB, application/pdf)
2015-07-02 15:52 UTC, Glutanimate
Details
Sample document rendered under LibreOffice 4.4.4 (44.56 KB, application/pdf)
2015-07-02 15:53 UTC, Glutanimate
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Glutanimate 2015-07-02 15:51:40 UTC
Created attachment 117007 [details]
sample document demonstrating unicode rendering under different fonts

**Overview**

All LibreOffice releases since the introduction of the 4.2.x have trouble rendering Unicode characters when substituting missing glyphs in a font family.

**Detailed description**

Some fonts do not contain the full unicode character-set. In these cases up until ~4.1.6.2 LibreOffice would substitute missing glyphs with other fonts, usually DejaVu Sans or Serif. This seems to have stopped working properly in more recent releases. The sample document attached to this bug report demonstrates how some Unicode characters (most out of the "arrow" subset) render under different fonts.

**Affected releases**

tested LO release           affected?

3.6.6                       no
3.7.2                       no
4.0.6.2                     no
4.1.6.2                     no
4.2.8.2                     yes
4.3.7                       yes
4.4.4                       yes
5.0.0.2                     yes

**Steps to reproduce**

1. Download the attached sample .odt document
2. Open the document in LibreOffice <=4.1.6.2 and >=4.2.8.2, respectively.
3. Compare how LO renders the Unicode characters depending on the font they are formatted in
    - on LO <=4.1.6.2 all characters will render properly (compare attached pdf file unicode_subistitution_sample_1_LO4162.pdf)
    - on LO >=4.2.8.2 Unicode characters formatted in Helvetica LT Std and Myriad Pro will render with overlapping artifacts or not render at all. Some Unicode characters formatted in DejaVu Serif will also be missing. (compare attached pdf file unicode_subistitution_sample_1_LO444.pdf)

**Additional information**

OS: Ubuntu 12.04 LTS
Comment 1 Glutanimate 2015-07-02 15:52:45 UTC
Created attachment 117008 [details]
Sample document rendered under LibreOffice 4.1.6.2
Comment 2 Glutanimate 2015-07-02 15:53:02 UTC
Created attachment 117009 [details]
Sample document rendered under LibreOffice 4.4.4
Comment 3 Buovjaga 2015-07-02 17:56:45 UTC
Confirmed that it is ok in 4.1.6.2 and not in 4.4.4.

Ubuntu 15.04 64-bit 
Version: 4.4.4.3
Build ID: 40m0(Build:3)
Locale: en_US.UTF-8

Version: 4.1.6.2
Build ID: 40ff705089295be5be0aae9b15123f687c05b0a
Comment 4 Joel Madero 2015-07-18 06:09:08 UTC
 612c411072ede9debf0ceafc3aef125b930b3762 is the first bad commit
commit 612c411072ede9debf0ceafc3aef125b930b3762
Author: Matthew Francis <mjay.francis@gmail.com>
Date:   Sat Mar 14 22:29:35 2015 +0800

    source-hash-3350fabd23859eb5fdbe77c6db9248866d4e2995
    
    commit 3350fabd23859eb5fdbe77c6db9248866d4e2995
    Author:     Miklos Vajna <vmiklos@collabora.co.uk>
    AuthorDate: Mon Jun 30 13:46:04 2014 +0200
    Commit:     Miklos Vajna <vmiklos@collabora.co.uk>
    CommitDate: Tue Jul 1 13:30:12 2014 +0200
    
        SwXAutoStyle::GetPropertyValues_Impl: fix handling of CharAutoStyleName
    
        Without this, all character properties defined inside automatic styles
        are ignored.
    
        Change-Id: I3c25f5d36503a9f40834dd88573a40c34380b967

:040000 040000 8e243f4c177a61cc8517f600627028046c90bd3e b425e995154a715fac9791a97f3c3a881eb8b6ec M	opt


# bad: [cf6ea17155fabb2a120ba07c150735591ac861d7] source-hash-3f94c9e9ddfd807b449f3bb9b232cf2041fa12d2
# good: [fc71ac001f16209654d15ef8c1c4018aa55769f5] source-hash-c15927f20d4727c3b8de68497b6949e72f9e6e9e
git bisect start 'latest' 'oldest'
# bad: [8cf60cc706948588e2f33a6d98b7c55d454e362a] source-hash-f340f0454627939f1830826fb5cc53a90e6c62a4
git bisect bad 8cf60cc706948588e2f33a6d98b7c55d454e362a
# good: [d9885f526fc7a09cc8f9f8ee643af1b966be24bb] source-hash-d1465c64c6f64ad8dd25e40cdc69649b24b305ea
git bisect good d9885f526fc7a09cc8f9f8ee643af1b966be24bb
# bad: [c779cf7967f4d14c5e649a5c696b07279d550efd] source-hash-e9c5022580f14c0ca97503f8b3cc56b530fff174
git bisect bad c779cf7967f4d14c5e649a5c696b07279d550efd
# bad: [030e0477638f9d9110ad62f88fd4b881521e0541] source-hash-1a6e47e3fda10e6d220b67d766ec6fbdfd852b80
git bisect bad 030e0477638f9d9110ad62f88fd4b881521e0541
# bad: [e5b21da1e7dc9ff7754029c50b197590d67638a1] source-hash-579d9ae1c5873f78dbe0364c83842de8556f6ccb
git bisect bad e5b21da1e7dc9ff7754029c50b197590d67638a1
# bad: [3aaaff0da6a48aee7a5e827ffc0fc467dd35aacf] source-hash-0ed069ff8dc941bf902edfefd5c62f1ece3f8ad8
git bisect bad 3aaaff0da6a48aee7a5e827ffc0fc467dd35aacf
# bad: [1c207b72b05a1725334a6da57518e44e3121d79e] source-hash-b807a02850e4b5cc25707e5af8eb36d76c76af61
git bisect bad 1c207b72b05a1725334a6da57518e44e3121d79e
# bad: [bb2988508caa0c52668ef6ee487c865944fb4bf0] source-hash-4776da791f69fa14104f054923be25618c297398
git bisect bad bb2988508caa0c52668ef6ee487c865944fb4bf0
# bad: [dd70af829a4a695ca92bd2228876bfa11ae5211e] source-hash-e50ef195bc95f3f410119f623928382cb88b45d2
git bisect bad dd70af829a4a695ca92bd2228876bfa11ae5211e
# bad: [8f4990439ec6ccc1689fc0792030d66debf9f31f] source-hash-545adb6bdbb2bb5d111f24d8f5b87ec5f262ffe8
git bisect bad 8f4990439ec6ccc1689fc0792030d66debf9f31f
# bad: [612c411072ede9debf0ceafc3aef125b930b3762] source-hash-3350fabd23859eb5fdbe77c6db9248866d4e2995
git bisect bad 612c411072ede9debf0ceafc3aef125b930b3762
# good: [9ca2fb7440b5ed2855b337e99dd02a6e172cbb66] source-hash-9e9e9e4748695a626609b380d8cc4a191c54d83a
git bisect good 9ca2fb7440b5ed2855b337e99dd02a6e172cbb66
# good: [0de5b1eb8036f0a919e4ec76928e785d3098f90b] source-hash-3d399b0e45720354fc64dc3d121ee486e01eff89
git bisect good 0de5b1eb8036f0a919e4ec76928e785d3098f90b
# first bad commit: [612c411072ede9debf0ceafc3aef125b930b3762] source-hash-3350fabd23859eb5fdbe77c6db9248866d4e2995
Comment 5 Matthew Francis 2015-08-26 03:52:21 UTC
(see comment 4)

Adding Cc: to vmiklos@collabora.co.uk; Could you possibly take a look at this one? Thanks
Comment 6 Miklos Vajna 2015-09-04 11:01:09 UTC
Hmm, I don't understand how that above commit would be related to this bug.

I've re-bibisected this using bibisect-44max, and here is what I got:
 7fce465423b50b5065e63b00534acfc034fc60d9 is the first bad commit
commit 7fce465423b50b5065e63b00534acfc034fc60d9
Author: Matthew Francis <mjay.francis@gmail.com>
Date:   Sat Mar 14 20:19:08 2015 +0800

    source-hash-a6b00d16eb27a5e7e31c721671001a909ecef960
    
    commit a6b00d16eb27a5e7e31c721671001a909ecef960
    Author:     Matúš Kukan <matus.kukan@collabora.com>
    AuthorDate: Tue May 27 16:37:30 2014 +0200
    Commit:     Matúš Kukan <matus.kukan@collabora.com>
    CommitDate: Tue May 27 19:38:23 2014 +0200
    
        Related bnc#822625: Cache FontEntry with the original FontSelectPattern.
    
        Otherwise we never hit cache directly, only after expensive call to
        ImplFindByFont.
    
        Change-Id: If15b368feeba94c8fff8ee7cbe049fc4a2069768

:040000 040000 70c1231738ea8a1b62e13e8e5a257cf6c5fbb1ac 87837459bbd0299468cd3065146bddff2454916f M      opt
Comment 7 Miklos Vajna 2015-09-04 11:24:08 UTC
I'll still try to take care of this, though.
Comment 8 Commit Notification 2015-09-04 14:08:13 UTC
Miklos Vajna committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=e047caf4bb461fb168d9fe08228c049938ca1698

tdf#92505 vcl: fix substituting missing glyphs

It will be available in 5.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 9 Commit Notification 2015-09-11 07:44:37 UTC
Miklos Vajna committed a patch related to this issue.
It has been pushed to "libreoffice-5-0":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=170eddb183293bc5740e3d2af09f4bc0a65b985d&h=libreoffice-5-0

tdf#92505 vcl: fix substituting missing glyphs

It will be available in 5.0.3.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 10 Commit Notification 2015-09-14 10:24:42 UTC
Miklos Vajna committed a patch related to this issue.
It has been pushed to "libreoffice-4-4":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=ffd7f291ef1daacc5f4d14e13594c94d3a438a1b&h=libreoffice-4-4

tdf#92505 vcl: fix substituting missing glyphs

It will be available in 4.4.6.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 11 Glutanimate 2015-12-04 20:15:05 UTC
I just gave LO 4.4.6 a try and can confirm that everything is working fine again. Miklos, Joel, and everyone else involved, thanks a lot for putting in the time to fix this!
Comment 12 Robinson Tryon (qubit) 2015-12-17 09:15:39 UTC Comment hidden (obsolete)