Problem description: Special spaces such as thin space (u2006) and hair space (u200a) aren’t displayd correctly but as normal spaces. Steps to reproduce: 1. Insert e.g. u2006 in your text by pressing Ctrl-Shift+u+2006 Current behavior: A normal-width space is inserted Expected behavior: A thin space should be inserted Operating System: Linux (Other) Version: 4.1.0.2 rc Last worked in: 4.0.4.1 rc
Last working version was 4.1.0rc1. Sorry for misinformation.
This bug is still there in 4.1.0-rc3 – did a fresh install, new profile, but the problem persists.
apparently using the wrong font, problem happens with /usr/share/fonts/liberation/LiberationSerif-Regular.ttf but not the bundled one. weirdly it also only happens with an installed LO but not a dev-install. fixed on master, now get about 30 more duplicate fonts discarded.
Michael Stahl committed a patch related to this issue. It has been pushed to "master": http://cgit.freedesktop.org/libreoffice/core/commit/?id=1c48e4efa2369e5708798bdefb46b74a86415d00 fdo#66715: fontconfig: try harder to ignore duplicate fonts The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Michael Stahl committed a patch related to this issue. It has been pushed to "libreoffice-4-1": http://cgit.freedesktop.org/libreoffice/core/commit/?id=ff8a6b661dc304cb3a858facbd0dbda66ff01641&h=libreoffice-4-1 fdo#66715: fontconfig: try harder to ignore duplicate fonts It will be available in LibreOffice 4.1.1. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
This bug is *not* resolved in 4.1.1rc1: special spaces work again for only some fonts, for most fonts installed on my system, though, they still don’t.
indeed, problem still happens with system fonts, for example "Bitstream Charter" on Fedora 19, which worked with LO 4.0.4.2. bibisect says: 3def5194ddaf9c4d766b71527874bd1a973b43e5..9a7603187eb5cc580d33212ee147f9ac89de55f4 which contains this: commit bff8fa97e16f0f06fddc5545ea36c8bd2b18a580 Author: Khaled Hosny <khaledhosny@eglug.org> AuthorDate: Mon May 6 11:08:29 2013 +0200 Enable HarfBuzz by default HarfBuzz integration should be functional now, so to give it more wider testing it is made now a required dependency on non-Windows non-Mac OSs. By default text layout is now done by HarfBuzz but ICU LayoutEngine is kept as a fallback and can be enabled with SAL_USE_ICULE env variable. ... and runninng current libreoffice-4-1 branch exhibits the problem (thin spaces have full width, presumably because the font does not contain glyphs for them (Bitstream Charter does not according to fontforge)), but if one sets the SAL_USE_ICULE variable the thin spaces work, so likely the harfbuzz layout is not perfect yet and missing some fallback. Khaled, do you have some time to look at this?
This is a HarfBuzz “feature”, when a character is not supported by the font it tries to use Unicode normalization to find an equivalent character that is supported by the font. It seems all those special spaces decompose to the ASCII space, U+0020, so HarfBuzz ends up using it and we end up thinking the font supports that character and do not trigger our font fallback mechanisms. We can query the font’s “cmap” table to find what characters are supported before passing the text to HarfBuzz, but then we lose HarfBuzz’s normalization fallback completely which is quite a useful feature in many other cases. Behdad, is there a way to handle the spaces differently on HarfBuzz’s side?
(In reply to comment #8) > This is a HarfBuzz “feature”, when a character is not supported by the font > it tries to use Unicode normalization to find an equivalent character that > is supported by the font. It seems all those special spaces decompose to the > ASCII space, U+0020, so HarfBuzz ends up using it and we end up thinking the > font supports that character and do not trigger our font fallback > mechanisms. We can query the font’s “cmap” table to find what characters are > supported before passing the text to HarfBuzz, but then we lose HarfBuzz’s > normalization fallback completely which is quite a useful feature in many > other cases. So, how is your font fallback triggered? Do you check for notdef glyphs in harfbuzz output or what? Because harfbuzz doesn't choose the font, harfbuzz client does. > Behdad, is there a way to handle the spaces differently on HarfBuzz’s side? Of course, just use a custom unicode-funcs that has whatever you want for the decompose_compatibility().
(In reply to comment #9) > (In reply to comment #8) > > This is a HarfBuzz “feature”, when a character is not supported by the font > > it tries to use Unicode normalization to find an equivalent character that > > is supported by the font. It seems all those special spaces decompose to the > > ASCII space, U+0020, so HarfBuzz ends up using it and we end up thinking the > > font supports that character and do not trigger our font fallback > > mechanisms. We can query the font’s “cmap” table to find what characters are > > supported before passing the text to HarfBuzz, but then we lose HarfBuzz’s > > normalization fallback completely which is quite a useful feature in many > > other cases. > > So, how is your font fallback triggered? Do you check for notdef glyphs in > harfbuzz output or what? We just check for glyph index == zero and trigger fallback in this case. > > Behdad, is there a way to handle the spaces differently on HarfBuzz’s side? > > Of course, just use a custom unicode-funcs that has whatever you want for > the decompose_compatibility(). I should have thought of that :) We currently use the built-in ICU unicode-funcs, not sure if one can change just decompose_compatibility(), but I’ll give it a try
(In reply to comment #10) > (In reply to comment #9) > > (In reply to comment #8) > > > This is a HarfBuzz “feature”, when a character is not supported by the font > > > it tries to use Unicode normalization to find an equivalent character that > > > is supported by the font. It seems all those special spaces decompose to the > > > ASCII space, U+0020, so HarfBuzz ends up using it and we end up thinking the > > > font supports that character and do not trigger our font fallback > > > mechanisms. We can query the font’s “cmap” table to find what characters are > > > supported before passing the text to HarfBuzz, but then we lose HarfBuzz’s > > > normalization fallback completely which is quite a useful feature in many > > > other cases. > > > > So, how is your font fallback triggered? Do you check for notdef glyphs in > > harfbuzz output or what? > > We just check for glyph index == zero and trigger fallback in this case. Good to know. Thanks. > > > Behdad, is there a way to handle the spaces differently on HarfBuzz’s side? > > > > Of course, just use a custom unicode-funcs that has whatever you want for > > the decompose_compatibility(). > > I should have thought of that :) We currently use the built-in ICU > unicode-funcs, not sure if one can change just decompose_compatibility(), > but I’ll give it a try Sure you can. That's what the parent arg in hb_unicode_functs_create() is for. And you can chain to the parent, check the result to see if it's a sole space character, and return false.
Given the nature of compatibility decomposition, I think we are better off avoiding it entirely and allowing for font fallback instead, so I just did that.
Khaled Hosny committed a patch related to this issue. It has been pushed to "master": http://cgit.freedesktop.org/libreoffice/core/commit/?id=1f9d1e15634b1ef1dea37dc00c7bacb4f97996d6 fdo#66715: Avoid compatibility decomposition with HarfBuzz The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
You can use a static ufuncs. No need to create one every time.
(In reply to comment #14) > You can use a static ufuncs. No need to create one every time. Makes sense, I should do the same for font functions as well.
(In reply to comment #15) > (In reply to comment #14) > > You can use a static ufuncs. No need to create one every time. > > Makes sense, I should do the same for font functions as well. http://cgit.freedesktop.org/libreoffice/core/commit?id=94a742fbc092eb75dbcffcbce0ac96679c90add4
Khaled Hosny committed a patch related to this issue. It has been pushed to "libreoffice-4-1": http://cgit.freedesktop.org/libreoffice/core/commit/?id=fbaedf8bd45d3cc4409c92a4b798d9d9788567ed&h=libreoffice-4-1 fdo#66715: Avoid compatibility decomposition with HarfBuzz It will be available in LibreOffice 4.1.2. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.