Hello May be related to Bug 59184 (linux Issue) I open this one for Windows Description : Unicode character are not displayed Platform 4.1.3.1 and 4.2.0.0.alpha0+ Windows 7 1. Deleting profile (Delete the profile is not necessary to reproduce the problem. This step is only to exclude a possible cause). 2. Launch LibreOffice 3. File> Open => "Test document with unicode caracter" attachment 72757 [details] Unicode characters are not displayed *and* the preview does not work in the font list (see UnicodeFileOpen.png screenshot) attachment 89147 [details] Quit LibreOffice 1. Deleting profile 2. Double click in Explorer windows on the document The document loads, the unicode characters are displayed (and preview works) (see UnicodeOpenExpl.png screenshot). attachment 89146 [details] The problem is also reproduced (qa-fr) with LO Portable 3.6.6.2, 4.1.3.2 LO, LO 4.2.0.0.alpha0+ on Windows 7 Home Premium. Regards Pierre-Yves
Hello, I confirm this behaviour for LO Portable 3.6.6.2, 4.1.3.2 LO, LO 4.2.0.0.alpha0+ on Windows 7 Home Premium. Jacques
** Please read this message in its entirety before responding ** To make sure we're focusing on the bugs that affect our users today, LibreOffice QA is asking bug reporters and confirmers to retest open, confirmed bugs which have not been touched for over a year. There have been thousands of bug fixes and commits since anyone checked on this bug report. During that time, it's possible that the bug has been fixed, or the details of the problem have changed. We'd really appreciate your help in getting confirmation that the bug is still present. If you have time, please do the following: *Test to see if the bug is still present on a currently supported version of LibreOffice (4.4.1 or later) https://www.libreoffice.org/download/ *If the bug is present, please leave a comment that includes the version of LibreOffice and your operating system, and any changes you see in the bug behavior *If the bug is NOT present, please set the bug's Status field to RESOLVED-WORKSFORME and leave a short comment that includes your version of LibreOffice and Operating System Please DO NOT *Update the version field *Reply via email (please reply directly on the bug tracker) *Set the bug's Status field to RESOLVED - FIXED (this status has a particular meaning that is not appropriate in this case) If you want to do more to help you can test to see if your issue is a REGRESSION. To do so: 1. Download and install oldest version of LibreOffice (usually 3.3 unless your bug pertains to a feature added after 3.3) http://downloadarchive.documentfoundation.org/libreoffice/old/ 2. Test your bug 3. Leave a comment with your results. 4a. If the bug was present with 3.3 - set version to "inherited from OOo"; 4b. If the bug was not present in 3.3 - add "regression" to keyword Feel free to come ask questions or to say hello in our QA chat: http://webchat.freenode.net/?channels=libreoffice-qa Thank you for your help! -- The LibreOffice QA Team This NEW Message was generated on: 2015-04-18
Hi Still reproduced on windows 7/64 & Version: 4.4.3.1 Build ID: b2f347f2ac68821efc00b6f1793cda90af748118 Locale: fr_FR Regards Pierre-Yves
The issue is with the font set--those glyphs are simply not defined in the Liberation Serif font set. Easily verified with a Unicode block display utility like BableMap. And, when sample document is edited and paragraph styles modified to use a font that contains glyphs for those Unicode pages they display correctly. Segoe UI Symbol for example provides glyphs for all. Liberation Sans and OpenSymbol do not. Suppose it is possible a replacement table had been defined, seems less likely that glyphs had been dropped from the fonts. Anyhow, this is working correctly in current master and 5.1 and 5.0 releases. See attached clip.
Created attachment 121756 [details] clip from current master of document with Segoe UI Symbol glpyhs
(In reply to V Stuart Foote from comment #4) > > Anyhow, this is working correctly in current master and 5.1 and 5.0 releases. > > See attached clip. Whoops not so fast. Seems it is *not* correct in 5.0 or 5.1 font faces with defined glyphs are rendering correctly on master (but only checked back to 14 December so not sure when it was picked up). Back to NEW. @Chris, Tor noticed you've been poking at salfont and salgdi3 -- had this behavior somehow been a loss in dynamic glyph fallback that has now cleared? =-=-= ref Herbert D.'s work on AOO https://bz.apache.org/ooo/show_bug.cgi?id=101552
V: Chris's latest work is in master only, not in 5.1 and even less in 5.0.
We need to adjust the summary and the STR Fallback handling of Unicode font replacement on Windows affected by method of launch. On Windows -- since it is apparently working correctly in Linux (bug 59184 with http://cgit.freedesktop.org/libreoffice/core/commit/?id=5a0984fd383e54c130016d15a0b72c230968c61e&h=libreoffice-4-1 ) -- using the OS to launch the document (i.e. Open from Windows Explorer) is fully performing fallback font replacement in LibreOffice. But, not the same if LibreOffice is already running. Fallback font replacement is not occurring for Unicode panes. STR with test document attachment 72757 [details] A. Correct fallback 1. open Windows Explorer 2. locate downloaded attachment 3. open (with double click, or context menu) result: Document opens, paragraph styled with Liberation Serif but containing Unicode code points not included in that font face are rendered with alternates (Segoe UI Symbol). 4. close, reopen from Start Center or MRU list. result: Document opens, fallback again performed, code points rendered with replacement graphemes rendered. B. Incorrect fallback 1. close LibreOffice 2. Open to StartCenter 3. open (Open, Ctrl+O, File -> Open) the attachment; result: Document opens, but paragraphs styled with Liberation Serif and containing Unicode code points not included in Liberation Serif panes are not rendered by fallback to alternate fonts. Additionally with current builds of master/5.2.0alpha0 > 2016-01-03 the metrics being used seems to have shifted resulting in excessive line spacing. Note: when launched from CLI with swriter.exe <attachment> builds through 5.1 perform fallback correctly. But are affected by Chris S' current work > 2016-01-03 on master.
Drat. Sorry, I honestly didn't notice this bug till now, got lost in a mail filter. I'll look at this tomorrow. I'd say it's been caused by some refactoring I've done, which I'm going to test and then revert anyway.
Hmmm... The fact it's working in master (partially) may be a happy accident. There are a few things I did, the first was that I streamlined the ImplFontAttributes, ImplDevFontAttributes and ImplFontMetricData classes (which I am now seriously considering to have been a bit of a mistake!) into ImplFontAttributes. The other thing I've done is put accessors and mutators around ImplFontAttributes (now FontAttributes) because I ultimately want to move away from making PhysicalFontFace and FontSelectPattern inherit from FontAttributes; a font has-a collection of FontAttributes after all, arguably it is not an is-a relationship. Pointers to the code for my own reference: LogicalFontInstance::GetGallbackForUnicode is the function that finds the fallback glyph if it's not in the existing font instance. http://opengrok.libreoffice.org/xref/core/vcl/source/font/fontinstance.cxx#GetFallbackForUnicode Could it be that initialising the constructor attributes explicitly has worked around some sort of undefined behaviour? Will need to bisect this tomorrow. I suppose I'm finally going yo have to install lode on my Windows laptop and OS X machine. :-)
Or... Fallback was not working due to a bug in the hash map implementation? When I followed the recommended practice of extending the std namespace with a custom hash I have a sneaking suspicion that when I rewrote it to make it clearer to myself it potentially got the hashcode working.... http://cgit.freedesktop.org/libreoffice/core/commit/vcl/source/font?id=50261ea4b7242e365717c77a9370afd1044a97e3
Better cgit reference to commit: http://cgit.freedesktop.org/libreoffice/core/commit/?id=50261ea4b7242e365717c77a9370afd1044a97e3
Ah. Sadly, I cannot take any glory for fixing this bug. I suspect this was fixed in bug 92505, I believe there was a bug in the font cache (actually, if it was this then I'm concerned that initialising FontSelectPattern with a copy constructor is blasting a member variable...). Commit by Miklos Vajna: http://cgit.freedesktop.org/libreoffice/core/commit/vcl/source/outdev/font.cxx?id=e047caf4bb461fb168d9fe08228c049938ca1698
Sorry for bug spam. Test this out tomorrow.
Created attachment 121863 [details] zip with screenclips of opening Unicode using fallbacks differ by method of launch -- StartCenter or OS Chris, * (In reply to Chris Sherlock from comment #13) > Ah. Sadly, I cannot take any glory for fixing this bug. I suspect this was > fixed in bug 92505, I believe there was a bug in the font cache (actually, > if it was this then I'm concerned that initialising FontSelectPattern with a > copy constructor is blasting a member variable...). > Still seems there is an issue in selecting the fallback font used for replacement of missing codepoint glyphs, from BMP Unicode as in 92505, and as here from the SMP Unicode plane Attaching screen clip examples (with 2016-01-06 master and 5.1.0.1--as named in zip) using the testdoc from bug 92505, using only BMP codepoints, but still displays distinction in fallback replacements made depending on if opening from OS or if launching from StartCenter with LO already running. With the SMP (the 1F300 codepoints of this issue and bug 59184) no fallback replacements occur when LibreOffice is already running on launch from StartCenter. But on initial launch from CLI or OS, a full slate of replacements glyphs are found. Is choice of fallback fonts in Windows somehow tied to user profile/locale? Recall browsing through code that excluded some unlikely fonts based on selected language--that was Caolán's work on the Linux fontconfig.cxx (http://cgit.freedesktop.org/libreoffice/core/commit/?id=5a0984fd383e54c130016d15a0b72c230968c61e&h=libreoffice-4-1)
I'll have to dig into the code... Sorry if this a silly question, but those latest screenshots - is the original document attached to this bug? I want to load it up and see what happens on an OS X and Linux system.
Created attachment 121921 [details] sample with both BMP and SMP codepoints that have fallback font problems @Chris, * (In reply to Chris Sherlock from comment #16) > I'll have to dig into the code... > > Sorry if this a silly question, but those latest screenshots - is the > original document attached to this bug? I want to load it up and see what > happens on an OS X and Linux system. They were all launches of attachment 117007 [details] (from bug 92505) But, I've attached a new combined sample document combining issues of BMP codepoints (from 0380, 2100, 2700, 2900) and SEP codepoints (from 1F300). Problems with fallback font replacement in both Unicode planes.
Created attachment 122044 [details] Autocorrect support for Emoji inserts clock face codepoints As in bug 97191 -- the clock face Emoji codepoints 1F559, 1F55A, 1F55B are another issue with fallback font handling. The autocorrect replacement is done, but without glyph available in the grapheme for current font--the space is rendered empty in the document. User thinks there is a problem and clobbers the codepoint rather than a Ctrl-Z to revert the autocorrect.
On Windows 10 Pro 64-bit en-US with Version: 5.2.0.0.alpha0+ (x64) Build ID: 042f16a19e3d5f884759dae71264433b988df0e6 CPU Threads: 8; OS Version: Windows 6.19; UI Render: GL; TinderBox: Win-x86_64@62-TDF, Branch:MASTER, Time: 2016-03-16_10:03:38 Locale: en-US (en_US) SimpleWinLayout rendering with the DirectWrite implementation for OpenGL rendering has not corrected the fallback font mechanism. I did not expect it to, but kind of hoped so. The method of launch is still influencing if font fallback replacement occurs when opening attachment 121921 [details] Continues to affect both OpenGL rendering and default rendering.
Ah, we already have a bug report for this phenomenon, good. And all back from 2013, even. I have recently noticed this while investigating several other bugs.
STR as in comment 8 continue valid on Windows 10 Pro 64-bit en-US with installed Version: 5.2.0.4 (x64) Build ID: 066b007f5ebcc236395c7d282ba488bca6720265 CPU Threads: 8; OS Version: Windows 6.19; UI Render: GL; Locale: en-US (en_US) and with a current master /a parallel install (Windows Explorer openwith and navigate to soffice.exe) Version: 5.3.0.0.alpha0+ Build ID: 626a1aa960ed36f5c3370e2aab1e72d41df875cf CPU Threads: 8; OS Version: Windows 6.19; UI Render: GL; TinderBox: Win-x86@42, Branch:master, Time: 2016-08-04_01:38:13 Locale: en-US (en_US); Calc: CL
Hi Chris, I'm setting this ticket back to NEW as it has been inactive for more than 3 months. Feel free to assign it back to you if you're still working on this. Regards
Created attachment 128383 [details] sample document mixed language multiple fonts Looking at experimental HarfBuzz based common layout (available on master from 2016-10-21, and in 5.3.0alpha1) and find fallback font use on Windows builds are still affected by the means of launching Writer module to edit a file. Attaching a new sample document to better challenge HarfBuzz layout. It uses a common paragraph rendered in Symbola (ttf-ancient-fonts), DejaVu Sans, Arial Unicode, and Noto Emoji. With a SAL_USE_COMMON_LAYOUT variable set when launched from Start Center receive one layout. And when launched from a command windows with swriter.exe tdf71603_fallback_testing_All_BMP.odt you get quite different layout result.
@Khaled, * Not sure it is reasonable to expect the new layout engine to rectify this old Windows font glitch--but it is certainly affected by it. Opening the new test document from the command line or in a new session from the Start Center should show the issue well.
This is a font fallback issue which is orthogonal to the work done in bug 89870.
Khaled Hosny committed a patch related to this issue. It has been pushed to "master": http://cgit.freedesktop.org/libreoffice/core/commit/?id=03bff1b6b953e4b7a54d2fb7bbf366bea7e959d9 tdf#71603: Improve font fallback on Windows a bit It will be available in 5.3.0. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Khaled Hosny committed a patch related to this issue. It has been pushed to "master": http://cgit.freedesktop.org/libreoffice/core/commit/?id=5d39c2013374727b1c8f147b8b99d54402a7ff02 tdf#71603: Create a new DC for the font fallback It will be available in 5.3.0. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Took a bit of debugging, but should be fixed now (see details in the above commit message).
On Windows 10 Pro 64-bit en-US with Version: 5.3.0.0.alpha1+ Build ID: 5d39c2013374727b1c8f147b8b99d54402a7ff02 CPU Threads: 8; OS Version: Windows 6.2; UI Render: GL; Layout Engine: new; TinderBox: Win-x86@62-merge-TDF, Branch:MASTER, Time: 2016-11-02_01:01:09 Locale: en-US (en_US); Calc: CL Version: 5.3.0.0.alpha1+ Build ID: 5d39c2013374727b1c8f147b8b99d54402a7ff02 CPU Threads: 8; OS Version: Windows 6.2; UI Render: GL; Layout Engine: old; TinderBox: Win-x86@62-merge-TDF, Branch:MASTER, Time: 2016-11-02_01:01:09 Locale: en-US (en_US); Calc: CL Documents open the same now if launched from Start Center/Open, and if launched form command line wint the new HarfBuzz and old layout engine. Also with the default GDI SimpleWinLayout. There is different fallback selection occurring between old layout and new HarfBuzz based layout--but it is now consistent by method of launch. A nice bonus of the patch is that the AutoCorrect Options dialog now shows the replacement glyph for all emoji and special characters fixing bug 92940