Description: Was testing OpenGL with HarfBuzz enabled and disabled, and noticed odd fallback behavior for the MS system provided bitmap font (Courier, MS Sans Serif, MS Serif) when OpenGL rendering is enabled. A bibisect results to this range: https://cgit.freedesktop.org/libreoffice/core/log/?qt=range&q=6b65a0e83c4798f117be61af91dbaebdc85e94b7..acd3ebefccd0b4570100c426574d83cfa9464f20 So looks that on abandoning Uniscribe shaping and reverting to DirectWrite with GDI for SimpleWinLayout and DirectWrite for OpenGL in WinLayout, that the MS "bitmap" fonts have been rendered blank in the Special Character dialog grid when OpenGL rendering is enabled. Introduction of HarfBuzz shaping with OpenGL results in this font being completely replaced by fallback. Not sure that is correct--but it was different. Steps to Reproduce: Windows builds (only?). 1. enable OpenGL Rendering, restart 2. open Writer 3. open Special Character dialog (default Liberation Serif) 4. navigate the drop list to Courier 5. grid populated? 6. if move to Courier New "ttf" font--its grid populated affecting builds through Version: 5.3.0.0.alpha1+ Build ID: f7c61b08d526c79ecd1522dff79386059b6125e0 CPU Threads: 8; OS Version: Windows 6.19; UI Render: GL; TinderBox: Win-x86@39, Branch:master, Time: 2016-10-25_00:39:40 Locale: en-US (en_US); Calc: CL Actual Results: font grid is empty Expected Results: font grid showing just the Reproducible: Always User Profile Reset: n/a Additional Info: But when enabling HarfBuzz SAL common layout -- (set SAL_USE_COMMON_LAYOUT=1) -- and restart. With OpenGL enabled, same STR shows a full grid of a fallback fonts filling the entire BMP! Worth fixing the "old" OpenGL rendering? Seems like maybe a better way with the new common layout would be to ignore the MS bitmap fonts? They don't show in the paragraph/character drop list of fonts. And with HarfBuzz we drop PS Type1 support--so dropping the *.fon bitmap fonts from the UI may not be too unreasonable. =-ref-= https://support.microsoft.com/en-us/kb/2396756 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:49.0) Gecko/20100101 Firefox/49.0
More than just visibility in the Special Character grid, if text strings with any of these characters are inserted into a document using the Special Character dialog--on enabling OpenGL the text is not rendered visible. And when SAL common layout is enabled--the replacement font is not well rendered --or even the wrong glyphs are substituted, not sure.
Created attachment 128287 [details] ODT test file with strings of MS bitmap fonts A test document, toggle OpenGL rendering on and off--with and without SAL common layout.
We should simply disable bitmap-only fonts, as a matter of fact I did this already last night for the new layout. Can you try with a build that has commit 48304cb5b4d8df71463837e2e48aa7c4d9594df9? (I don’t have a Windows box handy to test right now).
Can you also attach screenshots for the different setups?
Created attachment 128290 [details] zip of four screen clips of rendering MS bitmap fonts (In reply to Khaled Hosny from comment #4) > Can you also attach screenshots for the different setups? attached in a zip archvie
Should add that with OpenGL enabled, it looks like on the Writer canvas the MS Courier is getting a replacement with Courier New TTF. That doesn't happen in the Special Character dialog which goes blank with OpenGL and SAL common layout not active. With common layout, fall back of the entire BMP occurs in the Special Character dialog both with and without OpenGL. =-Sidebar-= Points out a need to add control to the Special Character dialog to offer full BMP/SMP (and other Unicode planes) of a "composite" font -- i.e. what fall-back will be if that font is selected. And a default to show just the glyph coverage of that single font.
confirmed for Version: 5.2.3.1 Build ID: 01ec8f357e651ca9656837b783cf7e6a32ee4d92 CPU Threads: 4; OS Version: Windows 6.1; UI Render: default; Locale: ru-RU (ru_RU); Calc: group
So, this applies to the new common layout, but it will do nothing for releases through 5.2--should the old DirectWrite with OpenGL rendering need to be corrected? (In reply to Khaled Hosny from comment #3) > We should simply disable bitmap-only fonts, as a matter of fact I did this > already last night for the new layout. Can you try with a build that has > commit 48304cb5b4d8df71463837e2e48aa7c4d9594df9? (I don’t have a Windows box > handy to test right now). Testing that on Windows 8.1 Ent 64-bit en-US with Version: 5.3.0.0.alpha1+ Build ID: 9c34797cc98030614b384b6ea0f233d59ae82253 CPU Threads: 8; OS Version: Windows 6.29; UI Render: GL; TinderBox: Win-x86@39, Branch:master, Time: 2016-10-26_03:58:19 Locale: en-US (en_US); Calc: CL Enabling SAL common layout composing the Writer canvas is now ignoring system installed bitmap fonts (on Windows Courier, MS Serif and MS Sans Serif). The WinLayout fallback seems to be making reasonable font selections--Mono spaced for Courier, a Serif font, and a Sans Serif respectively. Unfortunately we have no way to determine exactly the font being substituted. But the Tools -> Options -> Fonts: font table substitution can be used to manually select the replacement font to be used. The Special Character dialog also is ignoring the system installed fonts. With common layout, the dialog is offering system font Segoe UI as the replacement font as the edit cursor is positioned into on a word with the "not installed" font [1]. =-Note-= [1]. Adjusting the Special Character dialog to offer the substituted font at the text cursor rather than Segoe system default at the text cursor would be helpful
(In reply to V Stuart Foote from comment #8) > So, this applies to the new common layout, but it will do nothing for > releases through 5.2--should the old DirectWrite with OpenGL rendering need > to be corrected? I guess we can do the same an ignore bitmap-only fonts when OpenGL is active.
Completely agreed - disabling bitmap fonts, ideally globally is the way to go =) We don't want documents that look different with different rendering backends. There is no good DirectWrite API to render those anymore. Microsoft dropped them (for good reason) in 2013: https://www.poremsky.com/office/type-1-and-other-older-fonts-not-supported-in-office-2013/ =) If we're not using DirectWrite - we will get no accurate physical font API, and so we will have horrible font-fallback behaviour that is de-coupled from glyph-widths. Khaled - sounds like you've fixed this - and VCL is no longer picking up and listing these fonts ? - if so, I'm happy =) I'd love to do that globally for all backends. If so - can we close this ?
@Khaled, * So yes the new HarfBuzz based common layout handles the bitmap/raster fonts nicely with fallback. But can we really call it fixed at this point? Still should do something to likewise "ignore" the bitmap/raster fonts with existing DirectWrite and OpenGL rendering. Otherwise this mishandling will be with us for remainder of 5.2 and likely all of the 5.3 release until we shift entirely to HarfBuzz based shaping at 5.4?
(In reply to V Stuart Foote from comment #11) > @Khaled, * > > So yes the new HarfBuzz based common layout handles the bitmap/raster fonts > nicely with fallback. > > But can we really call it fixed at this point? Still should do something to > likewise "ignore" the bitmap/raster fonts with existing DirectWrite and > OpenGL rendering. > > Otherwise this mishandling will be with us for remainder of 5.2 and likely > all of the 5.3 release until we shift entirely to HarfBuzz based shaping at > 5.4? https://gerrit.libreoffice.org/#/c/30345/
Created attachment 128314 [details] ODT test document with the six fonts to ignore A test file composed with the six fonts that will be ignored. Composed on Windows 10 with 5.2.2.2
Khaled Hosny committed a patch related to this issue. It has been pushed to "master": http://cgit.freedesktop.org/libreoffice/core/commit/?id=8f0f5e0c709d01555a4069f8665889924ed181c7 tdf#103514: Always ignore bitmap fonts on Windows It will be available in 5.3.0. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Do we want to back-port this for 5.2?
A good candidate to backport for 5.2.4
Created attachment 128326 [details] ODT test document with the six fonts to ignore (FODT) as FODT for convenience.
Created attachment 128327 [details] ODT test document with the six fonts to ignore And just so its around the ODF archive version as well.
Back-port to 5.2 up for review... https://gerrit.libreoffice.org/#/c/30356
Khaled Hosny committed a patch related to this issue. It has been pushed to "libreoffice-5-2": http://cgit.freedesktop.org/libreoffice/core/commit/?id=e80621fb04093d5ab950485357f6058dcbaf8aaf&h=libreoffice-5-2 tdf#103514: Always ignore bitmap fonts on Windows It will be available in 5.2.4. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Khaled Hosny committed a patch related to this issue. It has been pushed to "master": http://cgit.freedesktop.org/libreoffice/core/commit/?id=ef4b9032de55e6b1b182e4ead1bbe6e590df296e tdf#103514: Try harder to ignore non-SFNT fonts It will be available in 5.4.0. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Khaled Hosny committed a patch related to this issue. It has been pushed to "libreoffice-5-3": http://cgit.freedesktop.org/libreoffice/core/commit/?id=fcbbd6089da7e60bf616041517a2aee3017ae448&h=libreoffice-5-3 tdf#103514: Try harder to ignore non-SFNT fonts It will be available in 5.3.0.1. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Khaled Hosny committed a patch related to this issue. It has been pushed to "libreoffice-5-2": http://cgit.freedesktop.org/libreoffice/core/commit/?id=f420f4c2e84deb182e4f675c1a0d45bede110ff2&h=libreoffice-5-2 Revert "tdf#103514: Always ignore bitmap fonts on Windows" It will be available in 5.2.5. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Khaled: just to be sure, is it on purpose that the revert has only be done on 5.2 branch? I mean, are 5.3 and master branches ok without the revert? (unless I missed something)
It was reverted on 5.2 only to avoid regression on a stable branch, while we try to find a proper fix for 5.3 (which is not released yet).
(In reply to Khaled Hosny from comment #25) > It was reverted on 5.2 only to avoid regression on a stable branch, while we > try to find a proper fix for 5.3 (which is not released yet). Ok, thank you for the explanation.
*** Bug 105315 has been marked as a duplicate of this bug. ***
(In reply to Michael Meeks from comment #10) > Completely agreed - disabling bitmap fonts, ideally globally is the way to > go =) We don't want documents that look different with different rendering > backends. There is no good DirectWrite API to render those anymore. > Microsoft dropped them (for good reason) in 2013: > https://www.poremsky.com/office/type-1-and-other-older-fonts-not-supported- > in-office-2013/ =) If we're not using DirectWrite - we will get no accurate > physical font API, and so we will have horrible font-fallback behaviour that > is de-coupled from glyph-widths. > > Khaled - sounds like you've fixed this - and VCL is no longer picking up and > listing these fonts ? - if so, I'm happy =) I'd love to do that globally for > all backends. > If so - can we close this ? That was a terrible choice. Honestly, why is it not OPTIONAL? Now my Calc table is all the way over with a wrong font.
(In reply to De-M-oN from comment #28) > > That was a terrible choice. Honestly, why is it not OPTIONAL? > Now my Calc table is all the way over with a wrong font. No it is not possible to make it OPTIONAL. It is no longer technically feasible to support those fonts as the libraries the project has moved to cross platform do not support them. Meaning, rework your sheets to use supported fonts. As a temporary work around to fully restructuring the sheets, you can use a Font Replacement for unsupported bitmap and PS Type 1 fonts--Tools -> Options -> Fonts: "Apply replacement table" checkbox. You'll need to know the system font name and pick an installed TTF/OTF replacement font. But it is temporary to be able to view/configure the document with visible fonts--but can not be saved.
(In reply to V Stuart Foote from comment #29) > No it is not possible to make it OPTIONAL. It is no longer technically > feasible to support those fonts as the libraries the project has moved to > cross platform do not support them. Meaning, rework your sheets to use > supported fonts. Can you list the relevant libraries? Perhaps an issue could be filed against those, for adding bitmap and Type 1 font support?
(In reply to Eyal Rozenberg from comment #30) > (In reply to V Stuart Foote from comment #29) > > No it is not possible to make it OPTIONAL. It is no longer technically > > feasible to support those fonts as the libraries the project has moved to > > cross platform do not support them. Meaning, rework your sheets to use > > supported fonts. > > Can you list the relevant libraries? Perhaps an issue could be filed against > those, for adding bitmap and Type 1 font support? Mainly HarfBuzz, but HarfBuzz is not going to supportsuch fonts. We can still support them in LibreOffice, though, as HarfBuzz is flexible enough to allow us to do that on top if it but it needs someone to volunteer doing it. I can give code pointers for anyone interested.