Bug 89870 (HarfBuzz) - Unify text layout to use HarfBuzz on all platforms
Summary: Unify text layout to use HarfBuzz on all platforms
Status: RESOLVED FIXED
Alias: HarfBuzz
Product: LibreOffice
Classification: Unclassified
Component: graphics stack (show other bugs)
Version:
(earliest affected)
unspecified
Hardware: Other All
: medium enhancement
Assignee: Khaled Hosny
QA Contact:
URL: https://summerofcode.withgoogle.com/p...
Whiteboard: target:5.3.0 target:5.4.0
Keywords:
Depends on: 103365 103401 103403 103514 103588 103590
Blocks: 52582 58941 66819 Font-Rendering 87731 103596 104281 104403 104930 107718 35320 39351 39805 44688 46055 53312 60268 64559 65344 66818 66916 67317 69895 70317 70775 72940 78340 83819 85852 86311 86768 87679 88021 91946 94981 95060 95836 96266 97329 98269 98527 98785 98879 99677 99878 100120 100169 100746 100813 100936 100973 101599 102224 102271 102916 102944 103103 103137 103217 103361 103468 103908 105448 109142
  Show dependency treegraph
 
Reported: 2015-03-07 09:07 UTC by Khaled Hosny
Modified: 2017-08-15 13:11 UTC (History)
13 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Khaled Hosny 2015-03-07 09:07:20 UTC
Currently we use three different layout systems; Uniscribe on Windows, Core Text on Mac OS X and HarfBuzz everywhere else.

This have the advantage of being consistent with other applications on each platform. However, it has several disadvantages; first we need to maintain 3 different text layout code paths with little sharing between them, which increases the complexity of our text layout code and opens the way to platform-specific bugs. We also are limited on Windows and Mac OS X by what both platforms provide in terms of complex text layout support, so we have don’t have the same language support on, say, Windows XP compared to Windows 10.

HarfBuzz is a free software text layout engine, it is actively developed and supported, both Firefox and Chrome switched to using it on all platforms to avoid the situations outlined above, and I think LibreOffice should do the same.
Comment 1 Phil Krylov 2016-02-07 13:32:43 UTC Comment hidden (me-too)
Comment 2 Volga 2016-02-10 17:47:17 UTC
I also strongly support it. Since Firefox[1] and Chrome used HarfBuzz on all platforms to handle OT layout, and Scribus developer launched ScribusCTL project[2] to intergrate HarfBuzz into Scribus 1.5.1 SVN, I think LibreOffice should following this practice. It will be helpful to make maintain text layout code easier.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=449292
[2] http://www.scribus.net/scribusctl-project-started/
Comment 3 Volga 2016-03-15 16:35:45 UTC
On planet.inkscape.org Inkscape developers have announed they have implemented OT feature controlling based on corresponding attributes of Pango 1.37.1+, which relys on recent version of HarfBuzz.
Comment 4 Volga 2016-03-15 16:47:00 UTC
On Windows 10 LO 5.0.x can access 'liga' and 'kern' feature, but in LO 5.1.1 they are “disabled”, what’s wrong? Using HarfBuzz on all platforms to process OT feature would making text layout behaviors independent of platform, also can lose the burden of maintain text layout codes, and get so easy solutions of bugs related to typography and font features.
Comment 5 akash96j 2016-06-24 14:41:33 UTC
I am working on this on a feature branch.

https://cgit.freedesktop.org/libreoffice/core/log/?h=feature/commonsallayout
Comment 6 Volga 2016-07-02 16:40:39 UTC
> We also are limited on Windows and Mac OS X by what both platforms provide in 
> terms of complex text layout support, so we have don’t have the same language >support on, say, Windows XP compared to Windows 10.

Wikipedia has an article about this problem:
https://en.wikipedia.org/wiki/Uniscribe

Seen from that, you can see the Uniscribe library has different behaviors between defferent versions of Windows. HarfBuzz had optimizations for Windows platform, and the developer want to make it performanced closer to recent version of Uniscribe, so intergrating HarfBuzz on Windows version would get greatly expected effects even if on Windows XP.

For OS X, I have no idea, because I have not found Wikipedia has information about the different behaviors of Core Text within different versions of OS X, but I think this problem may also appearing on OS X, and I have noticed HarfBuzz has improves for XCode recently.
Comment 7 Volga 2016-07-02 16:59:20 UTC
(In reply to akash96j from comment #5)
> I am working on this on a feature branch.
> 
> https://cgit.freedesktop.org/libreoffice/core/log/?h=feature/commonsallayout

Great, after this is successed, I hope the LO developers can make deep collaborations with Harfbuzz developers to fix more typography and font feature related bugs.
Comment 8 Volga 2016-07-25 03:16:55 UTC
This work should intergrate HarfBuzz version 1.3.0+, which works with symbol fonts as Wingdings or MT Extra.
Comment 9 Volga 2016-09-22 03:05:30 UTC
Is it possible to test it on LibreOfficeDev?
Comment 10 Khaled Hosny 2016-09-23 14:48:59 UTC
If you can build from git, the code is on feature/commonsallayout branch.
Comment 11 Volga 2016-09-27 16:14:24 UTC
Recently HarfBuzz fixed building issues with hb-coretext on older OS X versions and graphite2 (without pkg-config) in 1.3.2, and working around the MATH table.
Comment 12 Volga 2016-09-27 18:11:05 UTC
Seen from this commit:
https://cgit.freedesktop.org/libreoffice/core/commit/?h=feature/commonsallayout&id=279407d78c43da63ee88639f00482b1ea703f363

I think Tangut should also apppied the same glyph orientation as CJK texts does in vertical layout.
Comment 13 Volga 2016-09-28 11:00:00 UTC
On W3C website I have also found a tab include some vertical and bi-orientational scripts in Unicode 6.0 and their transformation from horizontal to vertical orientation. 
https://drafts.csswg.org/css-writing-modes-3/#script-orientations
Comment 14 Volga 2016-09-28 11:04:36 UTC
(In reply to General Kutuzov from comment #13)
> On W3C website I have also found a tab include some vertical and
> bi-orientational scripts in Unicode 6.0 and their transformation from
> horizontal to vertical orientation. 
> https://drafts.csswg.org/css-writing-modes-3/#script-orientations

On this page I found Meroitic characters, Egyptian Hieroglyphs and Yi also require this orientation.
Comment 15 Khaled Hosny 2016-09-28 11:13:15 UTC
Thanks for digging this up, I have been wanting to locate the W3C stuff and use it as a reference.
Comment 16 Volga 2016-10-01 14:10:29 UTC
(In reply to Khaled Hosny from comment #15)
> Thanks for digging this up, I have been wanting to locate the W3C stuff and
> use it as a reference.

OK I am glad that you want to apply this glyph orientation for vertical text on these scripts. BTW I found vertical text direction has different direction of columns when I seen from this stuff, vertical CJK use RTL direction for every colums, Mongolian, and Manchu use LTR direction for every lines. Recently I found SignWriting is also use left-to-right, top-to-bottom direction, and I found an example in Wikimedia Incubator:
https://incubator.wikimedia.org/wiki/Wp/ase
Comment 17 Khaled Hosny 2016-10-01 15:40:44 UTC
Right now we are trying to only support vertical text features currently supported by LibreOffice, supporting different column order or BTT direction is out of scope, but feel free to report separate issues for missing features so they can be supported in the future.
Comment 18 Volga 2016-10-05 06:08:47 UTC
For Tangut script, the Unicode Standard said “its implementation requires technology support similar to that used for CJK”, thus Tangut should use the same orientation as CJK, you can also use it as a reference.
Comment 19 Iandol 2016-10-19 06:21:17 UTC
Great to see the GSOC student finished this: https://somefoobar.wordpress.com/2016/08/24/gsoc-project-comes-to-an-end/

What is the plan to enable this into builds by default? I couldn't find it in https://wiki.documentfoundation.org/UnderTheHood/5.3
Comment 20 akash96j 2016-10-19 06:37:26 UTC
(In reply to Iandol from comment #19)
> Great to see the GSOC student finished this:
> https://somefoobar.wordpress.com/2016/08/24/gsoc-project-comes-to-an-end/
> 
> What is the plan to enable this into builds by default? I couldn't find it
> in https://wiki.documentfoundation.org/UnderTheHood/5.3

The code was merged with master yesterday. Currently the new layout engine is disabled by default. It will be enabled by default after some time to ensure no major issues arise.
In the meantime if you want to test the new engine, you can enable it at run time by exporting SAL_USE_COMMON_LAYOUT.
Comment 21 Iandol 2016-10-19 06:55:35 UTC
Great, thanks for your work on improving text rendering for LO! :coffee: Hope the remaining issues (integer > float etc.) get fixed quickly!!!
Comment 22 Andrea 2016-10-20 11:48:00 UTC
Sorry for my deep profanity.
How may I enable it?
Comment 23 Khaled Hosny 2016-10-20 12:07:36 UTC
(In reply to Andrea from comment #22)
> How may I enable it?

Set the environment variable SAL_USE_COMMON_LAYOUT, e.g.:
$ export SAL_USE_COMMON_LAYOUT=1
$ lowriter

or:
$ env SAL_USE_COMMON_LAYOUT=1 lowriter
Comment 24 Volga 2016-10-29 09:33:26 UTC
Recently HarfBuzz have been found some bugs affects some Brahmic scripts as Kannada and Khmer, also affect Old Hangul, maybe focus on furture version of HarfBuzz would be better.
Comment 25 Khaled Hosny 2016-11-03 01:11:52 UTC
The new layout engine have been enabled by default on master with https://gerrit.libreoffice.org/gitweb?p=core.git;a=commitdiff;h=d79f43d0bebbe7b1f8d68d50734737aea9fe839d

To disable it one needs to set SAL_NO_COMMON_LAYOUT environment variable (note the NO this time).

We need to test all the bugs marked as blocked by this and close the ones that are fixed (some might not be completely fixed by this bug, so they should not be mass closed).
Comment 26 Yousuf Philips (jay) 2016-11-03 14:37:39 UTC
(In reply to Khaled Hosny from comment #25)
> The new layout engine have been enabled by default on master with
> https://gerrit.libreoffice.org/gitweb?p=core.git;a=commitdiff;
> h=d79f43d0bebbe7b1f8d68d50734737aea9fe839d
> 
> To disable it one needs to set SAL_NO_COMMON_LAYOUT environment variable
> (note the NO this time).

Running today's master with a blank profile, the about dialog still shows that i'm running the old layout engine.

Version: 5.3.0.0.alpha1+
Build ID: 5d39c2013374727b1c8f147b8b99d54402a7ff02
CPU Threads: 2; OS Version: Linux 3.19; UI Render: default; VCL: gtk2; Layout Engine: old; 
TinderBox: Linux-rpm_deb-x86_64@70-TDF, Branch:master, Time: 2016-11-02_00:55:24
Locale: en-US (en_US.UTF-8); Calc: group
Comment 27 Khaled Hosny 2016-11-03 14:53:44 UTC
(In reply to Yousuf Philips (jay) from comment #26)
> Build ID: 5d39c2013374727b1c8f147b8b99d54402a7ff02

That is not an up to date build (83 commits behind master, and 47 behind the commit that flipped the switch), did you build it yourself?
Comment 28 Yousuf Philips (jay) 2016-11-03 15:27:35 UTC
(In reply to Khaled Hosny from comment #27)
> That is not an up to date build (83 commits behind master, and 47 behind the
> commit that flipped the switch), did you build it yourself?

No i didnt build it, its cloph's daily linux build from yesterday[1], but if it is behind the time you made the switch, i'll grab cloph's build from today [2]. Your patch was dated on the Nov 1st, so i assumed cloph's Nov 2nd build would have it in. :D

[1] http://dev-builds.libreoffice.org/daily/master/Linux-rpm_deb-x86_64@70-TDF/2016-11-02_00.55.24/

[2] http://dev-builds.libreoffice.org/daily/master/Linux-rpm_deb-x86_64@70-TDF/2016-11-02_23.37.13/

Would be useful to have a checkbox in Tools > Options > LibreOffice > Advanced to easily enable and disable this option and its saved in the user profile.
Comment 29 V Stuart Foote 2016-11-03 16:33:13 UTC
(In reply to Yousuf Philips (jay) from comment #28)

> [2]
> http://dev-builds.libreoffice.org/daily/master/Linux-rpm_deb-x86_64@70-TDF/
> 2016-11-02_23.37.13/
> 

Jay, that won't have it either, Cloph includes the build_info.txt for his TBs so simple to grab the SHA1 signature and enter it for the range field in the cgit instance, e.g. https://cgit.freedesktop.org/libreoffice/core/log/?qt=range&q=c8be45889217c555e4bec92af838d0524ceba4e0 to see what commit that ends at.

And to see what is coming in commits for the next build click the "log" tab. The toggle to HarfBuzz is actually the next commit.

> Would be useful to have a checkbox in Tools > Options > LibreOffice >
> Advanced to easily enable and disable this option and its saved in the user
> profile.

Hmm, that could be kind of helpful. The SAL_NO_COMMON_LAYOUT environment variable can be a little tough for users to set--but don't know that a checkbox on the Advanced panel is the best place, while the Expert Configuration is probably too far down in the weeds. Maybe on the Tools -> Options -> View panel? 

And, this is somewhat transient in that the old layout code will probably be stripped out by 5.4.
Comment 30 Khaled Hosny 2016-11-03 16:47:54 UTC
I’m not very thrilled about doing all the UI work for something people shouldn’t normally need to control. The runtime control is there for developers and testers to compare the two engines and catch any regressions. But if we are going to do that, it should be a simple configuration option that can be controlled from the Expert Config; the whole point of this dialog is to not have a dedicated UI for every switch we have.
Comment 31 Commit Notification 2016-11-03 17:24:56 UTC
Khaled Hosny committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=6324efd70dfa8c6be84cba1fa29658e3373cbbe3

tdf#89870: Add config option for layout engine

It will be available in 5.3.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 32 V Stuart Foote 2016-11-04 03:25:47 UTC
(In reply to Commit Notification from comment #31)
> 
> http://cgit.freedesktop.org/libreoffice/core/commit/
> ?id=6324efd70dfa8c6be84cba1fa29658e3373cbbe3
> 

That works well, Thanks Khaled!

Tools -> Options -> Advanced: Expert Configuration search for "TextLayoutEngine" and edit to change "new|old" as needed. 

Version: 5.3.0.0.alpha1+
Build ID: a6ce5d391476e4b6a2cb2d92ff45548c1d75684b
CPU Threads: 8; OS Version: Windows 6.2; UI Render: GL; Layout Engine: old; 
TinderBox: Win-x86@62-merge-TDF, Branch:MASTER, Time: 2016-11-04_00:03:22
Locale: en-US (en_US); Calc: CL
Comment 33 kompilainenn 2016-11-05 09:24:07 UTC
hi, Khaled, thank you for your great job.
but, i have one question: LibreOffice is using old HarfBuzz or new HarfBuzz?
and, finally: please, add checkbox in GUI for activate/deactivate HarfBuzz. finding of this option in Expert Configuration is uncomfortably.
Comment 34 Khaled Hosny 2016-11-05 11:41:01 UTC
(In reply to kompilainenn from comment #33)
> hi, Khaled, thank you for your great job.
> but, i have one question: LibreOffice is using old HarfBuzz or new HarfBuzz?

New HarfBuzz, old HarfBuzz is long dead.

> and, finally: please, add checkbox in GUI for activate/deactivate HarfBuzz.
> finding of this option in Expert Configuration is uncomfortably.

People should be using the new layout engine, the old code is left for debugging and it will be dropped completely in 5.4.
Comment 35 V Stuart Foote 2016-11-05 17:05:25 UTC
@Khaled, would you mind if since HarfBuzz _is_ unified as a common layout, we re-label this as the META tracker it has become and leave it set to NEW?

Keeping it around will let folks see at a glance where things are.
Comment 36 Khaled Hosny 2016-11-05 17:09:57 UTC
I would rather have a different issue for this that tracks only the regressions caused by the new layout engine.
Comment 37 kompilainenn 2016-11-06 21:06:10 UTC Comment hidden (no-value)
Comment 38 Khaled Hosny 2016-11-06 21:11:50 UTC
(In reply to kompilainenn from comment #37)
> (In reply to Khaled Hosny from comment #34)
> > (In reply to kompilainenn from comment #33)
>  
> > > and, finally: please, add checkbox in GUI for activate/deactivate HarfBuzz.
> > > finding of this option in Expert Configuration is uncomfortably.
> > 
> > People should be using the new layout engine, the old code is left for
> > debugging and it will be dropped completely in 5.4.
> 
> IMHO, many people will to want disable this new engine in LO 5.3, because it
> has too many bugs for OS Windows.

If it ends up being so buggy, we won’t be enabling it by default.

> i don't believe, that all bugs for HarfBuzz in Windows will fixed before
> release LO 5.3. sorry

You don’t need to believe in something for it to happen.
Comment 39 Volga 2016-11-12 07:46:03 UTC
(In reply to kompilainenn from comment #37)
> (In reply to Khaled Hosny from comment #34)
> > (In reply to kompilainenn from comment #33)
>  
> > > and, finally: please, add checkbox in GUI for activate/deactivate HarfBuzz.
> > > finding of this option in Expert Configuration is uncomfortably.
> > 
> > People should be using the new layout engine, the old code is left for
> > debugging and it will be dropped completely in 5.4.
> 
> IMHO, many people will to want disable this new engine in LO 5.3, because it
> has too many bugs for OS Windows.
> 
> i don't believe, that all bugs for HarfBuzz in Windows will fixed before
> release LO 5.3. sorry

I guess you should investigate which text and/or fonts misbehaviored with new text layout engine, and when & where do you download the build (Help > About LibreOfficeDev).
Comment 40 Commit Notification 2016-11-25 19:10:32 UTC
Khaled Hosny committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=e838d143178ee82c786d11fd792d8d6735858933

Revert "tdf#89870: Add config option for layout engine"

It will be available in 5.4.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 41 Volga 2017-03-29 08:55:46 UTC Comment hidden (off-topic)