Bug 70666 - EDITING: Diacritic placement problem with Graphite font
Summary: EDITING: Diacritic placement problem with Graphite font
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: graphics stack (show other bugs)
Version:
(earliest affected)
4.1.3.1 rc
Hardware: Other Windows (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: BSA target:4.3.0 target:4.2.4
Keywords: regression
Depends on:
Blocks:
 
Reported: 2013-10-20 11:38 UTC by Floater
Modified: 2016-05-30 18:55 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Bug illustration (23.73 KB, application/pdf)
2013-10-20 11:38 UTC, Floater
Details
Expected behaviour (41.61 KB, application/pdf)
2013-10-20 11:41 UTC, Floater
Details
odt file illustrating the bug (19.37 KB, application/vnd.oasis.opendocument.text)
2013-10-20 11:42 UTC, Floater
Details
Graphite font file (435.66 KB, application/x-font-ttf)
2013-10-20 11:43 UTC, Floater
Details
Layout problem (double diacritics) (10.17 KB, application/vnd.oasis.opendocument.text)
2014-04-04 15:35 UTC, László Németh
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Floater 2013-10-20 11:38:31 UTC
Created attachment 87868 [details]
Bug illustration

Problem description: Diacritics either do not appear (under char marked yellow in the demo doc), or occasionally, are not positioned properly - (marked red in attached doc)

Steps to reproduce:
1. ....
2. ....
3. ....

Current behavior: As in attached .odt document.

Expected behavior:As in the attached pdf created from using Worldpad

              
Operating System: Windows XP
Version: 4.1.3.1 rc
Comment 1 Floater 2013-10-20 11:41:19 UTC
Created attachment 87869 [details]
Expected behaviour
Comment 2 Floater 2013-10-20 11:42:35 UTC
Created attachment 87870 [details]
odt file illustrating the bug
Comment 3 Floater 2013-10-20 11:43:47 UTC
Created attachment 87871 [details]
Graphite font file
Comment 4 tommy27 2013-11-23 03:59:41 UTC
issue confirmed under Win7 64bit with 4.1.3.2 
works fine with 4.1.2, 4.1.1 and 4.1.0

so it's a very recent regression of the 4.1.3.x codeline
I add Writer expert to CC list
Comment 5 Michael Stahl (allotropia) 2014-02-04 13:23:56 UTC
regression from:

commit bb09c7c14bb1caf7b08b39944bda61382b158c64
Author:     László Németh <nemeth@numbertext.org>
AuthorDate: Fri Sep 6 11:08:02 2013 +0200

    fdo#68313 fix combining diacritics problem with Graphite fonts
Comment 6 László Németh 2014-02-04 13:53:46 UTC
I will fix it soon. Thanks for your bug report!
Comment 7 Commit Notification 2014-02-04 21:06:03 UTC
Laszlo Nemeth committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=a831930986166a8c5cb231821426f5cf4d976df2

fdo#70666 fix Graphite ligature replacement at line breaks



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 8 László Németh 2014-02-06 08:03:56 UTC
The problem was the special status of the character (not a combining character, but it acts like a combining character, see http://codepoints.net/U+0952). Now every ligatures are replaced, except the lower case letters to fix the problem of the hyphenation within the f and other western ligatures.
Comment 9 László Németh 2014-04-04 12:12:38 UTC
Martin: I have forgot to use the original gr_count_unicode_characters() in my last patch. It works well, but is there a performance issue here?

[Also I am very interested in the word and line end dependent font features, yet. Maybe I have to add an extra space here to fix the explicit space related italic correction feature of Linux Libertine G.]
 
> mst__ 13.45.40
nemeth, hello do i see it right that https://bugs.freedesktop.org/show_bug.cgi?id=70666 is fixed only on master but not release branches? any reason why not?
> mst__: It seems, I have forgot it. (Precisely, I had an idea to change my last patch to add the Graphite Unicode helper function, as in the original code of Martin Hosken, maybe it would be an optimization, but I haven’t done it. I will ask Martin in the issue.)
Comment 10 László Németh 2014-04-04 15:35:10 UTC
Created attachment 96907 [details]
Layout problem (double diacritics)
Comment 11 László Németh 2014-04-04 15:37:31 UTC
Michael Stahl has noticed that this issue hasn't been solved completely, yet, because it reports an older problem, too.

I have attached a test file for the second problem, containing the same words, but two spaces before the word breaks the correct layout with an extra diacritics).

> mst__: My fix is good for the problem caused by me.  It seems, LibreOffice has never supported that diacritics correctly: before Graphite 2 integration, there was only bad positions, with Graphite 2, double diacritics (in good and the old wrong positions). So there are two different problems in this issue.  Interestingly, removing the text before the word fix the double diacritics problem, too.
Comment 12 martin_hosken 2014-04-05 12:23:31 UTC
On Fri, 04 Apr 2014 12:12:38 +0000
bugzilla-daemon@freedesktop.org wrote:

> https://bugs.freedesktop.org/show_bug.cgi?id=70666
> 
> --- Comment #9 from László Németh <nemeth@numbertext.org> ---
> Martin: I have forgot to use the original gr_count_unicode_characters() in my
> last patch. It works well, but is there a performance issue here?
> 
> [Also I am very interested in the word and line end dependent font features,
> yet. Maybe I have to add an extra space here to fix the explicit space related
> italic correction feature of Linux Libertine G.]
> 
> > mst__ 13.45.40
> nemeth, hello do i see it right that
> https://bugs.freedesktop.org/show_bug.cgi?id=70666 is fixed only on master but
> not release branches? any reason why not?
> > mst__: It seems, I have forgot it. (Precisely, I had an idea to change my last patch to add the Graphite Unicode helper function, as in the original code of Martin Hosken, maybe it would be an optimization, but I haven’t done it. I will ask Martin in the issue.)

If you are using UTF16, it is still worth using the gr_count_unicode_characters() in case there are any surrogate pairs in the text. If there are none, then the character count is the same as the number of words, but if there are any surrogate pairs then the count will be less than the number of words. In addition, the text reading code is strict about using the number of characters passed in. So if your number is too high, it will try to read beyond the end of the string even if there is a guard NULL there.

BTW in case folks are feeling edgy about graphite's speed, there is special speedup code in there such that, for example, if you use Charis with plain old ASCII text, it will execute twice as fast as the new harfbuzz. OK, so to get that kind of speedup one does have to think about about one's gdl, but it's not that hard. And even without the speed up, it's about the same speed. gr_count_unicode_characters() is no slouch either.

Yours,
Martin
Comment 13 László Németh 2014-04-09 09:16:12 UTC
(In reply to comment #12)
> On Fri, 04 Apr 2014 12:12:38 +0000
> bugzilla-daemon@freedesktop.org wrote:
> 
> > https://bugs.freedesktop.org/show_bug.cgi?id=70666
> > 
> > --- Comment #9 from László Németh <nemeth@numbertext.org> ---
> > Martin: I have forgot to use the original gr_count_unicode_characters() in my
> > last patch. It works well, but is there a performance issue here?
> > 
> > [Also I am very interested in the word and line end dependent font features,
> > yet. Maybe I have to add an extra space here to fix the explicit space related
> > italic correction feature of Linux Libertine G.]
> > 
> > > mst__ 13.45.40
> > nemeth, hello do i see it right that
> > https://bugs.freedesktop.org/show_bug.cgi?id=70666 is fixed only on master but
> > not release branches? any reason why not?
> > > mst__: It seems, I have forgot it. (Precisely, I had an idea to change my last patch to add the Graphite Unicode helper function, as in the original code of Martin Hosken, maybe it would be an optimization, but I haven’t done it. I will ask Martin in the issue.)
> 
> If you are using UTF16, it is still worth using the
> gr_count_unicode_characters() in case there are any surrogate pairs in the
> text. If there are none, then the character count is the same as the number
> of words, but if there are any surrogate pairs then the count will be less
> than the number of words. In addition, the text reading code is strict about
> using the number of characters passed in. So if your number is too high, it
> will try to read beyond the end of the string even if there is a guard NULL
> there.
> 
> BTW in case folks are feeling edgy about graphite's speed, there is special
> speedup code in there such that, for example, if you use Charis with plain
> old ASCII text, it will execute twice as fast as the new harfbuzz. OK, so to
> get that kind of speedup one does have to think about about one's gdl, but
> it's not that hard. And even without the speed up, it's about the same
> speed. gr_count_unicode_characters() is no slouch either.
> 
> Yours,
> Martin

Martin, many thanks for your explanation! It seems, OUString of LibreOffice is based on UTF-16, so I will restore gr_count_unicode_characters() call (as I planned), combined with the hyphenation fix.
Comment 14 Commit Notification 2014-04-09 14:47:39 UTC
Laszlo Nemeth committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=4a8cf5221f425118a7ec79e0cbf9cb0fb47bf823

fdo#70666 avoid only bad Graphite ligature replacement



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 15 Commit Notification 2014-04-09 14:55:20 UTC
Laszlo Nemeth committed a patch related to this issue.
It has been pushed to "libreoffice-4-2":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=9f132a147c5d0f92b66837bb5816eb2a2b700f2c&h=libreoffice-4-2

fdo#70666 fix Graphite ligature replacement at line breaks


It will be available in LibreOffice 4.2.4.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 16 Commit Notification 2014-04-09 14:55:32 UTC
Laszlo Nemeth committed a patch related to this issue.
It has been pushed to "libreoffice-4-2":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=bffdb3eefe6ae5a6a758c5401d3558724fbaaa05&h=libreoffice-4-2

fdo#70666 avoid only bad Graphite ligature replacement


It will be available in LibreOffice 4.2.4.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 17 Michael Stahl (allotropia) 2014-05-06 13:50:41 UTC
appears to be fixed now, thanks! have filed a follow-up bug 78340 about
different rendering with non-printing characters enabled, which
had confused me quite a bit while verifying this bug.