Bug Hunting Session
Bug 80724 - FORMATTING: Underline problem with Malayalam and other complex glyph Unicode fonts
Summary: FORMATTING: Underline problem with Malayalam and other complex glyph Unicode ...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: UI (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: Other All
: medium enhancement
Assignee: Not Assigned
URL:
Whiteboard: BSA target:5.3.0
Keywords:
: 80723 (view as bug list)
Depends on:
Blocks: Font-Rendering
  Show dependency treegraph
 
Reported: 2014-06-30 18:28 UTC by Sooraj Kenoth
Modified: 2016-09-23 13:49 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Sample odt file. Font Rachana, updated on 26-Oct-2013 (8.76 KB, application/vnd.oasis.opendocument.text)
2014-06-30 18:28 UTC, Sooraj Kenoth
Details
Underline issue screenshot (6.59 KB, image/png)
2014-07-01 06:55 UTC, Sooraj Kenoth
Details
Screenshot on underline issue (12.14 KB, image/png)
2014-07-01 07:12 UTC, Sooraj Kenoth
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sooraj Kenoth 2014-06-30 18:28:04 UTC
Created attachment 102027 [details]
Sample odt file. Font Rachana, updated on 26-Oct-2013

Problem description: 

While putting underline in Malayalam, the underline strikes the stacked letter. It heavily affects the readability.


Expected behaviour:

There must be a readable gap between character and underline.
Operating System: Debian
Version: 4.1.6.2 release
Comment 1 tommy27 2014-06-30 18:37:59 UTC
*** Bug 80723 has been marked as a duplicate of this bug. ***
Comment 2 Yousuf Philips (jay) (retired) 2014-07-01 04:47:46 UTC
Hi Sooraj,

I believe this is a problem with the font and as such would be out of our scope to fix. I opened the odt file in Calligra Words and the font rendered the same way.
Comment 3 Sooraj Kenoth 2014-07-01 06:55:34 UTC
Created attachment 102052 [details]
Underline issue screenshot
Comment 4 Sooraj Kenoth 2014-07-01 07:10:37 UTC
Hi tommy27,

This is the first time I am reporting a bug to Libreoffice. Sorry for the duplicate submision. Can you remove 80723 for me?

Hi Jay Philips,

Thank you for the quick responce. I had a feel that it was a problem with the font. But I would like to confirm it before I proceed.

But even if it is a problem with font, it create another issue. If I am typing mulitiple language or multiple fonts in same document, I am getting a zigzag underline. I think there is some issue in the method in which libre office out the underline.

I am trying to attach the screenshot.
Comment 5 Sooraj Kenoth 2014-07-01 07:12:14 UTC
Created attachment 102053 [details]
Screenshot on underline issue
Comment 6 Yousuf Philips (jay) (retired) 2014-07-01 12:13:54 UTC
(In reply to comment #4)
> But even if it is a problem with font, it create another issue. If I am
> typing mulitiple language or multiple fonts in same document, I am getting a
> zigzag underline. I think there is some issue in the method in which libre
> office out the underline.

Hi Sooraj,

The position and size of an underline depend on the settings in the font which mention where the underline must go as well as how thick it should be, so yes the zigzag would be a normally thing if you have the same font but with different font sizes and font formatting (bold, italics, etc).

If you find that libreoffice's font behavior is different from other word processor on linux (calligra words, abiword, etc) or different between libreoffice versions, we would be able to investigate that issue.
Comment 7 Sooraj Kenoth 2014-10-08 07:31:07 UTC
(In reply to Jay Philips from comment #6)

Sorry for the late reply. I was trying to solve this issue from the font developer side. According their arguments, underlining is not a part of glyph. They are simple lines. The designers usually set only guide line or notional markings to define the position and width of underline. The actual decision has to be taken by the document processor or the application which uses the font. I agree with these arguments. Because the shape, size and position of underline is defined by the document and not by the font. 

> The position and size of an underline depend on the settings in the font
> which mention where the underline must go as well as how thick it should be,
> so yes the zigzag would be a normally thing if you have the same font but
> with different font sizes and font formatting (bold, italics, etc).

This what I am trying communicate. According to my current knowledge level this is a bug.

 
> If you find that libreoffice's font behavior is different from other word
> processor on linux (calligra words, abiword, etc) or different between
> libreoffice versions, we would be able to investigate that issue.

I didn't check it with other word processors. But I found this as a very common but a serious issue. Also I think libreoffice is much more than a simple word processor. I am from Kerala, India. In our context multilingual document is fairly common. Almost all local language document will contain at least one more language generally it is English, especially while writing abbreviations or technical terms.

Some older posts on same issue,
<https://forum.openoffice.org/en/forum/viewtopic.php?f=7&t=31726>
<https://bugs.freedesktop.org/show_bug.cgi?id=68573>
<https://issues.apache.org/ooo/show_bug.cgi?id=89284>

For the time being I found a work around for this. But is not a solution. In Libreoffice4.2 there is a feature called character border in character settings. There I can create a border only at bottom position, then it will look like an underline, with adjustable position and size. I just created two styles, one with border at bottom and other without any border. Also set two short-cuts to apply these styles. But this has lot of limitations.

I wish the libreoffice team will take forward this issue.
Comment 8 Joel Madero 2014-10-25 07:49:44 UTC
Please try in another word processor (literally you can try many easily). If you can reproduce the same problem on every other word processor (try regular text editors, maybe kde office, etc . . .) then it's likely NOTOURBUG. That being said, setting to UNCONFIRMED for now.
Comment 9 Joel Madero 2014-10-25 07:51:16 UTC
Also just so you're aware of how the project works (I saw your email to the QA list) - our project is powered by volunteers and is a meritocracy. So - even if we confirm your bug, it will require a volunteer to fix it and therefore there is no guarantee of when it will be fixed. All in all it will probably get less attention than many other bugs. Just trying to make sure that you're expectations are realistic.
Comment 10 Sooraj Kenoth 2014-10-25 13:08:37 UTC
(In reply to Joel Madero from comment #8)

Based on your suggestion, I have tried other word processors like Calligra, MS-Word and Xetex(Latex). In both Calligra and MS-word there exists some kind of underline porblem. In MS-word, the problem is very similar to that in Libre Office. At the same time, in Calligra, I am getting a consistent underline for a particular font size even when I use bold/italics/different languages/different fonts. In Xetex, I am getting a perfect output.

When Jay Philips said it is an issue with font, I checked with font developers and designers. That is how I learnt about Free Type and how underlining is done. Based on the information I have gathered, I still continue to believe that the problem I have outlined above is not one that can be solved by font developers and designers. This belief tems from my context.

I come from a context where different fonts and different languages are found in the same document, at times in the same sentence, given the linguistic diversity and traditions in our society. (I live in India, a country with 15 official languages and over 100 languages). Here, the decision on where to place the underline has to be a centralised one taken within the word processor. Expecting font designers to arrive at a common standard is next to impossible given the diversity of languages, scripts and script traditions.
The problem I have described above is something that most users of Latin scripts do not face. This is an issue that is faced with non Latin scripts like Indic and Arabic scripts. For Libre Office (or any other word processor) to be truly globally inclusive, we have to find a way to plug this issue.

(In reply to Joel Madero from comment #9)
> Also just so you're aware of how the project works (I saw your email to the
> QA list) - our project is powered by volunteers and is a meritocracy.

I apologise if you sensed an arrogance in my statements. I am also a volunteer in the Free Software group, but I am very new to this community. I reported this as an issue when it was acutely felt as a major drawback in going ahead with a full Free Software migration at the Kerala Legislative Assembly. (Kerala being a state in India with a population of about 30 million and the Assembly being the apex legislative body for the state). A full Free Software migration at the Assembly would be an important achievement for Free Software.  Not solving the issue of underlining can actually result in a roll back of this migration and reuse of proprietary software. As a Free Software supporter, it is important for me to try and  avoid such a roll back.

I am not a programmer. I do understand that solving the problem requires further effort. But as you would know, the first step would be to list it as a bug so that it at least becomes part of the queue of bugs to be resolved and the resolution becomes part of the main project.
Comment 11 V Stuart Foote 2014-10-25 13:39:22 UTC
While arguably not our bug, correct handling of glyphs and typography necessary for useful for Localization is a worthwhile enhancement.

Setting new enhancement.

Workaround as in comment 7
Comment 12 Joel Madero 2014-10-25 16:46:02 UTC
Sooraj Kenoth - hey! Thanks for doing the additional testing :-D I wasn't telling you about the project because you sounded ignorant, I just wanted to make sure that expectations were realistic :) 

@V Stuart Foote - hm - "arguably not our bug" then why would we pretend like we should fix it instead of telling Sooraj to go to the appropriate project and report the bug there? If it's not our bug, someone else should fix it so it's fixed in all software, and not just ours with some workaround that just deals with someone else's problem. 

Asking for Bjoern's input on this one. If it is our problem, it most definitely is a bug and not an enhancement request.

@Bjoern - thoughts on this? Our bug? If not, any clue whose bug it is?
Comment 13 V Stuart Foote 2014-10-25 18:30:59 UTC
@Joel, *,
(In reply to Joel Madero from comment #12)
> @V Stuart Foote - hm - "arguably not our bug" then why would we pretend like
> we should fix it instead of telling Sooraj to go to the appropriate project
> and report the bug there? If it's not our bug, someone else should fix it so
> it's fixed in all software, and not just ours with some workaround that just
> deals with someone else's problem. 

While we build installers for Malayalam (ISO 639-1 code ml, MS local ID code 1100) we do not have a development, or active Pootle based translation (i18n) effort for that local.

Not sure of all the specifics but believe all glyph support is going to be provided by OS (with local/font extension) and will be dependent on Unicode glyphs for the individual fonts.  Unicode fontset glyphs are not cast both with and without underline or strike through. Rather, included with each glyph will be font vertical metrics detailing the font designers suggestion of where to position an underline.

So, each application will need to underline/strikethrough the glyphs. But handling the vertical metrics for typography in internationalization is simply not implemented in LibreOffice when needed for L10n support. Not a bug, just not implemented.

AFAIK most general desktop applications, LibreOffice included, do not consider vertical metrics of the font design while rendering type--and instead use a default position and weight, with objectionable results as in this issue. That is the potential enhancement.

It is not a bug, but for improved support of L10n and i18n, LibreOffice would need to implement typographic support for vertical metrics of Unicode fontset glyphs. Unfortunately, likely not trivial. 

Stuart
Comment 14 Sooraj Kenoth 2014-11-01 12:43:24 UTC
(In reply to V Stuart Foote from comment #13)
> (In reply to Joel Madero from comment #12)

Thank for accepting this issue. I could not understand the meaning of what you said. Still I am happy that you have taken this issue as genuine. Right now I am working for a better workaround by using Macros.
Comment 15 V Stuart Foote 2014-11-01 14:01:50 UTC
This is Khaled's other issue of https://bugs.freedesktop.org/show_bug.cgi?id=68573#c6

A tease in Herbert Dürr's note 3 in OOo#89284 from 2009 suggesting it had come close to being implemented and then was backed out.

-=p.s.=-
Did a little QA housekeeping on this enhancement, sorry for the noise of extra mailings.
Comment 16 Commit Notification 2016-09-18 08:47:34 UTC
Mark Hung committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=eb8217e17e09613ab1af43a89b52f04e7cb781e8

tdf#80724 try to reuse calculated underline font as much as possible.

It will be available in 5.3.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.