Bug 82018 - FORMATTING: Japanese text rendered incorrectly
Summary: FORMATTING: Japanese text rendered incorrectly
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
4.3.0.4 release
Hardware: x86 (IA32) macOS (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:4.5.0
Keywords:
Depends on:
Blocks: Font-Rendering
  Show dependency treegraph
 
Reported: 2014-08-01 17:11 UTC by Matthew Francis
Modified: 2015-02-18 23:20 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Document which illustrates the issue (14.68 KB, application/vnd.openxmlformats-officedocument.wordprocessingml.document)
2014-08-01 17:11 UTC, Matthew Francis
Details
Screenshot of the issue (119.28 KB, image/png)
2014-08-01 17:12 UTC, Matthew Francis
Details
Updated document which illustrates the issue (8.99 KB, application/vnd.openxmlformats-officedocument.wordprocessingml.document)
2014-08-04 12:26 UTC, Matthew Francis
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Matthew Francis 2014-08-01 17:11:45 UTC
Created attachment 103824 [details]
Document which illustrates the issue

In LO 4.3.0.4 on OSX:

With a certain line of Japanese text in Writer, some characters are printed directly on top of other characters, rendering the text illegible.

Please see the attached ODF and screenshot for details.
Towards the end of the line, the あs and イs overlap considerably.


"Text Grid > Grid (lines and characters)" is enabled in the page options, but "Alignment > Snap to text grid (if active)" is not enabled in the paragraph options. The text itself contains several changes in style.
(* The text in the attached file is nonsense used for illustrative purposes, but this is a bug that I have observed in an actual document I am working with)

Working with Japanese text in general, I have seen that there are some circumstances in which the text layout and the cursor position lose sync; this may or may not be the same bug, but this is the first time I have seen the text itself laid out in an illegible/unreasonable manner.
Comment 1 Matthew Francis 2014-08-01 17:12:44 UTC
Created attachment 103825 [details]
Screenshot of the issue
Comment 2 Yousuf Philips (jay) (retired) 2014-08-04 02:10:41 UTC
Hi Matthew,

Thank you for the bug report. I tested it in 4.2.7 and 4.3.1 on Linux Mint and couldnt reproduce, so i'm forwarding this to our Mac QA team members.

From your screenshot, i noticed that you didnt have the MS Gothic font installed, and wondered if this could be the cause of the issue.
Comment 3 Matthew Francis 2014-08-04 03:56:41 UTC
(In reply to comment #2)

Thanks

The attached demonstration was minimised from what was originally a Word .doc, so it is possible that font substitution is a factor
Comment 4 retired 2014-08-04 09:24:53 UTC
I can confirm what Matthew sees (looks identical on OSX 10.9.4 and LO 4.3.0.4). 

I cannot find that MS font here as well (font name is displayed in italic and when I search for it in the font list, I do not find it).

So unsure what to make of this. Maybe LO should clearly state when a font is missing. But that's another bug then.

Jay can you move this forward accordingly?
Comment 5 Matthew Francis 2014-08-04 12:26:14 UTC
Created attachment 103990 [details]
Updated document which illustrates the issue


Font fallback appears to have been a red herring. In an attempt to further isolate the cause, I took the original test document, and edited its XML content as follows:

- Removed all the content from styles.xml
- Removed all the content from meta.xml
- Removed all extraneous declarations from content.xml, leaving only the sample text itself
- Progressively removed the content of settings.xml until the problem disappeared

This revealed that the direct cause of the problem is the following line in settings.xml:

<config:config-item config:name="CharacterCompressionType" config:type="short">1</config:config-item>

With CharacterCompressionType == 1 or 2, the problem appears
With CharacterCompressionType == 0 (or the line omitted), the problem does not appear


The attached updated test file contains only built-in fonts, and shows the issue occurring in two ways.
- In the first paragraph, in which there is only normal text, the text is rendered far beyond the right margin
- In the second paragraph, in which normal text switches to bold text half way through, the text is rendered overlapping as in the original test document
Comment 6 Yousuf Philips (jay) (retired) 2014-08-04 18:35:26 UTC
As Foss confirmed it, i'm setting it to NEW. Thanks Matthew for going into the XML to isolate the issue.
Comment 7 Matthew Francis 2014-08-05 17:38:42 UTC
Found another bug which touches on this one - bug 81144

The document attached there exhibits another example of the "CharacterCompressionType" bug, but also seems to have a separate rendering problem.

Commented on the other bug as well.
Comment 8 Matthew Francis 2014-08-12 00:18:08 UTC
This looks fixed on a recent built from git - although I can't immediately see by what. It would be nice to know just to make sure
Comment 9 retired 2014-08-12 08:06:05 UTC
In 4.3.1 http://www.libreoffice.org/download/pre-releases/ font is Tahoma and no longer MS Gothic (which does not exist on OS X.

If that means, this issue is fixed, Matthew, could you update it accordingly?
Comment 10 Matthew Francis 2014-08-12 08:44:45 UTC
(In reply to comment #9)
> In 4.3.1 http://www.libreoffice.org/download/pre-releases/ font is Tahoma
> and no longer MS Gothic (which does not exist on OS X.
> 
> If that means, this issue is fixed, Matthew, could you update it accordingly?

If you know of a particular bug or commit that affected this, could you possibly point me to it? I can't immediately spot the change in question.

I'm currently attempting to propel several CJK bugs forward, and I'm not quite sure how some of the rendering related issues are connected - it would be useful to be able to see why specifically this bug is (apparently) fixed before closing it.

Now that I have a local build working, I can bisect the source to find the relevant change if necessary, it just may take some time to do so at up to a couple of hours per rebuild.
Comment 11 retired 2014-08-12 11:48:16 UTC
Matthew, sorry I don't know about any commit. But if it works it works. To make sure this is not an accident, it might be worth re-checking with the latest nightly build if it is fixed there as well.

http://dev-builds.libreoffice.org/daily/master/
Comment 12 Matthew Francis 2014-08-13 14:42:08 UTC
After further bisection, the mystery fix appears to be

http://cgit.freedesktop.org/libreoffice/core/commit/?id=60075ca574b9f1e2f34908a7cb240005a35ee8b9

Norbert Thiebaud <nthiebaud@gmail.com>	2014-07-20
vcl quartz: restore old outline front drawing


As hoped, the surrounding patch set also gives some potential clues to bug 81144
Comment 13 Commit Notification 2015-02-18 23:20:12 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=ef966b007deca9e76c24bc2afb74ba94a3173771

assert on laying out fdo#82018-3.docx

It will be available in 4.5.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.