Bug 112093 - RTL Regression in justified Hebrew text
Summary: RTL Regression in justified Hebrew text
Status: VERIFIED DUPLICATE of bug 105913
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
(earliest affected) release
Hardware: All All
: medium normal
Assignee: Not Assigned
Depends on:
Reported: 2017-08-29 20:11 UTC by Schlomo Schapiro
Modified: 2018-06-17 11:26 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:

Demo file showing the problem on page 4 & 5 (3.96 MB, application/vnd.oasis.opendocument.text)
2017-08-29 20:14 UTC, Schlomo Schapiro
Screenshot of correctly rendered Hebrew justified text (175.09 KB, image/png)
2017-08-29 20:15 UTC, Schlomo Schapiro
Screenshot of broken Hebrew justified text (156.71 KB, image/png)
2017-08-29 20:15 UTC, Schlomo Schapiro
the corrected file (2.94 MB, application/vnd.oasis.opendocument.text)
2017-08-31 14:16 UTC, Jacques Guilleron
screenshot which marks with red lines where the problem is (300.63 KB, image/png)
2017-09-01 13:26 UTC, Schlomo Schapiro
screencast showing how to reproduce the problem from scratch (3.23 MB, video/mp4)
2017-09-01 13:26 UTC, Schlomo Schapiro
how it looks in master (163.97 KB, image/png)
2018-06-15 14:07 UTC, Xisco Faulí

Note You need to log in before you can comment on or make changes to this bug.
Description Schlomo Schapiro 2017-08-29 20:11:29 UTC
I have a document with justified Hebrew text that looks correct in LOO till 5.2 and is broken in LOO 5.3 and 5.4.

The problem is that the text is justified only on the left side but not correctly on the right side. Also the text is rendered differently (spacing, line breaks) so that a text block that previously fit on a page doesn't fit any more on a page. This is a 36 page brochure where this is really significant, I first noticed the problem by the fact that the brochure suddenly had 37 pages instead of 36.

The attached screenshots show the problem, look at the ragged right line in the bad example.

To verify that the problem is caused by LOO I downloaded the following versions and tried them all on the same system: = OK = OK = OK = BAD = BAD

My system is Ubuntu 16.04.3 LTS x86_64. The following attachments should help to analyze the bug:

- ODT file that has the problem (pages 4 and 5 in the screenshots)
- screenshot showing the bad rendering
- screenshot showing the good rendering in the older versions

To try out yourself I embedded the Hebrew fonts in the ODT so that you don't have to install them yourself.
Comment 1 Schlomo Schapiro 2017-08-29 20:14:34 UTC
Created attachment 135852 [details]
Demo file showing the problem on page 4 & 5
Comment 2 Schlomo Schapiro 2017-08-29 20:15:08 UTC
Created attachment 135853 [details]
Screenshot of correctly rendered Hebrew justified text
Comment 3 Schlomo Schapiro 2017-08-29 20:15:36 UTC
Created attachment 135854 [details]
Screenshot of broken Hebrew justified text
Comment 4 Jacques Guilleron 2017-08-31 14:14:46 UTC
Hi Schlomo,

There’s nothing lost. This is due to a change since LO with the text frames.
Previously, in Frame properties > Borders, when you set a space with content, if syncronize was ticked, the spacing was increased at the same time into the other borders, but kept greyed and without importance.
Now, the other borders have to be set to their right value.
Because of the old setting, you have now a reduced space in our text frame and have to set the spacing to 0 cm for the other borders than these right one, to retrieve your layout.
I join a corrected document to be quite clear.
If you have questions, I am available.
Comment 5 Jacques Guilleron 2017-08-31 14:16:41 UTC
Created attachment 135904 [details]
the corrected file
Comment 6 Schlomo Schapiro 2017-09-01 13:25:24 UTC
Thanks for taking a look. I checked your "fixed" document and indeed the overflow into the next page is fixed. So this is probably indeed related to the spacing in the text frames.

I am sorry for the somewhat misleading example. My problem is much more with the way how the justified (flush left and right) text is displayed on the right side.

On the right side the beginning of the words are not aligned on a straight vertical line but some lines have a slight indentation. This destroys the justified look and is for Hebrew a big problem as the text is read from right to left so that now the start of the text is not on the same position.

To help you identify the problem better I took a screenshot from your fixed document and marked with red lines the problematic margin. Green lines show the correct left margin.

To further demonstrate that this issue is actually not related to my example document I also created a screencast of how to recreate the problem:

1) open a webbrowser and go to http://www.i18nguy.com/unicode/shma.html
2) scroll down till after the yellow boxes with English text till you come to the Hebrew text.
3) copy 2 paragraphs of the main text (skip the heading)
4) open a new LOO editor (where CTL is enabled and set to Hebrew)
5) paste the 2 paragraphs without formatting
6) select all the text and switch to RTL and set justified
7) cycle through different fonts. Ezra SIL shows it strongly (available in Ubuntu), Andale Mono also shows it. Libertine doesn't.
8) switch between RTL and LTR and observe how the text layout changes slightly. With LTR the right margin is slightly better justified than in RTL, but the problem still persists.

See the screencast for a demo.

My conclusion from observing this is:
* The font has a strong influence on the intensity of the problem
* RTL / LTR setting also has an influence
* The problem is not related to any specific document but to the way how LOO renders the Hebrew justified text.

As this worked till before LOO 5.3 and since this clearly looks very bad I hope that this information will help you to solve the issue. Please don't hesitate to ask for more, I'll be also happy to try out anything you suggest.
Comment 7 Schlomo Schapiro 2017-09-01 13:26:01 UTC
Created attachment 135929 [details]
screenshot which marks with red lines where the problem is
Comment 8 Schlomo Schapiro 2017-09-01 13:26:36 UTC
Created attachment 135930 [details]
screencast showing how to reproduce the problem from scratch
Comment 9 Jacques Guilleron 2017-09-02 09:51:59 UTC
Thank you for the precisions. 
The misalignment is clearly visible on page 3 and begins with:
LO Build ID: 690f553ecb3efd19143acbf01f3af4e289e94536
CPU Threads: 2; OS Version: Windows 6.1; UI Render: default; Layout Engine: new; 
Locale: en-US (frFR); Calc: CL
and upper versions.
No issue in:
LO Build ID: f4ca1573fcf445164c068c1046ab5d084e1b005f
Threads CPU : 2; Version de l'OS :Windows 6.1; UI Render : par défaut; 
Locale : fr-FR (fr_R); Calc: CL
Comment 10 Schlomo Schapiro 2018-06-14 12:10:30 UTC
I checked again in version 6.0.4 and the regression bug is still there. Hebrew justified text is NOT flush on both left AND right margin.

What can I do to help you guys fix this? It forces me (and presumably other RTL users) to keep around an ancient version.
Comment 11 Xisco Faulí 2018-06-15 14:06:34 UTC
I can reproduce it in

Build ID: 1:6.0.4~rc2-0ubuntu0.16.04.1
CPU threads: 4; OS: Linux 4.13; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); Calc: group

but not in

Build ID: 1cf27cb44380f4a2bd6778c2dbbdef2a2a00cbc8
CPU threads: 4; OS: Linux 4.13; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); Calc: group threaded

Comment 12 Xisco Faulí 2018-06-15 14:07:13 UTC
Created attachment 142771 [details]
how it looks in master
Comment 13 Xisco Faulí 2018-06-15 14:07:36 UTC
Dear Schlomo Schapiro,
Could you please try to reproduce it with a master build from http://dev-builds.libreoffice.org/daily/master/ ?
You can install it alongside the standard version.
I have set the bug's status to 'NEEDINFO'. Please change it back to 'UNCONFIRMED' if the bug is still present in the master build
Comment 14 Xisco Faulí 2018-06-15 14:17:44 UTC
Problem fixed by https://cgit.freedesktop.org/libreoffice/core/commit/?id=2815499d7c0b32fa05fcd697e7b2c2d897f78dfb, which is the commit fixing bug 105913
The fix will be avilable for LibreOffice 6.1, due to be released in August, 2018

*** This bug has been marked as a duplicate of bug 105913 ***
Comment 15 Schlomo Schapiro 2018-06-17 11:24:37 UTC
I can confirm that the Hebrew justified text problem is solved in 

LibreOfficeDev 90e4c55d01637178418c33ffe818263114a53374

However, the positioning of the vowels (the dots under the letters) is different from and I also found that ligatures are not handled correctly. I will post those findings to bug 105913 (which IMHO is not yet properly fixed).

To help with debugging this issue I created a GitHub project with a sample file and some tools to quickly check against a new LOO version: https://github.com/schlomo/libreoffice-bug-rtl-justified-regression
Comment 16 Xisco Faulí 2018-06-17 11:26:26 UTC
Hi Schlomo Schapiro,
Could you please create another report for this?