Bug 156211 - Glitch in the kashida for justified Arabic/Persian text with double quotes
Summary: Glitch in the kashida for justified Arabic/Persian text with double quotes
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
5.3.0.3 release
Hardware: All All
: medium normal
Assignee: Jonathan Clark
URL:
Whiteboard: target:24.2.0 target:25.2.0 target:24...
Keywords: bibisectRequest, regression
Depends on:
Blocks: Undo-Redo Paragraph Character Kashida-Justification, Tatweel
  Show dependency treegraph
 
Reported: 2023-07-09 18:54 UTC by Hossein
Modified: 2024-08-26 11:05 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
Text with Persian double quotes (24.71 KB, application/vnd.oasis.opendocument.text)
2023-07-09 18:54 UTC, Hossein
Details
Screenshot from the glitch in the justified Persian text on Windows (69.69 KB, image/png)
2023-07-23 10:51 UTC, Hossein
Details
Screenshot from the glitch in the justified Persian text on Linux (37.79 KB, image/png)
2023-07-23 13:01 UTC, Hossein
Details
Result after remove/undo (25.53 KB, image/png)
2024-08-15 09:22 UTC, Hossein
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Hossein 2023-07-09 18:54:12 UTC
Created attachment 188277 [details]
Text with Persian double quotes

Description:
If you remove a text between Persian double quotes «» and then press ctrl+z, the other text between «» faces problem of bad horizontal lines (kashida).

Steps to Reproduce:
1. Open attachment
2. Select text between «», in this case لذا and remove it
3. Press ctrl+z

Actual Results:
Bad horizontal line (kashida) in the other text between «», in this case برای همين. This problem IS visible in the PDF output if you do exporting.

One side note: I found a Firefox an RTL/CTL bug while writing the above paragraph which has a Persian text that is broken across two lines!

Expected Results:
The text should be rendered as it was when loading the document, without badly positioned horizontal line.

Reproducible: Always


User Profile Reset: No


Additional Info:
$ instdir/program/soffice --version
LibreOfficeDev 24.2.0.0.alpha0 81726f5af5fda25f0d92ffc8458d7f24eb16f408

I can not open Help > About as it leads to the LibreOffice hang.
Comment 1 Stéphane Guillou (stragu) 2023-07-09 21:46:33 UTC
Issue is zoom-dependent. I re-saved the example document at 550× zoom to be able to bibisect it. I was looking at the horizontal line poking out to the right of the quoted string.

Repro in:

Version: 7.3.7.2 / LibreOffice Community
Build ID: e114eadc50a9ff8d8c8a0567d6da8f454beeb84f
CPU threads: 8; OS: Linux 5.15; UI render: default; VCL: gtk3
Locale: en-AU (en_AU.UTF-8); UI: en-US
Calc: threaded

Not in 7.2:

Version: 7.2.7.2 / LibreOffice Community
Build ID: 8d71d29d553c0f7dcbfa38fbfda25ee34cce99a2
CPU threads: 8; OS: Linux 5.15; UI render: default; VCL: gtk3
Locale: en-AU (en_AU.UTF-8); UI: en-US
Calc: threaded

Bibisected with linux-64-7.2 repo to first bad commit 74abd7e58426f871b3d9c240c781ffd08d3eedcb which points to core commit:

commit 56186db93b7777f7b99a8fbfce82b775e24b7695
author	Caolán McNamara <caolanm@redhat.com>	Tue Dec 14 21:27:14 2021 +0000
committer	Caolán McNamara <caolanm@redhat.com>	Wed Dec 15 11:30:55 2021 +0100
tdf#145934 workaround rounding causing 'dancing characters'
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/126870

However, before that commit, I could follow the steps and then change the zoom to 490× and see a rogue horizontal line on the "ی".

So I don't think that's coming from Caolán commit.

The horizontal bar on the "ی" at 490× I can reproduce in 5.3.0.1.
5.3.0.0.beta2 had horizontal lines all over the place at fileopen.
I could not reproduce in 5.2.
Comment 2 Commit Notification 2023-07-23 04:03:36 UTC
Khaled Hosny committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/9d9e3b439883c3c315501f56bb613e080863db64

tdf#156211: Don’t round Kashida width when doing subpixel positioning

It will be available in 24.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 3 ⁨خالد حسني⁩ 2023-07-23 04:26:15 UTC
The situation seems to be improved/fixed on hipdpi screens, but not in lodpi ones. I’m investigating why screen resolution is still making a difference here.
Comment 4 Hossein 2023-07-23 10:51:48 UTC
Created attachment 188522 [details]
Screenshot from the glitch in the justified Persian text on Windows

U=(In reply to ⁨خالد حسني⁩ from comment #3)
> The situation seems to be improved/fixed on hipdpi screens, but not in lodpi
> ones. I’m investigating why screen resolution is still making a difference
> here.
I still see the problem in hidpi display with 3x scaling

Version: 24.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 56edc523a4aa2a99ce15c66d42a8c7ba55bf1580
CPU threads: 20; OS: Windows 10.0 Build 22621; UI render: Skia/Vulkan; VCL: win
Locale: en-US (en_DE); UI: en-US
Calc: CL threaded
Comment 5 ⁨خالد حسني⁩ 2023-07-23 11:39:49 UTC
It is fixed for me on macOS.
Comment 6 Hossein 2023-07-23 13:01:21 UTC
Created attachment 188524 [details]
Screenshot from the glitch in the justified Persian text on Linux

I also see (In reply to ⁨خالد حسني⁩ from comment #5)
> It is fixed for me on macOS.
Also, I still reproduce the problem on Linux with HIDPI and 2x scaling:

Version: 24.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 56edc523a4aa2a99ce15c66d42a8c7ba55bf1580
CPU threads: 12; OS: Linux 5.19; UI render: default; VCL: gtk3
Locale: en-US (en_US.UTF-8); UI: en-US
Calc: CL threaded
Comment 7 Hossein 2023-07-23 13:16:03 UTC
I think this can be related to how the document is processed with undo-redo and also when loaded.
Here, if you paste the text "لذا" instead of "redo", which should lead to same content, you will get different output.

In other words:

Remove, then ctrl+z -> glitch
Remove, then paste removed text -> rendering is OK

Sometimes, it is reverse. The initial rendering is correct. But when you change and then undo, you will get the correct rendering.

tdf#146713 - RTL language overlap problem when mixed with LTR language
https://bugs.documentfoundation.org/show_bug.cgi?id=146713

In other words:

Initial output -> glitch
Change, and then undo -> rendering is OK

In both cases, the problem is visible in the PDF output, so it should be possible to safely test if the output remains the same before and after change and undo.
Comment 8 ⁨خالد حسني⁩ 2023-07-27 09:47:38 UTC
To be honest, I still have no idea what is going on. It seems like under certain circumstances Writer gets the wrong text widths, how and why this happens is not something I was able to identify yet.
Comment 9 ⁨خالد حسني⁩ 2023-08-10 11:50:18 UTC
Unassigning. I don’t seem to make any progress every time I look into it.
Comment 10 Hossein 2024-05-23 11:58:53 UTC
Still reproducible with the latest LO 24.8 dev master built today:

Version: 24.8.0.0.alpha1+ (X86_64) / LibreOffice Community
Build ID: 1396347c8e9b036f15bc646b9475073344e69e74
CPU threads: 12; OS: Linux 6.2; UI render: default; VCL: gtk3
Locale: en-US (en_US.UTF-8); UI: en-US
Calc: CL threaded

@Jonathan:
Hello, this issue represents an interesting open question, as it may pave the way to fix other remaining issues with text justification, listed in tdf#150285.
The example here is somehow minimal, and a single "text removal + undo" reproduces it. As per discussion and related commits, it seems to be related to kerning / subpixel positioning, relevant to these two files that you have recently touched:
vcl/source/outdev/text.cxx
vcl/source/outdev/font.cxx
Comment 11 Commit Notification 2024-08-14 22:31:43 UTC
Jonathan Clark committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/dda85e275d70d6365009042b8e207337f2e712c2

tdf#156211 sw: Fix spurious kashida inserted after undo

It will be available in 25.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Hossein 2024-08-14 23:30:40 UTC
Thanks Jonathan,
I confirm the fix with your patch:

Version: 25.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: dda85e275d70d6365009042b8e207337f2e712c2
CPU threads: 12; OS: Linux 6.2; UI render: default; VCL: gtk3
Locale: en-US (en_US.UTF-8); UI: en-US
Calc: threaded

One observation: if you remove the text as described in comment 0, and then undo/redo multiple times, the result of the initial justification is not similar to the 2nd time onward. You can reload the file in the end to see the difference in justification and kashida insertion.
Comment 13 Hossein 2024-08-15 09:22:44 UTC
Created attachment 195837 [details]
Result after remove/undo

Looking carefully into the result, I see that kashida is not inserted correctly in the final result after remove/undo.
Comment 14 Jonathan Clark 2024-08-15 13:56:42 UTC
(In reply to Hossein from comment #13)
> Created attachment 195837 [details]
> Result after remove/undo
> 
> Looking carefully into the result, I see that kashida is not inserted
> correctly in the final result after remove/undo.

Hossein and I discussed this elsewhere, but to summarize:

During layout, Writer scans each line and portion to detect whether there is enough room to insert kashida glyphs. I strongly suspect this algorithm may produce different results depending on zoom level. Changing the zoom level doesn't trigger another layout, so in the case that there is room for kashida glyphs at one zoom level but not another, you'll see results like that screenshot.

However, this would be a separate issue from the more deterministic undo/redo issue described here.
Comment 15 Commit Notification 2024-08-20 08:48:08 UTC
Jonathan Clark committed a patch related to this issue.
It has been pushed to "libreoffice-24-8":

https://git.libreoffice.org/core/commit/2879a1c441548bcac66ea5e6274840e65a9b05da

tdf#156211 sw: Fix spurious kashida inserted after undo

It will be available in 24.8.1.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.