Bug 134226 - Arabic diacritics and shadda overlap when shadda is in a separate span
Summary: Arabic diacritics and shadda overlap when shadda is in a separate span
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
7.0.0.0.beta1+
Hardware: All All
: medium normal
Assignee: Jonathan Clark
URL:
Whiteboard: target:24.8.0
Keywords:
Depends on: 61444
Blocks: Font-Rendering Arabic-and-Farsi DOCX-RTL Diacritics
  Show dependency treegraph
 
Reported: 2020-06-22 09:17 UTC by Eyal Rozenberg
Modified: 2024-08-04 22:00 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
A document exhibiting this bug (8.91 KB, application/vnd.oasis.opendocument.text)
2020-06-22 09:18 UTC, Eyal Rozenberg
Details
Screenshot of the rendering of the document by LO 7.0 beta 1 (31.78 KB, image/png)
2020-06-22 09:21 UTC, Eyal Rozenberg
Details
Screenshot of Arial font in Windows font previewer (14.53 KB, image/png)
2020-11-12 23:08 UTC, Ming Hua
Details
Screenshot on my Windows 10 system, LO 6.4.7 (14.43 KB, image/png)
2020-11-13 11:02 UTC, Ming Hua
Details
screenshot (108.10 KB, image/png)
2023-09-21 05:27 UTC, BogdanB
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Eyal Rozenberg 2020-06-22 09:17:13 UTC
Description:
On my system, if I use a CTL font which (to my knowledge) doesn't have Arabic glyphs, it seems some kind of fallback font is used. And with this fallback, I see some Arabic diacritics misplaced.


Note: The bug might not have to do with the use of a "fallback" font, and might just involve the "fallback font" itself. I can't tell, since I can't tell which font is used for fallback (which perhaps would merit another bug).

Steps to Reproduce:
1. Create new document
2. Type in شقة
3. Add the shadda diacritic to the q in the word shaqqa (to get شقّة).

Actual Results:
The shadda it overlaps the two dots of the quf.


Expected Results:
The shadda should appear above the two dots of the quf.


Reproducible: Always


User Profile Reset: No



Additional Info:
using GNU/Linux Devuan Beowulf.
Comment 1 Eyal Rozenberg 2020-06-22 09:18:10 UTC
Created attachment 162300 [details]
A document exhibiting this bug
Comment 2 Eyal Rozenberg 2020-06-22 09:21:17 UTC
Created attachment 162301 [details]
Screenshot of the rendering of the document by LO 7.0 beta 1
Comment 3 Dieter 2020-10-08 14:56:42 UTC
Observation:

I've opend your file, marked the word with the misplaced diacritics (Font Arial) and selected the font Arial again and diacritics are displayed correct.
Comment 4 Dieter 2020-10-08 14:57:12 UTC
Tested with

Version: 7.0.2.2 (x64)
Build ID: 8349ace3c3162073abd90d81fd06dcfb6b36b994
CPU threads: 4; OS: Windows 10.0 Build 19041; UI render: Skia/Raster; VCL: win
Locale: he-IL (de_DE); UI: en-GB
Calc: threaded
Comment 5 Xisco Faulí 2020-11-11 14:13:13 UTC
Hello Eyal,
Could you please try to reproduce it with the latest version of LibreOffice from https://www.libreoffice.org/download/libreoffice-fresh/ ?
I have set the bug's status to 'NEEDINFO'. Please change it back to 'UNCONFIRMED' if the bug is still present in the latest version.
Comment 6 Eyal Rozenberg 2020-11-11 22:17:54 UTC
(In reply to Xisco Faulí from comment #5)
Do you mean the 7.1a1 version?
Comment 7 Xisco Faulí 2020-11-11 22:19:01 UTC
(In reply to Eyal Rozenberg from comment #6)
> (In reply to Xisco Faulí from comment #5)
> Do you mean the 7.1a1 version?

7.0.3
Comment 8 Eyal Rozenberg 2020-11-12 21:56:59 UTC
Bug still manifests with :

Version: 7.0.3.1
Build ID: d7547858d014d4cf69878db179d326fc3483e082
CPU threads: 4; OS: Linux 5.2; UI render: default; VCL: gtk3
Locale: he-IL (en_IL); UI: en-US
Calc: threaded

on my Devuan GNU/Linux Beowulf. By the way, perhaps I should mention I also have the X font cache look at a Windows installation's font directory; and I have a few fonts I've installed myself (not Arial of course).
Comment 9 Ming Hua 2020-11-12 23:01:26 UTC
First, I can reproduce both the problematic rendering like the screenshot in attachment 162301 [details] and the "workaround" mentioned in comment 3 (choose text, re-assign font Arial, the rendering becomes correct) with 6.4.7 on Windows:
Version: 6.4.7.2 (x64)
Build ID: 639b8ac485750d5696d7590a72ef1b496725cfb5
CPU threads: 2; OS: Windows 10.0 Build 18363; UI render: default; VCL: win; 
Locale: zh-CN (zh_CN); UI-Language: en-US
Calc: threaded

Second, I question Eyal's judgement that this is caused by fall-back font.  Not that I know any Arabic myself, but to my layman's eyes, the Arabic is rendered very similarly on my Windows as Eyal's screenshot, and Eyal says:

(In reply to Eyal Rozenberg from comment #8)
> By the way, perhaps I should mention I also
> have the X font cache look at a Windows installation's font directory; and I
> have a few fonts I've installed myself (not Arial of course).
But any Windows installation should have Arial font installed.  So I checked my Arial font, it does contain Arabic glyphs, and they look just like how the sample document is rendered in LO (and Eyal's screenshot).

I therefore contend that this has nothing to do with fall-back font, but about rendering of Arial font when it's used by default (instead of by direct formatting).
Comment 10 Ming Hua 2020-11-12 23:08:49 UTC
Created attachment 167247 [details]
Screenshot of Arial font in Windows font previewer

Here is a screenshot of how Arial font is shown in Window's own font previewer, using the same sample text شقّة .  The five different rendering correspond to different "facets" of the same font, in this case from top to bottom, Regular, Italic, Bold, Bold Italic, and Black.
Comment 11 Eyal Rozenberg 2020-11-12 23:29:24 UTC
(In reply to Ming Hua from comment #10)

In all those renderings, the two-dots and the shadda don't overlap. They may be too close together for my taste, but that's not the same thing.
Comment 12 Ming Hua 2020-11-13 10:51:29 UTC
(In reply to Eyal Rozenberg from comment #11)
> In all those renderings, the two-dots and the shadda don't overlap. They may
> be too close together for my taste, but that's not the same thing.
Like Dieter discovered in comment #3 and I reproduced, if you select the text where the two diacritics overlap, and manually set it to "Arial" font again, the dots and shadda will become separate, while the rest of the rendering stay the same.

All I'm saying is what you see is indeed Arial font from your X font's cache of the Windows fonts, not some fall-back font.  Whether it's the "same thing" or not depends on context.
Comment 13 Ming Hua 2020-11-13 11:02:06 UTC
Created attachment 167266 [details]
Screenshot on my Windows 10 system, LO 6.4.7
Comment 14 Eyal Rozenberg 2020-11-13 13:44:04 UTC
Ok, narrowing the scope of this issue - unless someone reproduces it with a different font.
Comment 15 Dieter 2022-02-07 15:51:47 UTC
I think with respect to the previous discussion (comment 9 - 14) and the concentration on Arial, we can set status to NEW
=> NEW
Comment 16 ⁨خالد حسني⁩ 2022-08-12 19:18:20 UTC
If you open the XML file, the broken line looks like this:
<text:p text:style-name="P4"><text:span text:style-name="T7">شق</text:span><text:span text:style-name="T1">ّ</text:span><text:span text:style-name="T7">ة<text:tab/></text:span><text:span text:style-name="T2">Arial</text:span></text:p>

The text before the shadda is in a span with style T7, the shadda is in span of its own with style T1, and the text after it is in a span with style T7. I don’t know for sure what create these spans, but I think it is related to tracking changes. Clear Direct Formatting usually removes these extra spans (but also any other direct formatting).

Unfortunately, LO does not merge spans which laying out text, even if they eventually use the same font and have the same formatting. This means each span is processed independently, and mark positioning and other feature will be broken.

I think there is quite a few bugs caused by the same root issue, and some recentish (in the past 5 years or so) change in writer code code these extra possibly needless spans.
Comment 17 ⁨خالد حسني⁩ 2022-08-12 19:22:08 UTC
See also bug 96078 and search bugzilla to rsid
Comment 18 Eyal Rozenberg 2022-08-12 20:24:58 UTC
(In reply to Khaled Hosny from comment #17)
> See also bug 96078 and search bugzilla to rsid

What's rsid?
Comment 19 ⁨خالد حسني⁩ 2022-08-12 20:27:54 UTC
(In reply to Eyal Rozenberg from comment #18)
> (In reply to Khaled Hosny from comment #17)
> > See also bug 96078 and search bugzilla to rsid
> 
> What's rsid?

No idea, that what the different styles are setting and what the above bug is talking about:

  <style:style style:name="T2" style:family="text">
   <style:text-properties officeooo:rsid="000fd7fa"/>
  </style:style>

  <style:style style:name="T7" style:family="text">
   <style:text-properties officeooo:rsid="000e7bbb"/>
  </style:style>
Comment 20 ⁨خالد حسني⁩ 2022-08-13 16:12:35 UTC
See bug 52028
Comment 21 BogdanB 2023-09-21 05:27:16 UTC
Created attachment 189726 [details]
screenshot

Seems ok in 24.2. See my screenshot. Please retest.

Version: 24.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: a34dcd03254480927c403d904c0e754802d97b90
CPU threads: 4; OS: Linux 5.15; UI render: default; VCL: gtk3
Locale: ro-RO (ro_RO.UTF-8); UI: en-US
Calc: threaded
Comment 22 Eyal Rozenberg 2023-09-21 08:37:55 UTC
(In reply to BogdanB from comment #21)

1. Were you seeing the bug manifest with earlier versions?
2. I believe that on your system, a different fallback font is used, and that may explain why you're not seeing the bug manifest.
3. I'm still seeing the bug with:

Version: 7.6.1.2 (X86_64) / LibreOffice Community
Build ID: 60(Build:2)
CPU threads: 4; OS: Linux 5.15; UI render: default; VCL: gtk3
Locale: en-GB (en_GB.UTF-8); UI: en-US
Ubuntu package version: 4:7.6.1~rc2-0ubuntu0.22.04.1~lo2
Comment 23 Jonathan Clark 2024-05-22 17:26:14 UTC
This bug was fixed as part of the fix for bug 124116, in the following commit:

https://git.libreoffice.org/core/commit/ab0a4543cab77ae0c7c0a79feb8aebab71163dd7

tdf#124116 Correct Writer text shaping across formatting changes

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.