Bug 97622 - Unnecessary extra space inserted between RTL and Latin characters (including numbers)
Summary: Unnecessary extra space inserted between RTL and Latin characters (including ...
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: All All
: medium normal
Assignee: ⁨خالد حسني⁩
URL:
Whiteboard: target:6.1.0
Keywords:
Depends on:
Blocks: RTL-CTL
  Show dependency treegraph
 
Reported: 2016-02-07 16:58 UTC by Eyal Rozenberg
Modified: 2018-09-30 15:17 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Screenshot of bug manifesting (6.59 KB, image/png)
2016-02-07 16:59 UTC, Eyal Rozenberg
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Eyal Rozenberg 2016-02-07 16:58:32 UTC
Reproduction: 
1. Create a new document
2. Set your input language to Hebrew
3. Type a Hebrew letter (say, Aleph: א)
4. Type a digit (say, 2)
5. Press Enter
6. Set your language to English
7. Type the sequence "a2"

Result: There's a lot of space between the Aleph and the 2, while the a and the 2 have almost no space between them. 

This is surely incorrect behavior; it is also not what users expect from other apps, be it MS Word or even variable-space simple-text editors (gedit, kwrite, kate, leafpad, etc. etc.)

Notes:
- That this happens essentially regardless of the chose Hebrew font, or at least for ~10 fonts I tried.
- I've experienced this at least as early as 4.x , probably earlier, I just don't remember since I did not heavily use LO/OO earlier than that.
Comment 1 Eyal Rozenberg 2016-02-07 16:59:11 UTC
Created attachment 122425 [details]
Screenshot of bug manifesting
Comment 2 Eyal Rozenberg 2016-02-07 16:59:52 UTC
Comment on attachment 122425 [details]
Screenshot of bug manifesting

Note no space character was added between the aleph and the 2.
Comment 3 Yousuf Philips (jay) (retired) 2016-10-17 06:15:40 UTC
Hi Eyal,

Thank you for reporting the bug. I can confirm that it happens on 5.2.1.2 on Windows and master on Linux. The same happens with an arabic character and then a number.

Version: 5.3.0.0.alpha0+
Build ID: 45a7137c6796f33fbf5b8f7cb64e293260d991cb
CPU Threads: 2; OS Version: Linux 3.19; UI Render: default; 
TinderBox: Linux-rpm_deb-x86_64@70-TDF, Branch:master, Time: 2016-10-13_23:38:06
Locale: en-US (en_US.UTF-8); Calc: group

@Khaled: Any thoughts?
Comment 4 ⁨خالد حسني⁩ 2016-10-17 07:56:28 UTC
Looks like a bug indeed, and it happens in Writer only so likely a bug in Writer-specific code (that is to say I don’t have the slightest idea what is going on). Is this really inherited from OOo?
Comment 5 Eyal Rozenberg 2016-10-17 10:45:07 UTC
(In reply to Khaled Hosny from comment #4)
> Is this really inherited from OOo?

I don't mind checking if you tell me how to get OOo running on a Kubuntu 16.04 system.
Comment 6 Yousuf Philips (jay) (retired) 2016-10-17 12:24:24 UTC
(In reply to Khaled Hosny from comment #4)
> Is this really inherited from OOo?

Yes i tested 3.3.0 and i just checked AOO and its in there as well.

AOO412m3(Build:9782)  -  Rev. 1709696
2015-10-21 09:51 - Linux x86_64
Comment 7 Omer Zak 2017-11-11 20:11:40 UTC
Still happens in:

Version: 6.0.0.0.alpha1+
Build ID: 5d12237d79f289a1dcf8e07aa03df329e136f078
CPU threads: 8; OS: Linux 4.9; UI render: default; VCL: gtk3; 
Locale: en-US (en_US.utf8); Calc: group

OS: Debian 64bit Stretch (Debian 9.2, with some backported packages)


There is indeed excessive space between א and 2, even though no space character was inserted between them. This behavior was already mentioned in Comment 2.
Comment 8 eladhen2 2018-03-02 08:34:21 UTC
This bug is still present. Typing any Hebrew character followed by a number and you get a big space between the characters.

Version  6.0.1.1 on Linux Mint 18.3. Kernel 4.4, Locale: he-IL (he_IL.UTF-8)
Comment 9 Mark Hung 2018-03-04 15:05:45 UTC
What about turning off "Apply spacing between Asian, Latin and complex text" in Asian Typography tab of Paragraph style? The space shrinks ( although it is still wider comparing to normal English. ).
Comment 10 Eyal Rozenberg 2018-03-04 15:41:42 UTC
(In reply to Mark Hung from comment #9)
> What about turning off "Apply spacing between Asian, Latin and complex text"
> in Asian Typography tab of Paragraph style? The space shrinks ( although it
> is still wider comparing to normal English. ).

I don't have such a tab nor such a setting in my Paragraph Style dialog; LO 6.0.0.1 on GNU/Linux Mint 18.3.
Comment 11 eladhen2 2018-03-12 07:51:39 UTC
(In reply to Mark Hung from comment #9)
> What about turning off "Apply spacing between Asian, Latin and complex text"
> in Asian Typography tab of Paragraph style? The space shrinks ( although it
> is still wider comparing to normal English. ).

This actually works. Interesting. But this means the default behavior is wrong, as far as Hebrew is concerned. And this is something that you need to change manually in all styles... Really unapproachable...
Comment 12 eladhen2 2018-03-12 08:05:32 UTC
(In reply to eladhen2 from comment #11)
> (In reply to Mark Hung from comment #9)
> > What about turning off "Apply spacing between Asian, Latin and complex text"
> > in Asian Typography tab of Paragraph style? The space shrinks ( although it
> > is still wider comparing to normal English. ).
> 
> This actually works. Interesting. But this means the default behavior is
> wrong, as far as Hebrew is concerned. And this is something that you need to
> change manually in all styles... Really unapproachable...

Okay, I dug some more into this.

To get the Asian Typography tab you need to go to Tools > Options > Languages > and click the Asian checkbox.

This is an unreasonable requirement to get to a CTL setting.

I understand this Bug should change to a wrong default behavior and wrong placement of a setting (CTL setting that requires enabling Asian languages).
Comment 13 eladhen2 2018-03-12 09:46:43 UTC
(In reply to eladhen2 from comment #12)

> 
> Okay, I dug some more into this.
> 
> To get the Asian Typography tab you need to go to Tools > Options >
> Languages > and click the Asian checkbox.
> 
> This is an unreasonable requirement to get to a CTL setting.
> 
> I understand this Bug should change to a wrong default behavior and wrong
> placement of a setting (CTL setting that requires enabling Asian languages).

Another problem is while working with MS Word files. This setting isn't saved in Docx files, and so you need to reapply it manually every time you open a Docx file.

To reproduce: 
1. Edit a style and uncheck "Apply spacing between Asian, Latin and complex text" in the Asian Typography tab.
2. Save the file as Docx.
3. Close and reopen the file. Checking the edited style you'll see "Apply spacing between Asian, Latin and complex text" is checked again...
Comment 14 Eyal Rozenberg 2018-03-12 12:49:25 UTC
(In reply to eladhen2 from comment #11)
> This actually works. Interesting. But this means the default behavior is
> wrong, as far as Hebrew is concerned. And this is something that you need to
> change manually in all styles... Really unapproachable...

Actually, it's worse than that. It means that the dichotomy expressed in this boolean setting, i.e. you might want to add space between Asian and Latin, or Asian and RTL, but not between RTL and numbers. I believe there needs to be a better mapping of this option space by people who have experience both with an RTL language and an Asian language. Also, it's not clear that digits should be considered together with Latin characters .

For now, I suggest the "Spacing between RTL and Latin-or-Number characters" be _split_off_ into its own setting, with the other setting being "Spacing between Asian and everything else".
Comment 15 ⁨خالد حسني⁩ 2018-03-13 09:21:00 UTC
The “Apply spacing between Asian, Latin and complex text” is probably misnamed and misapplied, it should be “Add spacing between Asian and non-Asian text” because that is what CJK typesetting wants. It shouldn’t add any spacing if there is no CJK text on either side.
Comment 16 Eyal Rozenberg 2018-03-13 09:33:20 UTC
(In reply to Khaled Hosny from comment #15)
> The “Apply spacing between Asian, Latin and complex text” is probably
> misnamed and misapplied, it should be “Add spacing between Asian and
> non-Asian text” because that is what CJK typesetting wants. It shouldn’t add
> any spacing if there is no CJK text on either side.

This sounds quite reasonable to me (as a person who uses Hebrew and a bit of Arabic, on their own and with Latin text). If that is the case, it would indeed explain why we don't even see this option when not enabling Asian text support inn the preferences.
Comment 17 eladhen2 2018-03-13 09:46:25 UTC
(In reply to Khaled Hosny from comment #15)
> The “Apply spacing between Asian, Latin and complex text” is probably
> misnamed and misapplied, it should be “Add spacing between Asian and
> non-Asian text” because that is what CJK typesetting wants. It shouldn’t add
> any spacing if there is no CJK text on either side.

I don't know about other RTL languages, but in Hebrew this option serves no purpose and I don't see a reason for it to exist. It surely has no reason to be a default behavior.

This means (if this is the case with other RTL languages) that this option should not apply to RTL languages at all.
Comment 18 ⁨خالد حسني⁩ 2018-03-13 10:54:44 UTC
Currently the code inserts the extra space between any text of different scripts (of LO’s 3 scripts, Western, CTL and Asian), and I think it does not make much sense and should be applied only if one side of the script change is Asian.
Comment 19 Commit Notification 2018-03-24 20:30:24 UTC
Khaled Hosny committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=7024f73e088b741a3024f6934d04620d7ed73de4

tdf#97622: Apply Asian spacing only to Asian text

It will be available in 6.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 20 Eyal Rozenberg 2018-03-24 21:01:54 UTC
(In reply to Commit Notification from comment #19)
Thanks, that was one of the more puzzling annoyances for RTL language users of LO.
Comment 21 Commit Notification 2018-03-24 22:06:37 UTC
Adolfo Jayme Barrientos committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/help/commit/?id=a86659f64cb346127159064202d9b6d3513ffe3e

tdf#97622 Update help text to match new behavior
Comment 22 Lior Kaplan 2018-09-30 15:17:09 UTC
Verified fix LibO 6.1.1.