Bug 134045 - VIEWING: Arabic letters are not displayed in Math
Summary: VIEWING: Arabic letters are not displayed in Math
Status: UNCONFIRMED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Formula Editor (show other bugs)
Version:
(earliest affected)
6.4.1.2 release
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-06-16 15:44 UTC by libre officer
Modified: 2020-07-22 12:54 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
MATH liberation mono (25.57 KB, image/png)
2020-06-18 12:51 UTC, libre officer
Details
noto naskh arabic (26.81 KB, image/png)
2020-06-18 12:52 UTC, libre officer
Details
MATH noto serif (25.57 KB, image/png)
2020-06-18 12:53 UTC, libre officer
Details
WRITER-before typing (106.57 KB, image/png)
2020-06-18 12:54 UTC, libre officer
Details
WRITER-copypaste arabic letter (106.08 KB, image/png)
2020-06-18 12:55 UTC, libre officer
Details
LibreOffice in Arabic (39.08 KB, image/png)
2020-06-18 13:06 UTC, libre officer
Details
arabic displayed after a Thaana letter (170.08 KB, image/gif)
2020-06-18 13:56 UTC, libre officer
Details
Devanagari script appears as squares (840.05 KB, image/gif)
2020-06-18 14:04 UTC, libre officer
Details
Test writing systems (1.63 KB, text/plain)
2020-06-18 14:14 UTC, libre officer
Details
Language Settings LO7.0b (27.67 KB, image/png)
2020-06-21 13:15 UTC, libre officer
Details

Note You need to log in before you can comment on or make changes to this bug.
Description libre officer 2020-06-16 15:44:06 UTC
Description:
When I type any Arabic letter it shows blank in the resulting window.

Other scripts displays without a problem (tested: Hebrew, kanji, Devanagari).

Steps to Reproduce:
1. Open Math
2. Type any Arabic letter, for example:  س

Actual Results:
letter not displayed

Expected Results:
س


Reproducible: Always


User Profile Reset: No



Additional Info:
Version: 6.4.1.2 (x86)
Build ID: 4d224e95b98b138af42a64d84056446d09082932
CPU threads: 4; OS: Windows 10.0 Build 17763; UI render: default; VCL: win; 
Locale: fr-CH (fr_FR); UI-Language: en-GB
Calc: CL

Version: 6.4.1.2 (x86)
Build ID: 4d224e95b98b138af42a64d84056446d09082932
CPU threads: 4; OS: Windows 10.0 Build 17763; UI render: default; VCL: win; 
Locale: fr-CH (fr_FR); UI-Language: en-GB
Calc: CL

Version: 7.0.0.0.beta1 (x64)
Build ID: 94f789cbb33335b4a511c319542c7bdc31ff3b3c
Threads CPU : 4; OS : Windows 10.0 Build 17763; UI Render : Skia/Raster; VCL: win
Locale: fr-CH (fr_FR); Langue IHM : fr-FR
Calc: threaded
Comment 1 libre officer 2020-06-16 15:45:28 UTC
Version: 6.4.3.2 (x64)
Build ID: 747b5d0ebf89f41c860ec2a39efd7cb15b54f2d8
CPU threads: 4; OS: Windows 10.0 Build 17763; UI render: default; VCL: win; 
Locale: fr-CH (fr_FR); UI-Language: en-US
Calc: threaded
Comment 2 Regina Henschel 2020-06-17 00:09:03 UTC
@libre officer: You cannot confirm your own bug report.

Please try, whether copy&paste from Writer to the command window works.

Please try, whether defining an Arabic letter as symbol works, so that it can be used with %.

Do you use the Arabic letter inside quotation marks or as name for a variable?

Which fonts are set in menu Format > Fonts?

How do you "type" an Arabic letter? Or other way, how can someone in e.g. Germany reproduce the problem?
Comment 3 libre officer 2020-06-17 11:27:27 UTC
(In reply to Regina Henschel from comment #2)
> @libre officer: You cannot confirm your own bug report.
> 
> Please try, whether copy&paste from Writer to the command window works.

Does not work.
 
> 
> Please try, whether defining an Arabic letter as symbol works, so that it
> can be used with %.
> 
> Do you use the Arabic letter inside quotation marks or as name for a
> variable?

Yes, I can type in a quote "", but for the other mentioned non Latin scripts you don't need it. Arabic letters shouldn't be the only exception because it is confusing to the user.

> Which fonts are set in menu Format > Fonts?

Liberation : Serif, Sans and Mono.
 
> How do you "type" an Arabic letter? Or other way, how can someone in e.g.
> Germany reproduce the problem?

For example, try to copy-paste these letters س and ص in a formula (these are commonly used as the equivalent of x and y).

If you want to type Arabic words, you can either use https://www.branah.com/arabic or add Arabic keyboard in your OS and start typing with it.



Info:
Eastern Arabic numerals like ٠ ١ ٢ ٣ ٤ ٥ ٦ ٧ ٨ ٩ ١٠ can be typed directly
Comment 4 Regina Henschel 2020-06-17 16:11:59 UTC
(In reply to libre officer from comment #3)
> (In reply to Regina Henschel from comment #2)
> > @libre officer: You cannot confirm your own bug report.
> > 
> > Please try, whether copy&paste from Writer to the command window works.
> 
> Does not work.

That works here with Version: 6.4.5.0.0+ (x64)
Build ID: 70a2071ce91b71326659e645dd97996262ea309a
CPU threads: 8; OS: Windows 10.0 Build 18362; UI render: default; VCL: win; 
Locale: de-DE (en_US); UI-Language: en-US
Calc: CL

> 
> > Which fonts are set in menu Format > Fonts?
> 
> Liberation : Serif, Sans and Mono.

The characters, which you mention below have the code points U+0633 and U+0635. They are not contained in the Liberation fonts. It might be, that character replacement does not work on your system. You should try to set a font in the fields in section "Formula Fonts", that contains this subset, e.g one of the "Noto" fonts..

>  
> > How do you "type" an Arabic letter? Or other way, how can someone in e.g.
> > Germany reproduce the problem?
> 
> For example, try to copy-paste these letters س and ص in a formula (these are
> commonly used as the equivalent of x and y).

Copy and paste works without problems here.

Another way I can insert these characters: I write U+0633, mark it and then click on "Toggle Unicode Notation". (You need to add that command to a toolbar in Math.)

> 
> If you want to type Arabic words, you can either use
> https://www.branah.com/arabic

Copy and paste from there has no problems.

So I cannot reproduce your problem.

Have you tried to use a new user profile?
Have you tried, whether the problem is still there, if you use an Arabic User Interface?
Comment 5 libre officer 2020-06-18 12:51:58 UTC
Created attachment 162158 [details]
MATH liberation mono
Comment 6 libre officer 2020-06-18 12:52:43 UTC
Created attachment 162159 [details]
noto naskh arabic
Comment 7 libre officer 2020-06-18 12:53:43 UTC
Created attachment 162160 [details]
MATH noto serif
Comment 8 libre officer 2020-06-18 12:54:41 UTC
Created attachment 162161 [details]
WRITER-before typing
Comment 9 libre officer 2020-06-18 12:55:24 UTC
Created attachment 162162 [details]
WRITER-copypaste arabic letter
Comment 10 libre officer 2020-06-18 12:57:43 UTC
(In reply to Regina Henschel from comment #4)
> 
> > 
> > > Which fonts are set in menu Format > Fonts?
> > 
> > Liberation : Serif, Sans and Mono.
> 

This one works for example for

> The characters, which you mention below have the code points U+0633 and
> U+0635. They are not contained in the Liberation fonts. It might be, that
> character replacement does not work on your system. You should try to set a
> font in the fields in section "Formula Fonts", that contains this subset,
> e.g one of the "Noto" fonts..
> 


> 
> Another way I can insert these characters: I write U+0633, mark it and then
> click on "Toggle Unicode Notation". (You need to add that command to a
> toolbar in Math.)
> 


> 
> Have you tried to use a new user profile?
> Have you tried, whether the problem is still there, if you use an Arabic
> User Interface?
Comment 11 libre officer 2020-06-18 13:05:49 UTC
(In reply to Regina Henschel from comment #4)
> 
> > 
> > > Which fonts are set in menu Format > Fonts?
> > 
> > Liberation : Serif, Sans and Mono.
> 
> The characters, which you mention below have the code points U+0633 and
> U+0635. They are not contained in the Liberation fonts. It might be, that
> character replacement does not work on your system. You should try to set a
> font in the fields in section "Formula Fonts", that contains this subset,
> e.g one of the "Noto" fonts..
> 

This one displays Arabic when choosing for example `noto Arabic naskh`.

> Another way I can insert these characters: I write U+0633, mark it and then
> click on "Toggle Unicode Notation". (You need to add that command to a
> toolbar in Math.)
> 

Using default font, it doesn't display the Arabic character.

> 
> Have you tried to use a new user profile?
> Have you tried, whether the problem is still there, if you use an Arabic
> User Interface?

This one displays Arabic using default font (see attachment).
Comment 12 libre officer 2020-06-18 13:06:37 UTC
Created attachment 162163 [details]
LibreOffice in Arabic
Comment 13 libre officer 2020-06-18 13:56:55 UTC
Created attachment 162165 [details]
arabic displayed after a Thaana letter

Arabic word is displayed if followed by a Thaana letter (Maldivian language).
Note that if a space or other characters are inserted between, Arabic became invisible.
Comment 14 libre officer 2020-06-18 14:04:09 UTC
Created attachment 162166 [details]
Devanagari script appears as squares

Devanagari scripts appears as squares in the bottom windows, but displays correctly in the output top window.
Comment 15 libre officer 2020-06-18 14:14:11 UTC
Created attachment 162169 [details]
Test writing systems

first and 3rd paragraphs are LTR scripts.
2nd paragraph are RTL scripts.
Comment 16 Regina Henschel 2020-06-18 14:25:44 UTC
Your images show, that it is a problem with fallback for missing glyphs. The rectangle which can be seen in "Math noto serif" is the indicator for a missing glyph. And indeed, font "Noto serif" does not contain the needed subset "Basic Arabic". Fonts "Noto Naskh Arabic" and "Tahoma" contain this subset.

I have no idea, why glyph fallback does not work on your system but works on my system. But you can use a workaround: Format > Fonts in Math has a button "Default". This puts the settings to your user profile, so that all new formulas will use them. So you need to find a font, which has the needed subset and set this font in Format > Fonts. Then click button "Default".

If you have already got "Segoe UI" that might work. It is a more recent font than "Tahoma" or "Times New Roman".
You can add additional fonts and language support to Windows 10 from Settings > Apps > Apps & features > Optional features > Add.

I put the bug back to "Unconfirmed". Unfortunately nothing can be further done until someone can confirm the problem.
Comment 17 libre officer 2020-06-21 10:05:21 UTC
(In reply to Regina Henschel from comment #16)
> Your images show, that it is a problem with fallback for missing glyphs. The
> rectangle which can be seen in "Math noto serif" is the indicator for a
> missing glyph. And indeed, font "Noto serif" does not contain the needed
> subset "Basic Arabic". Fonts "Noto Naskh Arabic" and "Tahoma" contain this
> subset.

I tried `Tahoma` and it works, I think that this is to be set as a default font in formula.

> I have no idea, why glyph fallback does not work on your system but works on
> my system. But you can use a workaround: Format > Fonts in Math has a button
> "Default". This puts the settings to your user profile, so that all new
> formulas will use them. So you need to find a font, which has the needed
> subset and set this font in Format > Fonts. Then click button "Default".

This ones doesn't work, because it is set to `liberation`.

> If you have already got "Segoe UI" that might work. It is a more recent font
> than "Tahoma" or "Times New Roman".

`Segoe UI` and `Times New Roman` does shows only small rectangles.

> You can add additional fonts and language support to Windows 10 from
> Settings > Apps > Apps & features > Optional features > Add.

I tested in notepad the text I attached, and all the fonts are all displayed.

> I put the bug back to "Unconfirmed". Unfortunately nothing can be further
> done until someone can confirm the problem.
Comment 18 libre officer 2020-06-21 10:51:46 UTC
Please look at the attachment 162159 [details]

The text directions are different between input and output !
Comment 19 Regina Henschel 2020-06-21 11:20:21 UTC
(In reply to libre officer from comment #18)
> Please look at the attachment 162159 [details]
> 
> The text directions are different between input and output !

Please write a separate new bug report for the direction problem and attach not only a screenshot but the file itself too.

Only to be sure, 'Complex text layout' is enabled in Tools > Options > Language Settings > Languages ?
What is your setting for 'Ignore system input language'?
Comment 20 libre officer 2020-06-21 13:15:29 UTC
Created attachment 162251 [details]
Language Settings LO7.0b

(In reply to Regina Henschel from comment #19)
> (In reply to libre officer from comment #18)
> > Please look at the attachment 162159 [details]
> > 
> > The text directions are different between input and output !
> 
> Please write a separate new bug report for the direction problem and attach
> not only a screenshot but the file itself too.

ok, will do 👍

> Only to be sure, 'Complex text layout' is enabled in Tools > Options >
> Language Settings > Languages ?
> What is your setting for 'Ignore system input language'?

Check my `Language Settings LO7.0b` attachement
Comment 21 V Stuart Foote 2020-06-24 04:49:45 UTC
(In reply to libre officer from comment #20)

> 
> (In reply to Regina Henschel from comment #19)
> > (In reply to libre officer from comment #18)
> > > Please look at the attachment 162159 [details]
> > > 
> > > The text directions are different between input and output !
> > 
> > Please write a separate new bug report for the direction problem and attach
> > not only a screenshot but the file itself too.
> 
> ok, will do 👍
> 

assuming that was bug 134193
Comment 22 dante19031999 2020-07-19 11:32:46 UTC
This looks like a locale error on
    parsePredefinedToken
    parseAnyToken
    Both defined at /core/include/unotools/charclass.hxx.
If I'm not wrong these should be simplifications of
    parsePredefinedToken
    parseAnyToken
    Both defined at /core/i18npool/inc/cclass_unicode.hxx
with an automatic locale handele. These token parsers are used for
all the modules.
My hypothesis: the locale "error" makes the token parser go backguards.
Have no idea of arabic, so not gonna help.
Good look.
Comment 23 Eyal Rozenberg 2020-07-22 11:00:00 UTC
At this point the bug is very confusing and I'm not sure what it's about any longer.

At any rate, I am not able to reproduce the bug as initially reported, with:

Version: 6.4.0.3
Build ID: b0a288ab3d2d4774cb44b62f04d5d28733ac6df8
CPU threads: 4; OS: Linux 5.2; UI render: default; VCL: gtk3; 
Locale: he-IL (en_IL); UI-Language: en-US


I would appreciate:

1. A summary comment about what we're dealing with here.
2. Filing of separate bugs if there are separate issues.
3. Removing attachments no-longer-relevant attachments and perhaps rolling up screenshots into a single zip file if there are so many.
Comment 24 V Stuart Foote 2020-07-22 12:54:14 UTC
Back to unconfirmed.  

(In reply to dante19031999 from comment #22)
> This looks like a locale error on
> ...
> Have no idea of arabic, so not gonna help.

Sure in the general case maybe, but issue of comment 0 is a "simple" font fall back deficiency within the sm module which handles its own font assignment--outside the locale and language selections. It remains unconfirmed in that the sm module allows use of specific fonts for the nodes composing a formula. The font has to contain glyph for the Unicode point entered. 

Potentially sm framework refactoring needed for bug 101174 could improve sm font handling in general. While sm edit shell work needed for bug 134193 will require correct font handling to compose non-standard localized formula markup with CTL or CJK mode handling.