Bug 139107 - Availability of non-printing Unicode special characters should not depend upon the font
Summary: Availability of non-printing Unicode special characters should not depend upo...
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
6.4.7.2 release
Hardware: x86-64 (AMD64) All
: medium minor
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
: 162635 (view as bug list)
Depends on:
Blocks: Special-Character
  Show dependency treegraph
 
Reported: 2020-12-20 20:28 UTC by Shay G
Modified: 2024-08-26 12:04 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Shay G 2020-12-20 20:28:51 UTC
Description:
The RLO symbol is documented here: https://www.unicode.org/reports/tr9/#Directional_Formatting_Codes

I wasn't able to find it in the David CLM font. But it was on Liberation Serif.
The symbol should be on every font that supports RTL. It doesn't have and visual symbol anyway so no icon is required.

Steps to Reproduce:
1. Go to Insert -> Symbol
2. Search your favorite RTL font for Override.
3. No RLO on the list

Actual Results:
Feature is missing

Expected Results:
Can choose the RLO symbol


Reproducible: Always


User Profile Reset: No



Additional Info:
Show the RLO symbol

גרסה: 6.4.7.2
מזהה הבנייה: 6.4.7-7
תהליכי משנה במעבד: 8; מערכת הפעלה: Linux 5.9; עיבוד מנשק: בררת מחדל; VCL: kf5; 
מיקום: he-IL (en_US.UTF-8); UI-Language: he-IL
Comment 1 Eyal Rozenberg 2020-12-20 22:30:41 UTC
More generally, the issue is as follows:

The Insert Special Character dialog should allow the user to locate, select and insert Unicode characters which are non-printing, regardless of which font family is chosen in the dialog.

However - that is not the case. With some selected families - one cannot even locate, let alone select and copy, some of these characters; while with other families, more (or all) of them are available, e.g. 

Specifically, I tried this with the Right-Left Override character, and got

* Failure with: David CLM (bundled with LibreOffice), Nachlieli CLM, Linux Libertine, Georgia
* Success with: Liberation Sans, Liberation Serif, Noto Sans

and I also tried this with Right-to-Left Mark, and got:

* Failure with: Linux Libertine, Georgia
* Success with: David CLM, Nachlieli CLM, Liberation Sans, Liberation Serif, Noto Sans

(the choice of fonts is arbitrary, except that David CLM is LibreOffice' default for Hebrew text)
Comment 2 Eyal Rozenberg 2020-12-20 22:33:10 UTC
Remember:

Right-to-Left Override, or RLO, is Unicode character U+202D
Right-to-Left Mark, or RLM, is Unicode character U+200F

(the latter can be entered using the standard Hebrew keyboard layout using RightAlt+0)

For the unfamiliar, read this:
https://www.unicode.org/reports/tr9/#Directional_Formatting_Codes
Comment 3 Eyal Rozenberg 2020-12-20 22:34:06 UTC
And one final bit of spam for today: My findings were using

Version: 7.0.3.1
Build ID: d7547858d014d4cf69878db179d326fc3483e082
CPU threads: 4; OS: Linux 5.2; UI render: default; VCL: gtk3
Locale: he-IL (en_IL); UI: en-US
Comment 4 QA Administrators 2022-12-21 03:20:16 UTC Comment hidden (obsolete)
Comment 5 Shay G 2022-12-21 19:09:44 UTC
Bug re-confirmed.

Version: 7.3.7.2 / LibreOffice Community
Build ID: 30(Build:2)
CPU threads: 32; OS: Linux 6.0; UI render: default; VCL: kf5 (cairo+xcb)
Locale: he-IL (en_US.UTF-8); UI: he-IL
7.3.7-2
Calc: threaded
Comment 6 Mike Kaganski 2024-08-04 08:01:36 UTC
I disagree.

We have a special menu for insertion of marks: Insert->Formatting Mark->Right-to-Left mark. This feature provides exactly the necessary functionality to insert a font-agnostic Unicode non-printing codepoint to control the directionality.

On the other hand, there is the dialog in question, that is intended to show what is there in the fonts. And nothing else.

The issue may be its (and its command) title "Special Character(s)" - which has always confused me: it's not about anything special, it's about any characters of specific fonts. The similar dialog in Windows is called "charmap"; in Word, it's "Symbol" dialog.

But anyway: this request to add non-existing elements to fonts, based on "they are non-printing", is WF IMO.
Comment 7 Eyal Rozenberg 2024-08-05 10:29:14 UTC
(In reply to Mike Kaganski from comment #6)

Insert > Formatting Marks only offers some of the available control characters / formatting marks. Just for bidirectional control alone there are 12 control chars; the menu has just two. Do you suggest they all be added? And then - what about control characters from here?

https://en.wikipedia.org/wiki/C0_and_C1_control_codes

even just the 12 will make the menu already deterring for many/most users. It's a good idea, IMHO, that it only offers some marks.

> On the other hand, there is the dialog in question, that is intended to show
> what is there in the fonts. And nothing else.

That's really not true. This is the "Insert > Special Character..." dialog, not "Insert > Font Glyph..." ; users justifiably expect to be able to insert any special character from that dialog, without having to determine whether some typeface artificially contains that character or not.

Moreover, even if the dialog only said "character", and even if the Formatting Marks actually included RLO/LOR - it is still legitimate and reasonable to have cross-coverage of functionality from different parts of the UI. It's ok if the user found their way to the special character via browsing the Unicode ranges - and we should not hinder their desire to then say "ok, I want to put one of these in my document".

> The issue may be its (and its command) title "Special Character(s)" - which
> has always confused me: it's not about anything special, it's about any
> characters of specific fonts.

It has been implemented that way. This bug report requires a change in that implementation, and that change is merited IMHO. This can be achieved without worsening or slowing down the workflow for users looking for glyphs/symbols.

> But anyway: this request to add non-existing elements to fonts

We should definitely implement this in a way which does not confuse users about this fact. That can happen in multiple ways, e.g. :

1. Toggle/checkbox for searching for glyphs in fonts vs non-printing control characters where the font doesn't matter.
2. Some kind of visual indication that a control character in the table is presented, and can be inserted, irrespective of whether than font "has it" or not. maybe some inner frame, or background hatching, or color triangle at a corner etc.
Comment 8 Mike Kaganski 2024-08-05 10:36:07 UTC
(In reply to Eyal Rozenberg from comment #7)
> Insert > Formatting Marks only offers some of the available control
> characters / formatting marks. Just for bidirectional control alone there
> are 12 control chars; the menu has just two. Do you suggest they all be
> added? And then - what about control characters from here?
> 
> https://en.wikipedia.org/wiki/C0_and_C1_control_codes

If there are many control characters, then we need a new dialog for their insertion, instead of individual menu entries.
Comment 9 Mike Kaganski 2024-08-05 10:53:53 UTC
... and if we want to re-use exactly *this* dialog, then it only can de implemented by making this dialog multi-tab: one tab for the fonts' glyphs (having everything it has now), and another tab for "control characters", which would be font-agnostic, and would be there to implement that "users found this in UI, let's provide this here" idea (which is valid, but must not confuse by pretending that these characters are present in a font where they are not). Additionally, such distinction could emphasize their functional role; in that dialog, we could explain their functionality, and help to choose the one that is needed for some specific need, better than using a cell in the grid.
Comment 10 Heiko Tietze 2024-08-05 11:41:14 UTC
The request is not to insert the bell character or any other but RLO. Which is available and efficient. I agree with WF but would appreciate if Shay confirms.
Comment 11 V Stuart Foote 2024-08-05 19:06:00 UTC
The Special Character Dialog is our offering of a character map.

It has always been "single font", and was refactored to drop Unicode-centric chart representation (compressing out empty codepoints, but kept Unicode's 16 column table).

So for supporting workflows where NPC, e.g. directional controls, are not provided glyphs, we could offer a toggle to build a composite font chart--filling in glyphs when current single font does not cover them. And allowing user configuration to their preferences (both single font and fonts used for composite).

A good example of this approach, see Bablestone's (Adam West's) BableMap utility offering.  https://www.babelstone.co.uk/Software/BabelMap.html unfortunately Windows only.

Otherwise as Mike notes, the NPC are control/formatting characters and we provide  UI (Insert -> Formatting Mark) to input when desired without dependence on the SCD. The SCD, when the font supports it, *is* an alternate way of inserting the control character for the PS active font if it "covers" it. Otherwise our Alt+X conversion can be used to enter the Directional Formatting control glyphs [1] that receive fallback font (that may or may not be rendered with a visible glyph).

=-ref-=
[1] https://www.unicode.org/reports/tr9/#Directional_Formatting_Codes
Comment 12 Eyal Rozenberg 2024-08-05 20:15:38 UTC
(In reply to Heiko Tietze from comment #10)
> The request is not to insert the bell character or any other but RLO.

RLO is just the example. The request is that when one sees a control char in the dialog, one is able to realize the natural desire to insert that character; and is not hindered by the artificial, and irrelevant, fact of the font displayed in the dialog not having a (dummy) entry for that control character.

> insert... RLO ... Which is available and efficient. 

1. Where is it available?
2. Even if it were, the point is not efficiency, but rather not to frustrate the user's wishes. Preventing a very straightforward interaction with our UI because "well, you can do that another way which we like better" - is a bad idea.
Comment 13 Eyal Rozenberg 2024-08-05 20:19:38 UTC
(In reply to V Stuart Foote from comment #11)
> So for supporting workflows where NPC, e.g. directional controls, are not
> provided glyphs, we could offer a toggle to build a composite font
> chart--filling in glyphs when current single font does not cover them. And
> allowing user configuration to their preferences (both single font and fonts
> used for composite).

This is possible, and quite possibly a good idea in itself, but is much more far-reaching in terms of the UI implications (and perhaps coding effort). It also relates to bug 151215, and font selections in MSO (see bug 151215 comment 19). 

However - when there's no glyph necessary, i.e. for non-printing characters, and at least for the "special" characters this bug is concerned with - there is no need to build anything. In fact, even with a composite font - it may be the case that none of the components claims to cover the control chars, and we would be left with the same problem (of this bug)...
Comment 14 V Stuart Foote 2024-08-26 11:36:29 UTC
*** Bug 162635 has been marked as a duplicate of this bug. ***
Comment 15 V Stuart Foote 2024-08-26 11:50:58 UTC
Mike's comment 8 and comment 9 of adding an additional tab to the SCD to support charmap of font neutral NPC formatting and control characters would be viable UI.

It could be font aware by default, but support visualizing when glyphs are filled by fallback.

Layout might be a question, i.e. keep the 16 col of Unicode charts or a list with UCS string name. Several ways to slice it for efficient lookup, and a search (against UCS name) will still be needed. Then some decision about search of localized string (as provided now with the Insert -> Formatting Mark menu list).
Comment 16 Eyal Rozenberg 2024-08-26 12:01:53 UTC
(In reply to V Stuart Foote from comment #15)
> Mike's comment 8 and comment 9 of adding an additional tab to the SCD to
> support charmap of font neutral NPC formatting and control characters would
> be viable UI.

Definitely a possibility. Something similar-but-different would be having one of the "Character Blocks" be "Non-printing & Control characters", perhaps marking it different somehow; and when that's chose, the choice of font disappears, making it clear (?) that the character is font-independent. Or a toggle to control this effect. The "benefit" is that you avoid the work of designing the extra tab.

> Layout might be a question, i.e. keep the 16 col of Unicode charts or a list
> with UCS string name. Several ways to slice it for efficient lookup, and a
> search (against UCS name) will still be needed. Then some decision about
> search of localized string (as provided now with the Insert -> Formatting
> Mark menu list).

... although the different design might be useful.
Comment 17 Luke Kendall 2024-08-26 12:04:47 UTC
(In reply to Eyal Rozenberg from comment #16)
> (In reply to V Stuart Foote from comment #15)
> > Mike's comment 8 and comment 9 of adding an additional tab to the SCD to
> > support charmap of font neutral NPC formatting and control characters would
> > be viable UI.
> 
> Definitely a possibility. Something similar-but-different would be having
> one of the "Character Blocks" be "Non-printing & Control characters",
> perhaps marking it different somehow; and when that's chose, the choice of
> font disappears, making it clear (?) that the character is font-independent.
> Or a toggle to control this effect. The "benefit" is that you avoid the work
> of designing the extra tab.
> 
> > Layout might be a question, i.e. keep the 16 col of Unicode charts or a list
> > with UCS string name. Several ways to slice it for efficient lookup, and a
> > search (against UCS name) will still be needed. Then some decision about
> > search of localized string (as provided now with the Insert -> Formatting
> > Mark menu list).
> 
> ... although the different design might be useful.

That idea sounds very clear for the user, too!