Bug 151122 - Need way to avoid selecting fonts which don't cover the relevant language Unicode range
Summary: Need way to avoid selecting fonts which don't cover the relevant language Uni...
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: All All
: medium enhancement
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: Fonts-Name-Combobox Font-List
  Show dependency treegraph
 
Reported: 2022-09-22 09:57 UTC by Eyal Rozenberg
Modified: 2022-11-22 20:10 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:
Regression By:


Attachments
3 Screenshots of CTL-family fonts in the font selection dialog (169.42 KB, application/zip)
2022-09-23 19:27 UTC, Eyal Rozenberg
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Eyal Rozenberg 2022-09-22 09:57:46 UTC
When I am offered a font in the font selection dialog, I want to know which parts of the Unicode space (say, the basic multilingual plane) they cover. More precisely, I want to know which written scripts I can use them for. At the moment, this seems like anyone's guess, and when you use a font which is missing the glyphs you need, typically, some default is used; and since we don't know when/what fallback fonts are used (see bug 151121), we're left kind of in the dark.
Comment 1 V Stuart Foote 2022-09-22 14:01:16 UTC
Horrible idea. And we already provide for this with the Special Characters... dialog.

Selecting the Font from the listbox there will refresh the adjacent listbox for the font's covered Unicode blocks. And which further can then be selected to move the character chart focus to the start of the block.  You can pick from there.
Comment 2 Eyal Rozenberg 2022-09-22 15:22:38 UTC
(In reply to V Stuart Foote from comment #1)
> Horrible idea.

But I haven't even given a concrete idea, how can you know it's horrible beforehand?

And - how am I supposed to know if the fonts LO offers me even have the (combination of) languages I'm trying to write in? The only thing LO is willing to tell me is whether it's a "Western", "Complex", or "Asian" font family.

> And we already provide for this with the Special Characters... dialog.

Hmm, yes, that's true, that dialog doesn't show the silly fallback text.

But it's very inconvenient for someone to have to go back and forth between the two dialogs, to check.

Possible ways to get this indication:

* Filter the language-family font-family-list by chosen language and features.
  * Sub-option: Same, but allow requiring multiple languages.
* Stylistic indication of which fonts in the language-family font-family list support the chosen language.
* Have some button or hover-over-able widget open up a list, perhaps bulleted by symbolic icons, of the BMP regions covered by the font family. i.e. essentially the same list from the Insert Special Character dialog.

Orthogonally to the above:

* Add ability to toggle the fallback font use in the preview area of the font selection dialog, on or off.
Comment 3 V Stuart Foote 2022-09-22 16:09:09 UTC
(In reply to Eyal Rozenberg from comment #2)
> (In reply to V Stuart Foote from comment #1)
> > Horrible idea.
> 
> But I haven't even given a concrete idea, how can you know it's horrible
> beforehand?
> 

Because the suggestion in the 'Font list' context is that the drop list itself should included those details--they do not belong there in the UI!

> And - how am I supposed to know if the fonts LO offers me even have the
> (combination of) languages I'm trying to write in? The only thing LO is
> willing to tell me is whether it's a "Western", "Complex", or "Asian" font
> family.
> 
> > And we already provide for this with the Special Characters... dialog.

The Special Character Dialog (SCD) is where LibreOffice's delivery of these Unicode font coverage issues belong. I'd also argue that detailing font fall back and even font replacement (the Tools -> Options -> Fonts panels) likewise need linkage from the SCD.

Attempting to provide such details via the font selection listboxes (e.g. with tolltip and pop-up dialogs) is recipe for UI garbage. And is a "Horrible idea".

When you are assigning the font is not the time (or place in the UI) to be identifying the font to use.
Comment 4 V Stuart Foote 2022-09-22 16:12:32 UTC
> When you are assigning the font is not the time (or place in the UI) to be
> identifying the font to use.

... identifying the font feature/language coverage to use.
Comment 5 Heiko Tietze 2022-09-23 06:55:11 UTC
Don't see a clear use case. And in general I would keep the font selection mechanism as simple as possible and let the OS provide font management functions. If necessary a special dialog perhaps realized via extension could give insights to font attributes.
Comment 6 Eyal Rozenberg 2022-09-23 08:10:26 UTC
(In reply to Heiko Tietze from comment #5)
> Don't see a clear use case.

I gave the use case in Comment #2: I need to know whether the font family on the list covers the language I want to write in. The preview area does not provide that information, nor even a good hint of whether that's the case.
Comment 7 Eyal Rozenberg 2022-09-23 09:05:00 UTC
(In reply to V Stuart Foote from comment #3)
> Because the suggestion in the 'Font list' context is that the drop list
> itself should included those details--they do not belong there in the UI!
> 
> The Special Character Dialog (SCD) is where LibreOffice's delivery of these
> Unicode font coverage issues belong.
> 
> When you are assigning the font is not the time (or place in the UI) to be
> identifying the font to use.

(In reply to Heiko Tietze from comment #5)

> in general I would keep the font selection
> mechanism as simple as possible 


A single reply to all of these comments...


A font selection dialog (FSD; versus, say, a font choice text box elsewhere in the UI) is not intended merely for the user to communicate their apriori choice of font - but rather to allow the user to make the choice while in the dialog. While its true that this dialog is not a full-fledged "font explorer" - it does presume to allow making that choice - via the preview area. The idea is that you see what the font looks like for some piece of text, and make your decision.

But this has two underlying assumptions:

1. That the font you see in the preview area is the actual font you're selecting (rather than some kind of fallback).
2. (When assumption 1 holds) That if the font covers the glyphs used for the preview area, then it covers all glyphs for all of the languages in that language group you intended to use. (*)

These assumptions almost always hold if you're writing English, and usually hold if you're writing German or other Latin-based alphabets. But that  is absolutely not the case for other languages, especially in CTL language-group.

Let's ignore CTL for a second though. Suppose I want write in Russian. Some fonts have glyphs for the Cyrillic alphabet, some don't. I cannot choose from amongst a set of fonts unless I know that they are fonts which cover all glyphs related to the Russian language; and the font dialog does not let me know that this is the case.

You both seem to suggest that I should first open the SCD and check. That is a weird suggestion even if it were valid - to use a not-directly-linked dialog, from another menu, which officially has an entirely different purpose, as part of the font selection process. But even ignoring its weirdness - I can't use the SCD to choose between fonts, since I don't see how a stretch of text looks in a given font; and I can't use the FSD to choose between fonts since a great many of the fonts I'm offered I can't actually use. And I also cant use the FSD since the stretch of text presented in the preview is not in my language of interest.

(Yes, I can first type some text in Russian, then select it, then launch the FSD, and I'll get the preview I want, but that's cumbersome and newbie users won't even think of doing that. And there's the fact that the preview might not actually use the font itself but some fallback.)

So even by LO's current effective standard of what's necessary for selecting a font, this information needs to be related somehow. In fact, _some_ of it is already related - by placing some fonts in the Western language-family list and other fonts in the CTL language-family list: That's a partial indication of which parts of the Unicode plane are covered by the font family. What is necessary is additional refinement - somehow - to let the user choose only from amongst the fonts relevant to the languages they intend to write in.


Stuart worries about avoiding UI garbage. I agree that this is a danger and that careful thought is necessary. But something needs to be done, and users cannot be expected to do trial-and-error font selection.

So how about we focus on my first possible idea: 

* Filtering the list of fonts by the choice of language already present underneath the list: Only those fonts offering all glyphs associated with the chosen language will be displayed.
* (Possibly) allowing the choice of multiple languages in the widget(s) below the font list, for stricter filtering.

> If necessary a special dialog perhaps realized via extension
> could give insights to font attributes.

If something is necessary, then it needs to be in LO itself. If it's nice-to-have it may belong in an extension. I hope I've managed to convince that LO's own "standard" for what's necessary for font selection also necessitates some UI change to avoid mistaken selection of language-incompatible fonts.
Comment 8 Telesto 2022-09-23 14:41:17 UTC
@Eyal
You're clearly wanting to achieve something and struggling with fonts/ font replacement. I haven't taken much interest in fonts (or font family's etc). I'm only aware of font substitution because I encountered documents on the bug tracker with exotic fonts. (And couple of layout bugs caused by font substitution). So I really lack the understanding of the whole topic..

Would you mind to share what you're working on (some integral for all recent reports). And what the issues are you're encountering. It sounds highly specific what you want to do from a distance. And those requests are highly technical ..

I lack knowledge of limitations of fonts (how many symbols each font supports). And I have surely no clue about font issues with Hebrew/Arabic/Chinese/Japanese fonts (and font family's).
Comment 9 Eyal Rozenberg 2022-09-23 19:27:57 UTC
Created attachment 182649 [details]
3 Screenshots of CTL-family fonts in the font selection dialog

@telesto

To illustrate what CTL language users are facing here, please have a look at the three screenshots in the attached zip files. Each screenshot corresponds to the selection of a different CTL font (family).

Now, tell me what you can discern about the differences between these three fonts for Arabic script. When you answer me I hope my reply will make things clearer for you.
Comment 10 Telesto 2022-09-23 21:08:53 UTC
(In reply to Eyal Rozenberg from comment #9)
> Created attachment 182649 [details]
> 3 Screenshots of CTL-family fonts in the font selection dialog

Ah, OK "font selection dialog" = Format -> Character dialog -> Font tab with Complex Text Layout Turned on in Tools -> Options -> Language settings -> Languages (not active by default with Western locale)

You ask yourself, does the Hebrew font cover the same characters as the Western font.. Now I get it :-). Seems to be a PITA to figure out..

I guess that there are cases that the lack of a certain character to be trivial. You probably want same kind of side-by-side special character dialog if there are  mismatches; and for which symbols..

How is this problem handled with other application? Like MSO?
Comment 11 V Stuart Foote 2022-09-23 23:02:43 UTC
(In reply to Telesto from comment #10)
> 
> How is this problem handled with other application? Like MSO?

It is not, nor should it be. And that is the issue.

There are already some fabulous programs for doing font selection based on its Unicode coverage.  Point being these font management features do not belong in an office product bcz they are handled better either by os/DE provided apps--eg fontconfig and the fc- utilities or the ancient xfd or xfontsel, or by any number of GUI based character map utilities including the GNOME Character Map.

If on Windows give Andrew West's BableMap a drive [1]. It has *ALL* the features that Eyal is asking for, but it is specialized software. In fact the dev spun off the font manager from his text editor BablePad bcz font management and analysis does not belong with the editor.

And of course there are commercial products that provide similar function--on Windows High-Logic's MainType for example.  

=-ref-=
[1] https://www.babelstone.co.uk/Software/BabelMap.html
Comment 12 Eyal Rozenberg 2022-09-24 13:26:50 UTC
(In reply to Telesto from comment #10)
> You ask yourself, does the Hebrew font cover the same characters as the
> Western font.. Now I get it :-).

Oh, you made the wrong mistake :-P

First of all - that's meaningless. There's no need for the font used for Arabic (not Hebrew, the screenshots has Arabic sample text despite Hebrew being selected as the language which is another bug I should file I think) to cover the same characters as the one of Western scripts. In fact, they might as well be disjoint.

I was expecting you to say "the three fonts look almost similar in Arabic"

In fact, _none_ of the three fonts in the screenshots supports Arabic, or Hebrew, at all. It only looks like they do, and it even looks like the three fonts are slightly different - as apparently somehow the glyph spacing is taken from the font's general meta-data, so the result doesn't look the same. If you choose one of these fonts for CTL, you'll just get some fallback font glyphs used, and you would not be told which font (family) those fallback glyphs are taken from.

This is a problem for people, because:

* The user believes a certain font supports Arabic, while it doesn't.
* If the user sends this file to another person, that other person is likely to see it rendered differently - depending on which OS, VCL and font choices they have, which in turn determines the fallback font used.
* The chosen font may cover one CTL language's range of glyphs but not another one's. Even if the preview corresponded to the chosen language - how would the user choose a font which has glyphs for both, say, Hebrew and Arabic? Or Arabic and Shahmukhi (Punjabi)?

The third problem affects fewer people - but still, quite a few. The first two problems are faced by RTL/CTL users all the time. In fact, all three problems affect even people who only use LTR/Western languages, as in the example I gave with Cyrillic, above.
Comment 13 Heiko Tietze 2022-09-26 09:53:51 UTC
What bothers is the wish to get some very broad information at a control that is simply a picker. But if we turn it around and show incongruencies it makes sense. We draw the font name in italic if not available on the system and could so something similar if the listed font item unicode coverage does not match the paragraph language.
Comment 14 Eyal Rozenberg 2022-09-26 18:45:03 UTC
(In reply to V Stuart Foote from comment #11)

> There are already some fabulous programs for doing font
> selection based on its Unicode coverage.  Point being 
> these font management features do not belong in an office
> product bcz they are handled better either by os/DE
> provided apps--eg fontconfig and the fc- utilities or the
> ancient xfd or xfontsel, or by any number of GUI based
> character map utilities including the GNOME Character Map.

You seem to be suggesting LO remove its font selection dialog, or the font selection pane of the font dialog, entirely...

Font selection belongs in (almost) any application in which the user needs to select fonts. And while complex font exploration may perhaps be better suited for a different app, it is not a complex or esoteric task to want to choose a font which can be used for the language you want to write in; and my office productivity app must help me avoid choosing fonts (or font families) which don't cover the language I'm making the choice for.

> It has *ALL* the features that Eyal is asking for,

I asked for just one thing. And I'm not even the one who asks for it; I would argue every user asks for it: Not misdirected into choosing an invalid font for the language I'm writing in.
Comment 15 Eyal Rozenberg 2022-09-26 18:53:54 UTC
(In reply to Heiko Tietze from comment #13)
> What bothers is the wish to get some very broad information at
> a control that is simply a picker. But if we turn it around 

I just need to know the font I'm selecting covers my language(s). The title may overstate what's necessary, I've rephrased it.

> unicode coverage does not match the paragraph language.

There's a language choice underneath the picker - I believe it's better to go with that one. That can also allow the user to require coverage of multiple languages or no language.


I haven't made up my mind regarding whether I like italicizing, graying-out or filtering-out better.
Comment 16 Heiko Tietze 2022-11-22 12:20:54 UTC
I don't like "avoid selecting" and would prefer better feedback (that we have). Fiddling around with the list of available fonts makes it unclear for the user what is possible. Imagine a font is available for one paragraph style but not for another.
Comment 17 Eyal Rozenberg 2022-11-22 18:45:24 UTC
(In reply to Heiko Tietze from comment #16)
> I don't like "avoid selecting" and would prefer better feedback (that we
> have).

You mean "than we have", right?

> Fiddling around with the list of available fonts makes it unclear for
> the user what is possible. Imagine a font is available for one paragraph
> style but not for another.

How would this happen, unless you choose a different language? And if you do that, you would probably not be surprised to see a different list of fonts.

If we go for feedback-only, LO would still be frustrating you, making you have to guess which fonts covers a language until you get it right.
Comment 18 V Stuart Foote 2022-11-22 20:10:22 UTC
(In reply to Eyal Rozenberg from comment #17)
> 
> How would this happen, unless you choose a different language? And if you do
> that, you would probably not be surprised to see a different list of fonts.
> 
> If we go for feedback-only, LO would still be frustrating you, making you
> have to guess which fonts covers a language until you get it right.

You currently don't have to guess anything, just open the Special Characters dialog and review the character chart directly, even select the specific Unicode character block range to explore for a particular script. You'll see what the font "covers".