Bug Hunting Session
Bug 104318 - CTL, CJK, and Western Language controls in the GUI cause confusion, handling languages and script selection needs rework--comfortable for a "Global" user community
Summary: CTL, CJK, and Western Language controls in the GUI cause confusion, handling ...
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: UI (show other bugs)
Version:
(earliest affected)
5.1.4.2 release
Hardware: All All
: medium enhancement
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: Options-Dialog-Language
  Show dependency treegraph
 
Reported: 2016-12-01 10:30 UTC by Toby Anderson
Modified: 2018-10-24 20:33 UTC (History)
10 users (show)

See Also:
Crash report or crash signature:


Attachments
Mockup for comment 10 (120.35 KB, image/png)
2018-06-21 14:48 UTC, Heiko Tietze
Details
A screenshot of a sidebar with sections (25.15 KB, image/png)
2018-07-01 05:28 UTC, Mike Kaganski
Details
Mockup for comment 12 (67.67 KB, image/png)
2018-07-02 06:43 UTC, Heiko Tietze
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Toby Anderson 2016-12-01 10:30:12 UTC
Description:
When the CTL checkbox is not checked (tools -> Options... -> Language settings -> languages) then the user has no way to edit the basic fonts for CTL. Specifically the "Basic Fonts (CTL)" section of the options dialog is hidden and cannot be accessed. Nevertheless these basic fonts get applied to text that uses unicode characters considered to be in the CTL range.

Steps to Reproduce:
1. Go to tools -> Options... -> Language settings -> languages and make sure the checkbox "Complex text layout (CTL)" is unselected
2. Enter text in the document that uses a CTL script (e.g. Bengali script)
3. Attempt to format the text by applying a predefined style (not direct formatting) which uses a different font to the one listed in the (hidden) "Basic Fonts (CTL)" section of the options dialog

Actual Results:  
The font of the text remains whatever is set in the "Basic Fonts (CTL)" section of the options dialog

Expected Results:
The font of the text should become whatever the style specifies


Reproducible: Always

User Profile Reset: No

Additional Info:


User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36
Comment 1 Buovjaga 2016-12-07 17:13:27 UTC
Could you test with 5.3 beta to see, if it still happens?

You didn't mention your Linux distro, so..

For debs: http://dev-builds.libreoffice.org/pre-releases/deb/x86_64/LibreOfficeDev_5.3.0.0.beta1_Linux_x86-64_deb.tar.gz
For rpms: http://dev-builds.libreoffice.org/pre-releases/rpm/x86_64/LibreOfficeDev_5.3.0.0.beta1_Linux_x86-64_rpm.tar.gz

https://wiki.documentfoundation.org/Installing_in_parallel/Linux

Set to NEEDINFO.
Change back to UNCONFIRMED, if the problem persists. Change to RESOLVED WORKSFORME, if the problem went away.
Comment 2 Mike Kaganski 2016-12-12 17:00:36 UTC
The setting that you describe (tools -> Options... -> Language settings -> languages -> CTL) is not expected to change how existing documents are shown. It is to define if UI needs to support them. It is in the "Default Languages for Documents" section, and they don't pretend to do what you seem to expect.

Any information present in the document will be used to display it (provided that system has relevant fonts etc.) regardless of UI/defaults settings.

Closing as NOTABUG.
Comment 3 Toby Anderson 2018-06-18 18:13:10 UTC
"Any information present in the document will be used to display it (provided that system has relevant fonts etc.) regardless of UI/defaults settings."
This is the behaviour I expect, but it is not the behaviour I am getting. I am reopening this bug, and I will try to express it in a different way here.

On a fresh install with a new document I create a new style, based on body text. I set the font in the new style to be "Noto Serif Bengali" This is a font I have installed on my system. Next I type some Bengali text into my document: simply the word বাংলা. Then I select that text and choose the new style I made. I expect the font of that text to become Noto Serif Bengali (I am expecting too much?) It does not! The font remains Lohit Devanagari.

Surely this is a bug, right?

I'm now seeing this in version 6.0.3.2
I'm running Ubuntu 18.04

It's possible to change the font of the Bengali text using direct formatting, but not by applying a style.

Now, to help whoever might get assigned to fix this bug, this is the steps I eventually worked out I needed to take to get style's working on Bengali (and any CTL) textː

tools -> Options... -> Language settings -> languages
check the checkbox for complex text layout (CTL)
(at this point a slew of new options magically appears, but all the new options that are found in the "options" dialog don't actually appear until you close and reopen the dialogǃ But that's a bug for another day)
Open the Styles and Formatting dialog and modify the new style you made. Open the font tab.
Now there are two font settings in the style - one that applies when the style is on western text, and one that applies when it is on CTL text. The second one is set to Lohit Devenagari.

So the problem is not that it's completely impossible to use styles to set the font of CTL text, but merely that it is completely unfeasible that any normal user would be able to work out how to do that.

When that CTL checkbox in the options dialog is left unchecked then the styles only let you see and edit the font that is applied to western text, but they apply a different font to non-western text and the place that this is to be modified is completely hidden from view. I am suggesting that this is a bug.

I suppose the expected behaviour would be that when that CTL checkbox is not checked then all CTL text in a document should be treated as western for the purposes of applying styles.
Comment 4 Buovjaga 2018-06-18 18:20:47 UTC
(In reply to Toby Anderson from comment #3)
> tools -> Options... -> Language settings -> languages
> check the checkbox for complex text layout (CTL)
> Open the Styles and Formatting dialog and modify the new style you made.
> Open the font tab.
> Now there are two font settings in the style - one that applies when the
> style is on western text, and one that applies when it is on CTL text. The
> second one is set to Lohit Devenagari.
> 
> So the problem is not that it's completely impossible to use styles to set
> the font of CTL text, but merely that it is completely unfeasible that any
> normal user would be able to work out how to do that.
> 
> When that CTL checkbox in the options dialog is left unchecked then the
> styles only let you see and edit the font that is applied to western text,
> but they apply a different font to non-western text and the place that this
> is to be modified is completely hidden from view. I am suggesting that this
> is a bug.
> 
> I suppose the expected behaviour would be that when that CTL checkbox is not
> checked then all CTL text in a document should be treated as western for the
> purposes of applying styles.

Let's ask UX team's opinion
Comment 5 Mike Kaganski 2018-06-19 02:09:36 UTC
Let me try to explain what you see.

To make LibreOffice UI less cluttered for users that don't need some of the functionality (like people using exclusively Western writing systems, e.g. English language), LibreOffice has some options to hide some of its controls. I stress this word: *hide*. It doesn't mean that hiding something from the sight of user makes settings controlled by those controls to magically start behaving differently. It is judged that an average English speaker would not care what font is defined for CTL (complex text layout) or CJK (Chineese/Japaneese/Korean) languages - simply because that average speaker would *not* input the data in those languages.

SO when you create new styles, or use any pre-existing styles, when your UI is setup to hide those controls, what happens is LibreOffice keeps creating/remembering the default settings for those languages (it creates them as hard-coded in program, to make sensible default fonts that allow to use those styles with those writing systems). But those settings are, naturally, hidden. So user may be unaware of their presens and effect.

When a user who has a CTL/CJK keyboard layout/IME, but has CTL/CJK "support" (=controls display) disabled, starts to type in those writing system languages, LibreOffice still detects the language (say, Bengali) - remember, that we only hide controls, don't change any behaviour! - and starts applying the (currently hidden) parts of the style, e.g., the CTL font, regardless of what font was assigned to Western systems (the only visible control when CTL and CJK are hidden), even if user assigned a CTL-capable font to Western config.

This is not a bug. This works as designed (although one might propose some improvements here). Personally I feel that those controls would better be removed totally, so that all controls (including CTL/CJK) were shown at all times; possibly with better separation to tabs, so that user would have better distinction ("that tab is for CTL - no need to look there for English text"). But my opinion aside: it's usual complaint that "no average user would figure that"; but let's rwad that again: the use case is that LibreOffice first launch on a CTL/CJK system should automatically enable those settings - thus for average (was majority of) CTL/CJK user, this is not a problem; for a typical Western systems user, those controls are overkill and clutter, so they are hidden by default - and do the good job *by default*; and the minority who happen to use Werstern locale systems, but manually configure CTL/CJK input methods later, need some configuration, that is described in (surprise!) our documentation [1] (oh, I know: "users don't read documentation!"). Or searching for "Bengali" on Ask site [2] brings relevant answers right away. So - undoubtedly, there might be some improvement to this; but (a) what you see are not bugs at all; and (b) the proposals to improve must be specific, like "let's do this like that", not something like "It's not apparent to me (without reading docs) - so go change somehow!".

[1] https://documentation.libreoffice.org
[2] ask.libreoffice.org
Comment 6 Mike Kaganski 2018-06-19 02:32:39 UTC
Some errata/clarifications (sorry - I'm not a native English speaker):

presens -> presence
rwad -> read
was majority -> wast majority

"Personally I feel that those controls would better be removed totally": I mean that I would propose to remove the controls (the two checkboxes on Options-Language Settings-Languages) that allow to hide ("disable support for") Asian/Complex text layout controls.
Comment 7 Mike Kaganski 2018-06-19 02:57:46 UTC
(In reply to Toby Anderson from comment #3)
> I suppose the expected behaviour would be that when that CTL checkbox is not
> checked then all CTL text in a document should be treated as western for the
> purposes of applying styles.

Also let me point out that is't not something that concerns UX team only.

The proposed "expected behavoiur" will break something that has the most importance: the consistency of *rendering* of documents between systems, let alone behaviour.

Let's look at a typical situation where such a need might arise: a multilanguage user (A) using a Western-locale box to write CTL/CJK texts, and exchanging the documents with some user (B) who have CTL/CJK enabled on their systems. (For those Western-language speakers who don't type CTL/CJK, the proposal doesn't matter.)

So, when user (A) receives a document from user (B), what should the text look like? The user (B) has CTL/CJK enabled on system; so naturally, the styles are created with normal assignment of fonts to relevant writing systems parts of the styles. The styles are applied to the text; and so the text looks as intended. But user (A) has CTL/CJK disabled for our problem; so what should user (A) see? If the "expected behaviour" implemented, then only the Western-language part of the styles should be in effect - and so, everything that was configured by user (B) for CTL/CJK needs to be ignored. The look of text would change inevitably.

One could start invent some smart tricks (like treating existing styles with different settings one way, and creating new styles another way) - but that would not only introduce unneeded complexity to the code, but also make inconsistent behaviour between styles in a single document, which would confuse normal user even more. All this boils down to the need to display CTL/CJK (it's all that needed!) - so even simple "always show CTL/CJK" is a better solution. Introducing collapsing groups inside our dialogs (like in sidebar) used for such settings would improve usability more. But please *don't* implement the proposal from comment 4.
Comment 8 Toby Anderson 2018-06-19 04:28:06 UTC
I agree with Mike that my suggestion at the end of my previous comment is a bad idea. It wasn't well thought through.

I'm now willing to concede that this isn't technically a bug but instead is a usability issue, but I would say that it is still very extreme. I would expect there be countless users across Asia who do all their document formatting with direct formatting, or else switch to a pirated copy of Microsoft office, simply because "styles in LibreOffice just don't work"

You say this is in the documentation, but I wasn't able to find it there. Also I googled "LibreOffice styling bengali text" and none of the results on the first page gave me the answer to this.

I really think it's very important to fix this issue. Here's another suggestion: When entering CTL text for the first time in a document when the CTL options are hidden, or when attempting to apply a style to CTL text when those options are hidden, a dialog should appear saying something like "You are using text with complex layout in this document, but the CTL options are hidden. It is recommended that you make these options visible because complex layout text is treated differently for styling than western text. Would you like to make these options visible now? -> OK / Cancel / never show this again"
Comment 9 Toby Anderson 2018-06-19 04:30:04 UTC
Or maybe the CTL options should just automatically become visible when a document with CTL text is open. That's probably better.
Comment 10 Heiko Tietze 2018-06-19 07:43:22 UTC
(In reply to Mike Kaganski from comment #6)
> "Personally I feel that those controls would better be removed totally"

Me too. We could put CTL and CJK for example into tabs so that Western is the first option and the less often used don't clutter the UI. If needed the tab being active by default could get selected based on the locale.
Comment 11 Heiko Tietze 2018-06-21 14:48:18 UTC
Created attachment 143002 [details]
Mockup for comment 10

Tomaz, Caolan what do you think?
Basically I removed "For the current document only" and rearranged the controls.
Comment 12 Heiko Tietze 2018-06-28 08:26:04 UTC
Tomaz pointed out in the design meeting, that the character dialog depends on this setting too. If only one is checked we have list boxes for font name, weight, and size, while it's just a dropdown if two or more languages are active. And users who work on multi-language documents need all so "hiding" at tabs is not the best. Probably we can detect the lang of the current selection and show the respective tab?
Comment 13 Thomas Lendo 2018-06-30 20:26:56 UTC
I don't like the idea of showing all non-western options in all dialogs also in Western UI/document languages. It increases the mass of options for users they don't need it and may be confused.

Also removing the options controls and only showing these controls when clicking in complex text isn't perfect as I sometimes need to define CJK/RTL character options in some styles regardless if there is Chinese text in the current document.

If the user is using LibreOffice with western UI or has deactivated the visibility of CJK/RTL controls and is writing or clicking in such text, then an infobar prompt can be shown, informing the user that more text controls can be activated and pushing him/her to the options dialog and to the help page. A modal dialog shouldn't be used as users don't read it and close it without reading/understanding.
Comment 14 V Stuart Foote 2018-07-01 04:29:03 UTC
All controls should be visible and the UI simplified for consistency of working in all scripts.

Heiko's mockup comment 10 comment 11 would be reasonable UI to do so, but perhaps some logic that the "primary" active tab (Western, CJK, CTL) be drawn from the user locale.
Comment 15 Mike Kaganski 2018-07-01 04:35:16 UTC
A tangential question: why don't we use expanding sections (with "+" beside heading, like in sidebar) in dialogs?
Comment 16 V Stuart Foote 2018-07-01 05:02:59 UTC
(In reply to Mike Kaganski from comment #15)
> A tangential question: why don't we use expanding sections (with "+" beside
> heading, like in sidebar) in dialogs?

Well think we do on most dialogs, e.g. Tools -> Options, or Expert Configuration.

Specific glyph depends on DE where the slection choices are "+ or -" pairs, and some as "⏵or ⏷" pairs.

Heiko's RDE shows the selection in "⏵or ⏷" pairs.
Comment 17 Mike Kaganski 2018-07-01 05:28:37 UTC
Created attachment 143244 [details]
A screenshot of a sidebar with sections

(In reply to V Stuart Foote from comment #16)

Sigh... I wrote: *sections*, *like in sidebar*...
Comment 18 V Stuart Foote 2018-07-01 05:50:48 UTC
(In reply to Mike Kaganski from comment #17)
> Created attachment 143244 [details]
> A screenshot of a sidebar with sections
> 
> (In reply to V Stuart Foote from comment #16)
> 
> Sigh... I wrote: *sections*, *like in sidebar*...

Sorry Mike, I'm just dense. 

Yes we expand/collapse the "Content panels" held in the Sidebar Deck. There are a few collapsible "sections" in the dialogs, but "Tab" pages are simply a more functional GUI in more instances--here it could be either.
Comment 19 Khaled Hosny 2018-07-01 11:53:04 UTC
We should just remove the option that hides the CTL and CJK controls, and I have always argued for that. i don’t buy the argument that this complicates things for users who don’t use CTL or CJK scripts, besides being wester-centeric (since it assumes that anything not western is a second class citizen and needs permission to be displayed), it just also hides bad UX under the rug instead of actually addressing it.
Comment 20 Mike Kaganski 2018-07-01 12:25:44 UTC
(In reply to Khaled Hosny from comment #19)

+1
Comment 21 Heiko Tietze 2018-07-02 06:43:20 UTC
Created attachment 143263 [details]
Mockup for comment 12

(In reply to Heiko Tietze from comment #12)
> Tomaz pointed out in the design meeting, that the character dialog depends
> on this setting too.

Could be expandable sections too, of course, but then you need a scrollarea. Issue here is that users likely switch between Western and Asian quite often. Not too funny with the tabs (neither any other control) so we have to add code that switches automatically to the right tab depending on the current selection.
Comment 22 Tomaz Vajngerl 2018-07-02 07:46:43 UTC
we can't use scroll bars however...
Comment 23 Khaled Hosny 2018-07-02 10:55:22 UTC
Hiding CTL or CJK font setting by default and having to expand them or switch to a different tab is bad than status quo. Currently if I enable CTL and CJK controls then all my font settings are visible in one place and I can switch between them with little effort, which is not the case with the suggestion here.

I suggest simply dropping the setting and keeping things as they are now. This is the default for new installations for some time now already and I don’t see people screaming bloody murder about the extra controls.

On the long run, we should do away with the idea of Western, CTL and CJK font settings and do like, IIRC, what Microsoft Office is doing now and have a single font chooser that sets the fonts for all the three to the same font, the idea that there are only three categories of scripts is outdated and absured anyway, why I would want to set the same font for Devanagari, Thai and Arabic; there is absolutely nothing in common between the three.
Comment 24 Franklin Weng 2018-07-03 00:59:43 UTC
(In reply to Khaled Hosny from comment #23)
> 
> I suggest simply dropping the setting and keeping things as they are now.
> This is the default for new installations for some time now already and I
> don’t see people screaming bloody murder about the extra controls.

Nobody screaming doesn't mean they're fine with the options.  They may just don't know where to shout.  But I agree with your suggestion here.

> 
> On the long run, we should do away with the idea of Western, CTL and CJK
> font settings and do like, IIRC, what Microsoft Office is doing now and have
> a single font chooser that sets the fonts for all the three to the same
> font, the idea that there are only three categories of scripts is outdated
> and absured anyway, why I would want to set the same font for Devanagari,
> Thai and Arabic; there is absolutely nothing in common between the three.

Agree.  Don't make every language use the same font.  Sometimes it's not because, for example, Chinese and alphanumeric have nothing in common.  For me, the (half sized) alphanumeric characters in some Chinese fonts look so terrible!
Comment 25 Heiko Tietze 2018-07-04 19:36:05 UTC
We discussed the topic in the design meeting. The recommendation is to remove language dependent font families from tools > options at language as well as the modules. The default setting should be defined in officecfg/registry/data/org/openoffice/VCL.xcu or i18npool/source/localedata/data/* (l10n may be interested in supporting the default definition) with the usual way to override per styles: default (or below in the hierarchy). The same workflow should work for paragraphs with a different language.
Dev should consider side-effects such as search for kashida, which was only working when CTL was enabled (bug 116242).

From the minutes:

   + e.g. in tabs with Tools > Options > Language Settings > Default on top
   + remove “For the current document only”
   + drawback: displaces language settings (locale = english) and font settings (default = english)
   + how to deal with the character dialog? (Tomaz)
     + consequently we have to show all types at every time (Heiko)
     + could do that similar to fill styles (color, gradient, pattern...)
     + all types are needed in parallel so switching is bad (Tomaz)

   + flatten out "languages" needed/wanted (Stuart)
   + how to assign a default font to headings/text depending on locales
   + maybe add it to https://opengrok.libreoffice.org/xref/core/officecfg/registry/data/org/openoffice/VCL.xcu 
     or in files below https://opengrok.libreoffice.org/xref/core/i18npool/source/localedata/data/
     + see also tdf#112879
   + take defaults from this/these files (remove tools > options > language settings > languages: default languages)
   => remove also _all_ tools > options > writer > basic fonts (*) and let the user set-up defaults per styles: default
   + in multi-language documents/workflows the user is supposed to create paragraph styles with specific languages that take their default from the (non-configurable) defaults and may customize in the paragraph style dialog
   + when removing cjk/ctl dev should take care of spaghetti code similar to  search/kashida sensitive depends in the past on tools > options > languages on/off ( https://bugs.documentfoundation.org/show_bug.cgi?id=116242 )
Comment 26 Heiko Tietze 2018-07-05 08:17:30 UTC
(In reply to Heiko Tietze from comment #25)
> ...with the usual way to override per styles: default

That requires also to store Default independently from the document. When changing the default today it also affects tools > options .. > font but only for the current document. While changing tools > options ... > font is permanent. 

To rework the Default style has also a positive bearing on the understanding of styles and should be accompanied by a visual separation (e.g. per extra entry in the filter dropdown).
Comment 27 Eike Rathke 2018-07-09 11:58:41 UTC
(In reply to Heiko Tietze from comment #25)
> We discussed the topic in the design meeting. The recommendation is to
> remove language dependent font families from tools > options at language as
> well as the modules. The default setting should be defined in
> officecfg/registry/data/org/openoffice/VCL.xcu or
> i18npool/source/localedata/data/*
i18npool locale data doesn't make sense, we already have the language dependent fonts in VCL.xcu

However, in VCL.xcu we have ordered lists of fonts, of which an available one will be chosen at run time and then presented in Writer's Basic Fonts under Tools-Options. The font matching may also select a font of a family if the exact font name isn't found. My guess is you can't squeeze that functionality into some default style. Also I don't understand why you want to remove those settings.

> (l10n may be interested in supporting the
> default definition)
I don't grok that at all.