Bug 108795 - Cannot enter colon or semicolon in formula bar (French version)
Summary: Cannot enter colon or semicolon in formula bar (French version)
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
5.2.4.2 release
Hardware: All All
: medium normal
Assignee: Eike Rathke
QA Contact:
URL:
Whiteboard: target:6.0.0 target:5.4.2
Keywords:
Depends on:
Blocks:
 
Reported: 2017-06-26 18:57 UTC by Miguel
Modified: 2017-09-21 18:23 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
screenshot showing the bug (11.30 KB, image/png)
2017-06-26 18:57 UTC, Miguel
Details
Result in CSV (UTF8, separator=comma, limiter="), see that the inserted space is a nbsp, 2 bytes (C2 A0) in UTF-8 (163 bytes, text/csv)
2017-06-27 17:34 UTC, Miguel
Details
Example in ODS (LibreOffice Calc) format (9.32 KB, application/vnd.oasis.opendocument.spreadsheet)
2017-07-17 12:42 UTC, Miguel
Details
decimal "." for the french people (201.45 KB, application/vnd.oasis.opendocument.spreadsheet)
2017-08-21 15:07 UTC, Xavier Van Wijmeersch
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Miguel 2017-06-26 18:57:20 UTC
Created attachment 134298 [details]
screenshot showing the bug

You may know that in French typography, a non-breakable space is normally inserted before any colon or semicolon sign. So does LibreOffice Writer for French language, with no big trouble.

In LibreOffice Calc, French version, the same happens even in the cells configured in English language. OK, this is not the problem.

Let me click on a cell and type the text "List: one; two; three". Some spaces are correctly inserted and the cell now contains the text "List : one ; two ; three". If I don't want these spaces, I can now delete them: no problem.

Now, let me click on the formula bar and, again, type the text "List: one; two; three". The formula bar will show exactly this, but the cell will show "List  one  two  three;;:". Which means: spaces were inserted before the colons and semicolons, and then the colons and semicolons were transferred to the end of the string, in reverse order. This is bad!

OK then, I type Enter, go back to the same cell, and try to fix the problem in the formula bar. I remove the extra spaces, remove the colons and semicolons at the end, and try to insert colons and semicolons at the correct place, so that the formula bar shows "List: one; two; three". But now, the cell contains the text "List : on :  t :o  three", which is even worse and very strange.

I think that this bug exists since quite a long time, but I never took the time to report it before since there is a workaround (typing into the cell).

Since it is related to a French typographic rule, I suspect that it exists just in some languages, maybe just in the French version.

I am attaching a montage of screenshots
Comment 1 Eike Rathke 2017-06-27 17:07:54 UTC
I can't reproduce. I ran 5.3.0 and 5.3.4 and >=5.4.0.beta2 with
LC_ALL=fr_FR soffice
and in Calc typing "List: one; two; three" correctly gets auto-corrected to "List : one ; two ; three", both within a cell and the input line.
Comment 2 Miguel 2017-06-27 17:32:18 UTC
I have
LANGUAGE=fr_FR
LANG,LC_ADDRESS,LC_IDENTIFICATION,LC_MEASUREMENT,LC_MONETARY,LC_NAME,LC_NUMERIC,LC_PAPER,LC_TELEPHONE,LC_TIME=fr_FR.UTF-8

Could it be related to the UTF-8 encoding? You tried with fr_FR, which I think is an 8-bit encoding. Non-breaking space is 2 bytes in UTF-8, 1 byte in 8-bit encoding.

I tried and saved into CSV (UTF-8), see in second file. The inserted space actually is a non-braking space (UTF-8: C2 A0)

$ cat colon.csv 
Type in cell =>,List : one ; two ; three
Type in formula bar =>,List  one  two  three;;:
Type and correct in formula bar =>,"List   on ,  t wo  th;;:"

$ hexdump -C colon.csv 
00000000  54 79 70 65 20 69 6e 20  63 65 6c 6c 20 3d 3e 2c  |Type in cell =>,|
00000010  4c 69 73 74 c2 a0 3a 20  6f 6e 65 c2 a0 3b 20 74  |List..: one..; t|
00000020  77 6f c2 a0 3b 20 74 68  72 65 65 0a 54 79 70 65  |wo..; three.Type|
00000030  20 69 6e 20 66 6f 72 6d  75 6c 61 20 62 61 72 20  | in formula bar |
00000040  3d 3e 2c 4c 69 73 74 c2  a0 20 6f 6e 65 c2 a0 20  |=>,List.. one.. |
00000050  74 77 6f c2 a0 20 74 68  72 65 65 3b 3b 3a 0a 54  |two.. three;;:.T|
00000060  79 70 65 20 61 6e 64 20  63 6f 72 72 65 63 74 20  |ype and correct |
00000070  69 6e 20 66 6f 72 6d 75  6c 61 20 62 61 72 20 3d  |in formula bar =|
00000080  3e 2c 22 4c 69 73 74 c2  a0 c2 a0 20 6f 6e c2 a0  |>,"List.... on..|
00000090  2c c2 a0 20 74 c2 a0 77  6f c2 a0 20 74 68 3b 3b  |,.. t..wo.. th;;|
000000a0  3a 22 0a                                          |:".|
000000a3
Comment 3 Miguel 2017-06-27 17:34:43 UTC
Created attachment 134317 [details]
Result in CSV (UTF8, separator=comma, limiter="), see that the inserted space is a nbsp, 2 bytes (C2 A0) in UTF-8
Comment 4 Eike Rathke 2017-06-27 19:23:22 UTC
(In reply to Miguel from comment #2)
> Could it be related to the UTF-8 encoding? You tried with fr_FR, which I
> think is an 8-bit encoding.
Doesn't matter, internally it's Unicode anyway. However, same good result for
LC_ALL=fr_FR.UTF-8 soffice
Comment 5 Miguel 2017-06-27 22:15:45 UTC
(In reply to Eike Rathke from comment #4)
Well then, I have no idea which is the difference.
Just in case: I have XUbuntu 16.04.2 LTS, 32-bit version.

One thing: "About LibreOffice" tells me: "Locale : en-GB (fr_FR.UTF-8)"

Same bad behavior in LibreOffice 5.2.4.2
Comment 6 GerardF 2017-06-28 07:11:57 UTC
Can you have a try with spellchecking disabled ?
Comment 7 Miguel 2017-06-28 08:32:28 UTC
(In reply to GerardF from comment #6)
> Can you have a try with spellchecking disabled ?

* When I disable Tools - Automatic Spell Checking (Shift+F7), same bad behavior except that the red zigzag underline disappears
    (Outils - Correction orthographique automatique (Maj+F7))

* When I disable Tools - AutoCorrect - Localized Options - "Add non-breaking space before specific punctuation marks in French text", then no space gets inserted, so obviously the bug disappears.
    (Outils - Options d'autocorrection - Options linguistiques - "Ajouter un espace insécable avant les signes de ponctuation spécifiques dans les textes français.")
Comment 8 Eike Rathke 2017-07-17 12:24:05 UTC
(In reply to Miguel from comment #5)
> One thing: "About LibreOffice" tells me: "Locale : en-GB (fr_FR.UTF-8)"
That indicates your system environmen is fr_FR.UTF-8 but the LibreOffice locale is explicitly set to en-GB (English UK). Check Tools -> Options -> Language Settings -> Languages, Locale setting.

Now that doesn't make much sense because then the "French spacing" isn't triggered, unless you have cells explicitly formatted to French character attribution. But even then I can't reproduce.
Comment 9 Miguel 2017-07-17 12:41:18 UTC
(In reply to Eike Rathke from comment #8)
> Now that doesn't make much sense because then the "French spacing" isn't
> triggered, unless you have cells explicitly formatted to French character
> attribution. But even then I can't reproduce.

I am attaching an example of the file in LibreOffice format, for you to check whether there is such an explicit attribution.

Maybe there is something specific on my system, but it is 100% reproducible.
Comment 10 Miguel 2017-07-17 12:42:06 UTC
Created attachment 134685 [details]
Example in ODS (LibreOffice Calc) format
Comment 11 Jean-Baptiste Faure 2017-08-19 20:33:04 UTC
Set version to 5.2.4.2 from comment #5

@Miguel:  did you try to reset your LibreOffice user profile (rename ~/.config/libreoffice/4)?

Set status to NEEDINFO, please set it back to UNCONFIRMED once requested
informations are provided.

Best regards. JBF
Comment 12 Miguel 2017-08-21 10:04:56 UTC
@Jean-Baptiste Faure: if I rename ~/.config/libreoffice/4 , it works properly, either typing into a cell or into the formula bar. Tested with 5.2.4.2 and with 5.4.0.3.

Maybe because I am using a "partially French" configuration? i.e. with French language but without French numbers (using dot instead of coma as the decimal limiter)
Comment 13 Jean-Baptiste Faure 2017-08-21 10:26:53 UTC
(In reply to Miguel from comment #12)
> @Jean-Baptiste Faure: if I rename ~/.config/libreoffice/4 , it works
> properly, either typing into a cell or into the formula bar. Tested with
> 5.2.4.2 and with 5.4.0.3.

Ok, thank you for the test; so closing as NotOurBug.
Feel free to reopen if you disagree.

> 
> Maybe because I am using a "partially French" configuration? i.e. with
> French language but without French numbers (using dot instead of coma as the
> decimal limiter)

If you want use dot as decimal separator, perhaps you should use a fr locale that is doing so: check fr-BE, fr-CH and fr-CA. Doing so you will not have contradictions between locale and auto-correction.

Best regards. JBF
Comment 14 Miguel 2017-08-21 11:37:49 UTC
I think that there is really a problem for French people wanting to use "." as the decimal separator:

fr-CA exhibits the same initial bug. Further, it is not suitable for French (date format)

fr-CH is not suitable for France (date format and thousands separator). However it is interesting because it behaves properly, with "." as decimal separator and correctly inserted non-breakable spaces before ":" and ";".

Other French locales (including fr-BE) are not a solution: they use "," as decimal separator.

There is a real problem, since I do exactly as recommended here:
https://wiki.documentfoundation.org/Faq/Calc/121/fr
(using "Armenian" locale, with "EUR Interlingua" currency)
and the original bug is still there.

I reopen the bug, since:
1) it exists with "French (Canada)"
2) it exists with settings documented in the wiki
3) the discrepancy between "typing into a cell" and "typing into the formula bar" should not exist, whichever the configuration
Comment 15 Buovjaga 2017-08-21 12:15:45 UTC
Miguel: it seems you are now talking about bug 46448. Correct?
Comment 16 Miguel 2017-08-21 13:35:52 UTC
Buovjaga: no, it is a separate bug. Even without allowing custom separators as requested in bug 46448 (which would be nice indeed), there should be no bug when using a standard setting such as "French (Canada)", or when using combinations of standard settings such as "user interface: French (France), local parameters: Armenian, default currency: EUR-Interlingual", and there should be the same behavior when typing into a cell or into the formula bar.
Comment 17 Xavier Van Wijmeersch 2017-08-21 15:05:42 UTC
Hi Miguel

I think a found a workaround using Switzerland french, uncheck "same as local", also using for UI the us language. But not sure it's needed to change the UI language.
One question what you use for the thousand separator, must it the same as the "." separator
Comment 18 Xavier Van Wijmeersch 2017-08-21 15:07:23 UTC
Created attachment 135702 [details]
decimal "." for the french people

have a look to these maybe it helps
Comment 19 Miguel 2017-08-21 15:33:26 UTC
Hi Xavier,

Thank you, but as for me, I am not requesting a workaround, I am reporting a bug and an inconsistent behavior. So far, I will stay with Armenian language, Euro currency, and the bug that I know.

The French-Switzerland configuration is not suitable for France because:
1) the date is dd.mm.yyyy in Switzerland (like in Germany), whereas it is dd/mm/yyyy in France
2) the thousands separator is 1'000'000 in Switzerland, 1 000 000 in France

Anyway, I believe that this question belongs to bug 46448, there is no need to discuss it here.
Comment 20 Eike Rathke 2017-08-21 16:03:07 UTC
(In reply to Miguel from comment #14)
> fr-CA exhibits the same initial bug.

fr-CA likely can not expose the same bug because in fr-CA the only character that gets a NBSP prepended is the ':' colon, ';' semicolon does not. Maybe you were observing it for "List: one; two; three" -> "List  one; two; three:"?

Still, I could not reproduce with
LANGUAGE=fr_FR LANG=fr_FR.UTF-8 LC_ALL=fr_FR.UTF-8
UI: Par défaut - Français (France)
Locale: Français (Canada)
Currency: EUR Français
All in master branch that is, maybe it is specific to such combination in 5.2.x

Setting the locale to Armenian *no* correction is done to ":;" during cell input. Which is correct.

Setting the locale to Français (France) again produces correct results.

Is your UI language "Par défaut - Français (France)", or is it set fixed to "Français (France)"?
Do you change the locale within a running session, and if so with or without the document open where it happens? Or is it the same locale all the time?
Does the failure happen for you in a new document, or existing documents, or both, or ...?
Are the cells where it happens formatted (character attribution) to a different language/locale than the default locale?
Is the number format locale different?
Comment 21 Miguel 2017-08-21 16:22:01 UTC
(In reply to Eike Rathke from comment #20)
> 
> fr-CA likely can not expose the same bug because in fr-CA the only character
> that gets a NBSP prepended is the ':' colon, ';' semicolon does not. Maybe
> you were observing it for "List: one; two; three" -> "List  one; two;
> three:"?

Well, on my system (5.4.0.3 on Linux Ubuntu 32 bit), when set to French (Canada), both with default CAD currency or with EUR-FR, a space is added both before ":" and before ";". But the bug exists only with ";" (and only from the formula bar), the behavior is fine with ":".
Comment 22 Jean-Baptiste Faure 2017-08-21 17:16:02 UTC
As you just said in bug 46448, you problem with comma as decimal separator is linked to CSV import. You don't need to change the correct decimal separator for French (France) to import text files with English numbers. All what you need is explained in the FAQ: https://wiki.documentfoundation.org/Faq/Calc/136/fr 

Best regards. JBF
Comment 23 Eike Rathke 2017-08-21 20:19:45 UTC
(In reply to Miguel from comment #21)
> Well, on my system (5.4.0.3 on Linux Ubuntu 32 bit), when set to French
> (Canada), both with default CAD currency or with EUR-FR
Currency is completely irrelevant. Important is the language/locale of the character/font attribution of the cell.

> a space is added
> both before ":" and before ";".
Not with fr-CA.
Comment 24 Eike Rathke 2017-08-21 20:25:42 UTC
AHA! And now I finally can reproduce.. if the character/font attribution has a French locale that does not match the default locale, weird things happen.
In your case the cell probably has a fr-FR attribute while the default locale is fr-CA.
Investigating.
Comment 25 Miguel 2017-08-22 09:30:35 UTC
Hi Eike,

I had the user interface in "Français (France)", but I see no change with "Default: Français (France)".

I did all my tests with an already open document (observing that the new configuration is applied immediately to it, except for the currency format which is just partially updated), but I just checked that it is the same when I open a new document, and when I close and relaunch LibreOffice.

Regarding "French (Canada)", what I get is:
* in the formula bar, a space is added before ":" but not before ";"
* however, more spaces are added in the cell each time I type a ";" into the formula bar (the bug appears). Example:
  1) I type "one: un; two: deux; three: trois." into the formula bar
  2) while I am typing, the formula bar shows "One : un; two : deux; three : trois.", as expected for French-Canada (also notice the capital "O" at the beginning)
  3) but at the same time, the cell contains "one : un  two : deux  three : trois.;;" (notice the bug with the semicolons and the small "o" at the beginning), and keeps this value after I type Enter, which also makes the formula bar take this value.

After selecting "French (Canada)", LibreOffice was restarted and a new document was created. In contrast to your hypothesis, according to the "Cell Format" option, the cell is in language "Par défaut - Français (Canada)" [Default - French (Canada)].
Comment 26 Eike Rathke 2017-08-22 12:00:18 UTC
(In reply to Miguel from comment #25)
> In contrast to your hypothesis, according to the "Cell
> Format" option, the cell is in language "Par défaut - Français (Canada)"
> [Default - French (Canada)].
Which language are you looking at? Number format? Or Font? Font is relevant.

However, with the setting I mentioned I could reproduce and I think I found the cause.
Comment 27 Miguel 2017-08-22 13:10:05 UTC
(In reply to Eike Rathke from comment #26)
> Which language are you looking at? Number format? Or Font? Font is relevant.
> 
> However, with the setting I mentioned I could reproduce and I think I found
> the cause.

You are write, I was looking at the language in "Number", not in "Font".
Comment 28 Commit Notification 2017-08-23 22:31:52 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=06d14411a447cd798d1f0678a3c5e06f5278a1cb

Related: tdf#108795 a shared SvxAutoCorrect instance can not be stateful

It will be available in 6.0.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 29 Commit Notification 2017-08-23 22:31:59 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=311be7d70146e4963e4dfd1f39d6c71d45d241ca

Related: tdf#108795 never use an unresolved LANGUAGE_SYSTEM in SvxAutoCorrect

It will be available in 6.0.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 30 Commit Notification 2017-08-24 14:38:53 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=d31a8263d564ebd7886f6debdf36cedd031915ea

Resolves: tdf#108795 merge table EditEngine language attributes to input line

It will be available in 6.0.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 31 Eike Rathke 2017-08-24 15:41:28 UTC
Pending review https://gerrit.libreoffice.org/41525 for 5-4
Comment 32 Commit Notification 2017-09-05 14:25:02 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-5-4":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=c5fcab1ec72be1cf2868215ee71d3cb2668498c1&h=libreoffice-5-4

Resolves: tdf#108795 merge table EditEngine language attributes to input line

It will be available in 5.4.2.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 33 Miguel 2017-09-21 18:23:18 UTC
I have just tried in LibreOffice 5.4.2.1.

1) in French, non-breakable spaces are correctly inserted before colons ":" and semicolons ";", in all cases => OK

2) in French (Canada), the behavior is exactly the same => NOT OK, since in French (Canada) there should be no space added before semicolons.

3) if I type "abc: def" in the formula bar, and before I press Enter:
 a) the cell shows "abc : def" => OK, non-breakable space correctly inserted
 b) the formula bar shows "Abc : def" => NOT OK, because inconsistent with cell content
 c) after pressing Enter, both show "abc : def" => OK
 d) if I type the same text into the cell instead of the formula bar, this behavior does not happen, both places show "abc : def" => OK

So it seems to me that this version has been fixed so that it will show the (mostly) correct behavior, but that the internal machinery still is not consistent: there should be no difference between typing a text into a cell and into the formula bar.