Bug 118681 - SpecialCharacter dialog hex input handling of SMP codepoints is broken
Summary: SpecialCharacter dialog hex input handling of SMP codepoints is broken
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
6.1.0.1 rc
Hardware: All All
: medium normal
Assignee: Caolán McNamara
URL:
Whiteboard: target:6.2.0 target:6.1.0.2
Keywords:
Depends on:
Blocks: Special-Character
  Show dependency treegraph
 
Reported: 2018-07-10 22:05 UTC by V Stuart Foote
Modified: 2018-07-20 04:03 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description V Stuart Foote 2018-07-10 22:05:41 UTC
On Windows 10 Ent 64-bit en-US builds of
Version: 6.1.0.1 (x64)
Build ID: 378e26bd4f22a135cef5fa17afd5d4171d8da21a
CPU threads: 8; OS: Windows 10.0; UI render: GL or default; 
Locale: en-US (en_US); Calc: CL

Version: 6.2.0.0.alpha0+ (x64)
Build ID: d27e4007c697f5eb78181181a0ddcd663ac265b2
CPU threads: 8; OS: Windows 10.0; UI render: GL or default; 
TinderBox: Win-x86_64@42, Branch:master, Time: 2018-07-10_00:34:24
Locale: en-US (en_US); Calc: CL

Special character dialog handling of SMP Hexadecimal codepoint entry is off...

Select "Noto Sans Symbols" font and try to search select a 1D15F (MUSICAL SYMBOL QUARTER NOTE) glyph, or select "EmojiOne Color" and try to search select a 1F320 (SHOOTING STAR) glyph.

Can't do it: 

- keyboard entry is not sequential, text cursor of input field jumps to start with each Hex pair. Makes it difficult to reliably enter the codepoint--and finding a glyph in the SMP (e.g. for Emoji) is a real chore

- also an SMP codepoint input into the Hex field will not advance the Unicode Subsets into the entries in the SMP plane. Neither will click selecting a glyph from the chart, the Subset and glyph preview remain empty.
Comment 1 Caolán McNamara 2018-07-16 15:03:47 UTC
I think there's a few different problems bundled in here, I'll try to address the first one anyway.
Comment 2 Commit Notification 2018-07-16 16:23:36 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=cf6c748f30fd630b6b7256e0c7ba1783892f9d1e

tdf#118681 make sure we're using the right font char map

It will be available in 6.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 3 V Stuart Foote 2018-07-16 19:00:27 UTC
@Caolán, Thanks! Reading, IIUC that is the first of several steps to assure the Unicode subset handling in the listbox.

Problem entering string for the HEX value, i.e. aCodeString in selectCharByCode() [1] on Windows does not occur with the 6.0 builds. But it is bad on the 6.1 build and for master/6.2--so is that from changes to the get_text & get_preferred_size for the HEX entry [2].

I recall an issue in writing out the SMP glyphs to the the old "edit buffer"
in tdf#97839 in 5.3 before it was removed, bcz the length of the char could be more than one utf16 codepoint, as fixed in [3]--something similar here?

Also, did you want this Assigned?

=-ref-=
[1] https://opengrok.libreoffice.org/xref/core/cui/source/dialogs/cuicharmap.cxx#1048
[2] https://gerrit.libreoffice.org/#/c/52084
[3] https://cgit.freedesktop.org/libreoffice/core/commit/?id=847cdd8efd0662d61d288a4d944edc30e864d145
Comment 4 Caolán McNamara 2018-07-16 20:32:03 UTC
re "Problem entering string for the HEX value, i.e. aCodeString in selectCharByCode() [1] on Windows does not occur with the 6.0 builds", what exactly is the problem ?
Comment 5 V Stuart Foote 2018-07-16 21:49:55 UTC
(In reply to Caolán McNamara from comment #4)
> re "Problem entering string for the HEX value, i.e. aCodeString in
> selectCharByCode() [1] on Windows does not occur with the 6.0 builds", what
> exactly is the problem ?

The text cursor on the Hexadecimal field is repositioned to start of field during entry, before input is complete.  

Simple STR try to enter 1F320 SHOOTING STAR from Miscellaneous Symbols and Pictographs Unicode section.

1. open Writer, change font to Symbola [1]
2. open Special character dialog
3. enter 1f3 in Hex search field
4. LATIN SMALL LETTER DZ shown in preview, and on chart
Note position of entry cursor -> start of field

5. continuing enter 2 -> 21F3 value, so not helpful
6. have to stop entry and position to end of field enter 2 -> 1F32
7. GREEK SMALL LETTER IOTA WITH PSILI AND VARIA
Note position of entry cursor -> now end of field

8. enter 0 -> 1F320 value "Missing Glyph" and no change to Subset 

But kind of expect that 8. will change when https://cgit.freedesktop.org/libreoffice/core/commit/?id=cf6c748f30fd630b6b7256e0c7ba1783892f9d1e rolls.

So the issue with field cursor is just while entering values in the Hexadecimal field. Entering the Decimal value also returns glyphs for codepoints, but does not reposition the cursor to the start of the field so typing can continue.

=-ref-=
[1] install from http://users.teilar.gr/~g1951d/ if not on system, bonus current release support Unicode 11.0

http://users.teilar.gr/~g1951d/Symbola.zip
Comment 6 Commit Notification 2018-07-17 09:02:25 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-6-1":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=a09d357daf65e285323ff77653a6de3058fc44da&h=libreoffice-6-1

tdf#118681 make sure we're using the right font char map

It will be available in 6.1.0.2.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 7 Caolán McNamara 2018-07-17 14:54:12 UTC
wrt 1f3 the glyph exists so it gets selected, when a glyph is selected the text in the hex textentry is set to its number. There's code to not reset the text if the text would be the same as it started as, but 1f3 is not the same as 1F3, so it gets changed and the input position lost. I'll change it to be a case insensitive test
Comment 8 V Stuart Foote 2018-07-17 15:58:07 UTC
(In reply to Caolán McNamara from comment #7)
> wrt 1f3 the glyph exists so it gets selected, when a glyph is selected the
> text in the hex textentry is set to its number. There's code to not reset
> the text if the text would be the same as it started as, but 1f3 is not the
> same as 1F3, so it gets changed and the input position lost. I'll change it
> to be a case insensitive test

That was it. I went back and tested on Windows builds--entering with the 1F3 holds position while entering 1f3 converts to uppercase and positions to start. Guess the behavior was intended to clean up the HEX values to always display in ASCII upper case.

Also, yesterdays patch resolved the "missing glyph" and Unicode subset listbox search result.

Version: 6.2.0.0.alpha0+
Build ID: d6bd9c273483b12f1bb2ae398afdba977e3ec336
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk2; 
TinderBox: Linux-rpm_deb-x86_64@70-TDF, Branch:master, Time: 2018-07-17_12:03:08
Locale: en-US (en_US.UTF-8); Calc: group threaded

Thanks!
Comment 9 Commit Notification 2018-07-17 16:24:07 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=19b3a353662c547b9e1a445c1bc71cf3bc107250

Related: tdf#118681 use a case insensitive compare for hex digits

It will be available in 6.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 10 V Stuart Foote 2018-07-18 16:07:30 UTC
Issue of comment 5 all good now...

Version: 6.2.0.0.alpha0+ (x64)
Build ID: daafe79c55cd53decbeac2367f298d79371dcf3d
CPU threads: 8; OS: Windows 10.0; UI render: GL; 
TinderBox: Win-x86_64@42, Branch:master, Time: 2018-07-18_02:59:18
Locale: en-US (en_US); Calc: CL
Comment 11 Commit Notification 2018-07-18 19:00:31 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-6-1":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=805ea77cf21ff96f8ffd091f94726840771ee658&h=libreoffice-6-1

Related: tdf#118681 use a case insensitive compare for hex digits

It will be available in 6.1.0.2.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 V Stuart Foote 2018-07-20 04:03:24 UTC
clean Unicode entry now 1f320 or any other codepoint from SMP.


Version: 6.2.0.0.alpha0+ (x64)
Build ID: b7e139fa21607f488465fd87333db757ad0c91a2
CPU threads: 8; OS: Windows 10.0; UI render: GL; 
TinderBox: Win-x86_64@42, Branch:master, Time: 2018-07-20_01:39:32
Locale: en-US (en_US); Calc: group threaded