Bug 116242 - Diacritic-sensitive searches always linked to CTL language setting & replace fails with no CTL default
Summary: Diacritic-sensitive searches always linked to CTL language setting & replace ...
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
6.0.0.0.beta1
Hardware: All All
: medium normal
Assignee: Eike Rathke
URL:
Whiteboard: target:6.1.0 target:6.0.3 target:6.2.0
Keywords:
: 115829 116257 116285 116590 (view as bug list)
Depends on:
Blocks: RTL-CTL Find-Search
  Show dependency treegraph
 
Reported: 2018-03-06 16:12 UTC by Szasz-Fabian Jozsef
Modified: 2022-09-14 16:10 UTC (History)
8 users (show)

See Also:
Crash report or crash signature:


Attachments
test document to search for \u0163 ţ (8.62 KB, application/vnd.oasis.opendocument.text)
2018-03-06 20:09 UTC, V Stuart Foote
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Szasz-Fabian Jozsef 2018-03-06 16:12:37 UTC
There is a major bug in the Find & Replace:

Search: ţ (t with cedilla)

Replace: ț (t with comma)

Will replace all t (simple ascii t) characters to ț (t with comma below)

Similarly, all accented characters will match their non-accented simple versions: Searching ăáâ matches a, searching é matches e, searching sau matches său, etc.

This bug renders the Find & Replace dialog unusable... :-(

The [x] Match case checkbox has no effect on this bug.
Comment 1 V Stuart Foote 2018-03-06 20:09:02 UTC
Created attachment 140399 [details]
test document to search for \u0163 ţ

Why must we enable a default Complex text layout (CTL) on Tools -> Options -> Language Settings -> Languages to expose the Diacritic-sensitive check box on the Find & Replace dialog?  

Without it set, Find & Replace dialog search fails by glyph and for specific Unicode codepoints using the ICU Regular expression libs.

Open the attached document (no language set for paragraphs).  With no CTL default set active by the check box, the search for the "LATIN SMALL LETTER T WITH CEDILLA" by glyph, or by its \u0163 codepoint will fail.

Enable a CTL default language, check the Diacritic-sensitive box, and retest. Both a search by glyph, and a Regular expression search by codepoint work.

Seems like the diacritic-sensitive should be globally available, but may be wrapped up in the CTL handling.

Work around for functional search is to have a Complex text layout default language check box enabled.
Comment 2 Eike Rathke 2018-03-06 20:57:31 UTC
Seems searching for diacritic equivalents was implemented specifically for CTL/Arabic, see bug 52204, but the option is always enabled instead of disabled by default.
Also related is bug 91764.
Comment 3 Eike Rathke 2018-03-07 16:57:18 UTC
The actual problem seems to have been introduced with the change for bug 111846, that negated some logic but only part of it..
Investigating.
Comment 4 Commit Notification 2018-03-07 19:15:22 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=d99ae18b68d3889c999b59485006f85f7faa201c

Resolves: tdf#116242 handle negated diacritics correctly and show checkbox

It will be available in 6.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 5 Eike Rathke 2018-03-07 19:17:41 UTC
Unfortunately the dialog setting is stored in the configuration, so an update will still present the user with an unchecked Diacritics-sensitive box s/he has to enable once.
Comment 6 Eike Rathke 2018-03-07 19:21:37 UTC
Pending review https://gerrit.libreoffice.org/50914 for 6-0
Comment 7 V Stuart Foote 2018-03-08 04:58:22 UTC
(In reply to Eike Rathke from comment #5)
> Unfortunately the dialog setting is stored in the configuration, so an
> update will still present the user with an unchecked Diacritics-sensitive
> box s/he has to enable once.

/a install on Windows 10 Pro en-US 64-bit with bootstrap.ini
UserInstallation=$ORIGIN/../Data/settings of
Version: 6.1.0.0.alpha0+ (x64)
Build ID: 7a683c31e090e5a81debadcef025df9cd61c75f0
CPU threads: 8; OS: Windows 10.0; UI render: GL; 
TinderBox: Win-x86_64@42, Branch:master, Time: 2018-03-08_01:44:18
Locale: en-US (en_US); Calc: CL

Looks better. With this new default profile the Diacritic-sensitive checkbox is restored to the Find & Replace dialog, but is unchecked.

Checking the box enabled restores search by glyph of the OP and test doc. And also for regular expression searches.

But, shouldn't it be checked enabled by default to ensure a search by glyph will behave as expected.  Also for search with Regular expressions checkbox to find by ICU regex, \uHHHH syntax specifically, the transliteration must not occur--and would think the Diacritic-sensitive box needs to be enabled by default because of that as well.
Comment 8 V Stuart Foote 2018-03-08 09:02:00 UTC
*** Bug 116285 has been marked as a duplicate of this bug. ***
Comment 9 Eike Rathke 2018-03-08 12:22:02 UTC
(In reply to V Stuart Foote from comment #7)
> But, shouldn't it be checked enabled by default to ensure a search by glyph
> will behave as expected.
Geez, the ignore state is even the default config. I'll do a follow-up change.
Comment 10 Commit Notification 2018-03-08 12:34:14 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=a8959a26e18237c458a853f2ca7524e875d4c1b6

Change IsIgnoreDiacritics_CTL default to false, tdf#116242 follow-up

It will be available in 6.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 11 Telesto 2018-03-08 14:35:29 UTC
*** Bug 115829 has been marked as a duplicate of this bug. ***
Comment 12 Aron Budea 2018-03-10 06:52:43 UTC
*** Bug 116257 has been marked as a duplicate of this bug. ***
Comment 13 Jacques Guilleron 2018-03-10 10:28:46 UTC
Hi,

Works as expected with
LO 6.1.0.0.alpha0+
Build ID: 0d1479cbd442c3ec3bdd20333c341ea24dcca21e
CPU threads: 2; OS: Windows 6.1; UI render: default; 
TinderBox: Win-x86@42, Branch:master, Time: 2018-03-10_01:47:26
Locale: ro-RO (fr_FR); Calc: CL
Thank you Eike.
Comment 14 Commit Notification 2018-03-19 07:49:32 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-6-0":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=31f3e18d52c0cc124cd7a789a8f1fbc7c2534701&h=libreoffice-6-0

Resolves: tdf#116242 handle negated diacritics correctly and show checkbox

It will be available in 6.0.4.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 15 V Stuart Foote 2018-03-23 19:53:33 UTC
*** Bug 116590 has been marked as a duplicate of this bug. ***
Comment 16 Timur 2018-03-27 13:26:24 UTC
Duplicate bug was: "Search and replace is slow in 5.1 compared to LibO 5.0".
So, is backport to 5.4 possible here?
Comment 17 Commit Notification 2018-03-29 10:55:47 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-6-0-3":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=dbe02ba3775832ff35d512ad53b1e70028fb2659&h=libreoffice-6-0-3

Resolves: tdf#116242 handle negated diacritics correctly and show checkbox

It will be available in 6.0.3.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 18 Commit Notification 2018-10-11 16:30:47 UTC
Zdeněk Crhonek committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=80d1ac7d55c22de20edeec30f8f56e09f8455f8d

uitest findReplace; tdf#116242; tdf#98417; tdf#39022

It will be available in 6.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.