Bug 114000 - Regression - selected character set not honoured when importing from Lotus .123 format
Summary: Regression - selected character set not honoured when importing from Lotus .1...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: filters and storage (show other bugs)
Version:
(earliest affected)
5.4.2.2 release
Hardware: x86-64 (AMD64) Windows (All)
: medium normal
Assignee: David Tardon
URL:
Whiteboard: libwps target:6.1.0 target:6.0.0.1
Keywords:
Depends on:
Blocks:
 
Reported: 2017-11-23 00:30 UTC by Tomáš Hajný
Modified: 2017-12-07 15:06 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
Example file showing the problem as requested (5.62 KB, application/vnd.lotus-1-2-3)
2017-11-23 22:21 UTC, Tomáš Hajný
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Tomáš Hajný 2017-11-23 00:30:22 UTC
Description:
Accented characters are displayed incorrectly when opening files stored in Lotus .123 format regardless from selecting the correct character set. The chosen character set seems to be ignored. This is a regression - it used to work correctly in older versions (I believe that this was the case in 4.2.2, certainly in even older versions). The bug does not appear in the last Apache OpenOffice release (i.e. the same file with the same character set selected when opening is displayed correctly there).

Steps to Reproduce:
1.Open a file in Lotus .123 format stored in IBM codepage 852 (DOS / OS/2) containing accented characters (e.g. in Czech).

2.Select "Eastern Europe (DOS/OS/2-852)"

3.See the displayed accented characters - they are incorrect

Actual Results:  
E.g. "ß" displayed instead of "ž", "°" instead of "í", "Ï" instead of "ý", etc. Selected character set doesn't seem to have any impact - characters are displayed the same way if selecting the correct character set (IBM CP 852 - see above), or if selecting "Eastern Europe (Windows-1250/WinLatin 2)" (which is not correct, because it contains the same characters on different positions / with different ASCII codes).

Expected Results:
Correctly displayed characters (like in older LibreOffice or Apache OpenOffice versions).


Reproducible: Always


User Profile Reset: No


OpenGL enabled: Yes

Additional Info:
I tested with both 5.4.2.2 and 5.4.3.2. The following corresponds to the latter:

Verze: 5.4.3.2 (x64)
ID sestavení: 92a7159f7e4af62137622921e809f8546db437e5
Vlákna CPU: 4; OS: Windows 6.1; Vykreslování UI: výchozí; 
Národní prostředí: cs-CZ (cs_CZ); Calc: group


User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:49.0) Gecko/20100101 Firefox/49.0 SeaMonkey/2.46 Lightning/5.1b1
Comment 1 Aron Budea 2017-11-23 01:12:53 UTC
Please upload a sample file.
Comment 2 Tomáš Hajný 2017-11-23 22:21:13 UTC
Created attachment 137951 [details]
Example file showing the problem as requested
Comment 3 Tomáš Hajný 2017-11-23 22:22:21 UTC
Attachment provided as requested.
Comment 4 osnola 2017-11-24 11:42:22 UTC
Hello,
oops, this is my fault(*), I have committed a patch to correct this problem in libwps https://sourceforge.net/p/libwps/code/ci/2a3465442d1305c7fec55e986c9793a86e571baa/.

However, it is still possible to use the previous filter by choosing by hand the
 filter: "Lotus 1-2-3: (*.wk1, *.wk3, *.123)"

(*) I assumed faulty that this file was a mac's file :-~
Comment 5 Tomáš Hajný 2017-11-24 11:52:07 UTC
Please note that I didn't select a particular filter or its version (the previous comment by "osnola" kind of suggests that this might be the case), the filter was selected automatically when launching LibreOffice by double-clicking the .123 file.

BTW, it's also interesting that the character set conversion dialogue (part of the filter) isn't translated/localized to Czech (while all other dialogues are localized) - this is just a cosmetic thing, though, not really important for me.
Comment 6 David Tardon 2017-11-28 10:54:44 UTC
Okay, since this just missed libwps release, let's put it into LibreOffice as an extra patch.
Comment 7 Commit Notification 2017-11-28 10:57:04 UTC
David Tardon committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=54ec0aec1308bbbb8c9ab36fc76fe993bb23e5fb

tdf#114000 always use user-selected charset

It will be available in 6.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 8 David Tardon 2017-11-28 12:26:16 UTC
(In reply to Tomáš Hajný from comment #5)
> BTW, it's also interesting that the character set conversion dialogue (part
> of the filter) isn't translated/localized to Czech (while all other
> dialogues are localized) - this is just a cosmetic thing, though, not really
> important for me.

The writerperfect module, which contains this dialog, was not included for translation in 5.x by a mistake.
Comment 9 Commit Notification 2017-11-30 11:37:50 UTC
David Tardon committed a patch related to this issue.
It has been pushed to "libreoffice-6-0":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=804f9278451b50daae82618cacbeb8931e773caa&h=libreoffice-6-0

tdf#114000 always use user-selected charset

It will be available in 6.0.0.1.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 10 Tomáš Hajný 2017-12-07 15:06:03 UTC
I can confirm that the daily build libo-60-64~2017-12-05_02.52.19_LibreOfficeDev_6.0.0.0.beta1_Win_x64.msi behaves correctly with respect to this bug report - thanks for fixing it!