Bug 139196 - Export macros saves the macros in ANSI format, so special characters replaced with '?'
Summary: Export macros saves the macros in ANSI format, so special characters replaced...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: BASIC (show other bugs)
Version:
(earliest affected)
7.0.4.2 release
Hardware: x86-64 (AMD64) Windows (All)
: medium normal
Assignee: Andreas Heinisch
URL:
Whiteboard: target:7.2.0 target:7.3.0
Keywords:
: 136893 (view as bug list)
Depends on:
Blocks:
 
Reported: 2020-12-23 19:59 UTC by Zoltán Hegedüs
Modified: 2021-09-01 12:09 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
Example macros what consists special characters, what are not in ANSI character set. (6.52 KB, application/octet-stream)
2020-12-23 20:02 UTC, Zoltán Hegedüs
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Zoltán Hegedüs 2020-12-23 19:59:09 UTC
Editing macros, File menu: Export Basic: the created .bas file is in ANSI character set, so special characters are repleced to '?'. This causes data loss. There is no error message.
Comment 1 Zoltán Hegedüs 2020-12-23 20:02:29 UTC
Created attachment 168457 [details]
Example macros what consists special characters, what are not in ANSI character set.
Comment 2 Zoltán Hegedüs 2020-12-23 20:03:29 UTC
Cause of severity "major": this causes data loss, see the example attachment.
Comment 3 Urmas 2020-12-30 12:48:12 UTC
Confirmed in master.
Comment 4 Zoltán Hegedüs 2020-12-30 19:31:12 UTC
This can cause data loss, so this is not a minor problem. But I can not set the severity, so please anybody set to major.
Comment 5 Andreas Heinisch 2021-05-22 13:35:42 UTC
This could be easily solved by adding just the utf8 character encoding to the out/input stream. I am only concerned about the fact that exported macros using the ANSI format cannot be imported without any problems. Is there a way in LO to detect if a file is encoded with utf8?
Comment 6 Mike Kaganski 2021-05-22 17:09:17 UTC
(In reply to Andreas Heinisch from comment #5)

I'd just use UTF-8 with BOM.
Comment 7 Mike Kaganski 2021-05-22 17:12:58 UTC
... or use ucsdet_detect, as in https://gerrit.libreoffice.org/c/core/+/102884. But then I'd use a reasonably large detection chunk, maybe even the whole file.
Comment 8 Andreas Heinisch 2021-05-22 19:17:25 UTC
Proposed patch: https://gerrit.libreoffice.org/c/core/+/116004
Comment 9 Commit Notification 2021-05-25 20:21:07 UTC
Andreas Heinisch committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/178adcd8459af63ddb48927207baa5b4efbfda12

tdf#139196 - Import/export macros using utf-8 including BOM

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 10 Andreas Heinisch 2021-05-25 20:26:45 UTC
*** Bug 136893 has been marked as a duplicate of this bug. ***
Comment 11 Commit Notification 2021-05-27 07:37:02 UTC
Andreas Heinisch committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/46e924ac80a4abd7f48bd2df34ea4c38738fd745

tdf#139196 - Import/export macros using utf-8 including BOM

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Hans-Werner 2021-05-28 14:40:16 UTC
Tested with:

Windows 10 Home

Test_Save_vs_ExportBasic_MAKRO.bas
https://bugs.documentfoundation.org/show_bug.cgi?id=136893

LibreOffice
Version: 7.2.0.0.alpha1+ (x64) / LibreOffice Community
Build ID: 187136265d26c014e842550c2f1fc5997736e4fa
CPU threads: 4; OS: Windows 10.0 Build 19042; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: CL

It's working pretty fine - many thanks :-))
Comment 13 Commit Notification 2021-09-01 12:09:15 UTC
Andreas Heinisch committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/c356e5e0530e609a2ae9e8b11fb3c4ed43f966d4

tdf#139196 - Remove local BOM logic

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.