Bug 57113 - Macros: Unicode vs. password protected user libraries
Summary: Macros: Unicode vs. password protected user libraries
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: BASIC (show other bugs)
Version:
(earliest affected)
3.6.3.2 release
Hardware: All All
: medium normal
Assignee: Mike Kaganski
URL:
Whiteboard: BSA target:6.5.0
Keywords:
: 58426 127763 (view as bug list)
Depends on:
Blocks: Macro-UNOAPI
  Show dependency treegraph
 
Reported: 2012-11-14 10:07 UTC by Karel Behounek
Modified: 2023-07-24 12:02 UTC (History)
7 users (show)

See Also:
Crash report or crash signature:


Attachments
Test files (LO Calc) to demonstrate the issue. (19.10 KB, application/zip)
2012-11-15 06:59 UTC, Karel Behounek
Details
Illustration of behaviour (for comment 5) (300.93 KB, image/jpeg)
2014-05-22 08:13 UTC, Karel Behounek
Details
screenshot showing where the "password protect" is (24.90 KB, image/png)
2014-08-21 12:21 UTC, Kevin Suo
Details
An ODS file with password-protected macro library (15.69 KB, application/vnd.oasis.opendocument.spreadsheet)
2014-08-21 12:23 UTC, Kevin Suo
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Karel Behounek 2012-11-14 10:07:09 UTC
Problem description: 

When the user library is password protected, there are issues with unicode? (cyrillic in my case) strings.

Steps to reproduce:

1. create user library

2. enter some sample code

Dim oDoc As Object
Dim oSheet As Object
Dim oCell As Object

oDoc = ThisComponent
oSheet = oDoc.Sheets(0)
oCellTest = oSheet.getCellRangeByName("A1")
oCellTest.String = "Ленинский проспект"

2. run the macro - works fine (displays Ленинский проспект string in the A1 cell)

3. password protect the module

4. run the macro again - does not work fine (displays ???????? ??????? string in the A1 cell)

Current behavior:

The cyrillic characters are replaced with question marks when the module is password protected.

Expected behavior:

The cyrillic characters display just fine, no matter whether the module is password protected or not.

Platform (if different from the browser): 
              
Browser: Opera/9.80 (Windows NT 5.1) Presto/2.12.388 Version/12.10
Comment 1 Karel Behounek 2012-11-15 06:59:06 UTC
Created attachment 70101 [details]
Test files (LO Calc) to demonstrate the issue.

password-test-protected.ods: password protected user library demonstrating the described issue
password-test-unprotected.ods: unprotected user library, no issues (included for comparison)
Comment 2 Karel Behounek 2012-11-15 08:29:51 UTC
The problem seems to not be platform-specific, I've tested it in Windows XP 32-bit (Czech), Windows 7 64-bit (English) and OpenSUSE (Czech) and the behaviour is identical everywhere.

The problem is not caused by the presence of the password protection itself; once the password is entered ("becomes known to LO"), all works fine even in protected document. But if the end-user doesn't know the password protecting the BASIC library (this is intended), he/she is stuck.
Comment 3 Urmas 2012-12-17 18:45:51 UTC
*** Bug 58426 has been marked as a duplicate of this bug. ***
Comment 4 Joel Madero 2014-05-21 16:22:49 UTC
Ubuntu 14.04 x64
LibreOffice 4.2.4.2 release

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Just tested this on 4.2.4.2 release and it seems to work fine.


Marking as:
RESOLVED
WORKSFORME

If you still have this problem on 4.2.4.2 release please mark the bug as UNCONFIRMED and let us know your platform. Thanks!
Comment 5 Karel Behounek 2014-05-22 08:10:53 UTC
Thanks for the feedback, unfortunately it does not work for me.

Xubuntu 14.04 x86 (Ubuntu 14.04 LTS + Xfce 4.10), Czech localization in case it matters
LibreOffice 4.2.4.2 release

I tried both original test file included in attachment 70101 [details] (comment 2 in this thread) and a new file created from the scratch (in 4.2.4.2) and the behaviour is identical (broken) in both cases. On Windows platform it is the same.

Steps to reproduce:

- download the attachment 70101 [details] above
- open the file password-test-protected.ods from within it
- click on the "Run Test" button
- text string "This should display in cyrillic: ????????? ????????." appears in the A1 cell, i. e. question marks instead of cyrillic characters

When I do the same with the file password-test-unprotected.ods, the string is displayed correctly after pressing the "Run Test" button. See attached pair of screenshots.

It is important to NOT unlock the library (by entering the password) before pressing the "Run Test" button.
Comment 6 Karel Behounek 2014-05-22 08:13:04 UTC
Created attachment 99565 [details]
Illustration of behaviour (for comment 5)
Comment 7 Joel Madero 2014-05-22 14:22:58 UTC
Please don't change the version - it reflects the oldest confirmed version not the latest tested on. Thanks!
Comment 8 Kevin Suo 2014-08-21 12:21:32 UTC
Created attachment 105034 [details]
screenshot showing where the "password protect" is

Reproduced with version 4.3.1.1, Win7.

Set to NEW.

Password protect = protect the "libary", not the document, see screenshot.
Comment 9 Kevin Suo 2014-08-21 12:23:10 UTC
Created attachment 105035 [details]
An ODS file with password-protected macro library
Comment 10 QA Administrators 2015-09-04 02:49:36 UTC Comment hidden (obsolete)
Comment 11 Karel Behounek 2015-09-04 07:27:01 UTC
The bug is still present, tested on both Kevin Suo’s Chinese example and my Russian example, the behaviour is exactly the same as before (in earlier versions).

Windows 7 Professional SP1 (x64), LibreOffice 5.0.1.2.
Comment 12 QA Administrators 2016-09-20 10:29:29 UTC Comment hidden (obsolete)
Comment 13 Karel Behounek 2016-09-20 13:15:30 UTC
The bug is still present on a currently supported version. No changes in behaviour. Sample documents provided back in 2012 (comment #1 in this thread) are still usable for demonstrating the issue.

Tested on Windows 7 Professional SP1 (x64), LibreOffice 5.2.1.2.
Comment 14 Xisco Faulí 2017-09-29 08:52:55 UTC Comment hidden (obsolete)
Comment 15 Buovjaga 2019-08-11 10:41:07 UTC
Still confirmed

Arch Linux 64-bit
Version: 6.4.0.0.alpha0+
Build ID: 37fc9f51a8de11d40632e8cda17ccf1fa4b1f503
CPU threads: 8; OS: Linux 5.2; UI render: default; VCL: gtk3; 
Locale: fi-FI (fi_FI.UTF-8); UI-Language: en-US
Calc: threaded
Built on 6 August 2019
Comment 16 Mike Kaganski 2019-09-17 12:11:05 UTC
The problem happens at file save stage. The strings are written to the binary image (Module1.bin) in SfxScriptLibraryContainer::implStorePasswordLibrary; it calls SbModule::StoreBinaryData, which in turn calls SbiImage::Save.

The strings are 1-byte-encoded: the encoding used is initially taken from current thread text encoding (on Windows, that's always system encoding); the loaded module might change the encoding (see SbiImage::Load); and *both SbiImage::Load and SbiImage::Save* filter the possible encoding to only be 1-byte encoding (using resp. GetSOLoadTextEncoding and GetSOStoreTextEncoding). It means, for instance, that UTF-8 is impossible for the encoding of strings here.

If current system encoding happens to support the characters in the module, then the resulting image would contain the properly encoded characters (e.g., when I open protected document from attachment 70101 [details] on my Russian Windows, open the module entering the password, and then save the document), the image has the correct Cyrillic characters (and correct encoding number), so then the resulting file may be used without the password later. But I suppose that the test file was created on a system with non-Cyrillic thread encoding.

I don't know if it's possible to change the format to accept UTF-8 as encoding. Hope that the analysis would help.
Comment 17 Mike Kaganski 2020-01-01 18:19:43 UTC
https://gerrit.libreoffice.org/c/core/+/86065
Comment 18 Commit Notification 2020-01-01 18:23:11 UTC
Mike Kaganski committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/4abb191916916c7003deedcfdcf46287faccaf01

tdf#57113: store UTF-16 stringpool data after legacy 1-byte data

It will be available in 6.5.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 19 Mike Kaganski 2020-01-01 18:45:59 UTC
As noted in comment 16, the Unicode data was not stored in the binary image of password-protected library. So the fix consists of writing new data to the image, and reading it back. Hence, the documents saved in older versions will still have broken Unicode strings in fixed versions, until libraries loaded, and re-saved. Older versions, naturally, will be unable to see new data, and will only have broken strings.

To test the fix: open attachment 70101 [details] from comment 0, open the password-protected library, and re-save. Then re-open and test.
Comment 20 Mike Kaganski 2021-05-19 10:03:55 UTC
*** Bug 127763 has been marked as a duplicate of this bug. ***