Bug 125286 - FIREBIRD - Base not usable under Windows account with Chinese user name (UTF character support)
Summary: FIREBIRD - Base not usable under Windows account with Chinese user name (UTF ...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Base (show other bugs)
Version:
(earliest affected)
6.2.1.2 release
Hardware: x86-64 (AMD64) Windows (All)
: medium normal
Assignee: Mike Kaganski
URL:
Whiteboard: target:7.3.0 inReleaseNotes:7.3
Keywords:
Depends on:
Blocks: Database-Import
  Show dependency treegraph
 
Reported: 2019-05-14 14:52 UTC by Ming Hua
Modified: 2022-01-24 12:16 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ming Hua 2019-05-14 14:52:56 UTC
Description:
I've used LO for quite a few years, but never touched the Base part.  Recently I tried my hands on it, but found it not working at all under my non-privileged user account with a Chinse (non-Latin) user name on Windows 10.


Steps to Reproduce:
There are two symptoms:

1. Creating a new database in LO Base, after choosing options (don't register the database in the system, everything else default) in the wizard, the save file dialog pops out and asks for a file name.  After entering the file name, the dialog window disappears and the following error message dialog pops out:
firebird_sdbc error:
*I/O error during "CreateFile (create)" operation for file "C:\USERS\<Chinese User Name>\APPDATA\LOCAL\TEMP\LU1624462CC3W.TMP\LU1624462CC40.TMP\FIREBIRD"
*Error while trying to create file
*ϵͳ�Ҳ���ָ����·���� 
caused by
'isc_create_database'
(My user name is redacted as it's my real name.  The scrambled characters are just as shown on my screen.)
After this error no operation on the database seems possible.

2. Opening an existing database in LO Base, the following error message pops out:
firebird_sdbc error:
*cannot open backup file C:\Users\<Scrambled Characters>\AppData\Local\Temp\lu1624462cc3w.tmp\lu1624462cc3z.tmp/firebird.fbk
*Exiting before completion due to errors
caused by
'isc_service_query'
(Here the path doesn't exist, as the <Scrambled Characters> part is not my user name.  Since my user name has two characters and this scrambled part has three, I suspect there is some UTF-8 to UCS-2 conversion issue here.)

Under the administrative account with a English user name, Base seems to work fine on the surface.

I'm not sure the two symptoms are connected, so let me know if I need to file a separate bug.


Actual Results:
Base is not usable for users with Chinese account names.

Expected Results:
Base works for every account.


Reproducible: Always


User Profile Reset: No



Additional Info:
This issued existed for 6.2.1, and I've recently reproduced in 6.2.4 RC1.  The "About" information is as follows:
Version: 6.2.4.1 (x64)
Build ID: 170a9c04e0ad25cd937fc7a913bb06bf8c75c11d
CPU threads: 2; OS: Windows 10.0; UI render: GL; VCL: win; 
Locale: zh-CN (zh_CN); UI-Language: en-GB
Calc: threaded
Comment 1 Drew Jensen 2019-06-13 11:38:14 UTC
Well, so far I've only been able to look at this on Linux and on a machine set for English. Using LO 6.3 Beta 1

I tried this: 
Created a directory: 预算
Created a Base (Firebird embedded)  file named 预算 and stored it in that directory. No problem.
Added a table named 支付 with columns 键, 量 and 日期. Again with no problems.

So that is not a really satisfactory test - I'll see about getting a Chinese character account on a Windows 10 machine and try it all there. Beyond that I need to reach out to someone on the Chinese mailing lists for help in checking this.
Comment 2 Ming Hua 2019-06-13 14:35:59 UTC
(In reply to Drew Jensen from comment #1)
> Well, so far I've only been able to look at this on Linux and on a machine
> set for English. Using LO 6.3 Beta 1
> 
> I tried this: 
> Created a directory: 预算
> Created a Base (Firebird embedded)  file named 预算 and stored it in that
> directory. No problem.
> Added a table named 支付 with columns 键, 量 and 日期. Again with no problems.
Not that it's an important difference, but I'd like to comment that my problem comes from a Chinese *user account name*, not necessarily a Chinese directory.

I don't think there is anything comparable on Linux though.  A Chinese $HOME (/home/预算 for example) maybe, but I don't think Linux allows non-ascii username yet.
Comment 3 Julien Nabet 2019-06-24 15:38:35 UTC
(In reply to Ming Hua from comment #2)
> (In reply to Drew Jensen from comment #1)
> ...
> I don't think there is anything comparable on Linux though.  A Chinese $HOME
> (/home/预算 for example) maybe, but I don't think Linux allows non-ascii
> username yet.

Indeed, I made the same test as Drew (see comment 1) in Win10 with master sources updated today, no problem to create the file and reopen it.
Comment 4 Ming Hua 2019-06-26 17:35:08 UTC
Drew and Julien, thanks for looking into this bug.

I did exactly the same test in comment #1 on my computer under the English-named administrative account with LO 6.2.4.  No problems as well.

So yeah, that's probably not a satisfactory test.
Comment 5 Ming Hua 2019-06-29 08:11:34 UTC
One more data point -- I can reproduce both problems on Win10 with LO 6.3.0 beta2.
Comment 6 Ming Hua 2019-11-24 07:31:31 UTC
Both problems are still reproducible in 6.4.0 beta1.

Version: 6.4.0.0.beta1 (x64)
Build ID: 4d7e5b0c40ed843384704eca3ce21981d4e98920
CPU threads: 2; OS: Windows 10.0 Build 18363; UI render: default; VCL: win; 
Locale: zh-CN (zh_CN); UI-Language: en-US
Calc: threaded
Comment 7 himajin100000 2019-11-24 08:04:36 UTC
Similar report from a Japanese user.
https://ask.libreoffice.org/ja/question/186957/
Comment 8 Julien Nabet 2019-12-05 22:35:14 UTC
On pc Debian x86-64 with master sources updated today, I created a new user with non ascii characters and I don't reproduce this so I suppose it's indeed Windows only.
Here's a codepointer to start:
https://opengrok.libreoffice.org/xref/core/connectivity/source/drivers/firebird/Connection.cxx?r=f853ec31#172
172 m_sFirebirdURL = m_pDatabaseFileDir->GetFileName() + "/firebird.fdb";
173 m_sFBKPath = m_pDatabaseFileDir->GetFileName() + "/firebird.fbk";

I suppose "m_pDatabaseFileDir->GetFileName()" is ko but can't tell more.
We need someone who can build sources on Windows and who has a non ascii profile directory.

Stephan: comparing method osl_getFileURLFromSystemPath sal/osl/unx/file_url.cxx and sal/osl/w32/file_url.cxx, I noticed Unix version used RTL_TEXTENCODING_UTF8 and Windows version RTL_TEXTENCODING_ASCII_US, is it expected?
see
https://opengrok.libreoffice.org/xref/core/sal/osl/unx/file_url.cxx?r=0f874472#440

https://opengrok.libreoffice.org/xref/core/sal/osl/w32/file_url.cxx?r=895cd721#892
Comment 9 Julien Nabet 2021-05-14 15:53:21 UTC
I noticed these bugtrackers:
- https://github.com/FirebirdSQL/firebird/issues/2603
- https://github.com/FirebirdSQL/firebird/issues/3546
perhaps related.
Comment 10 Mike Kaganski 2021-09-10 17:45:17 UTC
I believe https://gerrit.libreoffice.org/c/core/+/121886 would *improve* this a bit. I.e., *if* the path only uses characters representable in current Windows codepage, then it should be possible to use such paths with Firebird DB.

Still, the issues mentioned in comment 9 would prevent from use of paths with characters not representable in the active Windows codepage. E.g., on my Russian system, I will not be able to use Chinese or Japanese characters in path, until FB implements Unicode path handling on Windows.
Comment 11 Commit Notification 2021-09-10 19:25:36 UTC
Mike Kaganski committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/d33f5abc4423aad4087eb77c920fcbd74e34533f

tdf#125286: Inform Firebird that we use UTF8 in paths

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Ming Hua 2021-09-19 05:12:35 UTC
(In reply to Mike Kaganski from comment #10)
> I believe https://gerrit.libreoffice.org/c/core/+/121886 would *improve*
> this a bit. I.e., *if* the path only uses characters representable in
> current Windows codepage, then it should be possible to use such paths with
> Firebird DB.
Indeed, I've installed a daily build including this commit, and it fixed both issues described in comment #0 for me.

Version: 7.3.0.0.alpha0+ (x64) / LibreOffice Community
Build ID: a9cc066a86c6bd3423c5802c5a4eded55a50c754
CPU threads: 2; OS: Windows 10.0 Build 19043; UI render: default; VCL: win
Locale: zh-CN (zh_CN); UI: zh-CN
Calc: threaded

As the simplified Chinese Windows code page 936 is rather large and covers many scripts, and I only have problem with Chinese characters in account names, I myself consider this issue FIXED.

If the developers want to leave this open for further efforts supporting more Unicode characters not representable in Windows code page, that's fine by me, but I probably won't be able to help with any further testing.

Thanks Mike for look into this, and everyone for helping!
Comment 13 Mike Kaganski 2021-09-19 07:38:42 UTC
(In reply to Ming Hua from comment #12)

Thank you for testing/confirmation!
As mentioned, the rest is rather on FB side; so let's close this one, as the change we actually needed on LO side.