Bug 133902 - Files with eg. Japanese characters in name can't be opened without C-UTF8 locale in system
Summary: Files with eg. Japanese characters in name can't be opened without C-UTF8 loc...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice Online
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
unspecified
Hardware: All All
: medium normal
Assignee: Andras Timar
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-06-11 17:26 UTC by Babbles
Modified: 2020-06-22 05:28 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
can't open file (7.14 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2020-06-15 15:44 UTC, Babbles
Details
can open file (7.14 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2020-06-15 15:45 UTC, Babbles
Details
Same as attachment 162014 (7.14 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2020-06-15 15:58 UTC, Aron Budea
Details
ZIP file. (6.06 KB, application/zip)
2020-06-15 16:03 UTC, Babbles
Details
locale-list (8.08 KB, text/plain)
2020-06-21 04:23 UTC, Babbles
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Babbles 2020-06-11 17:26:15 UTC
I'm upgrade to 6.2.10.16-16.
I can no longer open a file that I was editing in MS Excel.

> Failed to load the document.
> Make sure that the file format is supported and the file is not corrupted and try again.

Of course the files are not corrupted.
The new file user create can be opened and closed with no problem.
Comment 1 Babbles 2020-06-12 05:01:43 UTC
I was suspicious of .xls(97-2003 format) files. But this doesn't seem to matter. I can open it in xls files as well. Some files cannot be opened even in xlsx files.

The only thing that's not a problem is a file created by Calc itself.
Comment 2 Aron Budea 2020-06-15 03:58:10 UTC
Thanks for reporting, is there a chance you could share a sample you are not able to open anymore?
Comment 3 Babbles 2020-06-15 04:41:02 UTC
Sorry, problematic files cannot be shared. Users can't open it by Collabora, but they can open it by OnlyOffice. I'm currently moving to OnlyOffice as an emergency workaround.
Comment 4 Aron Budea 2020-06-15 05:16:03 UTC
I see, unfortunately I couldn't reproduce the issue myself with a number of examples I tried with.
Comment 5 Babbles 2020-06-15 05:21:56 UTC
The spreadsheet is not a complex structure. It is a simple spreadsheet that sets the width and height of the cells and the background color of the cells.
It's just a simple list and sums with the SUM function.

We still continue to use files that were originally created in Excel 2010.
Comment 6 Babbles 2020-06-15 08:59:49 UTC
Sorry, I grasped it incorrectly. There are simple files, but those were rare.
The file has a filter function in the header and a fixed window frame. Some of the files had several sheets.

I can't verify this for myself until all users have stopped using it. When the time comes, I'll switch to Calc to verify a few things.
Comment 7 Babbles 2020-06-15 14:39:49 UTC
Oh! I have found the cause!
If the file name contains Japanese (2-byte characters), the user can't open the file.
Once I changed all the file names to English characters, I was able to open the file.
This is both Calc and Writer.

CODE is increasingly tough on the Japanese. In Japanese display mode, it is difficult to recognize even a mouse click. This is reported in another topic.  https://bugs.documentfoundation.org/show_bug.cgi?id=132082

The previous version had a problem in Japanese display mode. In this version, there is a problem with Japanese file names.
Comment 8 Xisco Faulí 2020-06-15 15:26:58 UTC
(In reply to Babbles from comment #7)
> Oh! I have found the cause!
> If the file name contains Japanese (2-byte characters), the user can't open
> the file.
> Once I changed all the file names to English characters, I was able to open
> the file.
> This is both Calc and Writer.
> 
> CODE is increasingly tough on the Japanese. In Japanese display mode, it is
> difficult to recognize even a mouse click. This is reported in another
> topic.  https://bugs.documentfoundation.org/show_bug.cgi?id=132082
> 
> The previous version had a problem in Japanese display mode. In this
> version, there is a problem with Japanese file names.

Could you please attach a document with those characters that can't be open in LibreOffice Online ?
Comment 9 Babbles 2020-06-15 15:34:06 UTC
can't open:   日本語ファイル.xls
can open: nihongofail.xls
Comment 10 Xisco Faulí 2020-06-15 15:38:46 UTC
(In reply to Babbles from comment #9)
> can't open:   日本語ファイル.xls
> can open: nihongofail.xls

Thanks
Comment 11 Babbles 2020-06-15 15:44:59 UTC
Created attachment 162014 [details]
can't open file

I can't open this file.
Comment 12 Babbles 2020-06-15 15:45:37 UTC
Created attachment 162015 [details]
can open file

I can open this file.
Comment 13 Babbles 2020-06-15 15:52:39 UTC
> Created attachment 162014 [details]
> can't open file

I've attached a file with a Japanese file name, but this forum doesn't seem to be able to display Japanese in the file name.
日本語ファイル.xls
Comment 14 Aron Budea 2020-06-15 15:58:03 UTC
Created attachment 162016 [details]
Same as attachment 162014 [details]

Let me try attaching a file as well.
So far I'm suspecting something is unusual with the character encoding.
Comment 15 Aron Budea 2020-06-15 15:59:47 UTC
(In reply to Aron Budea from comment #14)
> Let me try attaching a file as well.
> So far I'm suspecting something is unusual with the character encoding.
Nope, shows the same wrong characters in BZ. Anyway, I have no problem opening the file in online after renaming it to 日本語ファイル.xlsx... very strange.
Comment 16 Babbles 2020-06-15 16:03:02 UTC
Created attachment 162017 [details]
ZIP file.

The file with a Japanese file name was zipped.
Comment 17 Babbles 2020-06-15 16:05:39 UTC
(In reply to Aron Budea from comment #15)

Does generating a Japanese file in a Japanese environment cause problems?
Comment 18 Aron Budea 2020-06-15 19:28:58 UTC
Hard to say, usually similar bugs could be solved by installing all locales on the server (it wouldn't depend on the client), see bug 122327, bug 114758 and bug 101581. Why would it start occurring after a CODE update, though? No idea...
Comment 19 Babbles 2020-06-16 01:57:21 UTC
(In reply to Aron Budea from comment #18)
> Husually similar bugs could be solved by installing all locales
> on the server

Can you tell me how to do this?
Comment 20 Aron Budea 2020-06-16 02:38:04 UTC
(In reply to Babbles from comment #19)
> Can you tell me how to do this?
It depends on the distro, in Debian-based distributions the package you need is 'locales-all'.
For RHEL/Centos perhaps this might help: http://blog.nashcom.de/nashcomblog.nsf/dx/locale-issue-on-linux-centos-rhel.htm

You specifically need C.UTF-8 locale, perhaps first you could check if it exists in the system.
Comment 21 Aron Budea 2020-06-18 09:12:33 UTC
Babbles, do you have an update on the locale situation?
Comment 22 Babbles 2020-06-18 10:03:51 UTC
(In reply to Aron Budea from comment #21)
> Babbles, do you have an update on the locale situation?

I'm not very knowledgeable in this area, so I may be doing it wrong.
I set it to c.UTF-8 or reinstalled glibc-common.
The situation is no different.

By the way, I'm using the Docker version 4.2.1.2 right now. This is the latest version, which works fine in the Japanese environment. Later versions cannot be used in Japanese display mode. And the latest version doesn't use Japanese file names.
Comment 23 Babbles 2020-06-18 10:08:20 UTC
(In reply to Babbles from comment #22)
> And the latest version doesn't use Japanese file names.
Sorry, that was a typo.
And the latest version can't use Japanese file names.
Comment 24 QA Administrators 2020-06-19 03:51:57 UTC Comment hidden (obsolete)
Comment 25 Babbles 2020-06-21 03:52:21 UTC
I have since updated to 6.2-17. loolwsd 4.2.4-5.
Even in this version I can't open a file with a Japanese file name.

I tried Docker(4.2.4.5) version. I was able to open a file with a Japanese file name here.
Is there still something wrong with loolwsd?
Comment 26 Aron Budea 2020-06-21 04:14:21 UTC
Please run the following command on the server, if it exists in your system, copy the result to a text file, and attach it to the bug report: localectl list-locales
Comment 27 Babbles 2020-06-21 04:23:55 UTC
Created attachment 162239 [details]
locale-list

(In reply to Aron Budea from comment #26)
> Please run the following command on the server, if it exists in your system,
> copy the result to a text file, and attach it to the bug report: localectl
> list-locales

It looks like most of the locales are installed.
Comment 28 Aron Budea 2020-06-21 04:58:50 UTC
(In reply to Babbles from comment #27)
> Created attachment 162239 [details]
> locale-list
It seems you're missing C.UTF8. Which distro are you using?
Comment 29 Babbles 2020-06-21 05:18:17 UTC
(In reply to Aron Budea from comment #28)
> It seems you're missing C.UTF8. Which distro are you using?

$ cat /etc/redhat-release
CentOS Linux release 7.8.2003 (Core)
Comment 30 Babbles 2020-06-21 12:09:27 UTC
I was able to open a file with a Japanese file name when using the Collabora demo site on Nextcloud. This is a function to specify a demo that Nextcloud provides. *Use a demo server menu.
demo server url: https://demo.ja.collaboraonline.com
Comment 31 Andras Timar 2020-06-21 12:28:31 UTC
CentOS 7 does not support C.UTF-8 locale. I have a patch on Gerrit that will help.
https://gerrit.libreoffice.org/c/online/+/96788
I will release a new CODE with this.
Comment 32 Babbles 2020-06-22 00:17:46 UTC
Oh, Thank you so much!!!!

I have updated to loolwsd 4.2.4-6.
I can now open files with Japanese filenames.
Comment 33 Andras Timar 2020-06-22 05:28:52 UTC
Thanks for the bug report!