Bug 68790 - Other: Build fails on Windows when system locale is "Japanese(Japan)" .
Summary: Other: Build fails on Windows when system locale is "Japanese(Japan)" .
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
4.2.0.0.alpha0+ Master
Hardware: Other Windows (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: BSA target:4.2.0
Keywords:
Depends on:
Blocks:
 
Reported: 2013-08-31 13:20 UTC by Tomofumi Yagi
Modified: 2013-11-21 20:45 UTC (History)
1 user (show)

See Also:
Crash report or crash signature:


Attachments
Full error message (68.17 KB, text/plain)
2013-08-31 13:20 UTC, Tomofumi Yagi
Details
AutoControls.hrc (44.93 KB, text/plain)
2013-08-31 13:24 UTC, Tomofumi Yagi
Details
Patch for adding BOM to .hrc file (4.09 KB, text/plain)
2013-08-31 14:32 UTC, Tomofumi Yagi
Details
Patch for escaping UTF-8 chars (869 bytes, patch)
2013-09-06 13:22 UTC, Tomofumi Yagi
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Tomofumi Yagi 2013-08-31 13:20:20 UTC
Created attachment 84969 [details]
Full error message

Problem description: 

When building LibreOffice 4.2.0.0.aplha0+(master), build fails in dbaccess module.

I'm using Windows 7 Ultimate and Visual Studio 2012 Express for Windows Desktop.

When using "--with-lang=en-US ja" autogen switch, build fails.
When using "--with-lang=ALL" switch, build fails,too.
But when not using "--with-lang=" switch, build succeeds.

Please refer Bug 66395 - Compilation error in Windows with UTF-8 unfriendly codepage( https://bugs.freedesktop.org/show_bug.cgi?id=66395 )


Steps to reproduce:

1. Open "Control Panel" -> "Regional and Language Options"
2. Select "Administrative" tab
3. Click "Change system locale".
4. Select "Japanese(Japan)" (Japanese version Windows has this value as default).
5. Click "OK" and restart computer.
6. Start to build libreoffice.
 
Build will fail with following message.
-----------------------------------------------------------------------
$ /opt/lo/bin/make 
/opt/lo/bin/make -j 8 -rs -f C:/lomas/core/Makefile.gbuild
[build DEP] LNK:Library/idbu.lib
[build ALM] animations
[build ALM] basebmp
[build ALM] basegfx
[build ALM] binaryurp
[build ALM] cli_ure
[build ALM] clucene
[build ALM] comphelper
[build ALM] configmgr
[build RDB] postgresql-sdbc
[build ALM] cppu
[build ALM] cppuhelper
[build CXX] dbaccess/source/ui/dlg/ConnectionHelper.cxx
[build CXX] dbaccess/source/ui/dlg/ConnectionPage.cxx
[build CXX] dbaccess/source/ui/dlg/ConnectionPageSetup.cxx
[build CXX] dbaccess/source/ui/dlg/DBSetupConnectionPages.cxx
[build CXX] dbaccess/source/ui/dlg/detailpages.cxx
[build CXX] dbaccess/source/ui/dlg/directsql.cxx
[build CXX] dbaccess/source/ui/dlg/dlgattr.cxx
[build CXX] dbaccess/source/ui/dlg/dlgsave.cxx
[build CXX] dbaccess/source/ui/dlg/dlgsize.cxx
(snip)
Note: including file:  C:\lomas\core\dbaccess\source\ui\inc\commontypes.hxx
C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc : warning C4819: The file contains a character that cannot be represented in the current code page (932). Save the file in Unicode format to prevent data loss
Note: including file: C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc
Note: including file:  C:\lomas\core\dbaccess\inc\dbaccess_helpid.hrc
Note: including file:  C:\lomas\core\dbaccess\source\ui\inc\dbu_dlg.hrc
Note: including file:  C:\lomas\core\dbaccess\inc\dbaccess_helpid.hrc
C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc(159) : error C2001: newline in constant
C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc(370) : warning C4819: The file contains a character that cannot be represented in the current code page (932). Save the file in Unicode format to prevent data loss
C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc(456) : error C2162: expected macro formal parameter
C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc(460) : error C2162: expected macro formal parameter
C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc(461) : error C2162: expected macro formal parameter
C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc(468) : error C2162: expected macro formal parameter
C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc(469) : error C2001: newline in constant
Note: including file: C:\lomas\core\include\connectivity/CommonTools.hxx
Note: including file: c:\lomas\core\dbaccess\source\ui\dlg\DriverSettings.hxx
Note: including file:  C:\lomas\core\dbaccess\source\inc\dsntypes.hxx
Note: including file:  C:\lomas\core\include\svl/poolitem.hxx
Note: including file: C:\lomas\core\dbaccess\source\ui\inc\dbadmin.hxx
Note: including file:  C:\lomas\core\include\sfx2/tabdlg.hxx
Note: including file:  C:\lomas\core\dbaccess\source\inc\dsntypes.hxx
Note: including file:  c:\lomas\core\dbaccess\source\ui\inc\IItemSetHelper.hxx
Note: including file:   C:\lomas\core\include\sal/types.h
Note: including file:   C:\lomas\core\include\comphelper/uno3.hxx
Note: including file:   C:\lomas\core\dbaccess\source\inc\dsntypes.hxx
Note: including file:  C:\lomas\core\include\comphelper/uno3.hxx
Note: including file: C:\lomas\core\include\comphelper/types.hxx
make[1]: *** [C:/lomas/core/workdir/wntmsci14/CxxObject/dbaccess/source/ui/dlg/ConnectionPage.o] Error 2
C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc : warning C4819: The file contains a character that cannot be represented in the current code page (932). Save the file in Unicode format to prevent data loss
Note: including file: C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc
Note: including file:  C:\lomas\core\dbaccess\inc\dbaccess_helpid.hrc
Note: including file:  C:\lomas\core\dbaccess\source\ui\inc\dbu_dlg.hrc
Note: including file:  C:\lomas\core\dbaccess\inc\dbaccess_helpid.hrc
C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc(159) : error C2001: newline in constant
C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc(370) : warning C4819: The file contains a character that cannot be represented in the current code page (932). Save the file in Unicode format to prevent data loss
C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc(456) : error C2162: expected macro formal parameter
C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc(460) : error C2162: expected macro formal parameter
C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc(461) : error C2162: expected macro formal parameter
C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc(468) : error C2162: expected macro formal parameter
C:\lomas\core\workdir\wntmsci14\SrsTemplatePartTarget\dbaccess\AutoControls.hrc(469) : error C2001: newline in constant
make[1]: *** [C:/lomas/core/workdir/wntmsci14/CxxObject/dbaccess/source/ui/dlg/detailpages.o] Error 2
make: *** [build] Error 2
-----------------------------------------------------------------------
              
Operating System: Windows (other)
Version: 4.2.0.0.alpha0+ Master
Comment 1 Tomofumi Yagi 2013-08-31 13:24:33 UTC
Created attachment 84970 [details]
AutoControls.hrc
Comment 2 Tomofumi Yagi 2013-08-31 14:32:22 UTC
Created attachment 84972 [details]
Patch for adding BOM to .hrc file

This problem is that MSVC interprets UTF-8 source code without BOM as local codepage when system locale is "Japanese(Japan)".
I attach a workaound patch in order to clarify the cause.
This patch tweaks transex3.
Tweaked transex3 outputs a *.hrc file with BOM.  
*.hrc files with BOM avoid this problem.
Comment 3 Tomofumi Yagi 2013-09-06 13:22:04 UTC
Created attachment 85353 [details]
Patch for escaping UTF-8 chars

I attached a patch for escaping UTF-8 chars in .hrc file.

After all,I think that *.hrc file with BOM is better.
 
Escaping UTF-8 chars method has two disadvantages.

One is that Converted strings are very long.
This often occurs preprocessor error(the input line is too long).

Two is that it makes code unreadable.

I show an example.

1) Text[ en-US ] = "~Host name";
2) Text[ ja ] = "ホスト名(~H)"; // String of Japanese(UTF-8)
3) Text[ ja ] = "\xe3\x83\x9b\xe3\x82\xb9\xe3\x83\x88\xe5\x90\x8d(~H)"; // Escaped string of Japanese(UTF-8)
Comment 4 Commit Notification 2013-09-14 16:52:37 UTC
Tomofumi Yagi committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=fff70bf98c7a5a63aa0db11e93a3512c6a9a9359

fdo#68790: fix build error on Win when system locale=="Japanese(Japan)"



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 5 Tomofumi Yagi 2013-09-20 12:18:27 UTC
Thank you so much for evaluation and merging of the patch.