Bug 62977 - FILEOPEN RTF: Doesn't handle \uN keywords with unsigned N
Summary: FILEOPEN RTF: Doesn't handle \uN keywords with unsigned N
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
3.5.0 release
Hardware: Other Windows (All)
: medium minor
Assignee: Miklos Vajna
URL:
Whiteboard: target:4.1.0 target:4.0.4
Keywords:
Depends on:
Blocks:
 
Reported: 2013-04-01 05:33 UTC by mimimimizk
Modified: 2013-11-20 13:27 UTC (History)
1 user (show)

See Also:
Crash report or crash signature:


Attachments
Rtf file it is garbled. (98.35 KB, application/rtf)
2013-04-01 05:33 UTC, mimimimizk
Details
RTF file garbled (6.35 KB, application/rtf)
2013-04-16 01:50 UTC, mimimimizk
Details
Display example Libreoffrce, word, word pad (screen shot) (190.00 KB, application/vnd.ms-excel)
2013-04-16 01:53 UTC, mimimimizk
Details
Xsl-fo file before converting in AppacheFOP (2.41 KB, text/xml)
2013-04-16 01:54 UTC, mimimimizk
Details

Note You need to log in before you can comment on or make changes to this bug.
Description mimimimizk 2013-04-01 05:33:02 UTC
Created attachment 77263 [details]
Rtf file it is garbled.

In the RTF file that was created using the ApacheFOP, some Japanese are garbled.
Occurred in 3.5 and later versions of events.

When you save in another application MSWord, WordPad, etc., you will be able to open it without garbled.

Please tell me the cause.
Comment 1 mimimimizk 2013-04-01 06:54:10 UTC
(In reply to comment #0)
> Created attachment 77263 [details]
> Rtf file it is garbled.
> 
> In the RTF file that was created using the ApacheFOP, some Japanese are
> garbled.
> Occurred in 3.5 and later versions of events.
> 
> When you save in another application MSWord, WordPad, etc., you will be able
> to open it without garbled.
> 
> Please tell me the cause.
Comment 2 Sam L 2013-04-14 03:34:38 UTC
mimimimizk, 
I am running version 4.0.2.2 of LibreOffice Writer on Debian Linux (Testing) and opening the .rtf file the Japanese text appears fine. Please provide information about your Operating System type and version and what Japanese font you're using. It may well be an issue with your system's Japanese font.

I had similar problems with Japanese text in the past on Linux and had to install a new font to resolve the problem. The font I had to install was "ttf-vlgothic".
Comment 3 Urmas 2013-04-14 09:29:53 UTC
Could you attach a PDF file with this document displayed correctly?

I can see several ?'s opening this in WordPad, for example.
Comment 4 mimimimizk 2013-04-16 01:50:55 UTC
Created attachment 78048 [details]
RTF file garbled

I will replace it with a smaller RTF files.
Comment 5 mimimimizk 2013-04-16 01:53:26 UTC
Created attachment 78049 [details]
Display example Libreoffrce, word, word pad (screen shot)
Comment 6 mimimimizk 2013-04-16 01:54:45 UTC
Created attachment 78050 [details]
Xsl-fo file before converting in AppacheFOP
Comment 7 mimimimizk 2013-04-16 05:51:37 UTC
I've uploaded a new RTF file and xsl-fo files before conversion rtf.
Also, I've uploaded a screen shot opens with the normal state and garbled.

The operating system you are using WindowsXP Professional Sp3.
I have to specify the Mincho "MS" to the type of font.
I have installed the fonts ttf-vlgothic, did not change the situation.

Thanks for the advice.
Comment 8 Urmas 2013-04-16 15:21:58 UTC
The reason for ?'s are due to Unicode character values as unsigned decimals, instead of signed as required by RTF.

@Reporter: Fonts should look better if the font table will be containing charset value: \fcharset128
You can ensure your XML source generates it or add it manually to RTF file immediately after each "\f1", "\f2", etc. within "{\fonttbl..." section.
Comment 9 mimimimizk 2013-04-19 10:47:35 UTC
where the unicode value has changed to a negative character that is more than 32767, garbled was solved.
(For example, \u34920 > \u-30616)
However, it is saved with a value of more than 32767 When you save in LibreOffice.
Is it to look for another way?
※ 15 page "Rich Text Format (RTF) Specification Version 1.9.1" on MicroSoft
There has been described to be written in unicode negative number more than 32767.

I tried to append \fcharset128 and \ansicpg932, but it did not resolve.
Comment 10 Urmas 2013-04-26 16:08:00 UTC
Probably due to check in core/writerfilter/source/rtftok/rtfdocumentimpl.cxx:3064
Comment 11 Caolán McNamara 2013-05-14 19:18:56 UTC
caolanm->vmiklos: Is this still relevant ?
Comment 12 Miklos Vajna 2013-05-15 07:46:47 UTC
Yes, it is. Actually it seems Word accepts values between SAL_MAX_INT16 and SAL_MAX_UINT16 for \u, and maps them to SAL_MIN_INT16..0, doing the same in writerfilter makes sense. Will do that in a bit.
Comment 13 Commit Notification 2013-05-15 08:38:03 UTC
Miklos Vajna committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=4a816a816e8b4e213394898bd0e307dd90fcbb64

fdo#62977 allow SAL_MAX_INT16..SAL_MAX_UINT16 when importing RTF_U



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 14 Miklos Vajna 2013-05-15 09:00:51 UTC
-4-0 review: https://gerrit.libreoffice.org/3913
Comment 15 mimimimizk 2013-05-16 01:06:04 UTC
As a result of tested on version4.1.0.0.alpha1+ of the master, the problem has been corrected.
Thank you very much.
Comment 16 Commit Notification 2013-05-16 06:50:26 UTC
Miklos Vajna committed a patch related to this issue.
It has been pushed to "libreoffice-4-0":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=53d40e18515c78e59583f1f8fadc82ffa2ab9e32&h=libreoffice-4-0

fdo#62977 allow SAL_MAX_INT16..SAL_MAX_UINT16 when importing RTF_U


It will be available in LibreOffice 4.0.4.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.