Bug 92296 - Character formating (bold, underline, color etc) corrupted when save as xlsx
Summary: Character formating (bold, underline, color etc) corrupted when save as xlsx
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
5.0.0.0.beta1
Hardware: All All
: medium major
Assignee: Katarina Behrens (CIB)
URL:
Whiteboard: target:5.1.0 target:5.0.6
Keywords: bibisected, bisected, regression
: 93115 98247 (view as bug list)
Depends on:
Blocks:
 
Reported: 2015-06-24 08:13 UTC by Kevin Suo
Modified: 2016-10-25 19:17 UTC (History)
9 users (show)

See Also:
Crash report or crash signature:


Attachments
the test ods file (8.70 KB, application/vnd.oasis.opendocument.spreadsheet)
2015-06-24 08:22 UTC, Kevin Suo
Details
the saved xlsx file (4.74 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2015-06-24 08:27 UTC, Kevin Suo
Details
screenshots showing the difference (272.10 KB, image/png)
2015-06-24 08:42 UTC, Kevin Suo
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Kevin Suo 2015-06-24 08:13:02 UTC
The attached ods file contains simple chars "abc def" in each cell, with different formating. 
* A1: without any additional formating to the chars.
* A2: the char "d" is bold.
* A3: the chars "def" are bold.
* A4: the char "d" is underlined.
* A5: the chars "def" are underlined.
* A6: the char "d" is in blue color.

(Please note there is a space between "abc" and "def")
When save this file as xlsx, the formatting is corrupted.


=== Steps to reproduce ===

1. Save the attached ods file as xlsx.

2. Reopen the xlsx file with LibreOffice.
Current Results (please compare with the above for expected results):
* A1: No corruption. -> Good.
* A2: The chars "abc" are bold, "d" is not bold. -> Bad
* A3: The chars "abc def" are all bold. -> Bad
* A4: The chars "abc" are underlined, "d" is not underlined. -> Bad
* A5: The char "abc def" are all underlined. -> Bad
* A6: The char "abc" are blue. "d" is not blue. -> Bad.

3. Reopen the xlsx file with MSO 2010.
Current Results:
You will see the same bug behaviour as shown in step 2, PLUS all the spaces in cells A2:A6 are missing.

Reproduced both in

Version: 5.0.0.1
Build ID: 9a0b23dd0ab9652e0965484934309f2d49a7758e
Locale: zh-CN (zh_CN)

and

Version: 5.0.0.0.beta3
Build ID: 96345c15d8ab19c49014f055fe41ba8e1f421e5c
Locale: zh-CN (zh_CN).

OS: Winsows 7 X86.
Comment 1 Kevin Suo 2015-06-24 08:22:48 UTC
Created attachment 116782 [details]
the test ods file
Comment 2 Kevin Suo 2015-06-24 08:27:35 UTC
Created attachment 116783 [details]
the saved xlsx file
Comment 3 Kevin Suo 2015-06-24 08:42:10 UTC
Created attachment 116788 [details]
screenshots showing the difference
Comment 4 Jacques Guilleron 2015-06-24 11:02:26 UTC
Hello Kevin,

I reproduce with all LO 5 releases & Windows 7 Home premium
Works with LO 4.4.4.2 Build ID: f784c932ccfd756d01b70b6bb5e09ff62e1b3285 so regression.

Jacques
Comment 5 Terrence Enger 2015-06-24 15:55:54 UTC
Working in the 50max bibisect repository, I see from `git bisect good` ...

    2faf2a5126e3ccf78be3d6619b571358bb7af742 is the first bad commit
    commit 2faf2a5126e3ccf78be3d6619b571358bb7af742
    Author: Matthew Francis <mjay.francis@gmail.com>
    Date:   Wed May 27 22:54:13 2015 +0800

        source-hash-2523972f6a066488c649ab97dcba4f458126f18b
    
        Bibisect: This commit covers the following irrelevant source commit(s)
        cfdd89f10fdc15b36bd0d1023e49ca3752feb51e
    
        commit 2523972f6a066488c649ab97dcba4f458126f18b
        Author:     Norbert Thiebaud <nthiebaud@gmail.com>
        AuthorDate: Sat May 9 19:39:55 2015 -0500
        Commit:     Caolán McNamara <caolanm@redhat.com>
        CommitDate: Sun May 10 19:43:06 2015 +0000
    
            remove a use of OUString::intern()
    
            maIconCache is defined as a std::unordered_map
            'interning' a OUString to give to it as a key seems completely
            superfluous since no-one will ever use it by address, even less
            compare the address.
    
            Change-Id: Iab5d106d7d51fcdfbd0a0dfd50ae28d686b9b2b7
            Reviewed-on: https://gerrit.libreoffice.org/15692
            Tested-by: Jenkins <ci@libreoffice.org>
            Reviewed-by: Caolán McNamara <caolanm@redhat.com>
            Tested-by: Caolán McNamara <caolanm@redhat.com>

    :040000 040000 ed730dac997bd3b47036945e349d224d12b820ba d2a7f7d865af57772da4e51f1ba6c7a7cceb7f08 M	opt

and from `git bisect log` ...

    # bad: [dda106fd616b7c0b8dc2370f6f1184501b01a49e] source-hash-0db96caf0fcce09b87621c11b584a6d81cc7df86
    # good: [5b9dd620df316345477f0b6e6c9ed8ada7b6c091] source-hash-2851ce5afd0f37764cbbc2c2a9a63c7adc844311
    git bisect start 'latest' 'oldest'
    # good: [0c30a2c797b249d0cd804cb71554946e2276b557] source-hash-45aaec8206182c16025cbcb20651ddbdf558b95d
    git bisect good 0c30a2c797b249d0cd804cb71554946e2276b557
    # good: [2ce02b2ce56f12b9fcb9efbd380596975a3a5686] source-hash-17d714eef491bda2512ba8012e5b3067ca19a5be
    git bisect good 2ce02b2ce56f12b9fcb9efbd380596975a3a5686
    # good: [40875247f0002056effdf6d2fbe43627691cd86c] source-hash-93f0b14458a618ad575cd446680e5c4aa7d87bdc
    git bisect good 40875247f0002056effdf6d2fbe43627691cd86c
    # skip: [61f66b1a251477193d796411ca95f50d606ead45] source-hash-3fd5f8919ec2256c70ff26c14cb9f8065c5cb2f1
    git bisect skip 61f66b1a251477193d796411ca95f50d606ead45
    # good: [e7374cd735af2344dae55be40946d96633d2f6ee] source-hash-8a91528a3e03fe6e2923c33327b687ecf57adb0b
    git bisect good e7374cd735af2344dae55be40946d96633d2f6ee
    # bad: [541837707e7b0c5f5335180de535043c43e78e8d] source-hash-0811de12ee6727bbb9d4265217833ba02301eed8
    git bisect bad 541837707e7b0c5f5335180de535043c43e78e8d
    # bad: [2faf2a5126e3ccf78be3d6619b571358bb7af742] source-hash-2523972f6a066488c649ab97dcba4f458126f18b
    git bisect bad 2faf2a5126e3ccf78be3d6619b571358bb7af742
    # good: [8f6f05e17a2972f3b9f83bbc4e10231626b44c13] source-hash-2c70139d6a8d8fabd671455d3edd32117783d4d7
    git bisect good 8f6f05e17a2972f3b9f83bbc4e10231626b44c13
    # good: [3629c94e7c28b3be63abd20c606ce2928ee00d2b] source-hash-d133a87c1e518f14b0fa0e0e8e372a9925c55f1a
    git bisect good 3629c94e7c28b3be63abd20c606ce2928ee00d2b
    # good: [944d3cc98286a9d17d6b400cb180fdcdd41008e3] source-hash-39707f9448d65ba14e04fc9377f87086dab58e89
    git bisect good 944d3cc98286a9d17d6b400cb180fdcdd41008e3
    # good: [16429ce8f5cc10184a338003387ab7b375869fc9] source-hash-892cb24be673e8441a75bdde950c2087a24bdf74
    git bisect good 16429ce8f5cc10184a338003387ab7b375869fc9
    # good: [a8b54baa305bc4121cd8a059aec583c94c30940f] source-hash-1f43eab48a0397d9f7c272116aeb150694b1721d
    git bisect good a8b54baa305bc4121cd8a059aec583c94c30940f
    # good: [56f483752729e9d0061347d5a4b28e23060e3af0] source-hash-dc6dc631e25b6ad3de5018b9be03b129d40d7375
    git bisect good 56f483752729e9d0061347d5a4b28e23060e3af0
    # good: [74abe1ef6a9336dcc1ae2291e34a22b9d3b6751f] source-hash-0b4e3f5ebd2511c7ba39cbcc83d5faddb842404d
    git bisect good 74abe1ef6a9336dcc1ae2291e34a22b9d3b6751f
    # good: [a5b688ef540d5189edde1faea5d8d64cf777f6da] source-hash-8908f10fe3b6566db469a1f2bd2674d8d0425165
    git bisect good a5b688ef540d5189edde1faea5d8d64cf777f6da
    # first bad commit: [2faf2a5126e3ccf78be3d6619b571358bb7af742] source-hash-2523972f6a066488c649ab97dcba4f458126f18b

I am setting O/S = All and whiteboard bibisected.
Comment 6 Matthew Francis 2015-08-26 03:50:19 UTC
(see comment 5)

Adding Cc: to nthiebaud@gmail.com; Could you possibly take a look at this one? Thanks
Comment 7 Norbert Thiebaud 2015-08-26 05:23:36 UTC
I've checked out 2523972f6a066488c649ab97dcba4f458126f18b

built it.. and ran the scenario..
can't reproduce here...
I did not switch to a Chinese locale though...
is that relevant ? 

in any case 2523972f6a066488c649ab97dcba4f458126f18b is
+++ b/vcl/source/gdi/impimagetree.cxx
@@ -181,7 +181,7 @@ bool ImplImageTree::doLoadImage(OUString const & name, OUString const & style, B
     }

     if (found)
-        maIconSet[maCurrentStyle].maIconCache[name.intern()] = std::make_pair(localized, bitmap);
+        maIconSet[maCurrentStyle].maIconCache[name] = std::make_pair(localized, bitmap);

     return found;
 }

there is _no_ way that is related with the described problem.
Comment 8 Matthew Francis 2015-08-26 06:21:06 UTC
Could be less reproducible than it first appears then, or could just have missed the target - I'll take a look when I get a minute
Comment 9 Matthew Francis 2015-08-26 11:02:10 UTC
The below appears to be the correct commit
Adding Cc: to yogesh.bharate@synerzip.com; Could you possibly take a look at this one? Thanks

    commit 8865ed2efecd03722d10e522265f31c99b13b2bb
    Author:     yogesh.bharate001 <yogesh.bharate@synerzip.com>
    AuthorDate: Mon Apr 27 15:08:16 2015 +0530
    Commit:     Joren De Cuyper <jorendc@libreoffice.org>
    CommitDate: Mon May 11 19:02:45 2015 +0000
    
        tdf#90812: rPr is not exported after roundtrip.
    
        Problem Description:
        XML Difference: In sharedStrings.xml
        Original file :
        <r>
          <rPr>
              <sz val="11"/>
              <color rgb="FFFF0000"/>
              <rFont val="Calibri"/>
              <family val="2"/>
              <scheme val="minor"/>
          </rPr>
           <t>Red</t>
        </r>
    
        Roundtrip file:
        <r>
          <t>Red</t>
        </r>
    
        rPr is missing in roundtrip file.
    
        Conflicts:
        	sc/qa/unit/subsequent_export-test.cxx
    
        Change-Id: I79efd0f8f1a735ef7e4ebd3fda220b3e339ea91c
        Reviewed-on: https://gerrit.libreoffice.org/15548
        Reviewed-by: Joren De Cuyper <jorendc@libreoffice.org>
        Tested-by: Joren De Cuyper <jorendc@libreoffice.org>
Comment 10 Robert Pollak 2015-10-05 14:58:53 UTC
I can reproduce this with LO 5.0.2.2 on German Windows 7 Pro. According to LO's info dialog, my locale is "en-US (de_AT)".
Comment 11 Katarina Behrens (CIB) 2015-11-04 11:48:58 UTC
@Kevin: The sample document you've posted is actually very useful and I'd like to use it in an unit (bugfix) test. What it means is that I'll be placed in LibO git repository. Can I do that?

(I know there are absolutely no confidential or personal data in it, but I thought I'd ask anyway)
Comment 12 Andras Timar 2015-11-04 14:53:54 UTC
*** Bug 93115 has been marked as a duplicate of this bug. ***
Comment 13 Kevin Suo 2015-11-05 01:23:31 UTC
(In reply to Katarina Behrens (CIB) from comment #11)
Of course you can use it, please go ahead, thanks.
Comment 14 Commit Notification 2015-11-05 18:11:54 UTC
Katarina Behrens committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=dd971d008780186c88cc33df3610ed8cc9f0a2f4

tdf#92296: Fix off-by-one formatting of text runs on OOXML export

It will be available in 5.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 15 Commit Notification 2015-11-09 14:58:20 UTC
Katarina Behrens committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=df6d3e9d91c25bd1ef7c7cba0f1af63895890411

Related tdf#92296, tdf#90812: Make this test more strict

It will be available in 5.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 16 Robinson Tryon (qubit) 2015-12-17 09:14:34 UTC Comment hidden (obsolete)
Comment 17 Fran 2016-02-16 13:50:30 UTC
Saving the attached ods file as xlsx or xls, closing it and opening the new xlsx or xls, some formating changes: original font (Liberation Sans) changes to Arial; Text alignment vertical changes from 'Default' to 'Align bottom'; Reference Edge in Text Orientation changes from 'Text Extension from Lower Cell Border' to 'Text Extension Inside Cell'; and if you activate 'Wrap text automatically' on a formated cell before saving the file as xlsx, when reopening it the 'Wrap text' option is unchecked.
Comment 18 Fran 2016-02-16 13:52:24 UTC
Sorry, I forgot the system info:

Version: 5.1.0.3
Build ID: 5e3e00a007d9b3b6efb6797a8b8e57b51ab1f737
CPU Threads: 4; OS Version: Windows 6.1; UI Render: default; 
Locale: es-ES (es_ES)

Thanks!
Comment 19 Kevin Suo 2016-02-17 02:49:37 UTC
(In reply to Fran from comment #17)
I confirm the original bug (the char formatting within the cell) is fixed in
Version: 5.1.1.1 (x64)
Build ID: c43cb650e9c145b181321ea547d38296db70f36e
CPU Threads: 4; OS Version: Windows 6.19; UI Render: default; 
Locale: zh-CN (zh_CN)

Mark as FIXED.

@Fran: Please file new bug for the other issues, as they are different issue.
Comment 20 Kevin Suo 2016-02-17 02:56:03 UTC
As this is a data-loss bug and also affects version 5.0, I request to backport this to version 5.0.

Still reproduciable with
Version: 5.0.5.2
Build ID: 55b006a02d247b5f7215fc6ea0fde844b30035b3
Locale: zh-CN (zh_CN)
Comment 21 Katarina Behrens (CIB) 2016-02-17 10:08:14 UTC
Out of those issues listed in comment 17, only font (Liberation Sans -> Arial) might be related to this one, everything else is handled by quite different code paths, so likely orthogonal to this

5.0 backport waits for approval here: 

https://gerrit.libreoffice.org/#/c/22420/
Comment 22 Fran 2016-02-17 10:26:37 UTC
I reopened this bug because the problems in the comment 17 only happen in cells with format in part of the text. 

Cells that contain unformatted text or text with the same format (all text underlined, all text red, ...) are saved OK. So I thought the new problem could be related to this one.

I'm sorry, I will open a new bug.

Thanks!
Comment 23 Fran 2016-02-17 11:21:33 UTC
In any case, this bug is not completely fixed. With LO 5.1.1 RC1, saving the test ods file to xlsx and opening it with MS Excel Viewer 2007, the spaces in cells A2:A6 are still missing.
Comment 24 Katarina Behrens (CIB) 2016-02-17 11:34:57 UTC
> In any case, this bug is not completely fixed. With LO 5.1.1 RC1, saving the
> test ods file to xlsx and opening it with MS Excel Viewer 2007, the spaces
> in cells A2:A6 are still missing.

That would be bug 96912
Comment 25 Commit Notification 2016-02-18 07:44:37 UTC
Katarina Behrens committed a patch related to this issue.
It has been pushed to "libreoffice-5-0":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=14279955b467241a70b61f993c334757c2b4a756&h=libreoffice-5-0

tdf#92296: Fix off-by-one formatting of text runs on OOXML export

It will be available in 5.0.6.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 26 Timur 2016-02-18 09:51:00 UTC
(In reply to Katarina Behrens (CIB) from comment #21)
> font (Liberation Sans -> Arial) might be related to this one
Could you look into that, or there should be another ticket?

Thank you for the backport.
Comment 27 m.a.riosv 2016-02-28 04:15:47 UTC
*** Bug 98247 has been marked as a duplicate of this bug. ***