Bug 77919 - FORMATTING: Bibliography, character styles are lost after update; also ODF export generates corrupted <text:index-entry-bibliography text:style-name="_33__20_"/> elements that eventually grow to enormous sequence of encoded characters
Summary: FORMATTING: Bibliography, character styles are lost after update; also ODF ex...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
4.2 all versions
Hardware: Other All
: medium major
Assignee: Matteo Casalin
URL:
Whiteboard: target:6.1.0 target:6.0.4 target:5.4....
Keywords: bisected, dataLoss, needUITest, regression
: 89644 94890 (view as bug list)
Depends on:
Blocks: Bibliography
  Show dependency treegraph
 
Reported: 2014-04-25 09:29 UTC by pierre-yves samyn
Modified: 2019-09-19 12:47 UTC (History)
15 users (show)

See Also:
Crash report or crash signature:


Attachments
Document with bibliography (13.82 KB, application/vnd.oasis.opendocument.text)
2014-04-25 09:29 UTC, pierre-yves samyn
Details
Screenshot Entries tab : tooltip ok, listbox to "None" (13.36 KB, image/png)
2014-04-25 09:32 UTC, pierre-yves samyn
Details
example of bibliography (10.98 KB, application/vnd.oasis.opendocument.text)
2015-12-11 02:51 UTC, Robert Gonzalez MX
Details
Bibliography test - known good file (10.88 KB, application/vnd.oasis.opendocument.text)
2017-12-31 09:44 UTC, Mihkel Tõnnov
Details

Note You need to log in before you can comment on or make changes to this bug.
Description pierre-yves samyn 2014-04-25 09:29:52 UTC
Created attachment 97948 [details]
Document with bibliography

Hello

The character styles assigned to the structure elements are lost after update of a bibliography 

Steps to reproduce
1. Open the attached document

In the bibliography, some character styles have been assigned to structure elements :
- Sh (short name)  Emphasis
- Au (author) Strong Emphasis
- Ti (title) Endnote Anchor

2. Right click on the bibliography then Edit Index
3. Select Entries tab, click on  Sh element in the structure line.

Expected result : tooltip & character style listbox display "Emphasis"

Actual result : tooltip Ok but character style listbox display <None>

Same with Au & Ti elements (correct style name in tooltip, None in the listbox).

3. Close the dialog
4. Right click on the bibliography then Update Index

Formating is lost...

My Platform : windows XP & Version: 4.2.3.3
Build ID: 6c3586f855673fa6a1576797f575b31ac6fa0ba3 

I set status to New because verified (qa-fr) on:
Windows 7/64 & Version: 4.2.4.1
Build ID: d4c441391e20647b3d2e8dde4d20aa868e77e515

Ubuntu 12.04 x86_64 Canonical : LibreOffice 3.5.7.2 Version ID 350m1(Build:2)

Ubuntu 12.04 x86_64 LibreOffice  4.2.4.1 [Build ID: d4c441391e20647b3d2e8dde4d20aa868e77e515]

Last known version where it worked on windows: 3.5.7.2

Regards
Pierre-Yves
Comment 1 pierre-yves samyn 2014-04-25 09:32:04 UTC
Created attachment 97949 [details]
Screenshot Entries tab : tooltip ok, listbox to "None"
Comment 2 Buovjaga 2015-03-03 15:28:00 UTC
Reproduced with attachment 97948 [details], already with 3.3.0.

Win 7 Pro 64-bit, LibO Version: 4.4.1.2
Build ID: 45e2de17089c24a1fa810c8f975a7171ba4cd432
Locale: fi_FI

Ubuntu 14.10 64-bit
LibreOffice 3.3.0 
OOO330m19 (Build:6)
tag libreoffice-3.3.0.4
Comment 3 Buovjaga 2015-04-17 18:28:09 UTC
*** Bug 89644 has been marked as a duplicate of this bug. ***
Comment 4 Buovjaga 2015-04-17 18:29:20 UTC
*** Bug 90621 has been marked as a duplicate of this bug. ***
Comment 5 Gordo 2015-04-18 12:41:51 UTC
I would like to add that every time a document with a bibliography is saved, even without adding any new biblio entries or updating the index, the index-entry-bibliography text:style-name in content.xml has extra data added to it.

In the Entries tab of Edit Index/Table, every time the tab is switched back and forth or the Type is selected back and forth, or even just visiting the Entries tab and OKing, causes extra data to be added to the style-name for each element in the Type.  You can see it in the tooltips.

Version: 4.4.2.2
Build ID: c4c7d32d0d49397cad38d62472b0bc8acff48dd6
Comment 6 Robert Gonzalez MX 2015-12-11 02:51:07 UTC
Hello.

I reproduce this in Version: 5.0.4.1
Build ID: 2def61bcbb29a7a8611b833682fe1291910b11ad
Locale: es-MX (es_MX)

Version: 5.1.0.0.beta1+
Build ID: 9b6d78cc68e9dc644bf2d06b358049de3cd3ee8f
Threads 8; Ver: Windows 6.2; Render: default; 

TinderBox: Win-x86@62-merge-TDF, Branch:libreoffice-5-1, Time: 2015-12-03_20:08:55
Locale: es-MX (es_MX)

On Windows XP SP3 and Windows 10
Comment 7 Robert Gonzalez MX 2015-12-11 02:51:57 UTC
Created attachment 121216 [details]
example of bibliography
Comment 8 Mihkel Tõnnov 2016-06-04 22:32:46 UTC
*** Bug 94890 has been marked as a duplicate of this bug. ***
Comment 9 Mihkel Tõnnov 2016-06-04 22:35:18 UTC
Still reproducible in 5.1.4.1.
Comment 10 Robert Gonzalez MX 2016-06-05 00:44:53 UTC
Hi there.

Reproducible with Version: 5.2.0.0.beta1
Build ID: 1e9933ef611c66bcded94b84052543c78cf1c223
CPU Threads: 8; OS Version: Windows 6.2; UI Render: default; 
Locale: es-MX (es_MX)
Comment 11 Mihkel Tõnnov 2017-02-19 22:04:52 UTC
Still affects 5.3.0.3.
As an added "bonus", now also last character of text fields gets eaten away (filed that as bug 106092).
Comment 12 Mihkel Tõnnov 2017-12-28 11:37:50 UTC
Still an issue in 6.0.0 rc 1.
Comment 13 Mihkel Tõnnov 2017-12-30 23:38:10 UTC
I tested old versions 3.6.7, 4.0.x, 4.1.x, and 4.2.0 beta 1:
* the last version where bibliography entries' character style is interpreted normally, is 4.1.6 rc 2,
* the first one where it breaks is 4.2.0 beta 1.
So this is a regression introduced in 4.2 branch.

(In reply to Buovjaga from comment #2)
> Reproduced with attachment 97948 [details], already with 3.3.0.
> 
> Win 7 Pro 64-bit, LibO Version: 4.4.1.2
> Build ID: 45e2de17089c24a1fa810c8f975a7171ba4cd432
> Locale: fi_FI
> 
> Ubuntu 14.10 64-bit
> LibreOffice 3.3.0 
> OOO330m19 (Build:6)
> tag libreoffice-3.3.0.4

That document was created with LibO 4.2.4 rc 1 - i.e. the document itself is already broken and can't be used for testing like this. You can try with a blank document in 3.3.0 if you want to confirm.

Setting version to 4.2 and also adding "regression" keyword.
Comment 14 Mihkel Tõnnov 2017-12-31 09:44:34 UTC
Created attachment 138761 [details]
Bibliography test - known good file
Comment 15 Mihkel Tõnnov 2017-12-31 09:49:00 UTC Comment hidden (obsolete)
Comment 16 Mihkel Tõnnov 2017-12-31 09:51:13 UTC Comment hidden (obsolete)
Comment 17 Mihkel Tõnnov 2018-01-01 17:35:20 UTC
Bibisected (properly this time), and identified a commit that indeed plausibly could have introduced this bug (unlike the commit pointed out by my first attempt where I used a sparse bibisect repository without realising it - sorry!)
 a89830a7c9075fb7180a502bf2cbd7c86490b387 is the first bad commit

    commit f4fd558ac9d61fe06aa0f56d829916ef9e5ee7b9
    Author:     Matteo Casalin <matteo.casalin@yahoo.com>
    AuthorDate: Fri Aug 9 22:11:53 2013 +0200
    Commit:     Tor Lillqvist <tml@iki.fi>
    CommitDate: Tue Aug 13 08:05:41 2013 +0000
    
        String to OUString and some reduction of scope
    
        Change-Id: Ia760c5f3f8c158bea30be3102841a66330e5180a
        Reviewed-on: https://gerrit.libreoffice.org/5339
        Reviewed-by: Tor Lillqvist <tml@iki.fi>
        Tested-by: Tor Lillqvist <tml@iki.fi>


Diff of changes by this commit:
https://cgit.freedesktop.org/libreoffice/core/commit/?id=f4fd558ac9d61fe06aa0f56d829916ef9e5ee7b9
Comment 18 Mihkel Tõnnov 2018-01-01 17:39:02 UTC
Matteo, this was your patch (admittedly long time ago), would you be able to take a look?
Comment 19 Matteo Casalin 2018-01-04 12:46:30 UTC
Hi Mihkel, thanks for bibisecting and reporting.
I will try to address it in the next few days.
Comment 20 Matteo Casalin 2018-01-21 21:30:30 UTC
Short update: I'm still working on it... I fixed two issues with that commit, but have not seen the "expected results" with the known good file, yet.
Comment 21 Michael Stahl (allotropia) 2018-04-08 14:48:14 UTC
current master: with attachment from comment #14 :

* the tooltip text is correctly generated in SwTokenWindow::CreateQuickHelp
  but i can't get the tooltip to actually show up on screen
  which looks like an unrelated issue or maybe it's intentional
  and there is some obscure setting that would enable tooltips

* the list box does show the style "Emphasis" for the "Au" button
  of the "Book" entry template, which is the only one in the document
  that has a character style applied

the attachment from the description has corrupted style name references
on its entry templates so it's no surprise that nothing is displayed.

but, on 5.4.6.2 i'm not seeing the style in the listbox;
would be nice to bisect the fix and backport it.
Comment 22 Michael Stahl (allotropia) 2018-04-08 14:51:35 UTC
don't have a 6.0 right now to test so adding backportRequest for both 5.4 and 6.0
Comment 23 Buovjaga 2018-04-08 17:06:58 UTC
Thanks, Michael. I feel weird adding a bibisectRequest to a bisected bug, but here goes :)
Comment 24 Mihkel Tõnnov 2018-04-08 22:34:14 UTC
(In reply to Michael Stahl from comment #21)
> current master: with attachment from comment #14 :
> 
> * the tooltip text is correctly generated in SwTokenWindow::CreateQuickHelp
>   but i can't get the tooltip to actually show up on screen
>   which looks like an unrelated issue or maybe it's intentional
>   and there is some obscure setting that would enable tooltips

I see the tooltip perfectly fine on current master - ?!

> ...
> would be nice to bisect the fix and backport it.

Bibisected; and while I couldn't get Git to show me a proper commit message for some reason, it did consistently identify this commit as the first "fixed" one:

    61d60487533bc93c1207747719d6382eaaa2c4f7 is the first fixed commit
    commit 61d60487533bc93c1207747719d6382eaaa2c4f7
    Author: Jenkins Build User <tdf@pollux.tdf>
    Date:   Sun Feb 11 16:09:25 2018 +0100

        source 34b98af8e5a4e568d8316700bea1ce604d825ce8

Diff of changes by this commit: https://cgit.freedesktop.org/libreoffice/core/commit/?id=34b98af8e5a4e568d8316700bea1ce604d825ce8

Matteo, you said back in January that you "fixed two issues with [old commit f4fd558ac9d61fe06aa0f56d829916ef9e5ee7b9], but have not seen the "expected results" with the known good file, yet" - does this new commit mean you figured it out? Or is there something still to be done?
Comment 25 Michael Stahl (allotropia) 2018-04-09 12:47:07 UTC
thanks for bisecting that!

backported to libreoffice-6-0 as commit c6e8460a5b47fa6fa971dde2a89e80662b6e97ae
backported to libreoffice-5-4 as commit df041d902eafb1e273eb360103252994bbd2cde2
Comment 26 Commit Notification 2018-04-09 19:47:52 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=829674fa46c8c07584022b84bbc8c3d877b895c2

Related: tdf#77919 GetPosPixel() is relative to parent, not grandparent

It will be available in 6.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 27 Commit Notification 2018-04-10 08:37:13 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-6-0":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=7d95ef515c604bf67e68fd4824aa0073d8a76d07&h=libreoffice-6-0

Related: tdf#77919 GetPosPixel() is relative to parent, not grandparent

It will be available in 6.0.4.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 28 Matteo Casalin 2018-04-11 20:32:32 UTC
(In reply to Mihkel Tõnnov from comment #24)
> Matteo, you said back in January that you "fixed two issues with [old commit
> f4fd558ac9d61fe06aa0f56d829916ef9e5ee7b9], but have not seen the "expected
> results" with the known good file, yet" - does this new commit mean you
> figured it out? Or is there something still to be done?

Hi Mihkel, all,
   commit https://cgit.freedesktop.org/libreoffice/core/commit/?id=34b98af8e5a4e568d8316700bea1ce604d825ce8 and (previous) commit https://cgit.freedesktop.org/libreoffice/core/commit/?id=e3e9870aacdd8df1f449b802cd9dc3fbd083182f are the two I was mentioning back at the beginning of January, which did not provide the expected results (with the "known good file"). I decided to push them anyway, not referring to this bug issue since in my tests they didn't solve it, and I was now slowly trying to see if they introduced the issue and some later commit added further deviations.
I didn't find any other regression in my commit from 2013 (these 2 are definitively enough) so since the issue is fixed I see nothing else still to be done.
Comment 29 Michael Stahl (allotropia) 2019-09-19 12:47:07 UTC
this bug was difficult to find, add some more detailed description of ODF impact