Bug 138518 - Update Cross References builds wrong directories and missformats paragraphs
Summary: Update Cross References builds wrong directories and missformats paragraphs
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
7.1.0.0.alpha1+
Hardware: x86-64 (AMD64) All
: medium normal
Assignee: Michael Stahl (allotropia)
URL:
Whiteboard: target:7.2.0 target:7.1.4
Keywords: bibisected, bisected, regression
Depends on:
Blocks: Fields-Cross-Reference
  Show dependency treegraph
 
Reported: 2020-11-26 21:13 UTC by Stefan_Lange_KA@T-Online.de
Modified: 2021-08-11 08:47 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments
zip file, contains a test odt document and some pdf files to show the test results (9.61 MB, application/x-zip-compressed)
2020-11-26 21:16 UTC, Stefan_Lange_KA@T-Online.de
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan_Lange_KA@T-Online.de 2020-11-26 21:13:43 UTC
Description:
In a specific text document (more than one directory "Table of Contents", see appended sample) the update of cross references results in wrong built directories and missformatted paragraphs (divided where it should not be divided). The problem exists in LOdev 7.1 beta and in LOdev 7.2 alpha, but not in LO 7.0.3 and in LOdev 7.0.4 (builds until 2020-11-25).

Steps to Reproduce:
1. open document "Test_paragraphs_divided.odt" from appended zip file 
--> The document looks good as it was created with LOdev 7.0.4: It consists of 112 Pages. All directories contain the correct page numbers and no paragraaphs (= chapters) are divided wrongly.
2. Update (= rebuild) all cross references e.g. by tab "References" -> "Cross References" -> All
3. Update all cross references repeatedly

Actual Results:
After the first update of cross referenes the document consists of 118 pages, because some chapters are divided wrongly (1.1.2.36, 1.1.2.46, 1.1.2,60, 1.1.2.61, 1.1.2.69, 1.1.3.12 and 2.11: divided and continued on the next page).
Both overview directories (built until level 1 resp. 3) contain some wrongly built page numbers: "Übersicht" -> "Obektive" at page 78 (correct 77) and "Fotoapparate - Übersicht" -> "Altix III" page 19 (corr. 18) and "Altix IIIA" page 70 (corr. 69), see "Test_paragraphs_divided_Update_CrossRef_All_x1_LOdev_7.2.0.pdf" in the zip file

Expected Results:
The directories should not be changed, because the were correct already, and the chapters should not be divided. So it is in LO resp. LOdev until 7.0.4 


Reproducible: Always


User Profile Reset: No



Additional Info:
When the cross references are updated repeatedly, the result are different from the result of the first update (2nd update: also 118 pages, but other wrong page number, 3rd update: 116 pages and other wrong page numbers, 4th update: 119 pages and other wrong page numbers), see "Test_paragraphs_divided_Update_CrossRef_All_x2_LOdev_7.2.0.pdf" etc. in the zip file!
Comment 1 Stefan_Lange_KA@T-Online.de 2020-11-26 21:16:11 UTC
Created attachment 167602 [details]
zip file, contains a test odt document and some pdf files to show the test results
Comment 2 Stefan_Lange_KA@T-Online.de 2020-11-26 22:30:05 UTC
Regression
----------
The problem didn't exist in all builds of LOdev 7.1.0.0.alpha0+ and in the builds of LOdev 7.1.0.0.alpha1+ until 2020-11-16.
It also cannot be reproduced with newest build of LOdev 7.0.5 from 2020-11-26.

The bug was introduced in LOdev 7.1.0.0.alpha1+ with the build from 2020-11-17:
Version: 7.1.0.0.alpha1+ (x64)
Build ID: 312a33b7636334f6ce3b6d1702bc5d3e45215601
CPU threads: 4; OS: Windows 10.0 Build 20262; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: threaded
Comment 3 Stefan_Lange_KA@T-Online.de 2020-12-12 09:58:12 UTC
To check if the problem still exists I have repeated the tests with recent versions of LOdev and LO.

The bug was reproduced with
Version: 7.1.0.0.beta1+ (x64)
Build ID: 8f7c26f166df33589f3522b43dbea4169a9ffd1e
CPU threads: 4; OS: Windows 10.0 Build 21277; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: CL
and with
Version: 7.2.0.0.alpha0+ (x64)
Build ID: 35e471bb4d1388cf5afcdcee214cf5111edf44e3
CPU threads: 4; OS: Windows 10.0 Build 21277; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: threaded
(both built on 2020-12-12)

The bug didn't occur in
Version: 7.0.4.2 (x64)
Build ID: dcf040e67528d9187c66b2379df5ea4407429775
CPU threads: 4; OS: Windows 10.0 Build 21277; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: CL
and in
Version: 7.0.5.0.0+ (x64)
Build ID: 2e6ecdfec95ac18a197cf31a67455cd103b73198
CPU threads: 4; OS: Windows 10.0 Build 21277; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: CL
(both built on 2020-12-11)
Comment 4 Dieter 2020-12-12 13:13:41 UTC
I confirm it with

Version: 7.1.0.0.beta1 (x64)
Build ID: 828a45a14a0b954e0e539f5a9a10ca31c81d8f53
CPU threads: 4; OS: Windows 10.0 Build 19042; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: en-GB
Calc: threaded

but not with

Version: 7.0.3.1 (x64)
Build ID: d7547858d014d4cf69878db179d326fc3483e082
CPU threads: 4; OS: Windows 10.0 Build 19042; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: en-GB
Calc: threaded
Comment 5 Stefan_Lange_KA@T-Online.de 2020-12-22 19:48:38 UTC
The bug still exists in the recent build of LOdev 7.2.0
Version: 7.2.0.0.alpha0+ (x64)
Build ID: 4041c68ea59181f1c4774c356809066d2051db41
CPU threads: 4; OS: Windows 10.0 Build 21277; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: threaded
(from 2020-12-22_00.24.32)
and - worse - also in the RC1 build of LO 7.1.0
Version: 7.1.0.1 (x64)
Build ID: b585d7d90ab863bf29b2d110c174c0c2a98f3ee4
CPU threads: 4; OS: Windows 10.0 Build 21277; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: threaded

IMHO this is a serious bug. With it Writer of LO 7.1 will be unusable - at least for me.
Comment 6 Stefan_Lange_KA@T-Online.de 2021-01-17 20:15:21 UTC
The bug is still present in
Version: 7.1.0.2 (x64) / LibreOffice Community
Build ID: 53d68d29d90fd16448721a60aad68c28ff0809f5
CPU threads: 4; OS: Windows 10.0 Build 21292; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: threaded
Comment 7 Stefan_Lange_KA@T-Online.de 2021-01-27 15:10:24 UTC
In the LOdev 7.2.0.0 build downloaded and installed from /daily/master/Win-x86_64@tb77-TDF/2021-01-20_05.01.14/ something was changed related to Bug 138518: 
When the update (= rebuild) of all cross references is started, first the wrong total number of pages is shown. But after this the number is changed in several steps and when the update action is finished the correct total number of pages is shown and there are no missformatted paragraphs. 
This is not an optimal behavior because it may need a lot of time (depending on the document size), but it is acceptable.

In the LOdev 7.1.1.0.0 build downloaded and installed from /daily/libreoffice-7-1/Win-x86_64@tb77-TDF/2021-01-27_13.50.45/ 
Version: 7.1.1.0.0+ (x64) / LibreOffice Community
Build ID: 1e6853f007994c803802fda563b70bb71063488b
CPU threads: 4; OS: Windows 10.0 Build 21296; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: threaded
the erronous behavior described in the bug report is still present.

When the change from LOdev 7.2.0.0 would be pushed to LOdev 7.1.1.0.0 I would close the bug.
Comment 8 Stefan_Lange_KA@T-Online.de 2021-01-29 15:34:41 UTC
The bug is still present in
Version: 7.1.0.3 (x64) / LibreOffice Community
Build ID: f6099ecf3d29644b5008cc8f48f42f4a40986e4c
CPU threads: 4; OS: Windows 10.0 Build 21301; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: threaded
Comment 9 Stefan_Lange_KA@T-Online.de 2021-01-29 16:19:06 UTC
The bug is also still present in 
Version: 7.1.1.0.0+ (x64) / LibreOffice Community
Build ID: f490652153498b6184c6589263212b3b3326900b
CPU threads: 4; OS: Windows 10.0 Build 21301; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: threaded
(from /daily/libreoffice-7-1/Win-x86_64@tb77-TDF/2021-01-29_15.51.08/)

Can it be that the change
https://gerrit.libreoffice.org/c/core/+/109554 "Update all" should do a full reformatting"
was pushed to LOdev 7.2 but not yet to LOdev 7.1.1 and LO 7.1.0?
I assume this change has solved the problem in LOdev 7.2.
Comment 10 Stefan_Lange_KA@T-Online.de 2021-02-01 11:08:22 UTC
Update to Comment 7: The problem is not solved completely in LOdev 7.2.0. 
There are no missformatted paragraphs and the total number of pages is correct, but the Tables of Contents are not updated correctly.
Example: In the test document "Test_paragraphs_divided.odt" after "Update all"  the paragraph "2 Objektive" is located on page 72 but in the main ToC for the paragraph is displayed page 78.
Additional problem: It is not signalized e.g. by hourglass cursor that the action of reformatting is still in progress resp. it is not signalized when the action is finished.
Comment 11 Stefan_Lange_KA@T-Online.de 2021-02-17 18:18:19 UTC
Bibisect information:

c98acbb9671253ab8bbf4be75c3fa9993c5deceb is the first bad commit
commit c98acbb9671253ab8bbf4be75c3fa9993c5deceb
Author: Norbert Thiebaud <nthiebaud@gmail.com>
Date:   Mon Nov 16 14:53:43 2020 -0800

    source sha:b9ef71476fd70bc13f50ebe80390e0730d1b7afb

    source sha:b9ef71476fd70bc13f50ebe80390e0730d1b7afb

and from https://git.libreoffice.org/core/+/b9ef71476fd70bc13f50ebe80390e0730d1b7afb:

commit	b9ef71476fd70bc13f50ebe80390e0730d1b7afb	[log]
author	Michael Stahl <Michael.Stahl@cib.de>	Fri Nov 13 20:52:28 2020 +0100
committer	Michael Stahl <michael.stahl@cib.de>	Mon Nov 16 16:51:19 2020 +0100
Comment 12 Commit Notification 2021-04-22 16:44:32 UTC
Michael Stahl committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/c799de145f7e289f31e3669646e5bd12814e6c5e

tdf#138518 sw: layout: avoid moving flys forward prematurely

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 13 Michael Stahl (allotropia) 2021-04-22 16:46:41 UTC
finally had some time to look into this one, should be fixed now on master

the fix requires commit  eb85de8e6b61fb3fcb6c03ae0145f7fe5478bccf
"sw: layout: if fly's anchor moves forward, move fly off SwPageFrame"
as well, but that one by itself wasn't sufficient.
Comment 14 Stefan_Lange_KA@T-Online.de 2021-04-23 11:52:05 UTC
I have made first tests with
Version: 7.2.0.0.alpha0+ (x64) / LibreOffice Community
Build ID: 0e35aa3de1cb97bf5a5f9918bfe09063097e064a
CPU threads: 4; OS: Windows 10.0 Build 21364; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: threaded

I have tested [References -> Update Indexes and Tables] as well with the attached test document as with other test documents of mine and also with my "productive" document.
Until now all was OK. I haven't found any problems neither with misformatted paragraphs nor with wrong updated directories.
Comment 15 Commit Notification 2021-04-23 15:05:40 UTC
Michael Stahl committed a patch related to this issue.
It has been pushed to "libreoffice-7-1":

https://git.libreoffice.org/core/commit/8feac9601cfe35ee0966776bab15d122206f154e

tdf#138518 sw: layout: avoid moving flys forward prematurely

It will be available in 7.1.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 16 Stefan_Lange_KA@T-Online.de 2021-04-24 11:12:47 UTC
Test with 
Version: 7.1.4.0.0+ (x64) / LibreOffice Community
Build ID: 8feac9601cfe35ee0966776bab15d122206f154e
CPU threads: 4; OS: Windows 10.0 Build 21364; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: CL
was OK too.
Many thanks for fixing this annoying bug! Now I can switch from LO 7.0 to LO 7.1.
Comment 17 Stefan_Lange_KA@T-Online.de 2021-04-24 14:35:45 UTC
Is there a way to push the fix to 7.1.3 too? RC2 is announced for the time from April 26 until May 2.
Comment 18 Commit Notification 2021-05-07 09:02:15 UTC
Michael Stahl committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/533a998e540b0f04068c876dde0e74adc3f79c93

tdf#138518 sw: layout: fix assert on ooo43913-1.doc

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 19 Commit Notification 2021-05-07 15:46:41 UTC
Michael Stahl committed a patch related to this issue.
It has been pushed to "libreoffice-7-1":

https://git.libreoffice.org/core/commit/7eb12f8a09dd67168ba42058b99d8ab58a5c7b0a

tdf#138518 sw: layout: fix assert on ooo43913-1.doc

It will be available in 7.1.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 20 Commit Notification 2021-05-07 16:38:24 UTC
Michael Stahl committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/59d96acec8c4d9e472daa3e2c287b3a754e01817

tdf#138518 sw: layout: unbreak fdo80206-1.doc

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 21 Commit Notification 2021-05-08 18:43:04 UTC
Michael Stahl committed a patch related to this issue.
It has been pushed to "libreoffice-7-1":

https://git.libreoffice.org/core/commit/dcea18e66e61c6f411b9bf6498a0ffc374161484

tdf#138518 sw: layout: unbreak fdo80206-1.doc

It will be available in 7.1.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 22 Stefan_Lange_KA@T-Online.de 2021-05-10 19:12:04 UTC
I have tested with my own documents (test document "Test_paragraphs_divided.odt" attached to this bug and my "productive" documents) and also with "kubus.docx" attached to Bug 142080 as well with
Version: 7.2.0.0.alpha0+ (x64) / LibreOffice Community
Build ID: 1675a68526c43c6c6e4dc850ee911f0c1de75c88
CPU threads: 4; OS: Windows 10.0 Build 21376; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: threaded
as with
Version: 7.1.4.0.0+ (x64) / LibreOffice Community
Build ID: a3e59beed1f1fbe783d29c9036d7849849ec109a
CPU threads: 4; OS: Windows 10.0 Build 21376; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: CL

Result: I haven't found any problems!

But because I hadn't any problems already after the fixes from 2021-04-22 and 2021-04-23 (Bug 138518) resp. 2021-05-05 (Bug 142080) and my results have not changed with the newest fixes I am not sure if the fixes work as designed.
Comment 23 Dieter 2021-05-21 17:31:38 UTC
(In reply to Stefan_Lange_KA@T-Online.de from comment #22)
> Result: I haven't found any problems!

So let's take this as verification. Michael, thanks for fixing it!
=> VERIFIED FIXED