Bug 145215 - The logic of Cross Reference representing Paragraph Numbering is different between MSOffice and LibreOffice
Summary: The logic of Cross Reference representing Paragraph Numbering is different be...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
7.3.0.0 alpha0+
Hardware: All Linux (All)
: medium normal
Assignee: Vasily Melenchuk (CIB)
URL:
Whiteboard: target:7.3.0 target:7.2.5
Keywords:
Depends on:
Blocks:
 
Reported: 2021-10-19 05:26 UTC by softdev1029
Modified: 2022-03-25 07:59 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
Test Doc (9.35 KB, application/vnd.oasis.opendocument.text)
2021-10-19 06:32 UTC, Kevin Suo
Details
Incorrect-Cross-Reference (617.65 KB, application/vnd.openxmlformats-officedocument.wordprocessingml.document)
2021-10-19 08:39 UTC, softdev1029
Details
Expected Results (pdf of last pages generated from MSO) (222.98 KB, application/pdf)
2021-10-19 09:33 UTC, Kevin Suo
Details
Incorrect-Cross-Reference-Simple-version (550.70 KB, application/vnd.openxmlformats-officedocument.wordprocessingml.document)
2021-10-25 16:27 UTC, softdev1029
Details
Incorrect-Cross-Reference-Simple-version-PDF-in-MSO-translated (170.03 KB, application/pdf)
2021-10-27 04:02 UTC, Kevin Suo
Details
Incorrect-Cross-Reference-Simple-version-PDF-in-lo-highlighted (343.26 KB, application/pdf)
2021-10-27 04:04 UTC, Kevin Suo
Details

Note You need to log in before you can comment on or make changes to this bug.
Description softdev1029 2021-10-19 05:26:59 UTC
Description:
The logic of Cross Reference representing Paragraph Numbering is different between MSOffice and LibreOffice.

For example, suppose that the following paragraphs exist.

2. Title
2.1 Sub Title
2.1.1 Paragraph
      (i) Sentence

Cross Reference with [Paragraph number in full context] is:

    2.1.1 (i) in MSOffice and 2.1 (i) in LibreOffice.



Steps to Reproduce:
1. Make paragraphs with multiple numbering levels
2. Make Cross Reference with [Paragraph number in full context]

Actual Results:
LibreOffice generates the Cross Reference different from MSOffice.

Expected Results:
LibreOffice should generate the same Cross Reference as MSOffice.


Reproducible: Always


User Profile Reset: No



Additional Info:
LibreOffice generates the Cross Reference different from MSOffice.
Comment 1 Kevin Suo 2021-10-19 06:32:05 UTC
Created attachment 175828 [details]
Test Doc

I do not reproduce with the above test document on master. Cross Referenced text is showing as "1.1.1.(i)" correctly. Could you please re-confirm?

Version: 7.3.0.0.alpha0+ / LibreOffice Community
Build ID: c7f0fddb4a1a0dcc0dd21d2a27d67d4e75a8195c
CPU threads: 8; OS: Linux 5.14; UI render: default; VCL: gtk3
Locale: zh-CN (zh_CN.UTF-8); UI: zh-CN
Build Platform: Fedora34@X64, Branch:master, bibisect-linux-64-7.3-CN
Calc: threaded
Comment 2 softdev1029 2021-10-19 08:39:39 UTC
Created attachment 175829 [details]
Incorrect-Cross-Reference

Please test the attached `Subscription Agreement.docx` document file.
At the end of file, it has a page to use multiple cases of Cross-Reference.

For example, `X-ref = Paragraph number = 2.2.2(i)`

Reproduce Steps:
1. Insert a new paragraph before `2.2.2` so that the paragraph `2.2.2` changes to `2.2.3`
2. Click ctrl+a to select all.
3. Click F9 to update all Cross References
4. Then, `X-ref = Paragraph number = 2.2(i)` but it should be `X-ref = Paragraph number = 2.2.3(i)`
Comment 3 softdev1029 2021-10-19 08:40:47 UTC
./soffice --version
LibreOfficeDev 7.3.0.0.alpha0 deae8bab12513ad09f432e5f65ea1768064b4d2e
Comment 4 softdev1029 2021-10-19 08:51:20 UTC
Build Platform: Ubuntu 20.04
Comment 5 Kevin Suo 2021-10-19 09:33:19 UTC
Created attachment 175830 [details]
Expected Results (pdf of last pages generated from MSO)

Attached are the last several pages with the cross references, generated from MSO 2010.

The cross references in the attached docx file, both on FILEOPEN and EDITING, are totally broken. There seems to be multiple issues:
1. Some are shown as errors (reference not found).
2. Some are shown correct values, but when you edit the reference field the dialog shown does not correspond to what it should be.
3. Some are shown wrong values. For instance, in the "Cross reference tests in relation to clause 2.2.1(i)" section, "2. X-ref = Paragraph number = 2.2.1(i)" is shown as "X-ref = Paragraph number = 2.(i)". It seems that only the headings level are used and the paragraph numbering levels are ignored (note that the source paragraphs use style "Heading 2" are thus there are only two levels shown in the reference result.
Comment 6 Kevin Suo 2021-10-19 10:13:00 UTC
Bibisected and found the behaviour has changed in the following range: e5ac6caf9efa3d3a8cb82dbc59ef00b193facdc4..7c95c16deba975b0b282ccea1fbfcfadd7aa28b3

7c95c16deba9 (tdf#143107 related: ensure correct order of which ids, 2021-06-28)
aa5c6d127559 (new ODF numbered list parameter loext:num-list-format, 2021-06-14)
99ef46fdb2a9 (tdf#143105 Don't export empty URLs to PDF, 2021-06-28)
f79a8f578ab5 (Update git submodules, 2021-06-28)

and the following two commit seems to be related:
commit aa5c6d127559912ad60a63fbd972b78fb8f9691b
Author: Vasily Melenchuk <vasily.melenchuk@cib.de>
Date:   Mon Jun 14 14:27:56 2021 +0300
    new ODF numbered list parameter loext:num-list-format

Before (and including) commit e5ac6caf9efa3d3a8cb82dbc59ef00b193facdc4, in the "Cross reference tests in relation to clause 2.2.1(i)" section, "2. X-ref = Paragraph number = 2.2.1(i)" is shown as "X-ref = Paragraph number = 2.2(i)". Although still not good, but its "better" than now.

From commit 7c95c16deba975b0b282ccea1fbfcfadd7aa28b, it is shown as  "X-ref = Paragraph number = 2.(i)", becoming worse.

Adding Vasily Melenchuk in cc, would you please take a look? Thanks.
Comment 7 softdev1029 2021-10-20 02:06:55 UTC
Hi, Kevin Suo, thanks.
I tried to test e5ac6caf9 or 7c95c16 commit but I can't find them in git history of the core repo.
Anyway,  in the "Cross reference tests in relation to clause 2.2.1(i)" section, "2. X-ref = Paragraph number = 2.2.1(i)" is shown as "X-ref = Paragraph number = 2.2(i)".
It is still not correct.

Can you let me know which files and which lines are related to these Cross-Reference features?
Then, I'd like to investigate them together.

If you give some more explanation how to implement the Cross-Reference feature, it will be highly helpful for me.

Thanks.
Comment 8 Kevin Suo 2021-10-20 04:17:26 UTC
> I tried to test e5ac6caf9 or 7c95c16 commit but I can't find them in git history of the core repo.

The bibisected source hash range is:
https://cgit.freedesktop.org/libreoffice/core/log/?qt=range&q=e5ac6caf9efa3d3a8cb82dbc59ef00b193facdc4..7c95c16deba975b0b282ccea1fbfcfadd7aa28b3

and the one which has caused the change of behaviour (i.e., from "2.2(i)" to "2.(i)") should be:
https://cgit.freedesktop.org/libreoffice/core/commit/?id=aa5c6d127559912ad60a63fbd972b78fb8f9691b

> Can you let me know which files and which lines are related to these Cross-Reference features?
Unfortunately, at this moment I have no idea.

> If you give some more explanation how to implement the Cross-Reference feature, it will be highly helpful for me.
I have no idea on the implementation details either.
Comment 10 softdev1029 2021-10-21 01:41:37 UTC
Hi Kevin Suo and Vasily Melenchuk,

Please let me know if you have any luck with fixing this issue.

Best regards
Comment 11 Kevin Suo 2021-10-21 02:13:07 UTC
(In reply to softdev1029 from comment #10)
It is out of my ability to fix this.
Vasily Melenchuk, any comments?
Comment 12 Commit Notification 2021-10-23 19:29:54 UTC
Vasily Melenchuk committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/b66ed4cb3707800b8bd9ab0f397c226037e10fbf

tdf#145215: docx: do not initialize prefix/suffix and include levels

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 13 Vasily Melenchuk (CIB) 2021-10-25 08:41:47 UTC
I DID NOT test the fix with original document: this 53 pages document with cross-references is freezing LO with dbgutil. I assume that smaller testcase (used as a test in patch) is reproducing the problem correctly.
Comment 14 Kevin Suo 2021-10-25 09:03:51 UTC
(In reply to Vasily Melenchuk (CIB) from comment #13)
Normally for me if the master dbgutil build freezes on FILEOPEN I will delete the ./instdir/usr dir.
Comment 15 Kevin Suo 2021-10-25 11:47:09 UTC
I tested and there seems still a problem in the above/blow section.
Comment 16 Kevin Suo 2021-10-25 11:51:13 UTC
softdev1029 would you please prepare a simplified test document for the above/below and the in-table cases?
Comment 17 softdev1029 2021-10-25 16:27:06 UTC Comment hidden (obsolete)
Comment 18 softdev1029 2021-10-27 01:56:13 UTC
Hi Kevin Suo and Vasily Melenchuk,

Please let me know if the simple test document I made is enough.

And please let me know how I can test it.

I cloned https://github.com/LibreOffice/core.git but I can't find the proper commit or branch.
Should I clone https://git.libreoffice.org/core instead?

Also, I checked https://dev-builds.libreoffice.org/daily/master/Linux-rpm_deb-x86_64@86-TDF/ but Ubuntu version doesn't exist yet.

How could I get the Ubuntu version?
Best regards
Victor
Comment 19 Kevin Suo 2021-10-27 03:44:00 UTC
(In reply to softdev1029 from comment #18)

There are two ways to test:

1. You can build the master branch from source, and then run the binary.

To build, you should:

(1) Install the build dependencies.
See:
https://wiki.documentfoundation.org/Development/BuildingOnLinux#Build_dependencies

(2) clone the source code:
$ git clone https://git.libreoffice.org/core

(3) cd to your source code directory, and run autogen.sh to generate the build configuration files:

$ cd /path/to/libreoffice/source/core

# Run autogen.sh with the default build options:
$ ./autogen.sh

Please note that for development builds you should enable the "--enable-dbgutil" option, so that you can see the debugging output in terminal as well as using the debug symbols when you do a gdb backtrace:

# Run autogen.sh with some build options, e.g.
$ ./autogen.sh --enable-dbgutil

You may see error message indicating missing build dependencies. If it is the case, then you should check and install the proper dependencies.

(4) When autogen.sh is successful, then make:
$ make

The first build may take up to 2-8 hours, depending on hour powerful your machine is. The following build (without running `make clean`) between commits will take several minutes up to several hours depending what has changed.

You may install ccache to speed-up your rebuild, see https://wiki.documentfoundation.org/Development/BuildingOnLinux#ccache
$ sudo apt-get install ccache

(5) After the `make` is finished, you can run the binary from the instdir:
$ ./instdir/program/soffice

Before `make`, if you want to checkout the commit which have fixed this bug, you can run:
$ git checkout b66ed4cb3707800b8bd9ab0f397c226037e10fbf

Detailed build instructions can be found in:
https://wiki.documentfoundation.org/Development/BuildingOnLinux
and
https://wiki.documentfoundation.org/Development/GenericBuildingHints

2. Alternatively, you can use a daily build:
https://dev-builds.libreoffice.org/daily/master/
But yes, I see from the link that currently the daily build is only until 2021-10-14, so you may have to wait if you are not able to build by yourself.
Comment 20 Kevin Suo 2021-10-27 04:02:21 UTC
Created attachment 175943 [details]
Incorrect-Cross-Reference-Simple-version-PDF-in-MSO-translated

This is how the simplified test document look like in MSO 2010. Since I use a Simplified Chinese language version of MSO, the above/below are shown as Chinese and thus I have translated them in the attached PDF file.
Comment 21 Kevin Suo 2021-10-27 04:04:23 UTC
Created attachment 175944 [details]
Incorrect-Cross-Reference-Simple-version-PDF-in-lo-highlighted

This is how the simplified test document looks like in LO master with the patch applied.

While most of the cross references are now correct, the above/below ones are still incorrect, see the yellow-highlighted parts in the attached PDF file.
Comment 22 Commit Notification 2021-11-18 07:13:48 UTC
Vasily Melenchuk committed a patch related to this issue.
It has been pushed to "libreoffice-7-2":

https://git.libreoffice.org/core/commit/43f8051986042a2c7943c716e764f4cfdd81723c

tdf#145215: docx: do not initialize prefix/suffix and include levels

It will be available in 7.2.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 23 Christian Lohmaier 2021-12-06 13:28:48 UTC
7.2.4 was a hotfix release, updating target in status-whiteboard