Created attachment 157557 [details]
Example case and proposal

Do you know about Create from "Additional Styles"?

On your Table of Contents:

1.  right-click, Edit Index, Type tab
2.  Click on Additional Styles  and Assign Styles

You can decide yourself which paragraph styles should be included in the table of contents.

Maybe that does what you want?

No, this is not what I want.
Please, have a look at the attached example.

Looking at your document, it becomes clear to me: Your aim is to create a kind of TOC extract for the current level, but with the context (here: chapter). I agree, that this is not possible with "create from additional styles"

Would be a good enhancement I think for authors of scientific literature.

cc: Design-Team for further input.

Yes, create a kind of TOC extract for the current level.

I left the upper levels in the subchapter ToC example, but I do *not* need them. In fact it would be very easy to remove them through "create from additional styles".

Created attachment 157613 [details]
Example case and proposal (updated)

Following your comments, I updated the attached proposal file.

I still don't get the use case. The option "For (Chapter)" allows to have a ToC restricted to the current chapter. For example

1. Heading A
 ToC
  1. 
   1.1
   1.2
   1.3
1.1 Heading Aa
1.2 Heading Ab
1.3 Heading Ab
2. Heading B
2.1. Heading Ba

and what I understand is that you want something like

1. Heading A
1.1 Heading Aa
 ToC
  1.1 
   1.1.1
   1.1.2
   1.1.3
1.1.1 HAa1
1.1.2 HAa2
1.1.3 HAa3
1.2 Heading Ab
1.3 Heading Ab
2. Heading B
2.1. Heading Ba

I would agree if this layout is needed in general or, even better, defined as a requirement for a layout. In other words: What scientific literature uses this style?

I understood the request as follows:

There is a "normal" TOC at the beginning or end of the book and an extract at the begionning of every subchapter, like

1. Heading A
1.1 Heading Aa
 ToC
  1.1 
   1.1.1
   1.1.2
   1.1.3

I remember that I saw this in scientific literature (not often), but I don't want to search within all my literature. But I hope, Emanuele can give an example.

Here is the use case.

The Italian Ministry of Interior is using Libreoffice since 2015 to write the Fire Safety Code. See it the current revision at: bit.ly/codicepi2019

This is a 300 pages fire safety regulatory text, that has:
- sections (Heading level 1),
- each section is composed of chapters (Heading level 2),
- each chapter is made of paragraphs (Heading level 3),
- and each paragraph is made of sub-paragraphs (Heading level 4).

Like this:

Section G
  Chapter G.1
    G.1.1 Definitions
    G.1.2 Language
    ...
  Chapter G.2
    G.2.1 Structure
    G.2.2 Procedure
    ...

Section S
  Chapter S.1
    S.1.1 Preface
    S.1.2 Performance levels
    ...
  Chapter S.2
    S.1.1 Preface
    S.1.2 Performance levels
    ...

Section V
  ...

What we need is a general ToC at the beginning of the full book:
(obtained automatically with Edit Index > Type > Create ToC for Entire document)

*General ToC*
  Section G
    Chapter G.1
    Chapter G.2
    ...
  Section S
    Chapter S.1
    ...
  Section V
    ...

we also need Section ToCs at the beginning of each section:
(obtained automatically with Edit Index > Type > Create ToC for Chapter, that is for level 1)

*Section G ToC*
  Chapter G.1
  Chapter G.2
  ...

*Section S ToC*
  Chapter S.1
  Chapter S.2
  ...

and we also need Chapter ToCs at the beginning of each chapter:

*Chapter G.1 ToC*
  Paragraph G.1.1
  Paragraph G.1.2
  ...

*Chapter G.2 ToC*
  Paragraph G.2.1
  Paragraph G.2.2
  ...

In the example document (bit.ly/codicepi2019), this third type of ToC is currently handmade, because the Edit Index > Type > Create ToC for... menu is limited to "Entire document" or "Chapter".

This is why it is not possible to create a ToC starting from any other lower level.

This request of enhancement is similar to what is described in:
https://bugs.documentfoundation.org/show_bug.cgi?id=112301

I remain at disposal for further info/test.

Many thanks for sharing your use case. One last question: More options lead to more complexity and we should try to work with the existing functions. 

Would it solve your problem when the "For (Chapter)" option not start at level 1 but where the cursor is placed? Meaning you put the cursor after 2.1 and it adds 2, at 3.1.1 it takes 3.1. By doing this we don't need to change the UI or much on the help. 

The alternative is "For (level) (2)" bringing more (hopefully unnecessary) flexibility on cost of the parameter. Do you need to insert, for example, 2.2.2 after 2.1?

Thank you for your immediate answer.

IMHO, taking the ToC level from the position of the cursor is fine, and should not break any existing document. It is indeed a great idea!

(In reply to Emanuele Gissi from comment #12)
> IMHO, taking the ToC level from the position of the cursor is fine, and
> should not break any existing document. It is indeed a great idea!

Sic fiat! Removing UX...

Any news?

I am updating the link to the use case of this feature.
It's the current Italian Fire Safety Code.

Here it is at Github:

https://github.com/codicepi/codicepi-edit

See the pdf export at:

https://github.com/codicepi/codicepi-edit/releases

Thank you!

Difficult to implement, Miklos? Could imagine a funding for companies.

My guess would be "not trivial, but possible".

Any news? Could I help out with financing?

Created attachment 175950 [details]
Another example of the bug with a master document odm

I am adding another example that possibly show very clearly why this bug impacts on the usability of master documents

(In reply to Emanuele Gissi from comment #18)
> Any news? Could I help out with financing?

You can contact a company that provides professional support. See for a list https://www.libreoffice.org/get-help/professional-support/. Adding also Andreas, maybe he's interested.

I have seen that the Document Foundation assigned a tender to implement master document fixes (#202106-02) to Allotropia:
https://blog.allotropia.de/2021/09/16/libreoffice-master-document-fixes/

I suggest adding this bug that work, I could finance the additional request.

(In reply to Emanuele Gissi from comment #21)
> I suggest adding this bug that work, I could finance the additional request.

Thorsten, what do you think?

I will take a look at this, after finishing some work.

Any good news?

I will give it a try in the next days and will report back. Busy times ...

Thank you, let me know if I can support you

So after investigating the request, I checked the code:

In void SwTOXBaseSection::UpdateOutline [1] the outlines will be updated for the index. The same function checks if a TextNode belongs to the own chapter or not if the create index for chapter is set [2]. 

If we change the current behaviour and create the index from the cursor position, the IsFromChapter and lcl_FindChapterNode functions [3] have to be changed in order to find the first parent node from the current node where the cursor is.

I am not an expert in the ToC index creation and I have to check how to retrieve the current cursor position in order to get the correct parent node for the index. Maybe we have to set also the starting level to make this working correctly.

[1] https://opengrok.libreoffice.org/xref/core/sw/source/core/doc/doctxm.cxx?r=5f9ffc31#1234
[2] https://opengrok.libreoffice.org/xref/core/sw/source/core/doc/doctxm.cxx?r=5f9ffc31#1253
[3] https://opengrok.libreoffice.org/xref/core/sw/source/core/doc/doctxm.cxx?r=5f9ffc31#857

Created attachment 176453 [details]
Result after the first WIP commit

WIP Patch for this enhancement: https://gerrit.libreoffice.org/c/core/+/125727

Above you have the current status of the implementation for this enhancement request. There are still some presentation and implementation issues. For instance, the ToC should not be intended if there is no top level entry for the current index (see ToC of Paragraph H4 and others). Then, do we take into account the level of evaluation ("Evaluate up to level" in th UI)?

Last but not least, I do not have enough insight in the ToC creation. For instance, how does the SwRootFrame const*const pLayout influence the generation of the ToC? How are other ToCs (UpdateTemplate, UpdateContent, UpdateTable, ...) influenced with this change? Maybe a developer with more insight could give some advice in order to proceed because I will sureley miss many points making this work :)

Thanks to Emanuele for the support!

WIP-Patch: https://gerrit.libreoffice.org/c/core/+/125727

Comments are highly appreciated :)

I applied your patch and recompiled LO.

All ToCs and indexes are created correctly.
No regression appears when ToCs and indexes are generated on the entire document, or first outline level.

Only problem left is that the "List of Figures" created per "Chapter" does not correctly select the figures contained by the cursor heading, and display all the figures of the document.

I attach the updated test case, with updated comments in it.
See the yellow inline comments for issues.

Created attachment 176935 [details]
Updated test case for the proposed patch

Created attachment 176936 [details]
Updated test case for the proposed patch (generated PDF)

(In reply to Andreas Heinisch from comment #29)
> WIP-Patch: https://gerrit.libreoffice.org/c/core/+/125727
> 
> Comments are highly appreciated :)

Pretty cool stuff. I can load the document on MSO and get the ToC at the correct place (MSO is not activated so I cannot test further). 

But while loading with older versions (7.2.3.2) works, the app goes into an infinite loop on interaction. Meaning, when I click on the ToC it occupies 100% CPU and I have to kill the application. Happens also when the ToC is placed on top.

(In reply to Heiko Tietze from comment #33)
> But while loading with older versions (7.2.3.2) works, the app goes into an
> infinite loop on interaction.

Apparently an issue with the user profile. Since it also happened for documents created with 7.2 I tested in safe mode and the issue is gone. No problem with the ToC.

Now the indices for figures should work as well. I have to make some additional tests and figure out how the pLayout and IsParaPropsNode work together.

I tested your patchset 5.
This is another step in the right direction.

Now the list of figures starts correctly from the first figure of the current heading.

There is only one problem left: the list of figures does not stop with the end of the heading, but continues until the last figure of the document.

I attach a screenshot of the test case to show what I mean.

Created attachment 177026 [details]
Screenshot of the list of figures (good start, broken ending)

Created attachment 177043 [details]
Page Frame of H4

Thx for the testing! Indeed I only checked the first and the last figure index, so I didn't find the non-working indices. In [1] there should be a check if the bottom of the frame area of the current chapter is greater or equal than the bottom of the frame area of the figure caption (pPgFrame->getFrameArea().Bottom() >= pMyFrame->getFrameArea().Bottom()).

However, the frame area of the chapter node contains only the first page and not the area until there is a new chapter node. So, if the figure caption is on another page, but the requested chapter is still valid, it won't be shown after the check will be added to [1]. If it's on the same page, it appears.

Maybe the experts here know a method where I can retrieve the area of the entire chapter? Otherwise, I have to get through all the chapter nodes with a level smaller than the chapter node requested and expand the bottom frame accordingly.

What do you think?

[1] https://gerrit.libreoffice.org/c/core/+/125727/5/sw/source/core/doc/doctxm.cxx#774

Created attachment 177160 [details]
Bug in the current implementation

After countless debugging session, I finally figured out what causes the problem of a wrong figure index. If you zoom out in writer the layout frame of a figure may be below of the layout frame of the requested header.

Is there another possibility to check if a figure is within a heading?

This could even lead to wrong figure indices in the current implementation, if a document is shown in a multipage layout.

I added a bug report about this problem (tdf#146441). I hope I may solve the enhancement for the single-page view, but in multiple-page and book view this will be still a problem.

I checked the other bug and confirmed it on:
Version: 7.2.4.1
Build ID: 20(Build:1)
CPU threads: 4; OS: Linux 5.15; UI render: default; VCL: gtk3
Locale: it-IT (it_IT.UTF-8); UI: it-IT
Calc: threaded

So, after some days of debugging and figuring out how the Nodes work, I am unable to solve the issues about the ToC about figures. Mainly they depend on Bug 146441. Until someone will solve it, the development of this enhancement unfortunately is stalled :(

After Bug 146441 is solved, this one can be adapted. I made some superficial tests of the indexes in Word where they work there as well.

So, IMHO I resolved the case for the list of figures for the single page view. I uploaded a new patch an tested it. Maybe someone could test it extensivley, since I still not entirley know the impact of the pLayout variable (a SwRootFrame) and the IsParaPropsNode check. It has something to do with merged paragraphs ...

Thank you Andreas!

I updated my complete test case (attached) and performed the following tests:

1. in single page view, update and export to pdf from patched LO;
2. in multi page view, update and export to pdf from patched LO;
3. in book view, update and export to pdf from patched LO;
4. export to docx, open with word online, update all indexes and export to pdf;

Then I compared the pdfs on https://draftable.com/
For your convenience I attach the following side-by-side comparisons:

1. single page vs multi page;
2. single page vs book;
3. single page vs word;

Here are the results of this test in brief:

- patched LO in single view: *OK, perfect!*
- patched LO in multi page and book view: list of figures is still broken (as per https://bugs.documentfoundation.org/show_bug.cgi?id=146441)
- in word the indexes are perfect, as in LO single view. There are some other problems in formatting pages and respecting caption numbering.

Ema

Created attachment 177408 [details]
Main test case odt

Created attachment 177409 [details]
Main test case pdf (single vs multipage side-by-side)

Created attachment 177410 [details]
Main test case pdf (single vs book side-by-side)

Created attachment 177411 [details]
Main test case pdf (single vs word online)

Ops, I swapped the "vs multipage" and "vs book" files. Sorry. But they are ok

Forgot to add. In the side-by-side comparison the differences are marked in red (deletions) and green (additions)

So I tried to fix the failing test for this enhancement. In _XDocumentIndex._update() the test fails using java. The very same test XDocumentIndex::testUpdate() (make CppunitTest_sw_apitests) succeeds without any issues using c++.

So I tried to reproduce the error in the beanshell macro editor:
import com.sun.star.uno.UnoRuntime;
import com.sun.star.text.XTextDocument;
import com.sun.star.text.XText;
import com.sun.star.text.XTextRange;
import com.sun.star.beans.XPropertySet;
import com.sun.star.text.XTextContent;
import com.sun.star.beans.XPropertySet;

import com.sun.star.lang.XMultiServiceFactory;

import com.sun.star.text.XDocumentIndex;
import com.sun.star.text.XTextCursor;


//////////////////////////// SETUP //////////////////////////////////
oDoc = XSCRIPTCONTEXT.getDocument();
XTextDocument xTextDoc = UnoRuntime.queryInterface(XTextDocument.class, oDoc);
XMultiServiceFactory xDocMSF = UnoRuntime.queryInterface(XMultiServiceFactory.class, xTextDoc);

Object idxDoc = xDocMSF.createInstance("com.sun.star.text.DocumentIndex");
XTextContent xTC = UnoRuntime.queryInterface(XDocumentIndex.class, idxDoc);

XText oText = xTextDoc.getText();
XTextCursor oCursor = oText.createTextCursor();

// Insert index into text document
oText.insertTextContent(oCursor, xTC, false);
oCursor.gotoEnd(false);

////////////////////////////// TEST ///////////////////////////////////////

XText xText = xTextDoc.getText();
XTextRange xTR = xText.getEnd();
xTR.setString("IndexMark");

Object idxMark = xDocMSF.createInstance("com.sun.star.text.DocumentIndexMark");
XTextContent xTCMark = UnoRuntime.queryInterface(XTextContent.class, idxMark);
xText.insertTextContent(xTR, xTCMark, true);

String contentBefore = xTC.getAnchor().getString();
xTC.update();
String contentAfter = xTC.getAnchor().getString();

xText = xTextDoc.getText();
xTR = xText.getEnd();

xTR.setString(contentBefore);
xTR = xText.getEnd();
xTR.setString(contentAfter);

return 0;

Even there the test case works as intended.

So I tried to move the cursor at the end of the file before the update of the index itself in _XDocumentIndex._update():

XTextDocument xTextDoc = (XTextDocument)tEnv.getObjRelation("TextDoc");
XText xText = xTextDoc.getText();
XTextCursor xTextCursor = xText.createTextCursor();
xTextCursor.gotoEnd(false);
String contentBefore = oObj.getAnchor().getString();
log.println("Content before: '" + contentBefore + "'");

oObj.update();

waitForEventIdle();

String contentAfter = oObj.getAnchor().getString();
log.println("Content after: '" + contentAfter + "'");

The tests fails with:
LOG> Content after: 'Alphabetical IndexNew'

Without the patch it works with:
LOG> Content after: 'Alphabetical IndexNew
IndexMark       1'

So imho, the test works in both cases, but I am not able to correct it. Maybe our test hero Xisco may provide some advices to fix this test.

So I checked other java and python index tests and imho, there is something wrong with the implementation of the test handling which has nothing to do with this enhancement.

For instance, check https://gerrit.libreoffice.org/c/core/+/128194/9/sw/qa/uitest/writer_tests5/tdf106899.py where the index does not update using the following commands:

xDocumentIndexes = document.DocumentIndexes
self.assertEqual(xDocumentIndexes.getCount(), 1)
self.assertEqual(xDocumentIndexes.hasByName("Alphabetical Index1"), True)
xDocumentIndex = xDocumentIndexes.getByName("Alphabetical Index1")

count = xDocumentIndexes.getCount()
for i in range(0,count):
    xDocumentIndexes.getByIndex(i).update            
xDocumentIndex.refresh()
xDocumentIndex.refresh
xDocumentIndex.update()
xDocumentIndex.refresh

Only the self.xUITest.executeCommand(".uno:UpdateAllIndexes") updates the index reliably. Maybe I am able to find out why these commands won't work, but python macros are not that easy to debug.

(In reply to Andreas Heinisch from comment #53)
> So I checked other java and python index tests and imho, there is something
> wrong with the implementation of the test handling which has nothing to do
> with this enhancement.

Maybe Xisco can help.

With no connection about this specific bug, I like very much the colaboration between all involved here, it's the perfect way of solving all bugs. 
- The reporter have tested the patches
- The develper is taking input from the reporter and others
- UX was very well involved here

And I feel the research on this bug will solve other bugs also.

Thanks to all! Great work toghether!

You are right. This is what makes me so proud of being part of this Community.
We are so close to solving this issue and many more.

Just the final push is missing.
But I am confident!

I will finish another issue. Then I hope I can reload the index like in [1] and retrieve the text in [2] where I had the same problems using python. Imho, the update index has some issues.

[1] https://gerrit.libreoffice.org/c/core/+/128194/11/sw/qa/uitest/writer_tests5/tdf106899.py#33
[2] https://gerrit.libreoffice.org/c/core/+/128194/11/sw/qa/uitest/writer_tests5/tdf106899.py#37

I finally managed to debug and overcome this error. I think the enhancement is no ready. I still have to refactor the code though (renaming variables, documenting the new functionality, etc.)

now ready ofc.

Great, indeed!
How can I help you? Testing?

Imho no testing needed. I just added, that if there is no chapter at all in the document, all figures are contained in the ToC. That's why the test failed in the first place.

After I refactored the code, I think it can be merged, but maybe some of the experts express their opinions about this change and the implementation of this enhancement.

I refactored the code and now I think we should again test the enhancement. I tested it locally, but I cannot test it extensivley with MS-Word. 

In addition, I would need some input what the next steps are in order to complete this enhancement (automatic testing, interoperability tests, updating the help, etc.). Thank you for your help!

(In reply to Andreas Heinisch from comment #62)
> In addition, I would need some input what the next steps are in order to
> complete this enhancement (automatic testing, interoperability tests,
> updating the help, etc.). Thank you for your help!

Ilmari or Seth can help you here.

(In reply to Heiko Tietze from comment #63)
> (In reply to Andreas Heinisch from comment #62)
> > In addition, I would need some input what the next steps are in order to
> > complete this enhancement (automatic testing, interoperability tests,
> > updating the help, etc.). Thank you for your help!
> 
> Ilmari or Seth can help you here.

I think the point was content - Andreas has experience editing Help.

Any update on this?
The patch is available and it works well.
How to integrate it into the next release?

I confirm I built the patch, exported some test cases to docx and imported the exported files to MS Word.

Andreas Heinisch committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/1cac7d18ab8561f129a30d6c93c0f9f1d7868e01

tdf#130318 - Use the actual cursor position to create ToC "for chapter"

It will be available in 7.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.

Thank you for all your help and thank you for merging it!

I will still have to implement a test case covering this new feature and adding a note in the release notes of LO 7.4.

I added the note in the release notes of LO 7.4:
https://wiki.documentfoundation.org/ReleaseNotes/7.4

I have been testing the LO daily live on Linux, and it seems to work well.

(In reply to Emanuele Gissi from comment #70)
> I have been testing the LO daily live on Linux, and it seems to work well.

=> VERIFIED FIXED