Bug 143268 - Slow response in document with many tables after Table tree is expanded in Navigator
Summary: Slow response in document with many tables after Table tree is expanded in Na...
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
3.3.0 release
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:7.3.0
Keywords: bibisectNotNeeded, haveBacktrace, perf, regression
Depends on:
Blocks: Writer-Tables
  Show dependency treegraph
 
Reported: 2021-07-08 18:48 UTC by David
Modified: 2021-09-27 13:41 UTC (History)
7 users (show)

See Also:
Crash report or crash signature:
Regression By:


Attachments
Document with many tables (53.92 KB, application/octet-stream)
2021-07-20 06:50 UTC, David
Details
Revised many tables document (55.09 KB, application/octet-stream)
2021-07-26 11:15 UTC, David
Details
Screen shot of document (44.84 KB, image/png)
2021-07-26 12:57 UTC, David
Details
Screen shot showing sidebar without table navigation (18.79 KB, image/png)
2021-07-26 15:15 UTC, David
Details
perf data (3.67 MB, application/octet-stream)
2021-08-18 20:58 UTC, David
Details
perf.data printout (651.39 KB, text/plain)
2021-08-19 14:21 UTC, David
Details
perf data (2.25 MB, application/octet-stream)
2021-08-20 00:16 UTC, David
Details
perf.data printout (463.33 KB, text/plain)
2021-08-20 00:19 UTC, David
Details
flamegraph image (196.32 KB, application/octet-stream)
2021-08-20 00:20 UTC, David
Details

Note You need to log in before you can comment on or make changes to this bug.
Description David 2021-07-08 18:48:43 UTC
I have a document that contains a large number of tables.  I've finally located the cause for a severe lag in the response of the document when scrolling or editing.  Version 6 with this document works fine.  Version 7 has the lag.  I haven't determined a specific version number where the problem began, but from what I recall, I think it was with 7.0.0.  

In trying to determine what I could do that might improve the speed, I found that enabling the Content Navigation View in the Navigator, with the cursor outside of a table (so that the headings view remains displayed), immediately caused the response to be as quick as the response on version 6.

I then discovered that because most of the editing is done in tables, the focus in the Navigator will switch to the Tables navigation and the extreme lag begins.  Version 6 does not change the focus from the Headings navigation even if editing inside a table.

I then discovered that in the context menu when right-clicking on the Headings entry provides an option to disable Outline Tracking.  Would it be possible that such an option to disable Table Tracking for the Tables view would prevent the focus from going to the Tables and thereby causing the lag as the program tries to keep up? 

Content Navigation View can be enabled to only have the headings displayed in the Navigator, but I do use the various other views also, such as sections and comments, so I like have them all displayed.
Comment 1 Buovjaga 2021-07-19 15:13:08 UTC Comment hidden (obsolete)
Comment 2 David 2021-07-20 06:50:34 UTC Comment hidden (obsolete)
Comment 3 Telesto 2021-07-20 06:56:39 UTC
Repro 
Version: 7.3.0.0.alpha0+ (x64) / LibreOffice Community
Build ID: 3d18cae102e16b85fb8787f5ec3b086bfa2bd7b8
CPU threads: 4; OS: Windows 6.3 Build 9600; UI render: Skia/Raster; VCL: win
Locale: nl-NL (nl_NL); UI: nl-NL
Calc: CL

So typing something into a table (holding a key) with sidebar -> navigator active
Comment 4 Buovjaga 2021-07-20 14:02:31 UTC
I can't investigate this because I don't see any lag in the bibisect repositories, even though I see it in 7.1.4 on Linux :(
Comment 5 David 2021-07-20 14:16:35 UTC
The simplest solution might be to add an option to disable Table Tracking, just like there is one to disable Outline Tracking for headings.
Comment 6 Buovjaga 2021-07-20 14:25:55 UTC
(In reply to David from comment #0)
> In trying to determine what I could do that might improve the speed, I found
> that enabling the Content Navigation View in the Navigator, with the cursor
> outside of a table (so that the headings view remains displayed),
> immediately caused the response to be as quick as the response on version 6.

Btw. if I do this with my 7.1.4, the lag does not go away. The lag for me is seen easiest by going to a line with a heading and pressing down on a character. Every couple of seconds it stops for a moment.
Comment 7 David 2021-07-20 15:13:27 UTC Comment hidden (obsolete)
Comment 8 David 2021-07-20 15:33:04 UTC
For the purpose of this bug, the document could be simplified by changing all the headings to use the Default Paragraph Style.  Having them set to a heading style probably complicates matters.
Comment 9 David 2021-07-23 13:01:26 UTC
Created new bug 143499 to address the lag in a document with many headings.
Comment 10 Kevin Suo 2021-07-26 06:34:42 UTC Comment hidden (obsolete)
Comment 11 David 2021-07-26 11:15:15 UTC
Created attachment 173852 [details]
Revised many tables document
Comment 12 David 2021-07-26 11:32:05 UTC
(In reply to Kevin Suo from comment #10)
> From the comments above, I read that the slowness when typing into the
> tables is never reproduced by someone else, correct?
> 
> The slowness is when you put your cursor within the headings paragraphs and
> then type something. If that is the case, then it is bug 143499 which I have
> already successfully bibisected. There is no slowness when you put cursor in
> the tables and then start typing.
> 
> Set this to NEEDINFO. David could you please clarify?
Incorrect!

A new document that does not have outline headings has been attached.  To see lag: enable navigator, disable *Heading* Content Navigation View.

Probably related to bug 143499, which deals only with the lag with many headings.  But because of the confusion, I made that a separate bug.

Reproduced by Telesto, comment 3:
>So typing something into a table (holding a key) with sidebar -> navigator active
Comment 13 Buovjaga 2021-07-26 11:42:35 UTC
(In reply to David from comment #12)
> A new document that does not have outline headings has been attached.  To
> see lag: enable navigator, disable *Heading* Content Navigation View.

I don't have to do anything extra to see the small lag every couple of seconds when pressing down on a character. On the other hand, the Content Navigation View is deactivated upon opening the document.

NixOS
Version: 7.3.0.0.alpha0+ / LibreOffice Community
Build ID: 67e47070a7580a17804adce812cc2f98bfe7b51f
CPU threads: 16; OS: Linux 5.13; UI render: default; VCL: x11
Locale: fi-FI (fi_FI.UTF-8); UI: en-US
Calc: threaded
Comment 14 David 2021-07-26 12:45:42 UTC
I have a quad-core, 3Ghz processor and the lag is severe in 7.x with Tables navigation active.  The lag will increase with the number of tables added. I show almost no lag in version 6.x, even on a much older machine.
Comment 15 David 2021-07-26 12:57:46 UTC
Created attachment 173854 [details]
Screen shot of document

Added screen shot showing sidebar view.
Comment 16 Buovjaga 2021-07-26 14:31:28 UTC Comment hidden (obsolete)
Comment 17 David 2021-07-26 15:15:06 UTC
Created attachment 173862 [details]
Screen shot showing sidebar without table navigation
Comment 18 David 2021-07-26 15:22:37 UTC
(In reply to Buovjaga from comment #16)
> David: do you experience the lag, if you launch from the command line with
> 
> SAL_USE_VCLPLUGIN=gen libreoffice

Yes. By a severe lag, I mean that with normal typing it is basically unusable, not that there is merely a hiccup every few seconds while holding down a key.

Added a screen shot of sidebar when there is no lag.  With the Table navigation view inactive, there is no noticeable lag whether in gen mode or not.
Comment 19 David 2021-08-06 11:10:47 UTC Comment hidden (obsolete)
Comment 20 Buovjaga 2021-08-06 11:16:56 UTC
(In reply to David from comment #19)
> The issue with the headings has been fixed (bug 143499).  Would a similar
> fix resolve the issue for this bug?

I think this might benefit from profiling data using a build with symbols: https://wiki.documentfoundation.org/Development/How_to_debug#Performance_debugging_.28perf.29
Comment 21 David 2021-08-06 11:28:03 UTC Comment hidden (obsolete)
Comment 22 Buovjaga 2021-08-06 11:29:38 UTC
(In reply to David from comment #21)
> According to what I understand from the instructions, LibreOffice needs to
> be built using -debug-symbols but that it shouldn't be the debug build.  Is
> there a pre-compiled version of such a build that can be downloaded and used
> for testing?

Unfortunately not.
Comment 23 David 2021-08-18 20:58:17 UTC Comment hidden (obsolete)
Comment 24 Buovjaga 2021-08-19 06:57:48 UTC
(In reply to David from comment #23)
> Created attachment 174399 [details]
> perf data
> 
> Perf data has been attached.  Is this the information needed?  The data has
> been recorded while pressing a key twice while within a table.

I'm not sure if it's possible to view this on another machine.

Can you install Hotspot, open the .data file and view the tab "Flame Graph"? https://repology.org/project/hotspot-perf/versions

For me, the calls appear with ?? and then the .so file name. Perhaps you see the actual function names?
Comment 25 David 2021-08-19 14:21:59 UTC
Created attachment 174418 [details]
perf.data printout

I attached a printout of what I get with 'perf report'.   

My output from hotspot is hundreds of megs for just a short recording.  Hotspot also reports a lot of missing debug symbols.  

I used the command './autogen.sh --enable-dbgutil --enable-symbols="sw/"' to configure the compile.  Should I compile with different parameters?
Comment 26 Buovjaga 2021-08-19 16:38:40 UTC
(In reply to David from comment #25)
> Created attachment 174418 [details]
> perf.data printout
> 
> I attached a printout of what I get with 'perf report'.   
> 
> My output from hotspot is hundreds of megs for just a short recording. 
> Hotspot also reports a lot of missing debug symbols.  
> 
> I used the command './autogen.sh --enable-dbgutil --enable-symbols="sw/"' to
> configure the compile.  Should I compile with different parameters?

Ahh, thanks for mentioning the parameters! You should drop --enable-dbgutil as it causes performance issues. Dropping it means you have to first run "make clean" to remove your workdir and instdir and then you have to build from scratch.
Comment 27 David 2021-08-20 00:16:38 UTC
Created attachment 174436 [details]
perf data

I've attached a new perf.data file.  It was generated using 'perf record -g --pid=[pid#]'.  

Libreoffice was compiled using './autogen.sh --enable-symbols'.
Comment 28 David 2021-08-20 00:19:17 UTC
Created attachment 174437 [details]
perf.data printout

A new printout of the perf data has been attached.  The nodes with the most cpu time have been expanded.
Comment 29 David 2021-08-20 00:20:48 UTC
Created attachment 174438 [details]
flamegraph image

A flamegraph image of perf.data has been attached.
Comment 30 David 2021-08-30 07:18:02 UTC
I did some more testing.  The regression is that with 7.x, the Table tree automatically expands when there are tables in the document.  In 6.x, the Table tree remains collapsed after a document is opened, even when typing in a table, until the tree is manually expanded.  When the tree in 6.x is expanded, then it has the same delayed response as 7.x, even if the tree is subsequently collapsed.

The delay is being caused by the SwTable::GetInfo subroutine in sw/source/core/table/swtable.cxx.

Adding 'return false;' to the beginning of 'case RES_AUTOFMT_DOCNODE:' case statement corrects the slow response time.  In fact, it is not obvious to me what negative ramifications there are to replacing the commands in this case statement with 'return false;'.  As far as I can tell, everything still seems to function properly and with a normal response time.

But is it necessary to call this subroutine on every key press?  It would seem to me that it should only be called if a table is inserted or removed or has a name change.  And if the commands in the specified case statement do need to be executed, is there a more efficient way of programming them?

Or is the easiest fix for this issue simply to replace the commands executed by this case statement with 'return false;'?
Comment 31 Buovjaga 2021-08-30 10:50:45 UTC
(In reply to David from comment #30)
> I did some more testing.  The regression is that with 7.x, the Table tree
> automatically expands when there are tables in the document.  In 6.x, the
> Table tree remains collapsed after a document is opened, even when typing in
> a table, until the tree is manually expanded.  When the tree in 6.x is
> expanded, then it has the same delayed response as 7.x, even if the tree is
> subsequently collapsed.
> 
> The delay is being caused by the SwTable::GetInfo subroutine in
> sw/source/core/table/swtable.cxx.
> 
> Adding 'return false;' to the beginning of 'case RES_AUTOFMT_DOCNODE:' case
> statement corrects the slow response time.  In fact, it is not obvious to me
> what negative ramifications there are to replacing the commands in this case
> statement with 'return false;'.  As far as I can tell, everything still
> seems to function properly and with a normal response time.
> 
> But is it necessary to call this subroutine on every key press?  It would
> seem to me that it should only be called if a table is inserted or removed
> or has a name change.  And if the commands in the specified case statement
> do need to be executed, is there a more efficient way of programming them?
> 
> Or is the easiest fix for this issue simply to replace the commands executed
> by this case statement with 'return false;'?

Thanks a lot for investigating! For me, the Table tree expands when I move the focus into a table. However, it does remember the expanded state between sessions.

Tracking the focus was added in 7.1 with https://git.libreoffice.org/core/commit/dda71436b8594fb0669b3ac06c02a3997a0674be
tdf#95378 Writer Navigator document content tracking

Let's ask Jim what he thinks.
Comment 32 Jim Raykowski 2021-09-12 21:22:50 UTC
AFAICT this slow behavior has always existed. It is caused by the way table names are inserted into the Navigator content tree. Please see sw/source/uibase/utlui/content.cxx SwContentType::FillMemberList case ContentTypeId::TABLE code block.
 
Refilling the table member list only when the number of tables change will not update the list when a table is renamed or it's visibility changes.

An improvement in response can be seen if the second argument to SwEditShell::GetTableFrameFormat is changed from true to false in the member loop.

GetTableFrameFormat const SwFrameFormat& rTableFormat = m_pWrtShell->GetTableFrameFormat(i, false);

Renaming a Table using the Rename object: dialog also shows this slowness when the document has a large number of tables.
Comment 33 Jim Raykowski 2021-09-13 02:17:01 UTC
(In reply to Jim Raykowski from comment #32)
> An improvement in response can be seen if the second argument to
> SwEditShell::GetTableFrameFormat is changed from true to false in the member
> loop.
>
No good! Changing the second argument to false causes tables that are deleted to show their name greyed out in the Tables list.
Comment 34 David 2021-09-13 20:05:50 UTC
(In reply to Jim Raykowski from comment #33)
> (In reply to Jim Raykowski from comment #32)
> > An improvement in response can be seen if the second argument to
> > SwEditShell::GetTableFrameFormat is changed from true to false in the member
> > loop.
> >
> No good! Changing the second argument to false causes tables that are
> deleted to show their name greyed out in the Tables list.

I've been using my own compiled version for about a week now.  I've added 'return false;' to the beginning of 'case RES_AUTOFMT_DOCNODE:' case in sw/source/core/table/swtable.cxx.  So far I have not seen any negative ramifications.  If these statements need to be executed for some reason (although why they need executed for every keystroke I'd have to have explained), the simplest work-around (though not the best solution IMO) to mitigate the affects of the table tracking would be to add a context menu to disable the table tracking (while still listing the tables and other targets).

Yes, it's always been slow, but before 7.0 table tracking wasn't always happening and so the affects weren't always noticed.
Comment 35 Jim Raykowski 2021-09-22 07:25:14 UTC
Here is a patch for consideration:
https://gerrit.libreoffice.org/c/core/+/122416
Comment 36 Buovjaga 2021-09-22 08:27:25 UTC
(In reply to Jim Raykowski from comment #35)
> Here is a patch for consideration:
> https://gerrit.libreoffice.org/c/core/+/122416

The lag I mentioned in comment 6 is definitely gone with this patch!
Comment 37 Commit Notification 2021-09-25 21:16:37 UTC
Jim Raykowski committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/a78190da867864485b5d0a91238616bd4090b5b5

tdf#143268 Fix slow response after Navigator fills table members

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 38 Buovjaga 2021-09-27 12:23:45 UTC
Verified with final patch, but would be cool to hear David's testing result.

You might use to this method to easily get a daily build: https://wiki.documentfoundation.org/Installing_in_parallel/Linux#Automated_installation

Version: 7.3.0.0.alpha0+ / LibreOffice Community
Build ID: b63c5ade3554a42def4bce94f9fd67ea66528214
CPU threads: 8; OS: Linux 5.14; UI render: default; VCL: kf5 (cairo+xcb)
Locale: fi-FI (fi_FI.UTF-8); UI: en-US
Calc: threaded
Comment 39 David 2021-09-27 13:06:29 UTC
Response is quick now!  Thanks Jim!