Bug 118165 - Adding a new paragraph to a text block is rather CPU hogging
Summary: Adding a new paragraph to a text block is rather CPU hogging
Status: RESOLVED DUPLICATE of bug 112989
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
5.4.0.3 release
Hardware: All Windows (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: haveBacktrace, perf, regression
: 118436 (view as bug list)
Depends on:
Blocks:
 
Reported: 2018-06-14 13:13 UTC by Telesto
Modified: 2018-07-02 14:12 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments
Example file (34.82 KB, application/vnd.oasis.opendocument.text)
2018-06-14 13:13 UTC, Telesto
Details
Callgrind output from master (1.15 MB, application/x-xz)
2018-06-24 11:19 UTC, Buovjaga
Details
Bibisect log (6.11 KB, text/plain)
2018-06-28 15:48 UTC, Telesto
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Telesto 2018-06-14 13:13:19 UTC
Description:
Adding a new paragraph to a text block is rather CPU hogging

Steps to Reproduce:
1. Open the attached file
2. Enable Spell check 
3. Have CPU monitoring tool with CPU graph
4. Wait until CPU until the is fully processed (CPU near 0%)
5. Scroll down to the last page
6. Press Enter at the beginning of the 'ddd...' & monitor the the before CPU usage before it drops back (1 minute with Libo6.1/ 35 seconds with LibO5.3


Actual Results:
60 seconds with Libo 6.1
35 seconds with LibO 5.3.5.2
8 second with 5.2 -> Difference between LibO5.2 <-> 5.3 (bug 112989)

Expected Results:
In this case: 35 seconds


Reproducible: Always


User Profile Reset: No



Additional Info:
Version: 6.2.0.0.alpha0+
Build ID: 51aa57cd8ed46d28262e0d315328231f0fa814f4
CPU threads: 4; OS: Windows 6.3; UI render: default; 
TinderBox: Win-x86@42, Branch:master, Time: 2018-06-12_05:54:56
Locale: nl-NL (nl_NL); Calc: CL
Comment 1 Telesto 2018-06-14 13:13:43 UTC
Created attachment 142733 [details]
Example file
Comment 2 Buovjaga 2018-06-24 11:19:28 UTC
Created attachment 143071 [details]
Callgrind output from master

Repro.

Arch Linux 64-bit
Version: 6.2.0.0.alpha0+
Build ID: 5b42a17dc99fba2ccf8dd8d0a8e0e4e836e30120
CPU threads: 8; OS: Linux 4.17; UI render: default; VCL: kde4; 
Locale: fi-FI (fi_FI.UTF-8); Calc: group threaded
Built on June 22nd 2018
Comment 3 Telesto 2018-06-28 11:40:51 UTC
*** Bug 118436 has been marked as a duplicate of this bug. ***
Comment 4 Telesto 2018-06-28 15:48:24 UTC
Created attachment 143191 [details]
Bibisect log

I bisected this to: https://cgit.freedesktop.org/libreoffice/core/commit/?id=559bb1420cd7fe699e139e20ea3b4205b9e6937d

author	Stephan Bergmann <sbergman@redhat.com>	2017-03-20 17:02:05 +0100
committer	Stephan Bergmann <sbergman@redhat.com>	2017-03-20 17:03:15 +0000
commit 559bb1420cd7fe699e139e20ea3b4205b9e6937d (patch)
tree 4b1e640bdb70ff81aeebb5eb6a9d180e6ebe4f7f
parent f22dbb10abbb2d02a57f9fee42a3c95f99277899 (diff)
These files are called unorc on all platforms

However, not 100% sure if this is accurate
Comment 5 Buovjaga 2018-06-28 16:01:39 UTC
(In reply to Telesto from comment #4)
> Created attachment 143191 [details]
> Bibisect log
> 
> I bisected this to:
> https://cgit.freedesktop.org/libreoffice/core/commit/
> ?id=559bb1420cd7fe699e139e20ea3b4205b9e6937d
> 
> author	Stephan Bergmann <sbergman@redhat.com>	2017-03-20 17:02:05 +0100
> committer	Stephan Bergmann <sbergman@redhat.com>	2017-03-20 17:03:15 +0000
> commit 559bb1420cd7fe699e139e20ea3b4205b9e6937d (patch)
> tree 4b1e640bdb70ff81aeebb5eb6a9d180e6ebe4f7f
> parent f22dbb10abbb2d02a57f9fee42a3c95f99277899 (diff)
> These files are called unorc on all platforms
> 
> However, not 100% sure if this is accurate

Doesn't seem like it, because those are minor makefile changes.
Comment 6 Telesto 2018-06-28 17:32:46 UTC
(In reply to Buovjaga from comment #5)
> Doesn't seem like it, because those are minor makefile changes.

Minor changes can have large impact :-). Bibisect 'pattern' is more or less normal (except for the skip). Anyway, I'm not in the mood for a third bibisect attempt. First attempt ended somewhere in the Calc code

Btw, the profile folder seems to influence the bibisect. Deleting it for every bibisect step seems to be necessary.
Comment 7 Telesto 2018-06-29 12:32:31 UTC
Adding CC to Stephan Bergmann

Is there any change that this is happening because of https://cgit.freedesktop.org/libreoffice/core/commit/?id=559bb1420cd7fe699e139e20ea3b4205b9e6937d

author	Stephan Bergmann <sbergman@redhat.com>	2017-03-20 17:02:05 +0100
committer	Stephan Bergmann <sbergman@redhat.com>	2017-03-20 17:03:15 +0000
commit 559bb1420cd7fe699e139e20ea3b4205b9e6937d (patch)
tree 4b1e640bdb70ff81aeebb5eb6a9d180e6ebe4f7f
parent f22dbb10abbb2d02a57f9fee42a3c95f99277899 (diff)
These files are called unorc on all platforms

I ended up here twice, but it doesn't seem like it
Comment 8 Stephan Bergmann 2018-06-29 12:57:36 UTC
(In reply to Telesto from comment #7)
> Is there any change that this is happening because of
> https://cgit.freedesktop.org/libreoffice/core/commit/
> ?id=559bb1420cd7fe699e139e20ea3b4205b9e6937d

Rather unlikely, esp. if this issue can also be reproduced on Linux (as I assume from comment 2) or macOS (where that change should have no effect at all).

> I ended up here twice, but it doesn't seem like it

With which bibisect repo?
Comment 9 Stephan Bergmann 2018-06-29 12:59:11 UTC
(In reply to Stephan Bergmann from comment #8)
> With which bibisect repo?

"bibisect-win32-5.4", according to attachment 143191 [details]
Comment 10 Stephan Bergmann 2018-06-29 14:38:33 UTC
(In reply to Stephan Bergmann from comment #8)
> (In reply to Telesto from comment #7)
> > Is there any change that this is happening because of
> > https://cgit.freedesktop.org/libreoffice/core/commit/
> > ?id=559bb1420cd7fe699e139e20ea3b4205b9e6937d
> 
> Rather unlikely, esp. if this issue can also be reproduced on Linux (as I
> assume from comment 2) or macOS (where that change should have no effect at
> all).

What that commit fixes is the following:  Prior to that commit, some extension-related data on Windows (and only on Windows) would not be available to LO, so that some extensions may have not worked properly.  (I'm not entirely sure, but at least superficially it looks like that was broken ever since <https://cgit.freedesktop.org/libreoffice/core/commit/?id=2e5af9dc4fe88fead050ebe08f2b2d044124e7ce> "properly generate rc files" from 2013.)

Per-language dictionaries come as extensions, so this commit might in principle make a difference here ("2. Enable Spell check" in comment 0) after all.  However:

* The bibisect-win32-5.4 repo appears to not include any bundled extensions at all, and it uses its own UserInstallation, so should not have any shared or per-user extensions installed unless you explicitly added some.  Telesto, are there any extensions listed under "Tools - Extension Manager..." with all three check boxes "Bundled with LibreOffice", "Installed for all users", and "Installed for current user" checked?

* Most dictionary extensions do not include any kind of data that would not be available to LO prior to my commit, anyway, so should not be affected by that commit.  Only some spelling-related extensions like Lightproof (which is included in the dictionaries for en, hu, pt-BR, and ru, I think, but not for the nl-NL locale reported in comment 0) would be affected.
Comment 11 Telesto 2018-06-29 15:07:25 UTC
(In reply to Stephan Bergmann from comment #10)
> (In reply to Stephan Bergmann from comment #8)
> > (In reply to Telesto from comment #7)
> > > Is there any change that this is happening because of
> > > https://cgit.freedesktop.org/libreoffice/core/commit/
> > > ?id=559bb1420cd7fe699e139e20ea3b4205b9e6937d
> > 
> Telesto,
> are there any extensions listed under "Tools - Extension Manager..." with
> all three check boxes "Bundled with LibreOffice", "Installed for all users",
> and "Installed for current user" checked?

I have a list of extensions installed (I forgot about that). I likely copied the dictionary's from a regular build to the "share\extensions" directory of the bibisect folder

So yes, all three check boxes "Bundled with LibreOffice", "Installed for all users", and "Installed for current user" are checked
Comment 12 Telesto 2018-06-29 15:18:10 UTC
Small addition: I can't reproduce the same issue on Ubuntu. There *some* slowness, but nothing compared to Windows and it's already around in 5.3. So probably not the same thing.
Comment 13 Stephan Bergmann 2018-06-29 15:19:10 UTC
(In reply to Telesto from comment #11)
> So yes, all three check boxes "Bundled with LibreOffice", "Installed for all
> users", and "Installed for current user" are checked

Sorry if I caused confusion there:  Whether those checkboxes are checked doesn't make a difference to whether those extensions are active.  The checkboxes only control what subset of extensions is listed in the Extension Manager dialog, so I only mentioned those checkboxes in case your Extension Manager's list was empty when you nevertheless had some extensions installed that were just not shown.
Comment 14 Stephan Bergmann 2018-06-29 15:23:44 UTC
(In reply to Telesto from comment #12)
> Small addition: I can't reproduce the same issue on Ubuntu. There *some*
> slowness, but nothing compared to Windows and it's already around in 5.3. So
> probably not the same thing.

If it really started to be slow on Windows with my commit 559bb1420cd7fe699e139e20ea3b4205b9e6937d, then that would imply that:

1  it is related to some (language/spelling specific) extensions that cause the slowness (and just didn't cause slowness before on Windows because those extensions were effectively broken on Windows prior to my commit), and

2  it would likely be similarly slow (before and after my commit) on other platforms, if you use the same/similar extensions (and locale) there.
Comment 15 Telesto 2018-06-29 18:32:09 UTC
@Aron
Would you mind to (un)confirm the bug report & to check my bibisect result (comment 4) 

Probably Win only. I can't reproduce the same problem on Ubuntu with dictionary's installed
Comment 16 Aron Budea 2018-06-30 00:16:53 UTC
(In reply to Telesto from comment #15)
> Would you mind to (un)confirm the bug report & to check my bibisect result
> (comment 4) 
I tested with 6.1 beta2 / Windows 7, took 3 measurements that turned out to be: 10s, 15s and 25s.
On the other hand, with 5.4.0.3 it was 53s.

Not sure if any conclusion can be drawn from this.
Comment 17 Buovjaga 2018-06-30 05:40:44 UTC
(In reply to Telesto from comment #15)
> Probably Win only. I can't reproduce the same problem on Ubuntu with
> dictionary's installed

I did another test to give exact numbers.

On Linux:
It takes 43 seconds with Master.
It takes 50 seconds with 5.3.6.1 (Appimage)

So yeah, no regression on Linux after all.

On Windows:
It takes 20 seconds with 5.3.0.
It never goes to maximum with 6.2 and the little CPU bump goes to 0 in a couple of seconds.

Spellchecking was functioning correctly in all builds.

So something seems to have happened very recently. I guess I should do a new Linux master build in case it is as recent as the past 2 days.

Version: 6.2.0.0.alpha0+ (x64)
Build ID: 7696d1d217b6cff63490e7f98403d0e499c33d17
CPU threads: 4; OS: Windows 10.0; UI render: default; 
TinderBox: Win-x86_64@42, Branch:master, Time: 2018-06-30_01:51:45
Locale: fi-FI (fi_FI); Calc: group threaded

Arch Linux 64-bit
Version: 6.2.0.0.alpha0+
Build ID: 4600b07c1d787f959618d9ecf54161e4ea4ffa61
CPU threads: 8; OS: Linux 4.17; UI render: default; VCL: gtk3; 
Locale: fi-FI (fi_FI.UTF-8); Calc: group threaded
Built on June 28th 2018
Comment 18 Buovjaga 2018-06-30 05:42:23 UTC
Actually: in Win master, spellchecking did not function at first with the default Dutch. I switched it to English (USA) to get spellchecking.
Comment 19 Buovjaga 2018-06-30 06:22:12 UTC
Still hammering CPU like before on Linux.

Arch Linux 64-bit
Version: 6.2.0.0.alpha0+
Build ID: cdaa5d8e9727a0a41f969ae29b0442f3ca2f67c0
CPU threads: 8; OS: Linux 4.17; UI render: default; VCL: gtk3; 
Locale: fi-FI (fi_FI.UTF-8); Calc: group threaded
Built on June 30th 2018
Comment 20 Telesto 2018-07-02 14:10:35 UTC
This one is a dupe of, or interconnected with bug 112989. It also happens in LibO5.3 with Harfbuzz enabled. It's OK without it. 

Still no clue why this is working in 5.4 until commit 559bb1420cd7fe699e139e20ea3b4205b9e6937d. 

This should be looked after a fix is published for bug 112989.

I would definitely vote for "Text layout performance" as Tender budget 2018. Spell-check & Text shaping isn't a great combo at this point.

---
Text layout performance

CostEstimate: 6 weeks Contact: Khaled

Our text layout performance is abysmal, all over the code base it is assumed that shaping text is cheap and we can do it over and over again. Want to measure the text? Shape it and discard the output afterwards. Want to measure part of the same text? Shape again. Want to find line breaks? Shape again. Want to finally draw it? Shape again. This might have been cheap for Latin script in the olden days when all we did is query font cmap table and put glyphs next to each other, which is not the case anymore and never been the case for more involved scripts. We need to work on this, and there are many possibilities; retaining shaping results much longer, improving the wasteful OutputDevice API, caching etc.

*** This bug has been marked as a duplicate of bug 112989 ***