Bug 115757 - LanguageTool extension maxes out CPU with a long document
Summary: LanguageTool extension maxes out CPU with a long document
Status: RESOLVED NOTOURBUG
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
4.3.0.4 release
Hardware: x86-64 (AMD64) All
: medium major
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: haveBacktrace, perf
: 125061 (view as bug list)
Depends on:
Blocks:
 
Reported: 2018-02-15 19:46 UTC by J. Schulte
Modified: 2019-09-18 13:23 UTC (History)
6 users (show)

See Also:
Crash report or crash signature:


Attachments
Example file (1.16 MB, application/vnd.oasis.opendocument.text)
2018-10-14 08:17 UTC, Buovjaga
Details
Callgrind output from master (8.02 MB, application/x-xz)
2018-10-14 08:18 UTC, Buovjaga
Details
perf flamegraph (118.08 KB, application/x-bzip)
2019-05-31 21:19 UTC, Julien Nabet
Details
perf flamegraph (27.85 KB, application/x-bzip)
2019-06-01 13:51 UTC, Julien Nabet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description J. Schulte 2018-02-15 19:46:50 UTC
Description:
A desire for one performance for that LibreOffice Writer

Steps to Reproduce:
Dear LibreOffice Team

I would like to suggest a suggestion for the LibreOffice Writer, which I would be very happy if you could fulfill them in the upcoming release.

My suggestion is about the performance of LibreOffice Writer. I write documents that have over 600 thousand words and are over 700 pages long, which causes the writer to lose performance and no longer run smoothly, even sometimes threatening to crash if I try to find spelling mistakes with the spell checker, which takes ten to fifteen minutes Because of the size of the document, even though I have a power PC and the CPU gets bored, LibreOffice is not usable at all during the spellchecker, which prevents me from reading or editing another document.

The performance has to be improved somehow, so this does not happen. It is very annoying when LibreOffice crashes during the process, because the spell checker does not follow and you have to start over again because it could not be saved.

Thank you for your attention and look forward to the upcoming releases of LibreOffice.

Yours sincerely, J. Schulte

Actual Results:  
 

Expected Results:
 


Reproducible: Always


User Profile Reset: No



Additional Info:
 


User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:58.0) Gecko/20100101 Firefox/58.0
Comment 1 Telesto 2018-02-15 21:24:44 UTC Comment hidden (obsolete)
Comment 2 J. Schulte 2018-02-15 22:50:03 UTC Comment hidden (obsolete)
Comment 3 Telesto 2018-02-16 08:28:36 UTC Comment hidden (obsolete)
Comment 4 Xisco Faulí 2018-02-16 10:25:08 UTC Comment hidden (obsolete)
Comment 5 J. Schulte 2018-02-16 17:17:28 UTC Comment hidden (obsolete)
Comment 6 J. Schulte 2018-02-17 15:18:56 UTC
I create a new Dokument for you and I come on 1,8 GB RAM and the performance ease up but it is not so bad as by my roman. Around that adjust the scenario, is it important that the extension "LanguageTool" installed and activ is, around on 1,8 GB. The performance does not differ if this "LanguageTool" on or off is. Only RAM is affected to get.

Test Docokemt: https://drive.google.com/file/d/1NPXKo2nsfn9steLOw1AuWvZxQE21wT-n
Comment 7 Telesto 2018-02-17 15:33:44 UTC
Repro with
Version: 6.1.0.0.alpha0+
Build ID: b87fe45e8b087a315a65b92bf9c168b1e4c5cc00
CPU threads: 4; OS: Windows 6.3; UI render: default; 
TinderBox: Win-x86@42, Branch:master, Time: 2018-02-16_23:14:35
Locale: nl-NL (nl_NL); Calc: CL

1. Open https://drive.google.com/file/d/1NPXKo2nsfn9steLOw1AuWvZxQE21wT-n
2. Wait until CPU usage drops to 0%
3. Use the right click menu to correct a spelling mistake
Comment 8 J. Schulte 2018-02-17 17:13:45 UTC Comment hidden (obsolete)
Comment 9 J. Schulte 2018-02-17 17:37:24 UTC Comment hidden (obsolete)
Comment 10 J. Schulte 2018-02-17 17:37:51 UTC Comment hidden (obsolete)
Comment 11 Telesto 2018-02-17 17:51:07 UTC
(In reply to J. Schulte from comment #8)
> with or no "languageTool"?

With "Automatic spell checking" enabled, default extensions, dictionary.. The CPU usage of soffice.bin should drop to 0% in around a minute or so after file-opening (depending on system specs).
Comment 12 J. Schulte 2018-02-17 18:03:18 UTC Comment hidden (obsolete)
Comment 13 J. Schulte 2018-02-17 18:21:53 UTC
https://www.youtube.com/watch?v=9De1273b8HM
Comment 14 Telesto 2018-02-17 18:44:45 UTC
(In reply to J. Schulte from comment #13)
> https://www.youtube.com/watch?v=9De1273b8HM

Hmmm, interesting. This is indeed something else.. Does this also happen in safe-mode: Help -> Restart in safe mode
Comment 15 J. Schulte 2018-02-17 18:52:25 UTC Comment hidden (obsolete)
Comment 16 Telesto 2018-02-17 18:54:24 UTC
@Áron
You might be interested in comment 13. It's leaking pretty bad, for some reason..
Comment 18 J. Schulte 2018-02-17 19:20:50 UTC
I upload a video with my Roman.
I hope you Understand my problem better now.

https://www.youtube.com/watch?v=qqQYBhdNrTs
Comment 19 Telesto 2018-02-18 11:23:58 UTC
Setting back to unconfirmed for now. I'm not able to reproduce the problem as illustrated in the youtube video's.
Comment 20 J. Schulte 2018-02-18 15:26:19 UTC
https://www.youtube.com/watch?v=YoU2DT8GDMo
Comment 21 J. Schulte 2018-02-18 20:13:53 UTC
When it shouldn´t help i will search for a better methode
Comment 22 Xisco Faulí 2018-02-27 11:39:16 UTC
You can't confirm your own bugs. Moving it back to UNCONFIRMED until someone
else confirms it.
Comment 23 Timur 2018-09-03 09:51:40 UTC
This is not a proper https://wiki.documentfoundation.org/QA/BugReport.
Non useful title and not specific repro steps. 
I can see in 6.2+ attached 750-pages ODT is somewhat slow and keep CPU at 13%. Guess because it's recounting words and characters. Using spell checker is also somewhat slow but it works. Can't say now if it's due to my i7 or 6.2+. I didn't test with LanguageTool. 
I set back to Unconfirmed. 
J., please:
- write reproducible steps and propose specific title
- test both with and without LanguageTool
- please test with previous (like 4.3.7, 4.4.7, 5.0.6, 5.1.6, 5.2.7) and future versions (current master 6.2+), using Separate Install GUI tool http://tdf.io/siguiexe. It downloads and extracts different LO versions, without installing, so you may test different versions
Comment 24 J. Schulte 2018-09-23 14:40:57 UTC
https://www.youtube.com/watch?v=jcfsletH_D0

Here is a video too performanc of Libreoffice with and without LanguageTool-Team
Comment 25 Buovjaga 2018-10-14 08:17:15 UTC
Created attachment 145699 [details]
Example file

Confirmed neverending hammering of CPU with LanguageTool 4.3. Without LT it goes to 0% shortly after opening the file.

Arch Linux 64-bit
Version: 6.2.0.0.alpha0+
Build ID: 00e10ae3189a4407ffb1a48f836cd52dc9a1b6df
CPU threads: 8; OS: Linux 4.18; UI render: default; VCL: gtk3_kde5; 
Locale: fi-FI (fi_FI.UTF-8); Calc: threaded
Built on 13 October 2018
Comment 26 Buovjaga 2018-10-14 08:18:03 UTC
Created attachment 145700 [details]
Callgrind output from master

I let it run for over an hour after the GUI appeared, hopefully captured enough data.
Comment 27 Telesto 2018-10-14 09:30:04 UTC
(In reply to Buovjaga from comment #25)
> Created attachment 145699 [details]
> Example file
> 
> Confirmed neverending hammering of CPU with LanguageTool 4.3. Without LT it
> goes to 0% shortly after opening the file.
> 
Is it a regression? At the side of LanguageTool and/or LibreOffice.. Not volunteering to bibisect :-)
Comment 28 J. Schulte 2018-10-14 14:09:41 UTC Comment hidden (obsolete)
Comment 29 J. Schulte 2018-10-14 14:30:26 UTC
https://www.youtube.com/watch?v=Li-Dwhula68

This is my last video, because i dont speak english and use google translate and i have no fun more.

[German]
Ich kann kaum englisch, weswegen ich mit den Fehler leben werde oder nach einer Alternative suchen werde, denn ich kann mit diesen Laggs und Hänger nicht leben und auch nicht tolerieren. LibreOffice 6 ist kein Fortschritt, gegenüber 5, weswegen ich ein Downgrade durcghführen werde. Ich gebe es auf.

Mein Problem ist, dass LibreOffice kein Multitaking unterstützt oder nur ungenügend. Sobald es hängt, hägen alle geöffneten Dokumenten und wenns abstürzt, stürzen alle Dokumente auf, was wirklich ungenügend ist.

Ihr könnt euch diesen Problem widmen, wobei ich sehr stark bezweifele, dass hier überhaupt was passieren wird, weswegen ich jetzt die Hoffnung aufgebe und euch viel Spaß wünsche.

Vielleicht kriegt ihr das Problem, durch die Hilfe von anderen Programmierer, hin, aber ich kann euch nicht helfen.

Grüße Dallun511
Comment 30 Buovjaga 2018-10-14 15:27:36 UTC
(In reply to Telesto from comment #27)
> Is it a regression? At the side of LanguageTool and/or LibreOffice.. Not
> volunteering to bibisect :-)

I don't think so. I tested on Win and even 4.3.0 keeps the CPU going. Strangely, I tried with bibisect repos, but in their builds the CPU calmed down after like max. 1 minute. I took care to install https://extensions.libreoffice.org/extensions/languagetool and that the German dict was installed and automatic spellchecking enabled.
Note: in 4.3.0 one enables the auto spellcheck from toolbar.
Here is the German dictionary for the convenience of future testers: https://extensions.libreoffice.org/extensions/german-de-de-frami-dictionaries

I also tested LT 3.9 on Linux master and no change in behaviour.
Comment 31 Alex Thurgood 2019-05-28 14:18:35 UTC
*** Bug 125061 has been marked as a duplicate of this bug. ***
Comment 32 Julien Nabet 2019-05-31 21:19:31 UTC
Created attachment 151822 [details]
perf flamegraph

On pc Debian x86-64 with master sources updated today, I could reproduce this.

Then I uninstalled the extension and remove local LO profile and it was still hanging (contrary to my other local repo specific for Flamegraph without enable-dbgutil).
So I attached a Flamegraph in my repo which has enable-dbgutil and which shouldn't have any trace of Languagetool.
Comment 33 Julien Nabet 2019-05-31 21:20:57 UTC
Noel: having noticed your great job about optimization, does the Flamegraph perf may help to find some bottleneck here even if I used enable-dbgutil (see previous comment)?
Comment 34 Noel Grandin 2019-06-01 06:49:15 UTC
@Julien, this is just one of those places where enable-dbgutil build is insanely slow. The optimised build should be fine.
Comment 35 Julien Nabet 2019-06-01 13:51:59 UTC
Created attachment 151830 [details]
perf flamegraph

On pc Debian x86-64 with master sources updated today + enable-symbols (not enable-dbgutil), I retrieved another perf Flamegraph about 1 min.
I installed LanguageTool + add sfl4j-api and slf4j-simple to classpath inside LO.

Is it more useful? If not, is there something which may help to pinpoint the pb?
Comment 36 Noel Grandin 2019-06-01 16:10:51 UTC
@Julien perf is not much use at looking inside java code. All we can tell is that the CPU time is inside the GrammarChecking java code. (i.e. inside LanguageTool)

Tools like VisualVM or the Netbeans Profiler can look inside Java code, and is probably what is necessary here.

But at this point, it is no longer a bug in LibreOffice, so you are perhaps better off logging this upstream at 
     https://github.com/languagetool-org/languagetool/issues
Comment 37 Buovjaga 2019-06-01 16:15:10 UTC
(In reply to Noel Grandin from comment #36)
> @Julien perf is not much use at looking inside java code. All we can tell is
> that the CPU time is inside the GrammarChecking java code. (i.e. inside
> LanguageTool)
> 
> Tools like VisualVM or the Netbeans Profiler can look inside Java code, and
> is probably what is necessary here.
> 
> But at this point, it is no longer a bug in LibreOffice, so you are perhaps
> better off logging this upstream at 
>      https://github.com/languagetool-org/languagetool/issues

I already created an issue last year: https://github.com/languagetool-org/languagetool/issues/1217
Comment 38 Buovjaga 2019-06-01 16:15:45 UTC
But sure, no point in keeping this open.
Comment 39 Julien Nabet 2019-09-18 13:23:40 UTC
BTW, LO includes Langtool 1.7 (see https://opengrok.libreoffice.org/xref/core/download.lst?r=6d15dcd1#138)
whereas current version is 4.6 (see https://languagetool.org/#libreoffice)