Created attachment 57846 [details] large autocorrect database this is a different issue than: https://www.libreoffice.org/bugzilla/show_bug.cgi?id=46765 that only affect LibO 3.5.0, while the current bug has been there since the OOo era... it seems that large autocorrect databases male LibO freeze when start typing. try this: enter the autocorr subfolder in your user profile (in Windows is under: User\LibreOffice 3\user\autocorr) and put the attached acor_.dat file (backup yours first) which has 64000 entries inside it. start LibO and open a blank writer file. digit "test" and hit space... you will see a first 2-3 second freeze after "tes" the the cursor moves to "test" and another longer freeze (6 seconds maybe) happens before you see the "space" so there are 2 freeze moments, the first one is shorter and unrelated to autocorrection (bug 46765... it's present even with no autocorrect database) while the second is longer and depends on autocorrection. both freeze moment happen just at first start, so it will annoy you only once when you first launch LibO, but give a bad impression about LibO performace these tests were done on LibO 3.5.0 Windows Vista 64bit SP1 IntelCore2 Duo CPI P8400 @2.26 GhZ, 4GB RAM but can be reproduced even on Windows XP.
Hello! I tested it with the patch which reduced loadtime of autocorrection tables. The first freeze occured just as you described (although it was 1 sec max), there was no freeze before space.
great!!! as I hoped, your path fixed both things!! thank you so much... I'm struggling with those autocorrect issues since years!!! I'm going to mark this bug as fixed
just forgot to say that Dezsi's patch fixes this old OOo issue I opened in May 2009: Issue 101726 - Speed up autocorrect replacement table loading time https://issues.apache.org/ooo/show_bug.cgi?id=101726 Great to see that LibO folks solved an issue that OOo never seriously tried to fix.
For the record, the patch is in master, -3-5, and -3-5-1: http://cgit.freedesktop.org/libreoffice/core/commit/?id=8e750a2d9c653e0693738c847fe2ee2a8ab04052 http://cgit.freedesktop.org/libreoffice/core/commit/?h=libreoffice-3-5&id=3a90b7fea7de8860dfdb92925df39dac3d0ed4fc http://cgit.freedesktop.org/libreoffice/core/commit/?h=libreoffice-3-5-1&id=14f0c12942a8557d3d9dcdaea2e80bad34f40d86 So, will be in 3.5.1, 3.5.2, and 3.6.0. Set target accordingly.
mmhhh.... it seems that the "huge database" related typing freeze is not gone yet... I still experience this in LibO 3.5.1 and 3.5.2... I thought that dezsi's patch which fixed the old "OOo autocorrect replacement table loading time issue", had an impact on the current issue as well... he told in Comment 1 that it did, so I closed the issue... but now I'm going to reopen since it's still here... if I put the previously attached huge acor_.dat file in my "User\LibreOffice 3\user\autocorr" folder, the freeze happens if I remove it, the freeze does not happen again. tested with LibO 3.5.2 on Vista 64bit.
Created attachment 59932 [details] LibO start typing without acor_.dat this is a fresh install of portable LibreOffice 3.5.2 (downloaded from http://www.winpenpack.com/main/download.php?view.1338) the user profile is clean no autocorrect file database is present once an empty Writer document is opened, there's a short freeze while typing the first word (i.e. "test") the freeze happens between "te" and "st" following words are written with no freeze this is Bug 46765
Created attachment 59934 [details] LibO start typing with acor_.dat this is the same fresh install of portable LibreOffice 3.5.2 as before the user profile is the same except for the presence of a large autocorrect file database (acor_.dat - see first attachment - containing more than 64000 entries) same test as before, after the first small freeze between "te" and "st", you will see a much longer freeze after "test" and before the cursor moves to the "space" while following words are again written with no freeze. this shows that at document start the first access to a large autocorrect database has impact on LibO performance
the previous tests were done on Windows Vista 64bit but have been also reproduced in Windows XP 32 bit. I ask for Linux and Mac users to try reproducing the scenario using the attached large autocorrect database (acor_.dat) whose Windows path is: User\LibreOffice 3\user\autocorr
I report here a private mail from Michael Meeks about this bug, giving instructions to debug: So - then we need a profile of the piece that is slow; do you have a Linux machine you can repeat this on ? If so, can you run libreoffice under callgrind like this: export OOO_DISABLE_RECOVERY=1 valgrind --tool=callgrind --simulate-cache=yes --dump-instr=yes ./soffice.bin -writer --splash-pipe=0 Select your dictionary (beware it runs 80x slower than normal or so ;-). Then you'd want to gzip up the callgrind.out.... and attach it to the bug - that'll help the developers ~immediately fix the problem, and save them lots of time. The fix is prolly only a few minutes if we have that data, and it's more likely to get done. A linux VM would be fine to do the profiling. .......... unfortunately I have no experience with Linux so I can not run this text. is there any "penguin" right here that may help?
I tried valgrind as in comment 9, but the callgrind.out.XXXX is empty (0 bytes) after exit libo. Am I missing something? Ubuntu 64 bit libreoffice 3.5.2.2 installed from ppa:libreoffice/ppa, with libreoffice-dbg installed. I tried, and the empty file callgrind.out.XXXX created every times (4 times currently), even if I just started Writer and close, doing nothing. Console window (Terminal) says 'Segmentation fault' after exit libo.
hi Korrawit. thanks 4 testing. I'm just curious to know if you experience on Linux the freeze I described on Windows
> I tried valgrind as in comment 9, but the callgrind.out.XXXX is > empty (0 bytes) after exit libo. Am I missing something? Thanks for trying that ! you're sure you used 'soffice.bin' and not 'soffice' ? do you have write access to the directory that soffice.bin is in (prolly best to give that to yourself and run it from there). Also - you need to ensure no other soffice.bin is running or the factory behaviour will defeat you so perhaps 'pkill -9 -f soffice' beforehand (?). What else ? Can you run './soffice.bin' successfully yourself without valgrind ? if not perhaps we need to run 'soffice' with --trace-children=yes as another argument to valgrind. Thanks so much for trying to get a trace !
Forgot to mention in comment 10 that I use ubuntu 10.04 64bit. @tommy27: Yes, I see the freeze on ubuntu too, so set platform to All. @michael: Thanks for your suggestions :) Anyway, I just try again with ubuntu 12.04 32bit, and it seems to work. So I don't know which part(s) that trigger zero-length-file problem. Please see next comment for the callgrind log.
Well, attachment is too large! So, http://www.mediafire.com/?6mi1gypc666rinr Hope it'll be useful, and feel free to blame me if I've done something wrong. :)
Created attachment 60609 [details] kcachegrind output ... Korrawit - you're a hero :-) this is exactly what we need. It's a nice test, 55% of the measured time is in this code, so it's easy to see; and I attach a photo of the calltrace where all the time is being consumed. Hopefully a fix will just drop out of that.
I have a rather simple prototype patch; just re-building to test it.
The patch fixes the N^2 in the insert/sort - unfortunately that only saves around 1/2 the time - the rest seems to be some diffuse wasteage of a reasonably normal and fairly unbelievable kind (harder to fix). This takes me down from 1200ms to load the list to 700ms or so - which should be better. Give it some testing, and if you're happy I'll get some review / cherry-picking action to -3-5. Ultimately this code needs a re-write to use STL anyway so ... :-) I'd prefer to wait for / encourage that: perhaps adding this specific instance of: editeng/source/misc/svxacorr.cxx and it's 'SvxAutocorrWordList' to the STL porting EasyHack might help if you'd like to dig that out.
Michael Meeks committed a patch related to this issue. It has been pushed to "master": http://cgit.freedesktop.org/libreoffice/core/commit/?id=b269dced445494891a8e6e8d1d62b931a31dddbd fdo#46805 - special-case appending items to autocorrect lists
thanks Micheal. I like to test the build with the fix. where can I download it? P.S. do you think is safe to be cherry-picked in 3.5.3 or 3.5.4?
Thanks Michael too :) tommy27: You're on Windows, right? If yes, try one in <http://dev-builds.libreoffice.org/daily/Win-x86@6-fast/master/>, but you'll have to wait for today builds. /me also clear out irrelevant target in whiteboard
ok, I downloaded the: master~2012-04-27_21.25.23_LibO-Dev_3.6.0alpha0_Win_x86_install_en-US.msi and made comparative testings. - start LibO - open empty Writer document, italian language - digit "test" and hit "space" here's the results. 1 large autocorrect database (acor_.dat) time to see the white space LibO 3.5.2 --> 9 seconds LibOdev 3.6.0 --> 5 seconds 2 large autocorrect databases (acor_.dat ; acor_it-IT.dat) time to see the white space LibO 3.5.2 --> 14 seconds LibOdev 3.6.0 --> 9 seconds as Michael said, his fix reduced the freeze time about 50% which is a good result after all. maybe it's too late for 3.5.3. could it be considered for 3.5.4?
Michael Meeks committed a patch related to this issue. It has been pushed to "libreoffice-3-5": http://cgit.freedesktop.org/libreoffice/core/commit/?id=a617eecdb6b2f8d6bb1b20674f8be8ce4d60f2d1&g=libreoffice-3-5 fdo#46805 - special-case appending items to autocorrect lists It will be available in LibreOffice 3.5.4.
just wanted to say a final thank you to Micheal and Korrawit
> just wanted to say a final thank you to Micheal and Korrawit No problem - Korrawit was the man with the profile ;-) Having said that - we should build another profile, of the code with the fix; and try to work out where/why it is still quite so bad, there is simply no excuse for a modern CPU to take -that- long doing anything; but it needs some staring at kcachegrind & playing with it I think. Why we try to keep these autocorrection entries in this specific order, I have no idea - I would have thought a hash-table would have been by far the best - at least until we have to present a list of them in the GUI.
mmmhhh... it seems that performance in LOdev 3.7 declined versus current 3.6 stable releases... so I'm REOPENING this bug. my tests say that the freeze time after typing the first word in a Writer document become longer in LOdev 3.7 -------------------- LibO 3.6.1 -------------------- acor_.dat (65K entries) 5 seconds acor_.dat 65K entries + acor_it-IT.dat 55K entries 9 seconds -------------------- master 3.7.x -------------------- acor_.dat 65K entries 13 seconds acor_.dat 65K entries + acor_it-IT.dat 55K entries 20 seconds acor_.dat 65K entries + acor_it-IT.dat 119 entries * 26 seconds * this test can't be done in 3.6.1 since only master currently supports autocorrect databases larger than 65K I suspect that the longer freeze of LOdev 3.7 is a side effect of the new "32-bit autocorrect databases" introduced by this committ: http://cgit.freedesktop.org/libreoffice/core/commit/?id=78a39502de36c32d7aaf6e5bbde1f8df80fdd21f so I'm adding Tomaz Vajngerl to the CC List, since he's the man who's doing a lot of work fixing autocorrect performance issues, and maybe he's could find a solution in this area he already knows very well @ Tomaz if you read Comment 17 and Comment 25 from Micheal Meeks, you'll see that he saw room for improvement for his partial 3.5.4 fix
(In reply to comment #26) > mmmhhh... it seems that performance in LOdev 3.7 declined versus current 3.6 > stable releases... so I'm REOPENING this bug. That sounds bad! > I suspect that the longer freeze of LOdev 3.7 is a side effect of the new > "32-bit autocorrect databases" introduced by this committ: So, could you please check whether this is real ;) I mean, do you have daily builds before and after this commit? If yes, you should check if the commit actually decrease speed. Or the decrease comes from another commit.
Guys - this is a most-annoying bug for 3.5. I would strongly prefer not to mangle the history of this bug, and the stats for 3.5 by moving all these around. If there is a -new- regression in 3.7 - please create a new bug for that, and link to this one rather than re-opening an old one and confusing the version picture :-) As a one-off, just for you I've filed bug#55570 - which is also tracked vs. the 3.7 regression tracker. It'd be great if someone could check that that has all the right links, and information necessary to work on this.