Bug 49350 - 10x speedup of saving a new entry to a large autocorrect replacement table
Summary: 10x speedup of saving a new entry to a large autocorrect replacement table
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Linguistic (show other bugs)
Version:
(earliest affected)
3.3.0 release
Hardware: All Windows (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:3.7.0
Keywords: difficultyInteresting, easyHack, skillCpp
Depends on:
Blocks:
 
Reported: 2012-05-01 12:57 UTC by tommy27
Modified: 2015-12-16 00:21 UTC (History)
7 users (show)

See Also:
Crash report or crash signature:


Attachments
large autocorrect database (317.43 KB, application/octet-stream)
2012-05-01 12:57 UTC, tommy27
Details
autocorrect save freeze (820.87 KB, video/bar)
2012-05-18 13:31 UTC, tommy27
Details
profile from Very Sleepy (31.52 KB, application/octet-stream)
2012-05-23 14:04 UTC, tommy27
Details
huge autocorrect database (119K entries) (679.06 KB, application/octet-stream)
2012-09-21 11:52 UTC, tommy27
Details

Note You need to log in before you can comment on or make changes to this bug.
Description tommy27 2012-05-01 12:57:48 UTC
Created attachment 60864 [details]
large autocorrect database

when you enter the autocorrect replacement table adding new entries, you have to click “OK” to save them when you are done.

If the autocorrect database is very large, it takes several seconds to close the replacement table and save the new entries 

steps to reproduce

1- download the attached acor_it-IT.dat file (more than 47500 entries)
2- put it into the autocorrect folder which is: /home/<user>/.libreoffice/3/user/autocorr (Linux) or 
\Users\<user>\AppData\Roaming\LibreOffice\3\user\autocorr (Windows)
3- open a Writer new sheet
4- Format -> Autocorrect -> Autocorrect options -> Replace
5- select Italian (Italy) 
6- add one random autocorrect entry and click New 
7- click OK

LibO freezes for several seconds before closing the replacement table.
When the autocorrect database is small it's almost instantaneous, but when you have a large database (like in this scenario) things become very slow. 

--------------------

it seems that OOo/LibO autocorrect feature has many performance issues when dealing with large autocorrect databases.

loading the replacement table was incredibly slow in OOo ( https://issues.apache.org/ooo/show_bug.cgi?id=101726 ) with almost 15 minutes needed in OOo 3.3 to load a 65000 entries list.... now thanks to Deszi Szabolcs patch in LibO 3.5.1 this huge time has been reduced to a few seconds. (great improvement indeed!!!)

http://cgit.freedesktop.org/libreoffice/core/commit/?h=libreoffice-3-5-1&id=14f0c12942a8557d3d9dcdaea2e80bad34f40d86

large autocorrect databases also cause some freezing while start typing ( Bug 46805 ) which has been partially solved by Michael Meeks with his 3.5.4 patch that reduced to 50% the freeze time.

http://cgit.freedesktop.org/libreoffice/core/commit/?id=a617eecdb6b2f8d6bb1b20674f8be8ce4d60f2d1&g=libreoffice-3-5


however, saving into the replacement table is still very slow in LibO despite the previous bug fixes. 

adding and saving new autocorrect entries is instead very fast if you use "right click autocorrect" suggestions. 

so the slowness seems related to the closure of the replacement table itself.
Comment 1 tommy27 2012-05-02 05:55:30 UTC
just to add that this issue has been reproduced on different Windows O/S (XP 32bit, Vista 64bit, 7 64bit).

is there any Linux user that can test it as well?
Comment 2 Roman Eisele 2012-05-07 23:00:48 UTC
More or less REPRODUCIBLE with LibreOffice 3.5.3.2 (Build-ID: 235ab8a-3802056-4a8fed3-2d66ea8-e241b80), German langpack installed, Italian spelling module installed (if this is of any importance), on MacOS X 10.6.8 with German UI.

I see the spinning curser for some seconds
* after step 7, as reported by tommy27
* after step 6 (when the new entry is added to the autocorrection table)
* when I repeat steps 4 to 7 immediately after step 7: when I open the 'Autocorrect options' dialog window a 2nd time, the selection 'Italian (Italy)' is already pre-selected (good!) and the large autocorrection table is loaded immediately, this takes some time.

Of course, on my machine I would not call any of these delays a 'freeze' -- they take between 1 and 3 seconds for me. But this may be related to the fact that my machine is not that slow ;-) and has plenty of RAM installed and a SSD (solid-state drive), therefore if there are any disc-access operations involved they will get handled rather fast.

On older machines these delays may take much more time and will be really annoying, I suppose.

Personally I don't think this is a 'most annoying bug' (no regression, no crash, no data loss, no misleading error message, etc.), but I agree it would be very nice if there was some speedup possible.
Comment 3 Roman Eisele 2012-05-07 23:08:55 UTC
(In reply to comment #0)
> 2- put it into the autocorrect folder which is:
> /home/<user>/.libreoffice/3/user/autocorr (Linux) or 
> \Users\<user>\AppData\Roaming\LibreOffice\3\user\autocorr (Windows)

On MacOS X, the autocorrect folder is located at
~/Library/Application Support/LibreOffice/3/user/autocorr
Comment 4 Michael Meeks 2012-05-09 11:51:40 UTC
ditto, for me it takes of the order of two seconds; not so good, but not -so- slow either. I agree that it's not a most-annoying bug on that basis.

Having said that - if someone invests the time to create a callgrind profile of just this bit, I'm most happy to look at improving it.
Comment 5 tommy27 2012-05-09 16:35:46 UTC
@Roman @Micheal

maybe you guys work on very fast machines which helps not noticing the issue which maybe is also less evident on MacOS and Linux.

however I have to sadly confirm than on my system (Windows Vista 64bit Intel Centrino 2, 2.27 GHz, 4GB RAM) the performance is very slow...

the freeze time after adding a new entry and hitting the OK button varies from 15 seconds to even 60 seconds if I added more entries...

having said that, I'm not criticizing the 3.5.x MAB removal... 
I accept this decision and I hope that some Linux user could provide that callgrind testing needed to debug.
Comment 6 tommy27 2012-05-17 03:57:03 UTC
I start thinking this issue is Windows related.
I tried with an high-performance PC (Win7 ultimate sp1 64bit; Intel Xeon @2.66 Ghz 2 processors; 8 GB RAM) and I still experience 15-60 seconds freeze.

this contrasts with reports from Micheal and Roman that experiences just a few seconds delay on Linux and Mac.

so the issue doesn't depend on PC power but rather on O/S.

the same freeze happens with hi-performance Win7-64bit, medium-performance Vista 64-bit and low-performance XP-32bit computers.

so it's not annoying on Mac and Linux
but it's annoying on Windows
Comment 7 tommy27 2012-05-18 13:31:55 UTC
Created attachment 61815 [details]
autocorrect save freeze

I'm attaching a screen recording showing a 40 seconds freeze after saving a single new entry to my autocorrect database. 

test done on Windows Vista 64bit (computer components are described in Comment 5).

that's very annoyoing so I'm thinking about adding it again to the 3.5.x MAB page.
Comment 8 Michael Meeks 2012-05-19 05:12:52 UTC
Please - install a Linux virtual machine, install a build with debug symbols, and run it under callgrind.

Post the callgrind.out.12235 file, and then we can trivially fix it.

The video is not useful - no-one doubts that this operation is slow; the big piece of the work is replicating your system to the point that we can do the profiling to get the above: that is most-likely 95% of the work to fix it - and all of it can be done by a motivated user.

As/when a profile is there, made vs. a build with debug info - I will personally fix it for you.
Comment 9 tommy27 2012-05-19 06:12:04 UTC
Ok Micheal, I'll try my best... I have no experience at all with Linux but I have to start from somewhere...
Comment 10 tommy27 2012-05-19 12:44:41 UTC
my only concern is that -as it seems- the issue is more prominent on Windows (15-60 seconds) than Linux/Max (2-3 seconds according to above posts).

so I wonder if a callgrind on Linux would disclose what's causing a longer freeze on Windows...
Comment 11 dingodog 2012-05-20 10:52:35 UTC
(In reply to comment #8)
> Please - install a Linux virtual machine, install a build with debug symbols,
> and run it under callgrind.
> 
> Post the callgrind.out.12235 file, and then we can trivially fix it.
[...]
are the precompiled binaries releases for Linux x86 built with debug symbols? I downloaded the pre-release of LibreOffice 3.5.4 (3.5.4rc1)

started callgrind

valgrind --tool=callgrind swriter

but no info were written in *.out resulting file

with:

valgrind --tool=callgrind --trace-children=yes swriter

I obtained these *.out files
- http://ge.tt/6pw1TvH/v/0

what specific callgrind switches are needed to get info you need?
Comment 12 tommy27 2012-05-20 12:27:40 UTC
@dingodog

probably the test should be done on particular LibO builds with debug symbols.
check this link: http://packages.debian.org/experimental/libreoffice-dbg
Comment 13 tommy27 2012-05-20 12:45:26 UTC
@ Micheal Meeks

in the meantime I tried to get a profile of the issue on a regular LibO 3.5.3 running on Windows using Very Sleepy 0.82 which should be a similar tool to valgrind for Linux.

my guess is that a better understanding of the bug could be achieved on the same O/S where the bug is worst... the freeze in Linux is very short compared to Windows.

here's the files that the Very Sleepy software generated: 
http://www.sendspace.com/file/u0ou9p  (the .zip contains a .csv file and a proprietary extension .sleepy file)

I do not know if those file can be interpreted correctly by you and and I do not know if I did it the right way (it was the first time I used such kind of software)

remember that I did not use a build with debug symbols...
but if it's necessary I'll download one for Windows from this link:

http://dev-builds.libreoffice.org/win32-debug/libreoffice-3-5/

and and I'll try replicating again the test on Very Sleepy
Comment 14 Michael Meeks 2012-05-21 01:34:13 UTC
> remember that I did not use a build with debug symbols...
> but if it's necessary I'll download one for Windows from this link:

unfortunately, without debugging symbols the output will be useless. It's fine to use a windows / sampling profiler, the problem is so big it would be hard for it not to find it ;-) but without a stack-trace it will just generate meaningless hexadecimal garbage.

Also - please attach small attachments here; sendspace seems to be some mass of advertising, and I couldn't work out how to download your .zip - it gave me a .exe ;-)

Thanks !
Comment 15 Michael Meeks 2012-05-21 01:35:03 UTC
Also - you can download a master build with symbols here:

http://dev-builds.libreoffice.org/daily/W2008R2@20-With-Symbol-Bytemark-Hosting/master/current/
Comment 16 Michael Meeks 2012-05-21 02:02:47 UTC
Hi dindodog,

> are the precompiled binaries releases for Linux x86 built with debug symbols? > I downloaded the pre-release of LibreOffice 3.5.4 (3.5.4rc1)

 No - the debuginfo is rather large, it would double the size of the download. So you would prolly want to just use the version from your linux distribution and install the 'debuginfo' package - the slowness is an old problem so there is no need for a really new build I think.

> what specific callgrind switches are needed to get info you need?

Good point :-) I use:

export OOO_DISABLE_RECOVERY=1
valgrind --tool=callgrind --simulate-cache=yes --dump-instr=yes ./soffice.bin -writer --splash-pipe=0

You need to be able to write to the application's directory so:

sudo chmod a+rw .

or something first. It'd be great to have a wiki page on profiling and/or to update any existing ones if there are any.

Thanks !
Comment 17 dingodog 2012-05-21 03:18:14 UTC
(In reply to comment #16)
> the debuginfo is rather large, it would double the size of the download.
> So you would prolly want to just use the version from your linux distribution
> and install the 'debuginfo' package - the slowness is an old problem so there
> is no need for a really new build I think.
[...]
any url for these packages from libreoffice servers?
Comment 18 tommy27 2012-05-23 14:04:16 UTC
Created attachment 62035 [details]
profile from Very Sleepy

Here's a profile output from LibO 3.5.3 dev with debug symbols captured with Very Sleepy 0.82.

I started the recording just before hitting "ok" to save the new added entry to the replacement table.
Comment 19 tommy27 2012-06-03 13:12:34 UTC
just curious to know if the profile I obtained with Very Sleepy on Windows is somewhat useful to debug.
Comment 20 Michael Meeks 2012-06-04 09:43:31 UTC
Rather a hard to read profile from Very Sleepy - what do the columns mean in the CSV file, eg. this looks bad:

SvTreeEntryList::remove	2.568197	79.886709	0.326782	10.164923	svtlo	[unknown]

It'd be nice to work out where that is getting called from and get a stack trace: the nice thing about callgrind of course is that it gives a full, hierarchical map of what is called from where, that makes fixing this stuff really really easy. No doubt something quite silly happening there.
Comment 21 Korrawit Pruegsanusak 2012-07-25 14:33:08 UTC
Using LibO 3.5.4 on Ubuntu 12.04 x64, the callgrind log is at: http://www.mediafire.com/?1urd2y4y5qvgorj

Hope it helps
Comment 22 Korrawit Pruegsanusak 2012-07-25 16:14:42 UTC
(In reply to comment #21)

Forgot to say: after step 6, I left the computer and got out to have a meal. When I got back, it finished and I did step 7. I don't know whether this alters the result log. Thanks again.
Comment 23 tommy27 2012-07-26 01:13:52 UTC
(In reply to comment #21)
> Using LibO 3.5.4 on Ubuntu 12.04 x64, the callgrind log is at:
> http://www.mediafire.com/?1urd2y4y5qvgorj
> 
> Hope it helps


thank you very much!!! I hope that Micheal Meeks is still available to fix it
Comment 24 tommy27 2012-08-18 11:24:09 UTC
(In reply to comment #23)
> (In reply to comment #21)
> > Using LibO 3.5.4 on Ubuntu 12.04 x64, the callgrind log is at:
> > http://www.mediafire.com/?1urd2y4y5qvgorj
> > 
> > Hope it helps
> 
> 
> thank you very much!!! I hope that Micheal Meeks is still available to fix it


hi Michael, I don't want to seem pushy, but I just wanted to know if you are aware of the backtrace provided by Korrawit
Comment 25 Michael Meeks 2012-08-20 10:17:20 UTC
> hi Michael, I don't want to seem pushy, but I just wanted to know
> if you are aware of the backtrace provided by Korrawit

Drat - I missed that :-) Poking at the trace (thanks for that) in kcachegrind - it seems like a big chunk of the problem is here:

SvLBoxString::InitViewData(SvLBox*,SvLBoxEntry *, SvViewDataItem *)

Which is called 484,000 times - and chews up 25% of the total time (including startup etc.) mostly in measuring text via. OutputDevice::GetTextWidth()

That's fun :-)

That in turn is called from SvLBox::RecalcViewData - which is called 11 times (seems odd), from SvTreeListBox::SetFont - which is called 9x times (odd).

That is called from SvTreeListBox::PaintEntry1 - which is called 80x times - so, perhaps if we can work out what's up with changing the font there, we could get a 10x speedup of that piece.

I've easy-hackified this, hopefully that'll increase interest.

Thanks for the trace !
Comment 26 tommy27 2012-08-22 15:59:27 UTC
Yes, let's hope that as an easy hack this issue will be soon fixed.

As an additional information, the “freeze” only happens when saving new entries in the “replacement table GUI mode”.

If you save a new entry with “right click autocorrect suggestions” there's no freeze at all, so I suspect that there's something wrong with the GUI mode.
Comment 27 Not Assigned 2012-09-20 19:22:05 UTC
Tomaž Vajngerl committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=3da2a6a58784a3c607ea4e440865478f1a4fe56e

fdo#49350 Speedup entry painting for SvTreeListBox



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 28 tommy27 2012-09-21 05:39:33 UTC
@Tomaz

tested on master 2012-09-20_23.38.37

I actually see no difference in freeze time after adding a new autocorrect entry.
Maybe that daily build did not yet include your freeze or the fix was not effective.
Comment 29 Tomaz Vajngerl 2012-09-21 08:39:44 UTC
Hi,

The fix is already in master 2012-09-20_23.38.37 - I tested this build now.

This fix primary speeds up changing of selection in auto-correct dialog. You can see the difference when holding up or down key so the selection changes in the list. This was too slow in Linux but bearable when I tested it in Windows. Now it is fast in both environments. 

Inserting should be a little faster too - for me inserting a new entry in Linux was 8 sec. and now it is <1 sec. In Windows I don't see any difference at inserting - it is fast <1 sec.

The biggest delay left for me now is when you click OK - on this action all the entries are synchronized and written into autocorr file and this takes too long IMHO. I will try to improve this.

Can you try again and can you put your "merged" autocorr file somewhere so I can test with 100.000+ entries. 

Regards, Tomaž
Comment 30 tommy27 2012-09-21 11:52:55 UTC
Created attachment 67494 [details]
huge autocorrect database (119K entries)

@ Tomaz

as you asked I'm sending you the new autocorrect file for testing.

regarding the present issue, the freeze I was talking about was exactly the one that happens when you hit the "OK" button.

on my Vista 64bit laptop the master build takes 5 minutes to close and save the replacement table using the 119K entries database... unacceptable...

with a smaller database (45K entries) current LibO 3.6.1 takes 40 seconds... still very annoying... (see attachment 61815 [details] )

regarding your fix now I've noticed the difference and I consider it a nice improvement. however the "bottle neck" is still represented by the "OK" freeze that hopefully you will fix as well
Comment 31 tommy27 2012-09-22 13:56:49 UTC
(In reply to comment #29)
> snip
> 
> The biggest delay left for me now is when you click OK - on this action all the
> entries are synchronized and written into autocorr file and this takes too long
> IMHO. I will try to improve this.

from tests I've done on master LibO 3.7.x with your fix, it seems that the "click OK" delay lenght depends upon 2 factors:

1- size of the autocorrect database
2- alphabetical position of the newly added entry

which means that it takes longer to save new entries in larger databases, and it takes longer to save a "Z" entry rather than an "A" entry.

adding a "A" entry to the replacement table takes:
-  8 seconds in a  55K acor.dat file
- 22 seconds in a 119K acor.dat file

adding a "Z" entry instead takes:
- 1 minute  and 40 seconds in a  55K acor.dat file
- 6 minutes and 10 seconds in a 119K acor.dat file

tests were performed on Windows Vista 64bit, Intel Centrino 2, 2.27 GHz, 4GB RAM).

so it seems to me that most of the time is used/wasted resorting in alphabetical order the whole autocorrect database.

on the other hand adding a new autocorrect entry from "mouse right click suggestions" is extremely fast (1 or 2 seconds; maybe slightly slower when adding to a 119K database rather than a 55K database).

I have a theory about this different behaviour between "mouse" and "replacement table" autocorrect methods but I need to do another test to confirm it or not.... stey tuned... more to come...


> Can you try again and can you put your "merged" autocorr file somewhere so I
> can test with 100.000+ entries. 
> 
> Regards, Tomaž

did you see my upload?
Comment 32 tommy27 2012-09-22 14:45:04 UTC
ok... some more test feedback...

DocumentList.xml (which contains all the autocorrect entries) works even if entries are not sorted alphabetically (I manually edited the file changing position of entries and then repacked it as .dat). 

You can use it even unsorted and the .xml remains unchanged until you add a new entry.... once you do it, either using mouse right click suggestions or manually adding to the autocorrect replacement table, the DocumentList.xml is updated and resorted in alphabetical order.

So, the “mouse” and “table” add new entries resorting alphabetically the .xml but the mouse does it a lot faster. We need to discover the reasons of this bad performance of the table.

Tomaz, regarding your fix, did you actually touch the areas that Micheal Meeks said could be responsible for a 10x speedup (see Comment 25 ) or did you tweak another part of the code?
Comment 33 Tomaz Vajngerl 2012-09-22 19:51:51 UTC
(In reply to comment #31)
> from tests I've done on master LibO 3.7.x with your fix, it seems that the
> "click OK" delay lenght depends upon 2 factors:
> 
> 1- size of the autocorrect database
> 2- alphabetical position of the newly added entry
> 
> which means that it takes longer to save new entries in larger databases, and
> it takes longer to save a "Z" entry rather than an "A" entry.
> 
> adding a "A" entry to the replacement table takes:
> -  8 seconds in a  55K acor.dat file
> - 22 seconds in a 119K acor.dat file
> 
> adding a "Z" entry instead takes:
> - 1 minute  and 40 seconds in a  55K acor.dat file
> - 6 minutes and 10 seconds in a 119K acor.dat file
> 
> tests were performed on Windows Vista 64bit, Intel Centrino 2, 2.27 GHz, 4GB
> RAM).
Ouch.. this is crazy.
 
> so it seems to me that most of the time is used/wasted resorting in
> alphabetical order the whole autocorrect database.
> 
> on the other hand adding a new autocorrect entry from "mouse right click
> suggestions" is extremely fast (1 or 2 seconds; maybe slightly slower when
> adding to a 119K database rather than a 55K database).
> 
> I have a theory about this different behaviour between "mouse" and "replacement
> table" autocorrect methods but I need to do another test to confirm it or
> not.... stey tuned... more to come...
No.. the actual problem is (or was) that the dialog makes its own copy of all entries and afterwards it tries to synchronize its own copy with the main structure (where the entries are stored and checked when you type) in a very non-efficient way. The behavior you saw was probably the side-effect of the synchronization. For 'z' entries the algorithm needed to go through the whole list to find a match - but for 'a' entries only some entries at the beginning.  

I have changed this so I only store what has been deleted and added to the list and now there is no need for synchronization at all. Using this approach it takes about 12 sec. when I change some entries (at the beginning or end). It generally works but I have to test all scenarios and make some more improvements to the code before I commit - so it may take some time.

It still takes too long to save to a file and I think that I know the reason. I think that every time a new entry is added, the acor file is written. Which then means that when you add 10 entries, acor file is (re)written 10 times. I am not yet sure about this but maybe you can test this.

> did you see my upload?
Yes thanks!

> Tomaz, regarding your fix, did you actually touch the areas that Micheal Meeks
> said could be responsible for a 10x speedup (see Comment 25 ) or did you tweak
> another part of the code?
Yes - the first fix was based on information on this page. I tweaked the usage of SetFont method in PaintEntry1.
Comment 34 tommy27 2012-09-23 06:45:05 UTC
(In reply to comment #33)
> 
> No.. the actual problem is (or was) that the dialog makes its own copy of all
> entries and afterwards it tries to synchronize its own copy with the main
> structure (where the entries are stored and checked when you type) in a very
> non-efficient way.   -- snip --

that was crazy and explain the worsening performance once the autocorrect database becomes larger

> I have changed this so I only store what has been deleted and added to the 
> list and now there is no need for synchronization at all. 
> Using this approach it takes about 12 sec. 

WOW... let me say that I would already be very happy with such time.
This is already a drastic improvement from the 6 minutes and 10 seconds I experienced with "z" entries

> when I change some entries (at the beginning or end). It
> generally works but I have to test all scenarios and make some more
> improvements to the code before I commit - so it may take some time.

don't worry, take the time you need. I'm struggling with OOo/Lib autocorrect performance issues from years and I can wait.

> It still takes too long to save to a file and I think that I know the reason. I
> think that every time a new entry is added, the acor file is written. Which
> then means that when you add 10 entries, acor file is (re)written 10 times. I
> am not yet sure about this but maybe you can test this.

mmmhhh... I do not confirm this... from my tests it seems that the acor_.dat file is saved only when you hit OK (I monitored the "last modified" infos on file properties), not after each new added entry... 

did you actually see time difference when hitting OK after adding a single or multiple new entries? does it always take 12 seconds, regardless of the number of newly added items?

P.S. if you have a home-made test build with your latest fixes, I'll be glad to beta test it as well
Comment 35 khagaroth 2012-09-23 07:44:33 UTC
> > It still takes too long to save to a file and I think that I know the reason. I
> > think that every time a new entry is added, the acor file is written. Which
> > then means that when you add 10 entries, acor file is (re)written 10 times. I
> > am not yet sure about this but maybe you can test this.
> 
> mmmhhh... I do not confirm this... from my tests it seems that the acor_.dat
> file is saved only when you hit OK (I monitored the "last modified" infos on
> file properties), not after each new added entry... 

I think what he meant was that _after hitting OK_, the file is saved repeatedly for each entry you added.
Comment 36 tommy27 2012-09-23 08:20:18 UTC
(In reply to comment #35) 
> 
> I think what he meant was that _after hitting OK_, the file is saved repeatedly
> for each entry you added.

It could be... however if it's like that, he should notice different saving times depending if a single entry or multiple ones have been added.

Tomaz said he was able to bring the saving time down to 12 seconds but he did not specify if that time refers to a single entry or more than one.
Comment 37 Tomaz Vajngerl 2012-09-23 09:50:09 UTC
Hmm..

First time it takes about 12 sec. when adding 3 new entries, second time it takes 6 sec. (probably because the file is cached). If I add only 1 entry it takes 2 sec., 2 entries 4 sec. - So it looks like it really rewrites the acor file for every add/delete. I will need to add a combined command which makes all changes in one step - then probably it will take 2-4 sec. regardless of how many changes are made. 

P.S. I remembered I use a debug environment for testing - which is not compiler optimized - so it could be even faster in an actual daily build / release. 

Regards, Tomaž
Comment 38 tommy27 2012-09-23 10:55:14 UTC
@Tomaz

I'm impressed in your progresses... you are really boosting the autocorrect feature performances!!!
Comment 39 Not Assigned 2012-09-23 20:04:59 UTC
Tomaž Vajngerl committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=0513e10635c85fc1aa214948de4992d4b76d555c

fdo#49350 Speedup "OK" action of auto-correct dialog



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 40 tommy27 2012-09-24 20:02:16 UTC
unfortunately I cannot beta-test your fix since latest Windows daily does not install...  see Bug 55290
Comment 41 Korrawit Pruegsanusak 2012-10-02 15:21:57 UTC
(In reply to comment #40)
> unfortunately I cannot beta-test your fix since latest Windows daily does
> not install...  see Bug 55290

In the meantime, you might use this method http://wiki.documentfoundation.org/Installing_in_parallel#Windows
Comment 42 Tomaz Vajngerl 2012-10-02 16:36:33 UTC
(In reply to comment #41)
> (In reply to comment #40)
> > unfortunately I cannot beta-test your fix since latest Windows daily does
> > not install...  see Bug 55290
> 
> In the meantime, you might use this method
> http://wiki.documentfoundation.org/Installing_in_parallel#Windows

Cool.. it installs. I tried it with W2008 build which is recent (http://dev-builds.libreoffice.org/daily/W2008R2@16-minimal_build/master/). This one also works in windows 7 (or other windows).
Comment 43 tommy27 2012-10-02 19:24:56 UTC
@Korrawit
thanks for the hint. I was finally able to install a new LOdev build.

@Tomaz
tested your fix on master~2012-09-26_20.38.45_LibO_Dev_3.7.0.0.alpha0_Win_x86_install_en-US.msi

my first comment is: WOW!! How fast!!!
you brought down the saving time from minutes to a few seconds!!!
this is a terrific improvement!!! it improves productivity and user experience... no more frustrating waiting time when saving new autocorrect entries!!!

so setting this issue as RESOLVED FIXED in LibO dev3.7

now, I hope that your two commits ( http://cgit.freedesktop.org/libreoffice/core/commit/?id=3da2a6a58784a3c607ea4e440865478f1a4fe56e and http://cgit.freedesktop.org/libreoffice/core/commit/?id=0513e10635c85fc1aa214948de4992d4b76d555c ) could be cherry-picked to LibO 3.6.3.

I do not know what the correct procedure is... should you ask in the developer's list a patch review in order to push the commits to the 3.6 branch?
Comment 44 Korrawit Pruegsanusak 2012-10-03 04:37:40 UTC
(In reply to comment #43)
> I do not know what the correct procedure is... should you ask in the
> developer's list a patch review in order to push the commits to the 3.6
> branch?

Yes. Write an email to developer list with subject:
  [REVIEW] $an_explanation_here
and message with link to the commits.
Comment 45 tommy27 2012-10-03 05:39:27 UTC
@Korrawit

just to make it clearer... has that mail to be sent from the committ author (Tomaz) or is something that I can even do it myself?
Comment 46 Korrawit Pruegsanusak 2012-10-03 05:46:52 UTC
(In reply to comment #45)
> just to make it clearer... has that mail to be sent from the committ author
> (Tomaz) or is something that I can even do it myself?

You can do it yourself. (I already *did* it)
Anyway, I'm not sure if the patches will pass the review, especially the second one, because it's large, but I might be wrong of course ;)
Comment 47 Michael Meeks 2012-10-03 09:00:23 UTC
Ho hum; we can try getting them reviewed for -3-6 of course; but I'd want the developer of the code to recommend a suitable patch themselves, preferably with a 'diff -w' posted for people to look at, rather than someone un-connected with the implementation.

That said - this looks like a really cool fix for a long-standing annoyance.

It would be great to add it to:

https://wiki.documentfoundation.org/ReleaseNotes/3.7

with suitable credits to developer & testers :-)
Comment 48 Roman Eisele 2012-10-03 09:52:33 UTC
(In reply to comment #47)
> It would be great to add it to:
> 
> https://wiki.documentfoundation.org/ReleaseNotes/3.7
> 
> with suitable credits to developer & testers :-)

I have already added a short notice there some days ago:
  https://wiki.documentfoundation.org/ReleaseNotes/3.7#Performance
but please improve and exand it!
Comment 49 tommy27 2012-10-10 11:53:24 UTC
@Tomaz Vajngerl

sorry for keeping bothering you but I still hope that you'll find the time to ask review in the developers list to cherry-picki this Bug 49350 and Bug 48279 to LibO 3.6.3.

@Roman

good idea, I'll prepare some graphs and text for the wiki page.
Comment 50 Tomaz Vajngerl 2012-10-10 17:06:48 UTC
(In reply to comment #49)
> @Tomaz Vajngerl
> 
> sorry for keeping bothering you but I still hope that you'll find the time
> to ask review in the developers list to cherry-picki this Bug 49350 and Bug
> 48279 to LibO 3.6.3.
> 

Hi, the problem is that I also have to prepare the patch, not just ask for review. To do this I have to switch to 3.6 development tree, rebuild everything, make changes, test if it works, prepare the patch, ask for review and inclusion and if I want to go back to regular development, switch back to 3.7 tree, rebuild everything again, continue development. For this I need time and hard drive space but currently I want to spend this on something more productive like removing lag when typing in writer when using large autocorrect databases. Also it is probably too late for 3.6.3 anyway.

Regards, Tomaž
Comment 51 tommy27 2012-10-10 17:32:09 UTC
@Tomaz
thank you very much. I was not aware of the effective work needed for a 3.6.x cherry-pick... now that it's clearer to me how the whole thing works, I agree with you: it's better to keep polishing the autocorrect feature in the 3.7.x branch and maybe backport to 3.6.x when everything is fine. I'll be glad to beta-test any of your next fixes.
Comment 52 Robinson Tryon (qubit) 2015-12-16 00:21:24 UTC
Migrating Whiteboard tags to Keywords: (EasyHack,DifficultyInteresting,SkillCpp )
[NinjaEdit]