Created attachment 99580 [details] template to use as starting document Problem description: When creating a larger number of documents (8000, real life example) using mailmerge, the process is very slow in the beginning and is even slowing down constantly with a growing number of created documents. LO seems to scan every single document, so in the end it will have to check an insane number of documents. It usually crashes before reaching a large number, though. While creating the documents the process sucks 100% CPU time and LO is quite unresponsive. Some benchmarks, taken on a rather fast i5 machine: Number of docs crated | time consumed (in minutes) 500 | 3:20 750 | 8:10 1000 | 17:15 1500 | 56:50 1750 | 92:20 Version tested: 4.4.0.0.alpha0+. The exact same problem also exists in versions 4.1.3, 4.1.5 and 4.2.4, all tested on a deb-based Linux. Steps to reproduce: Open attached document "mailmerge_testtemplate.ott", enter mailmerge wizard, choose attached document "mailmerge_datasource_8000.ods", match data fields and go! Current behavior: As stated above the time and ressources consumed are just crazy, it is unpossible to use mailmerge for creating a larger number of documents. Expected behavior: Documents should be created within a reasonable time span, the application should be responsive during that time. Operating System: Ubuntu Version: 4.3.0.0.alpha0+ Master
Created attachment 99581 [details] datasource for testing mailmerge containing 8000 recipients
Addendum: on a 64bit platform performance is significantly worse: 1000 documents took slightly more than one hour.
Confirmed in Linux Mint in 3.6.7 and 4.3 beta.
I guess there's no problem when I mark this as duplicate of bug 56355 ?? *** This bug has been marked as a duplicate of bug 56355 ***
(In reply to comment #4) > I guess there's no problem when I mark this as duplicate of bug 56355 ?? > > *** This bug has been marked as a duplicate of bug 56355 *** Well bug 56355 has no example document attached. And actually the MM speed depends on the type of document. Just look at Brenda Granados results in bug 56355. These are fine (1:20). But the MM document is very simple.
Is there any chance of a callgrind trace vs. a build with debugging symbols (the last part is critical). Something being -this- slow must have a really lame cause somewhere (some O(N^4) or somesuch). Of course that will take a lot of CPU time, I'd suggest doing a merge of the smallest, fastest document we can the largest number of times to find the high order in number of mails. Thanks !
Checked with: Version: 4.3.0.1 Build ID: 67f5430184326974072b65403ef1d9d934fc4481 Windows 8.1 Enterprise 64 bit Instead of using Wizard directly used File>Print approach. Writing directly to separate odt files per each record took: - 500 files - ~3:00 - 1000 - ~10:00 - 1500 - ~17:00 and so on... All this is at constant 1000 files in 10 minutes ratio. System is responsive, LO only at 20% of CPU (i5). Speed is 2 files per second, but I have low disk write speed. Unfortunately counter tends to got stuck sometimes (you have to move the windows few times). Selecting direct printing took: - 1000 records - ~4:00 and so on... All this is at constant 1000 records in 4 minutes ratio. Speed is x(x) records per second. All in all - still slow if you generate xxx(x) records/files, but definitely faster than using the Wizard.
FYI: the callgrind is attached to #80823.