Download it now!
Bug 79067 - MAILMERGE: mailmerge takes ages to create documents
Summary: MAILMERGE: mailmerge takes ages to create documents
Status: RESOLVED DUPLICATE of bug 56355
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
3.6.7.2 release
Hardware: Other Linux (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: BSA lhm-limux
Keywords:
Depends on: 80823
Blocks:
  Show dependency treegraph
 
Reported: 2014-05-22 12:20 UTC by Stefan Koehler
Modified: 2014-08-01 11:57 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
template to use as starting document (38.23 KB, application/vnd.oasis.opendocument.text-template)
2014-05-22 12:20 UTC, Stefan Koehler
Details
datasource for testing mailmerge containing 8000 recipients (393.71 KB, application/vnd.oasis.opendocument.spreadsheet)
2014-05-22 12:22 UTC, Stefan Koehler
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Koehler 2014-05-22 12:20:56 UTC
Created attachment 99580 [details]
template to use as starting document

Problem description: When creating a larger number of documents (8000, real life example) using mailmerge, the process is very slow in the beginning and is even slowing down constantly with a growing number of created documents. LO seems to scan every single document, so in the end it will have to check an insane number of documents. It usually crashes before reaching a large number, though.

While creating the documents the process sucks 100% CPU time and LO is quite unresponsive.

Some benchmarks, taken on a rather fast i5 machine:

Number of docs crated | time consumed (in minutes)

 500 |  3:20
 750 |  8:10
1000 | 17:15
1500 | 56:50
1750 | 92:20

Version tested: 4.4.0.0.alpha0+. The exact same problem also exists in versions 4.1.3, 4.1.5 and 4.2.4, all tested on a deb-based Linux.

Steps to reproduce:

Open attached document "mailmerge_testtemplate.ott", enter mailmerge wizard, choose attached document "mailmerge_datasource_8000.ods", match data fields and go!

Current behavior: As stated above the time and ressources consumed are just crazy, it is unpossible to use mailmerge for creating a larger number of documents.

Expected behavior: Documents should be created within a reasonable time span, the application should be responsive during that time.

              
Operating System: Ubuntu
Version: 4.3.0.0.alpha0+ Master
Comment 1 Stefan Koehler 2014-05-22 12:22:59 UTC
Created attachment 99581 [details]
datasource for testing mailmerge containing 8000 recipients
Comment 2 Stefan Koehler 2014-05-22 12:48:59 UTC
Addendum: on a 64bit platform performance is significantly worse: 1000 documents took slightly more than one hour.
Comment 3 Yousuf Philips (jay) (retired) 2014-05-31 04:13:59 UTC
Confirmed in Linux Mint in 3.6.7 and 4.3 beta.
Comment 4 Cor Nouws 2014-06-17 11:50:33 UTC
I guess there's no problem when I mark this as duplicate of bug 56355 ??

*** This bug has been marked as a duplicate of bug 56355 ***
Comment 5 Jan-Marek Glogowski 2014-06-17 17:56:23 UTC
(In reply to comment #4)
> I guess there's no problem when I mark this as duplicate of bug 56355 ??
> 
> *** This bug has been marked as a duplicate of bug 56355 ***

Well bug 56355 has no example document attached.

And actually the MM speed depends on the type of document.

Just look at Brenda Granados results in bug 56355. These are fine (1:20). But the MM document is very simple.
Comment 6 Michael Meeks 2014-07-09 11:13:04 UTC
Is there any chance of a callgrind trace vs. a build with debugging symbols (the last part is critical).

Something being -this- slow must have a really lame cause somewhere (some O(N^4) or somesuch).

Of course that will take a lot of CPU time, I'd suggest doing a merge of the smallest, fastest document we can the largest number of times to find the high order in number of mails.

Thanks !
Comment 7 bfoman (inactive) 2014-07-11 17:38:33 UTC
Checked with:
Version: 4.3.0.1
Build ID: 67f5430184326974072b65403ef1d9d934fc4481
Windows 8.1 Enterprise 64 bit

Instead of using Wizard directly used File>Print approach. Writing directly to separate odt files per each record took:
- 500 files - ~3:00
- 1000 - ~10:00
- 1500 - ~17:00
and so on... All this is at constant 1000 files in 10 minutes ratio.
System is responsive, LO only at 20% of CPU (i5). Speed is 2 files per second, but I have low disk write speed. Unfortunately counter tends to got stuck sometimes (you have to move the windows few times).
Selecting direct printing took:
- 1000 records - ~4:00
and so on... All this is at constant 1000 records in 4 minutes ratio.
Speed is x(x) records per second.
All in all - still slow if you generate xxx(x) records/files, but definitely faster than using the Wizard.
Comment 8 Jan-Marek Glogowski 2014-08-01 11:57:27 UTC
FYI: the callgrind is attached to #80823.