Bug 119942 - Mail Merge: data sent is off by one (wrong record index in merged result) if "Next Mail Merge Entry" used in Toolbar and saved as individual
Summary: Mail Merge: data sent is off by one (wrong record index in merged result) if ...
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
(earliest affected) release
Hardware: All All
: medium normal
Assignee: Not Assigned
Keywords: implementationError
Depends on: 132678
Blocks: Mail-Merge
  Show dependency treegraph
Reported: 2018-09-17 22:57 UTC by brief
Modified: 2022-09-29 09:53 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:

sample ods file (17.98 KB, application/vnd.oasis.opendocument.spreadsheet)
2018-09-17 22:58 UTC, brief
sample odt file (10.37 KB, application/vnd.oasis.opendocument.text)
2018-09-17 22:58 UTC, brief
Another minimal odt file (10.11 KB, application/vnd.oasis.opendocument.text)
2020-10-10 11:54 UTC, Kevin Suo
Another minimal ods file (datasource) (9.13 KB, application/vnd.oasis.opendocument.spreadsheet)
2020-10-10 11:55 UTC, Kevin Suo
merged result 7z from the minimal file (12.43 KB, application/x-7z-compressed)
2020-10-10 12:00 UTC, Kevin Suo

Note You need to log in before you can comment on or make changes to this bug.
Description brief 2018-09-17 22:57:44 UTC
On sending a mailmerge with a datasource of a calc ods file, the data in the mail body does not match the recipients data.

E.g. a mail is sent for mail8@host (data from line 8, column email) with data from line 7.

Steps to Reproduce:
0. Edit the ods file to contain email addresses.
1. Open the odt file.
2. Attach the ods file using the mail merge assistant.
3. Delete the fields and re-add them using the correct data source.
4. Send emails.

Actual Results:
Hi 7,
1  7,00 €
2 14,00 €

Expected Results:
Hi 8,
1  8,00 €
2 16,00 €

Reproducible: Always

User Profile Reset: Yes

OpenGL enabled: Yes

Additional Info:
Version: (x64)
Build-ID: 0c292870b25a325b5ed35f6b45599d2ea4458e77
CPU-Threads: 4; BS: Windows 6.3; UI-Render: Standard; 
Gebietsschema: de-DE (de_DE); Calc: group
Comment 1 brief 2018-09-17 22:58:36 UTC
Created attachment 144965 [details]
sample ods file
Comment 2 brief 2018-09-17 22:58:52 UTC
Created attachment 144966 [details]
sample odt file
Comment 3 brief 2018-09-17 23:02:00 UTC Comment hidden (obsolete)
Comment 4 brief 2018-09-17 23:15:39 UTC
After removing the surrounding table, the correct data is used.
Comment 5 Buovjaga 2018-10-13 11:25:40 UTC

You missed instructions: to match the fields to data source entries, double-click them (Vorname, asd, asd2) and pick the columns with the matching names.

Arch Linux 64-bit
Build ID: 6.1.1-1
CPU threads: 8; OS: Linux 4.18; UI render: default; VCL: gtk3_kde5; 
Locale: fi-FI (fi_FI.UTF-8); Calc: group threaded
Comment 6 QA Administrators 2019-10-16 02:29:46 UTC Comment hidden (obsolete)
Comment 7 Kevin Suo 2020-10-10 11:54:50 UTC
Created attachment 166250 [details]
Another minimal odt file

This bug exists for a long time, and still exists on the current master build.

The problem is, when the mail merge file (i.e., the odt file) contains a table, then the merged result would have the 1st record in the 1st result, but the 2nd result still contains the 1st record, the 3rd result contains the 2nd record, and so on, and finally the last record would contain both the last-1 and the last record in the same document.

The problem does not happen if the odt file does not contain a table.

Attached is a minimal odt file.

Steps to Reproduce:

1. Open the "Another minimal odt file", double-click on any of the two fields, then browse "Add database file", choose the "Another minimal ods file" attached to the next comment.

2. Update the two fields to use the datasource you have added in step 1. Save and reopen the odt file.
(LibreOffice may crash at this step, but this may be a different issue and I will report in a separate report)

3. Enable Mail Merge toolbar (View -> Toolbars -> Mail Merge), click once the "Next Mail Merge Entry" (i.e., -> button in the Mail Merge toolbar). Then click "Save Merged Documents -> Save as Individual documents" on the same toolbar. Choose a location and save.

4. Open the merged individual odt files.
--> The 1st result correctly contains the 1st record ("Kevin Suo"). The 2nd result wrongly still contains the 1st record ("Kevin Suo"). The 3rd result wrongly contains both the 2nd record ("Xiujuan Liu") and the 3rd record ("Aixi Suo") in the same file on two different pages.

The expected result is, 1st record in the 1st result, 2nd record in the 2nd result, and 3rd record in the 3rd result.
Comment 8 Kevin Suo 2020-10-10 11:55:24 UTC
Created attachment 166251 [details]
Another minimal ods file (datasource)
Comment 9 Kevin Suo 2020-10-10 12:00:38 UTC
Created attachment 166252 [details]
merged result 7z from the minimal file
Comment 10 marjan 2021-07-18 00:17:24 UTC
@Kevin Suo <suokunlong@126.com>: Thank you for your effectively minimized error report. I am suffering the same bug for many years. In good old days this used to work.

I am very sad that we don't have enough resources to kill this bug: Mail Merge fields in an array of an odt- document.

I have always used a header of all my letters with the 2x2 table with specific dimension to fit the envelope with the transparent "address-window".

I am suffering this bug for years. I am forced to write a bash script each time I make a significantly different Mail-Merge document to cut resulting one-PDF to individual letters, where I am forced to keep all copies of the same number of pages, ...

Unfortunately, I am a "complete newbie" in LibreOffice source code, but if I could get enough support, I might be able attack this bug myself or preferably in a group of co-sufferers.

@documentfoundation.org: Assignee: Not Assigned, really? 

Kevin Suo <suokunlong@126.com>, thanks again that I do not have to repeat the report, just quote it.
Comment 11 Kevin Suo 2021-07-18 00:38:18 UTC
Currently there is no one working on this.

Yes, of course you can try to fix this by yourself. You can search the code on https://opengrok.libreoffice.org/, then set breakpoints to where you are interested, then it will not be difficult to find the reason, if you know some cpp.
Comment 12 Michael Weghorn 2021-07-19 06:32:19 UTC
(In reply to marjan from comment #10)
> @Kevin Suo <suokunlong@126.com>: Thank you for your effectively minimized
> error report. I am suffering the same bug for many years. In good old days
> this used to work.

If this used to work in the past, this is a regression and doing a bibisect could help finding out what change introduced the problem. Once that is known, finding the root cause and a potential fix is usually much easier.

See this wiki page:
Comment 13 Michael Weghorn 2021-07-19 06:34:46 UTC
In case you want to take a look and debug into this yourself, I have taken those notes for potential starting points for mail merge a while ago:

> * Relevant code places:
>     * `sw/inc/dbmgr.hxx`, `sw/source/uibase/dbui/dbmgr.cxx`, in particular the
>       whole method `MergeMailFiles()`
>     * unit tests are in `sw/qa/extras/mailmerge/mailmerge.cxx` -> have a look
>       at existing ones in order to see how this works
>     * implementation for mail merge related dialogs: `sw/source/ui/dbui/mmresultdialogs.cxx`
>     * UNO API method implementations: `sw/source/uibase/uno/unomailmerge.cxx`
Comment 14 Kevin Suo 2021-07-19 07:23:09 UTC
Could someone reproduce the crash following the steps I have mentioned in comment 7? I see it on master dbgutil build.

The crash may have related to this index-off issue, but we may need to fix the crash first before we see whether they are of the same cause.
Comment 15 Timur 2021-08-31 11:37:00 UTC
Repro per Comment 7 in LO 6.4 and 7.3+, using ODT attachment 166250 [details]. If ODS attachment 166251 [details] is saved with original name, no need to "Add database file" or update fields.

Repro only with all this: using Mail Merge toolbar, "Next Mail Merge Entry", "Save Merged Documents -> Save as Individual documents". 
No repro without toolbar but with File-Print or with Mail Merge toolbar saving as Single document, regardless of "Next Mail Merge Entry" button. 

MM toolbar appeared in LO 5.2 but Next Entry worked in 5.3 and it's already wrong there. So I set implementationError.

With LO 7.3+, I just have reliable crash on LO exit after using MM. 
@Kevin: you may try to reset user profile and just use ODT from the same folder where you have ODS.
Comment 16 Timur 2021-08-31 11:43:59 UTC
It might be that original report and bug 139072 are bug 106959 that is a regression, while I tested per Comment 7 something else that's related to using toolbar.
Comment 17 Viniasco 2022-09-29 09:37:14 UTC

Le bogue existe toujours dans la dernière version de LibreOffice. 
Le publipostage fonctionne quand il se fait vers un fichier, mais est décalé quand il est vers des méls, lorsque que le .odt contient un tableau.