Bug Hunting Session
Bug 34187 - MAILMERGE Transfer status" remains on "0 of 0 sent" with attach as pdf
Summary: MAILMERGE Transfer status" remains on "0 of 0 sent" with attach as pdf
Status: CLOSED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
3.3.1 RC1
Hardware: Other All
: medium critical
Assignee: Caolán McNamara
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 34184
  Show dependency treegraph
 
Reported: 2011-02-11 09:14 UTC by Rainer Bielefeld Retired
Modified: 2011-05-03 21:11 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
This is the doucment I used for my latest tests. (91.84 KB, application/vnd.oasis.opendocument.text)
2011-02-11 09:14 UTC, Rainer Bielefeld Retired
Details
Crash Report created with OOo3.3.0 (12.78 KB, text/plain)
2011-02-19 04:04 UTC, Rainer Bielefeld Retired
Details
Trace on Mac OSX when mailmerge e-mailing hangs (46.07 KB, text/plain)
2011-03-11 05:41 UTC, Alex Thurgood
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Rainer Bielefeld Retired 2011-02-11 09:14:16 UTC
Created attachment 43256 [details]
This is the doucment I used for my latest tests.

I successfully sent an e-mail with this functionexactly  1 time, afterwards LibO crashed, and furthermore I have been unable to send Emails "LibreOffice 3.3.1 RC1 - WIN7  Home Premium (64bit) German UI  [OOO330m19 (build 7 / tag 3.3.1.1)]"

Everything works fine until Step 8, I select field with E-Mail-Address in my Address database spreadsheet and PDF-document, press "Send document" and send Message appears. Connection status is "The connection to the outgoing mail server has been established", but "Transfer status" remains on "0 of 0 sent", and generally after some minutes LibO will crash.
There is no unexpected CPU load.
Comment 1 Rainer Bielefeld Retired 2011-02-11 10:48:00 UTC
Added some stuff from Bug 31792 - MAILMERGE: Mail Merge Wizard hangs hoping that someone will be able to help me to gather useful information.

I downloaded MS Process Monitor v2.94, but it's not really self explicating :-/

May be it helps to find the reason of the problem:
After the crash I always have a document "Untitled" that can not be recovered; Ibeleive it's the merged document created by the wizard.
Comment 2 Rainer Bielefeld Retired 2011-02-11 10:52:50 UTC
!staff!
Comment 3 Rainer Bielefeld Retired 2011-02-11 23:36:43 UTC
I set "dbg = True" in "mailmerge.py", where can I find something like a report?
Comment 4 Cédric Bosdonnat 2011-02-14 01:44:29 UTC
(In reply to comment #3)
> I set "dbg = True" in "mailmerge.py", where can I find something like a report?

You need to run LibreOffice from the command line: the output is written there.
Comment 5 Don't use this account, use tml@iki.fi 2011-02-14 01:52:25 UTC
No it isn't, not on Windows. There is a distinction between "console" and "gui" executables. (These are just slightly misleading names for possible values of a flag in the .exe header, doesn't affect whether the program can have a GUI or all.)

The LibreOffice executables are marked as "gui" ones, which means that even if you start them from a command prompt, you won't see any output written to stdout or stderr.

You need to redirect the standard output and standard error of soffice.exe to a file. Start a command prompt (Windows-key+R, type cmd into the field, press Enter), and enter the command:

"\Program Files\LibreOffice 3\program\soffice.exe" >lo.log 2>&1

and afterwards look in lo.log

(Yes, that command >foo 2>&1 syntax really *is* cmd.exe syntax, too. Just like in Bourne-style shells on Unix. But in most other aspects, cmd.exe of course is not like Unix at all.)
Comment 6 Rainer Bielefeld Retired 2011-02-14 02:16:58 UTC
@Tor Lillqvist
I tried in a little different way:
opened Dos input window
changed directory to " ....\LibreOffice 3\program\"
Startet Libreoffice with "soffice" +ENTER
  that worked
Terminated LibO
Startet Libreoffice from Dos input window with "soffice >lo.log 2>&1" +ENTER
  Libo does not start, Message "Zugriff Verweigert" (= Access denied) 

May be that is some WIN7 Admin permissions issue? 
I am on IRC #libreoffece Username "RainerBielefeld", it would be kind if you could help.
Comment 7 Rainer Bielefeld Retired 2011-02-14 03:15:13 UTC
I did the test (with a horrible No of crashes after step 6).
Mail sending process reached  "Sending mail", connection to server exists, nothing happens, I see a lo.log with 0 bytes
After 1/4h LibO terminated sending process with "Runtime Error   ...\soffice.bin R6025 - pure virtual function call" 
Confirming this message causes LibO to crash. Unfortunatela with an empty lo.log
Comment 8 Rainer Bielefeld Retired 2011-02-16 00:15:46 UTC
Today I repeated Test from Comment 6 with OOo 3.1.1 WIN7 hoping to get some new information. 

I also only got an empty logfile but a fascinating result: OOo sent successfully using a not existing account with wrong not existing authentication data. May be outgoing server does not ask for new authentication when I had sent an other mail few moments before? But I also tried with a not existing mailserver name, and OOo sent successfully, I got the mail).
Or something really strange?
Comment 9 Rainer Bielefeld Retired 2011-02-19 04:04:09 UTC
Created attachment 43551 [details]
Crash Report created with OOo3.3.0

Same with " Ooo 3.3.0  – WIN7  Home Premium  (64bit) German UI [OOO330m20 (build 9567)]"
Comment 10 Tine 2011-03-04 03:53:38 UTC
Hi at all

send an e-mail doesn't work at

Aptosid 2.6.37-2.slh.3 amd64
LibreOffice 3.3.1
OOO330m19 (Build: 8 )
tag libreoffice-3.3.1.2, Debian package 1:3.3.1-1

after each dist-upgrade is the mail settings overwritten

current setting
LO - Extras - Options - Internet - E-Mail /usr/bin/icedove %u
it works with
Aptosid 2.6.37-1.slh.3 amd64
LibreOffice 3.3.1
OOO330m19 (Build: 8 )
tag libreoffice-3.3.1.2, Debian package 1:3.3.1-1

last setting
LO - Extras - Options - Internet - E-Mail icedove
it works with
LibreOffice 3.3.1
OOO330m19 (Build:7)
tag libreoffice-3.3.1.1, Debian package 1:3.3.1~rc1-1

current setting doesn't work now after each dist-upgrade
Comment 11 Cor Nouws 2011-03-07 05:23:47 UTC
For the guys wrestling with this bug: you may want to have a look at https://bugs.freedesktop.org/show_bug.cgi?id=32553#c17 (and previous comments in there).
However, I doubt if it really is the same issue. But you can try what happens if you leave out the table from the header / footer of the first test doc.
Comment 12 Rainer Bielefeld Retired 2011-03-11 00:45:11 UTC
This becomes more and more a blocker for me. Can it be that I am the only onein the world who uses e-mail-mailmerge?
Comment 13 Cor Nouws 2011-03-11 00:53:26 UTC
Hi Rainer,

(In reply to comment #12)
> This becomes more and more a blocker for me. Can it be that I am the only onein
> the world who uses e-mail-mailmerge?

I guess not - see my comment #11 ;-)
Comment 14 Alex Thurgood 2011-03-11 01:37:01 UTC
Absolutely not the only one :-)) - if I can no longer have individual documents as an output from the mailmerge (see other mailmerge bug reports), I could at least hope to be able to send them as e-mail ?

The problem is that this worked in previous versions of OOo. So what is the incentive to move forward, from a business use case point of view to something that not only no longer works, but apparently no one wants to (as in deemed not important enough) / can / assign resources / to fix ? I'll try and activate the debug option in the python script on my Mac OSX machine to see if anything useful comes out of it.


Alex
Comment 15 Cor Nouws 2011-03-11 01:46:25 UTC
Hi Alex,

(In reply to comment #14)
> if I can no longer have individual documents
> as an output from the mailmerge (see other mailmerge bug reports), I could at
> least hope to be able to send them as e-mail ?

I simply can do that.
But not with a document as described in bug 32553.
Comment 16 Alex Thurgood 2011-03-11 05:41:00 UTC
Created attachment 44350 [details]
Trace on Mac OSX when mailmerge e-mailing hangs

Enclosing a trace from Mac OSX from hang, force kill when trying to use mailmerge to send e-mail.
Comment 17 Michael Meeks 2011-03-14 04:21:10 UTC
Hi Alex,
Thanks for that ! the trace is interesting; clearly mail sending is done in the main thread, and blocks the main-loop, which is sub-optimal.

                                                                                                                                  20 init_sockobject + 1667 (in _socket.so) [0x2872c1a4]
                                                                                                                                    20 recvfrom$UNIX2003 + 10 (in libSystem.B.dylib) [0x9045e5fe]

One thing of course, is we get a C stack-frame, and not a python one (which is -much- harder to get sadly).

Also - we don't really know if this is where it hung [ waiting for a response from the mail server ], so - if you can attach gdb to the process while it is running (and/or hung).

gdb <pid of soffice.bin>
<ctrl-c> to break it
thread 1 # or to the 'SvMain' thread at leats
backtrace # and paste the output here.
finish # and keep typing finish until it hangs - and paste which frame that was in. That'd be great.

It looks as if the python mail library we use is somewhat slow & horrible wrt. send/recv errors - quite possibly we need to do this in a thread and then bang on the thread remotely [ by closing its socket eg. ] after some delay. In a nutshell you need to find some python hacker to come fix the bug - shouldn't be too hard for someone versed in the ways of python.
Comment 18 Alex Thurgood 2011-03-14 08:25:57 UTC
Hmmm, some python debugging information that was piped to standard out while I was attempting to do mailmerge.


exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 15: ordinal not in range(128), traceback follows
  /Applications/LibreOffice.app/Contents/basis-link/program/mailmerge.py:128 in function sendMailMessage() [print >> sys.stderr, "PyMailSMPTService subject", subject]


The 'xe9' character corresponds to e accute accent "é". 


Alex
Comment 19 Alex Thurgood 2011-03-14 08:33:56 UTC
Further to the python debugging message, the transmission of the e-mails is suspended, with a rather laconic internal error message that the e-mail address is invalid (which it isn't of course). So it seems that not only is the mailmerge script currently incapable of handling non-ASCII characters in the e-mail subject line, but if you do put such a character in, the mailmerge will fail silently, or more exactly, with an incorrect error message that the e-mail address is wrong.

Alex
Comment 20 Alex Thurgood 2011-03-14 08:39:45 UTC
To all :

I have now been able to send e-mails via mailmerge to several e-mail addresses three times in a row. The only difference in my MacOSX system from when my trace was posted on March 11th, 2011 and today is that I have installed the Apple Java update to 1.6.0_24. How has that made a difference ? I have no idea, but now mailmerge no longer hangs. The forementioned failed python ascii encoding of the subject doesn't cause LibO to hang, merely not to send any e-mail. If I take out the unsupported character e-mail mailmerge works. This is on 3.3.2 RC1.

Alex
Comment 21 Rainer Bielefeld Retired 2011-03-14 12:16:45 UTC
Thank yo for testing! My Java version is 1.6.0_24 (on WIN7), so I would have liked to text, but in step 6 and later my "LibreOffice 3.3.2RC1  – WIN7  Home Premium  (64bit) English UI [OOO330m19 (Build:201 / tag 3.3.2.1)]" does incredible numbers of crashes, I only get one in 20 to the sending procedure (and until now I only found the time for 15 tests without any success).

So OT my question: any idea how to debug this crash problem?
Comment 22 Rainer Bielefeld Retired 2011-03-14 12:53:46 UTC
I have to revise my calculation: the 16th test lead to Email Send Procedure, but I was nor successful, I see Transfer status without any progress for 30 minutes.

@Alex:
I also had days where I was able to send some messages, but the day after I again had the situation that no sending is possible.
Comment 23 Michael Meeks 2011-03-17 10:39:22 UTC
Alex - great hint around the charset issue; a rather helpful data-point for investigation, although perhaps a different bug.
Comment 24 Alex Thurgood 2011-03-17 10:43:48 UTC
Hi Michael,

Yes, I'll open a separate issue for that. I am going to try mailmerge again with a fresh install because although I said it worked 3 or 4 times in a row in my previous posting, it failed to work again later on, just kind of said the mails were sent, but actually sent nothing :-/

So I can confirm what others have been noticing - some times it works, some times it doesn't.


Alex
Comment 25 Alex Thurgood 2011-03-17 10:46:53 UTC
I did some profiling with Shark, the Apple provided utility that comes with XCode, but of course, needless to say, the b****y mailmerge thing worked flawlessly when I attached the running soffice process, so the results weren't of any use !!!


Alex
Comment 26 Caolán McNamara 2011-03-18 07:13:56 UTC
Re Comment #18

"exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in
position 15: ordinal not in range(128), traceback follows
  /Applications/LibreOffice.app/Contents/basis-link/program/mailmerge.py:128 in
function sendMailMessage() [print >> sys.stderr, "PyMailSMPTService subject",
subject]"

This is just in the debugging path, i.e. only happens when dbg is enabled, etc, this bug has gotten messy with different arch and potentially different bugs gets confused together so lets return to the original report which was...

'I select field with E-Mail-Address in my Address database spreadsheet and PDF-document, press "Send document" and send Message appears. Connection status is "The connection to the outgoing mail server has been established", but "Transfer status" remains on "0 of 0 sent"'

This I can reproduce, the key info is 0 of 0, not 0 of X where X is > 0, and this I fixed earlier today (http://cgit.freedesktop.org/libreoffice/writer/commit/?id=97bce7aad1c3a6f3fd23a3380c79ca9c1515624c) seeing as we're looking up export/save as filters by suffix, which is problematic and we end up exporting no document and therefore have nothing to send.
Comment 27 Caolán McNamara 2011-03-21 02:22:22 UTC
Got enough acks, and cherry-picked for 3-3 and 3-3-2, not certain it'll make the cut by time for 3.3.2, but we'll see.
Comment 28 Rainer Bielefeld Retired 2011-03-22 00:20:30 UTC
Only Task blocker
Comment 29 Rainer Bielefeld Retired 2011-04-24 10:43:40 UTC
Using "LibreOffice 3.4Beta2  – WIN7  Home Premium  (64bit) German UI [DEV300m103 (Build:1)]" I did a successufl test to use MIALMERGE email.

Second test was not successful (only 1 mail of to has been sent), in extended send status dialog I got message:
<type 'exceptions.TypeError'>: string payload expected: <type 'instance'>, traceback follows
  C:\Program Files (x86)\LibreOffice 3\Basis\program\python-core-2.6.1\lib\email\generator.py:175 in function _handle_text() [raise TypeError('string payload expected: %s' % type(payload))]
  C:\Program Files (x86)\LibreOffice 3\Basis\program\python-core-2.6.1\lib\email\generator.py:135 in function _dispatch() [meth(msg)]
  C:\Program Files (x86)\LibreOffice 3\Basis\program\python-core-2.6.1\lib\email\generator.py:109 in function _write() [self._dispatch(msg)]
  C:\Program Files (x86)\LibreOffice 3\Basis\program\python-core-2.6.1\lib\email\generator.py:84 in function flatten() [self._write(msg)]
  C:\Program Files (x86)\LibreOffice 3\Basis\program\python-core-2.6.1\lib\email\generator.py:201 in function _handle_multipart() [g.flatten(part, unixfrom=False)]
  C:\Program Files (x86)\LibreOffice 3\Basis\program\python-core-2.6.1\lib\email\generator.py:135 in function _dispatch() [meth(msg)]
  C:\Program Files (x86)\LibreOffice 3\Basis\program\python-core-2.6.1\lib\email\generator.py:109 in function _write() [self._dispatch(msg)]
  C:\Program Files (x86)\LibreOffice 3\Basis\program\python-core-2.6.1\lib\email\generator.py:84 in function flatten() [self._write(msg)]
  C:\Program Files (x86)\LibreOffice 3\Basis\program\python-core-2.6.1\lib\email\message.py:135 in function as_string() [g.flatten(self, unixfrom=unixfrom)]
  C:\Program Files (x86)\LibreOffice 3\Basis\program\mailmerge.py:225 in function sendMailMessage() [self.server.sendmail(sendermail, truerecipients, msg.as_string())]
Comment 30 Rainer Bielefeld Retired 2011-05-03 21:11:45 UTC
(In reply to comment #29)
This problem seems to be a completely different one, I filed
Bug 36763 - Mail Merge E-Mail stops with error "Invalid Address"
for it.