Bug 37531 - Libreoffice will not run in "batch mode" when there is another instance open
Summary: Libreoffice will not run in "batch mode" when there is another instance open
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
3.3.0 release
Hardware: All All
: medium major
Assignee: Stephan Bergmann
URL:
Whiteboard: target:5.3.0
Keywords:
: 40807 41686 44496 73405 103096 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-05-24 00:37 UTC by butrus.butrus
Modified: 2020-01-03 11:58 UTC (History)
25 users (show)

See Also:
Crash report or crash signature:


Attachments
Resolves 37531. Appends PID to named pipe path. Dirty. (2.79 KB, patch)
2012-03-16 17:56 UTC, jtang613
Details
This affects LyX and other desktop applications that rely on LibreOffice (108.18 KB, image/png)
2014-04-28 19:35 UTC, F.H.
Details

Note You need to log in before you can comment on or make changes to this bug.
Description butrus.butrus 2011-05-24 00:37:38 UTC
I use "libreoffice-writer -convert-to pdf ..." to do a batch conversion of my odt's. However, this won't work if another instance of LO is open: in such a case it will just open new LO Writer window.
Comment 1 vitriol 2011-05-24 00:42:06 UTC
Perhaps related to Bug 37411?
Comment 2 Yifan Jiang 2011-05-24 00:54:47 UTC
Yes, it also happens on 3.4 RC1 / SLED 11 sp1 i586.

cc for Petr/Muthu's review.
Comment 3 butrus.butrus 2011-05-24 05:11:41 UTC
Ooops, "(/usr/bin/)libreoffice-writer" is my private wrapper. I meant "/opt/libreoffice/program/swriter -convert-to pdf ...", of course. Sorry for confusion.
Comment 4 Petr Mladek 2011-05-24 09:58:56 UTC
Muthu, could you please have a look? Does it have any reasonable solution?
Comment 5 Brandon Simmons 2011-06-16 14:29:53 UTC
I was hit by this too. In headless mode 'soffice' exits quietly with success without doing anything:

Try:

#!/bin/bash
soffice --headless --convert-to pdf foo.doc &
soffice --headless --convert-to pdf foo.doc && echo "exit status: $?"
Comment 6 sasha.libreoffice 2012-03-06 04:32:12 UTC
*** Bug 41686 has been marked as a duplicate of this bug. ***
Comment 7 sasha.libreoffice 2012-03-06 04:33:43 UTC
Contents from there:
to replicate
1) Open Writer or Calc using gnome menu
2) call this command: libreoffice3.4 --convert-to pdf myfile.doc
3) nothing will happen, as if having the GUI open interrupted the execution of
the command.

I'm seeing the same issue.

When a GUI instance of LO is running, the convert-to CLI command fails to
execute - and usually spawns another GUI window.  This occurs with other CLI
commands as well, such as --print-to-file.

(Ubuntu 11.10 x86_64)
Comment 8 sasha.libreoffice 2012-03-06 04:36:04 UTC
*** Bug 44496 has been marked as a duplicate of this bug. ***
Comment 9 Petr Mladek 2012-03-06 05:44:47 UTC
Michael is quite familiar with the startup code. I wonder if he has an idea.
Comment 10 jtang613 2012-03-07 07:51:33 UTC
This bug seems to affect all CLI operations, not just convert-to.  When a GUI instance of LO is running, 'libreoffice --help' produces no output, but returns success.  A fix would be greatly appreciated.
Comment 11 Michael Meeks 2012-03-14 14:06:20 UTC
Muthu is the best guy here. Then again - getting an:

strace -f -s 4096 ./soffice --headless etc. etc for both the first and second processes would prolly show you what is going on quite easily.

The code is:

http://cgit.freedesktop.org/libreoffice/core/tree/desktop/unx/source/start.c

for the quick-start arg. passer, and hereabouts:

http://cgit.freedesktop.org/libreoffice/core/tree/desktop/source/app/cmdlineargs.cxx

for the 'main' process.

You may find, you have better joy if you run './soffice.bin' instead of './soffice' - though the paths will need setting up.

If you can make ./soffice.bin behave differently to ./soffice then there is a problem in 'ooqstart.bin' I guess, that we'd need to look at.

Failing that, as a hack around removing ooqstart.bin might help you.

I'm not going to have time to look at this just now, but patches much appreciated.
Comment 12 Muthu 2012-03-15 01:30:45 UTC
As far as I remember, lo doesn't do the conversion, because the (new) instance wouldn't do anything other than sending the (older) process a signal.

How important is this feature? I guess, people would mostly either use the UI or the batch converter and not both?
Comment 13 Bernard Moreton 2012-03-15 02:23:07 UTC
If a user is running CONVERT from the command line, I'd guess that such a person would be very likely to be running the UI as well...
Comment 14 sasha.libreoffice 2012-03-15 03:32:39 UTC
May be use two versions of LibreOffice which save profile in different places. So we can then use 2 independent versions of office. Or specify place with user profile in command line?
Comment 15 jtang613 2012-03-15 05:55:49 UTC
(In reply to comment #12)
> As far as I remember, lo doesn't do the conversion, because the (new) instance
> wouldn't do anything other than sending the (older) process a signal.
> 
> How important is this feature? I guess, people would mostly either use the UI
> or the batch converter and not both?

imho, all applications, including LO should allow multiple running instances to operate independently of each other.

Suppose for example: A user want to edit a spreadsheet while at the same time wants to perform a batch convert of 100+ odt files to pdf from the command-line.
Comment 16 Petr Mladek 2012-03-15 09:17:08 UTC
Hmm, it is possible to open another document from the command line. It just passes the file name to the already running LibreOffice instance. The running instance takes it and opens the document in a new window.

I wonder if something like this would be possible with the document conversions. We just need to pass the right commands to the already running LO.
Comment 17 jtang613 2012-03-15 13:45:28 UTC
(In reply to comment #11)
> Muthu is the best guy here. Then again - getting an:
> 
> strace -f -s 4096 ./soffice --headless etc. etc for both the first and second
> processes would prolly show you what is going on quite easily.
> 

Seems both instances of LO try to claim the same named pipe for IPC.

Instance-1 succeeds in creating and communicating using this pipe, but instance-2 seems to time out as shown below:

25304 access("/tmp/OSL_PIPE_1000_SingleOfficeIPC_bcb44c91593d45fbe896d17e4322b1a", F_OK) = 0
25304 connect(7, {sa_family=AF_FILE, path="/tmp/OSL_PIPE_1000_SingleOfficeIPC_bcb44c91593d45fbe896d17e4322b1a"}, 110) = 0
25304 getcwd("/home/user/Build", 4096) = 24
25304 sendto(7, "InternalIPC::Arguments1file:///home/user/Build,--convert-to,pdf,untitled.odt", 76, 0, NULL, 0) = 76
25304 sendto(7, "\0", 1, 0, NULL, 0)    = 1
25304 recvfrom(7,  <unfinished ...>
25308 <... futex resumed> )             = -1 ETIMEDOUT (Connection timed out)

Perhaps it's possible to append an instance number to the pipe name?
Comment 18 jtang613 2012-03-16 17:56:39 UTC
Created attachment 58580 [details]
Resolves 37531. Appends PID to named pipe path. Dirty.

This patch appends the current Process ID to the IPC named pipe path in order to provide each instance of LO with a unique pipe.  Admittedly, it's a bit of a hack and may break other stuff.  The original code seems to hint that multiple instances should run "out of the box" sharing the same IPC and backend... But that's not currently the case, so here we are.

Works for me. YMMV.

Feedback appreciated.
Comment 19 Muthu 2012-03-18 23:47:57 UTC
jtang: cool! can you forward that patch along with a brief to the mailing list for review, please? A lot of people would sure be interested. Thanks a lot again for the patch.
Comment 20 butrus.butrus 2012-03-19 05:13:54 UTC
(In reply to comment #16)
> Hmm, it is possible to open another document from the command line. It just
> passes the file name to the already running LibreOffice instance. The running
> instance takes it and opens the document in a new window.
> 
> I wonder if something like this would be possible with the document
> conversions. We just need to pass the right commands to the already running LO.

This is suboptimal as sometimes you need to use a particular version of LO for batch conversion to be able to quarantee the stability of the conversion (no new bugs etc.)

The best solution would be to just process all "request" that are to be done in the "batch mode" by the called binary.
Comment 21 Petr Mladek 2012-03-19 07:03:21 UTC
I had the feeling that the pipe is there to communicate with the primary LO instance. If you call second instance, it could use the pipe to tell the first instance what command is required. I might be wrong, though.
Comment 22 jtang613 2012-03-19 07:31:09 UTC
(In reply to comment #21)
> I had the feeling that the pipe is there to communicate with the primary LO
> instance. If you call second instance, it could use the pipe to tell the first
> instance what command is required. I might be wrong, though.

Yes, agreed, it does seem intended to serve this purpose - as I've noted above.  However, its implementation seems to fail at achieving this goal.
Comment 23 jtang613 2012-03-19 07:54:30 UTC
Feedback from libreoffice-dev indicates that the above patch may indeed break other aspects of LO - use at own risk.

LO runs with the expectation that there is only ever one instance operating on the ~/.config/libreoffice/3/user directory at any given time. Hence the client/server model, though currently broken, is used to offload requests from additional LO instances to the primary instance.

Two further options immediately come to mind:
1) Fix the IPC layer to allow CLI options to be background-processed by the primary instance.
2) Add a CLI argument to allow LO to use an "alternate-config-dir" (as Sasha suggests)

At first glance, option 2 seems to be the quickest solution that will not break other parts of LO.  However, the 'proper' long-term solution remains option 1. 

I'll take a shot at option 2 since it solves my immediate requirements.  Hopefully by week's end.  I'll leave option 1 to the Jedi Masters.
Comment 24 sasha.libreoffice 2012-03-19 08:21:21 UTC
For changing user profile use this argument:
-env:UserInstallation=${MY_CONF}
Explanations about this argument is here:
Bug 45026 - Command Line invokes empty document if LibreOffice Writer already running
Comment 25 jtang613 2012-03-19 08:35:38 UTC
(In reply to comment #24)
> For changing user profile use this argument:
> -env:UserInstallation=${MY_CONF}
> Explanations about this argument is here:
> Bug 45026 - Command Line invokes empty document if LibreOffice Writer already
> running

Excellent!  This undocumented / semi-documented argument does exactly what I'm looking for. The commands below will now run correctly.

soffice --headless --convert-to pdf foo.doc &
soffice -env:UserInstallation=file:///home/user/.libreoffice-alt --headless --convert-to pdf foo.doc &
Comment 26 Muthu 2012-03-19 23:23:32 UTC
Cool! Shall we close this as 'wontfix' then? (or as duplicate of bug 40526)

(In reply to comment #25)
> 
> Excellent!  This undocumented / semi-documented argument does exactly what I'm
> looking for. The commands below will now run correctly.
> 
> soffice --headless --convert-to pdf foo.doc &
> soffice -env:UserInstallation=file:///home/user/.libreoffice-alt --headless
> --convert-to pdf foo.doc &
Comment 27 jtang613 2012-03-20 04:51:35 UTC
(In reply to comment #26)
> Cool! Shall we close this as 'wontfix' then? (or as duplicate of bug 40526)

That depends. I would still consider it broken since the IPC is not performing it's intended function. Both the pipe renaming and alternate config directory are kludges to get around the broken IPC - they don't fix the core problem.
Comment 28 Thorsten Behrens (allotropia) 2012-07-26 12:55:15 UTC
Comment on attachment 58580 [details]
Resolves 37531. Appends PID to named pipe path. Dirty.

Setting flag on patch to prevent it showing up in searches - I understand that the patch as-is is not intended for integration?
Comment 29 Eltomito 2013-03-22 11:37:16 UTC
(In reply to comment #26)
> Cool! Shall we close this as 'wontfix' then? (or as duplicate of bug 40526)

I would like to see this fixed.

It seems counter intuitive to have the behavior of libreoffice
batch processing invoked from the command line change depending on
whether an instance of LO is already running or not.
It certainly felt broken to me when I ran into it.

And I do use LO gui and LO command line at the same time.
I edit .odt or .doc scripts in the LO gui and then use a script to convert them to .fdx. This script only understands .fodt, so it first invokes LO batch mode to convert the input to that. It is inconvenient to have to shut down LO whenever I need to do the conversion.

Actually, I was really happy when I found out about the batch conversion mode in LO. It's great, dunno what I'd do without it :)
Comment 30 Michael Meeks 2013-03-22 14:25:49 UTC
> I would like to see this fixed.

There is a simple way to get the bug addressed: send a patch ! :-)

There is quite a lot that is possible here; for example - we could encourage the headless mode not to use any user settings / extensions / auto-save, and ignore the home directory completely. That would remove lots of the (locking related) reasons that we have a factory process anyway; it'd be useful for liblibreoffice too.

Might help for multi-process instances as well.
Comment 31 Riccardo Magliocchetti 2013-06-06 20:34:13 UTC
*** Bug 40807 has been marked as a duplicate of this bug. ***
Comment 32 F.H. 2014-04-28 19:35:48 UTC
Created attachment 98136 [details]
This affects LyX and other desktop applications that rely on LibreOffice

This behavior affects *other* desktop applications that rely on LibreOffice's headless operation.

For example, LyX relies on "libreoffice --convert-to" to incorporate LibreOffice drawings, spreadsheets, charts, etc. in complex LyX documents. See attached screenshot.  With the current behavior, if an instance of LibreOffice happens to be running, LyX fails to process *all* LibreOffice files.
Comment 33 ign_christian 2014-09-27 03:32:51 UTC
*** Bug 73405 has been marked as a duplicate of this bug. ***
Comment 34 LRN 2016-02-01 07:15:03 UTC
I would like to see this fixed.

I'm using LO in batch processing mode from an application to convert ODT files (which can be generated easily enough, since they are ZIPed collections of XML files, mostly) to PDFs. It's confusing and limiting that an instance of LO running in parallel with my application will prevent this feature from working.

The -env:UserInstallation=file:///... option seems to be working for now, but i wouldn't like to have to use it.
Comment 35 Michael Meeks 2016-02-01 18:03:41 UTC
> I would like to see this fixed.

The general approach then is to work on fixing it yourself, or to fund someone to do that. However - the good news in this case is that this is fixed already - in a way that should give you more control too.

If you use the LibreOfficeKit API - checkout

https://github.com/LibreOffice/core/tree/master/include/LibreOfficeKit

Then you can yank all of that code into your own process and then there should be no pipes, no locking and you control the lifecycle =) I believe there are also python bindings as well as C, C++.

There is also an 'loconvert' package that uses LibreOfficeKit - which also be somewhat helpful here - and is supposedly faster than the pipe method.

I'm tempted to just close this bug =) would that make people mad ?

HTH,

Michael.
Comment 36 abma 2016-02-01 18:20:27 UTC
> If you use the LibreOfficeKit API - checkout
>
> https://github.com/LibreOffice/core/tree/master/include/LibreOfficeKit
> 
> Then you can yank all of that code into your own process and then there should > be no pipes, no locking and you control the lifecycle =) I believe there are also python bindings as well as C, C++.

The program i use uses the java api. As far as i can tell it doesn't matter which api is used, it matters that only one instance of libreoffice can use one config directory, so it also shouldn't matter if a different frontend is used.

it would be nice, if LO could be run headless and as gui without requiring a dedicated config dir for each instance.

as there are several duplicate reports it seems a feature which is wanted by a dozen of people.

not sure if this is a bug or feature request, its at least an unexpected limitation.
Comment 37 Bernard Moreton 2016-02-02 09:35:33 UTC
The problem is that someone using command-line conversion for the first time will probably come up against the problem, and will see the process as failing.  Yes, there are ways round the problem (and I'm glad to hear of another two for the first time); but LO functions should "just work".  I'd suggest that it should be left on the bug list, in the hope that it may be fixed at some time.
Comment 38 Jérôme Kunegis 2016-02-26 15:17:58 UTC
Just a further comment from someone "in the field" here.

I have an ODP file, which I convert to PDF by hand, and then I have a Makefile that does further things with that (mainly using pdftk, concatenating it with other PDF files with are autogenerated from completely different sources programmatically).  

Now, everytime I change the ODP file using Impress, I have to "export as PDF" from within Impress before I can execute Make.  It would be convenient if there was a simple command line that made LibreOffice export the ODP as PDF.  Note that I usually change the ODP file multiple times in a row, so LibreOffice is running while I execute Make, and I'm hitting this bug.  

Doing the "export as PDF" is not too time consuming, but simply comparing LibreOffice to other tools (such as Inkscape), it is a quite large incompleteness in the feature set that the conversion to PDF cannot be executed on the command line when the UI is running.  

Note that such command lines work trivially with virtually all other tools, such as the previously mentioned Inkscape.  Since this is called from within a Makefile, using an API in Python or some other programming language is not a solution.  In particular for a piece of software which is flagship for the free software movement and Unix-like environments, this lack of functionality is quite notable.

At the very least, I would except the "libreoffice" invocation to return a non-zero exit code and output a corresponding error message explaining the limitation, to indicate that it did not actually convert anything to PDF. The non-zero exit code is also important to tools such as Make.

Just for completeness, yes, I am aware of alternative solutions which allow the conversion to be done in a single line.
Comment 39 Ktr 2016-03-16 15:35:54 UTC
Hello everyone,

This bug might be hard to fix, but maybe making the headless mode output something like : "Hey I see you are running the GUI, close it so I can do my work" would be easy to implement, no ? And this way at least the user has a clue of why it doesn't work.

For information, this bug is still present in 5.1.0.3.

Regards,
~k
Comment 40 Mateusz Konieczny 2016-05-15 10:12:53 UTC
"How important is this feature? I guess, people would mostly either use the UI or the batch converter and not both?"

It is quite important - I was trying to use (for the first time) libreoffice --convert-to xls (I am looking for workaround for https://bugs.documentfoundation.org/show_bug.cgi?id=97631 - so I decided to try converting to xls).

As I has open Calc instance it was silently failing.

Without http://stackoverflow.com/questions/30349542/command-libreoffice-headless-convert-to-pdf-test-docx-outdir-pdf-is-not I would be looking for a solution for a long time.

It would be much better to see at least error message "please close all instance of Libre Office, for details see https://bugs.documentfoundation.org/show_bug.cgi?id=37531"

I moving it from NEEDINFO to UNCONFIRMED as I see no active requests for information of data.
Comment 41 Hans Deragon 2016-05-15 19:05:35 UTC
I feel it would be quite important.  Though most do not use batch mode, only the GUI, those who do use the batch mode probably are running a GUI version to test their results while trying to build their batch commands.  Sure, in production on a server, only the batch mode will run, but during development and qa, both for sure.
Comment 42 Colin Tedford 2016-05-27 23:24:20 UTC
This bug has bitten me multiple times and I've forgotten it multiple times because it's so surprising and unintuitive. I'm glad to finally have a workaround, but would love to see this fixed properly (and documented in the man page in the meantime).

(This is my first time commenting here and the interface is confusing and the Help link is 404 -- apologies if I accidentally changed anything. I'm using "Reply" b/c it gives me fewer fields.)
Comment 43 ch 2016-07-02 05:58:11 UTC
I stumbled upon this bug today. So its still present in:

    $ libreoffice --version
    LibreOffice 5.1.4.2 10m0(Build:2)

I would like to see this fixed.
Comment 44 Jason White 2016-07-13 19:00:50 UTC
Although I have an older version than ch@denx.de, I also would like to confirm that this bug is still present and is posing problems to users.

In my particular case batch mode is used to generate PDFs of documentation used in a large software project. Due to the quantity of documents in each release the use of batch mode is required. This bug prevents the generation of PDFs from the commandline while a Libreoffice window is still open.

I use this command: libreoffice5.0 --headless --convert-to pdf *.odt

My Version is:
libreoffice5.0 --version
LibreOffice 5.0.3.2 e5f16313668ac592c1bfb310f4390624e3dbfb75

Hopefully this will be resolved Soon.
-Jason White
Comment 45 daniel.oconnor 2016-07-15 13:57:48 UTC
As a very small but useful improvement, could the --help text be adjusted

From:

-env:<VAR>[=<VALUE>]
      Set a bootstrap variable.
      Eg. -env:UserInstallation=file:///tmp/test to set a non-default user profile path.

to:

-env:<VAR>[=<VALUE>]
      Set a bootstrap variable.
      Eg. -env:UserInstallation=file:///tmp/test to set a non-default user profile path. Tip: set unique Userinstallations to multiple instances to run in isolation.


Alternatively, could a simple --isolated flag be added as a shorthand for -env:UserInstallation=file:///tmp/randomtmpfile ?

I think one of the big problems is discoverability of the -env behaviour - many of the top stackoverflow answers for this have not managed to discover it; and simply advise "don't run two instances"?
Comment 46 daniel.oconnor 2016-07-15 14:00:29 UTC
(In reply to Mateusz Konieczny from comment #40)
> As I has open Calc instance it was silently failing.

I raised #100932 as a suggestion on improving what we think of as 'silent failure'
Comment 47 daniel.oconnor 2016-07-15 14:30:01 UTC
Wording change:

https://github.com/CloCkWerx/core/commit/c96431ec3fa344c0646b1428f3e1014d5404d4dd
Diff:
https://github.com/CloCkWerx/core/commit/c96431ec3fa344c0646b1428f3e1014d5404d4dd.diff

(If people like it, it would be really helpful if a more experience developer could shepherd it through gerrit, I seem to be running a bit low on disk space to do a proper checkout)
Comment 48 Commit Notification 2016-07-15 16:16:59 UTC
Stephan Bergmann committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=08960441558743f222280637985690bb8b41b861

Related tdf#37531: Handle conversion args sent over pipe

It will be available in 5.3.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 49 Commit Notification 2016-07-15 21:00:06 UTC
Stephan Bergmann committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=8d8668be3444560de127274201a3ab68b1dff950

Related tdf#37531: --convert-to output relative to caller's CWD

It will be available in 5.3.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 50 Commit Notification 2016-07-15 21:25:06 UTC
Stephan Bergmann committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=a2c557d80ac68c06ea59586245a7431e061938f0

Related tdf#37531: Make --cat headless/hidden like --convert-to

It will be available in 5.3.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 51 Stephan Bergmann 2016-07-15 21:29:47 UTC
(In reply to Colin Tedford from comment #42)
> (This is my first time commenting here and the interface is confusing and
> the Help link is 404 -- apologies if I accidentally changed anything. I'm
> using "Reply" b/c it gives me fewer fields.)

[resetting Version and Hardware to previous values]
Comment 52 Stephan Bergmann 2016-07-15 21:34:20 UTC
Things should work now, except for "'soffice --cat foo.odt' prints to the called soffice's stdout, not the caller's (though this will unlikely ever be fixed)" (<https://cgit.freedesktop.org/libreoffice/core/commit/?id=08960441558743f222280637985690bb8b41b861> "Related tdf#37531: Handle conversion args sent over pipe").
Comment 53 Eugen Dedu 2017-05-02 11:55:16 UTC
*** Bug 103096 has been marked as a duplicate of this bug. ***