Bug Hunting Session
Bug 91480 - allow to build shared libraries in bundled projects on Windows
Summary: allow to build shared libraries in bundled projects on Windows
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
unspecified
Hardware: Other All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: difficultyBeginner, easyHack, skillCpp, topicCleanup
Depends on:
Blocks:
 
Reported: 2015-05-22 14:32 UTC by David Tardon
Modified: 2017-02-14 08:58 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments
libwps patch for testing (1.64 KB, text/plain)
2015-05-22 14:32 UTC, David Tardon
Details

Note You need to log in before you can comment on or make changes to this bug.
Description David Tardon 2015-05-22 14:32:40 UTC
Created attachment 115830 [details]
libwps patch for testing

There are lots of bundled projects that can be built as part of libreoffice build. Many of these use autotools for build. To build these on Windows, we use wrappers that translate gcc command line options to msvc syntax. This allows us to build the projects uniformly on all platforms. The problem with this is that the wrappers only work for static libraries. There is a handful of projects that need to be built as shared and the number is increasing.

We need to fix up the wrappers to allow building shared libraries as well. The code is rather simple and lives in solenv/gcc-wrappers. Attached is a minimal patch that adapts libwps to be built as shared on Windows; this can be used for testing. The unpacked sources of libwps can be found in workdir/UnpackedTarball/libwps (Not right away: tarballs of bundled projects are unpacked as a part of the build). This task does not require a full libreoffice build: "make libwps.all" is enough for testing if the wrappers work or not. If they do, the built .dll would be in workdir/UnpackedTarball/libwps/src/lib/.libs.
Comment 1 Ashod Nakashian 2015-07-09 22:03:27 UTC
After spending some time tackling this, it's clear that it isn't as straightforward as I was led to believe.

The main issue is that, while the compiler (CC and CXX) is replaced by the gcc- and g++-wrapper, the linker isn't replaced. LD isn't used at all, instead, these external projects (the handful I looked into) explicitly use LIBTOOL, which points to a script generated by configure. So defining LIBTOOL doesn't help here.

In all the cases I've checked, setting --enable-shared and --disable-static do not produce the desired result. At least in the case of libwps it's clear that shared libraries are not supported (or are broken if they were at some point). If the project in question compiles successfully, it always generates a static library.

So the solution is to patch configure (in each project) such that shared libraries would be supported. To do that we would most probably need to wrap MS link.exe in the same way that gcc-wrapper wraps cl.exe. But first we need to make sure that Makefile does call LD, which can then be proxied as necessary.

Another way is to patch the VS projects where they are available to produce DLL, then compile using devenv.com (part of VS) instead of $(MAKE). This should be the simplest solution, but many projects might not even have these VS projects, although those could be created as well.


If there is a smarter way that I missed, I'd be interested to be hinted at the right direction.
Comment 2 David Tardon 2015-07-10 08:02:27 UTC
(In reply to Ashod Nakashian from comment #1)
> After spending some time tackling this, it's clear that it isn't as
> straightforward as I was led to believe.

Sorry I misled you. This was intended to be an "exploration"-style task, but I see the description does not say that anywhere...

Thanks for the analysis, though!

> The main issue is that, while the compiler (CC and CXX) is replaced by the
> gcc- and g++-wrapper, the linker isn't replaced. LD isn't used at all,
> instead, these external projects (the handful I looked into) explicitly use
> LIBTOOL, which points to a script generated by configure. So defining
> LIBTOOL doesn't help here.

Actually, LD is used. But indirectly by libtool.

> In all the cases I've checked, setting --enable-shared and --disable-static
> do not produce the desired result. At least in the case of libwps it's clear
> that shared libraries are not supported (or are broken if they were at some
> point). If the project in question compiles successfully, it always
> generates a static library.

So, that is probably because libtool finds no known dynamic linker. It defaults to building static libraries in that case instead of just failing as any sensible tool would do. You can check the build log (in workdir/UnpackedTarball/*/build.log by default); there should be an info message from libtool that it can't build dynamic libs for some reason and that it switches to static.

> So the solution is to patch configure (in each project) such that shared
> libraries would be supported. To do that we would most probably need to wrap
> MS link.exe in the same way that gcc-wrapper wraps cl.exe. But first we need
> to make sure that Makefile does call LD, which can then be proxied as
> necessary.

I agree that a ld-wrapper seems to be needed. But not with the rest. Dynamic libraries are already supported, through libtool. But libtool either cannot use link.exe directly (-> a wrapper) or it needs extra options for building dynamic libs (my first suspect would be -no-undefined, but libwps already passes that one).

> Another way is to patch the VS projects where they are available to produce
> DLL, then compile using devenv.com (part of VS) instead of $(MAKE). This
> should be the simplest solution, but many projects might not even have these
> VS projects, although those could be created as well.

Well, while some of the projects do have VS project files, these are typically out-of-date...
Comment 3 Ashod Nakashian 2015-07-10 15:30:01 UTC
Thanks for the detailed feedback David.

> So, that is probably because libtool finds no known dynamic linker.

At least in the case of libwps shared lib was disabled explicitly for non-gcc compilers (actually, it assumes non-gcc to mean mvs/msc). I have a patch to enable it, but, as you note, it still defaults to static linking. I think I know where to look: libtool.

> I agree that a ld-wrapper seems to be needed.

I'll experiment with a wrapper and investigate how we could get it to cooperate with libtool to create shared libs.

Btw, are we waiting for Windows support of shared libs to enable/test on other platforms, or does shared work on Linux? Do we know these libraries can/do link as shared? I haven't tried building on Linux, and it's easier to ask :)


> But not with the rest.

You mean only with some projects? Not sure I get your point here.
Comment 4 David Tardon 2015-07-11 08:21:12 UTC
(In reply to Ashod Nakashian from comment #3)
> Thanks for the detailed feedback David.
> 
> > So, that is probably because libtool finds no known dynamic linker.
> 
> At least in the case of libwps shared lib was disabled explicitly for
> non-gcc compilers (actually, it assumes non-gcc to mean mvs/msc).

Do you speak about libwps itself here, or about its build inside libreoffice? By the reference to msvc I assume the latter... In that case, libwps is not being built using its native build mechanism on Windows, because that currently cannot produce a shared library! The patch I attached switches the libwps build to the native build mechanism, so it can be used for experimentation.

> I have a
> patch to enable it, but, as you note, it still defaults to static linking. I
> think I know where to look: libtool.

As I have said, you should check the build log first. There might be some info message(s) from libtool telling why it switched back to static linking.

> 
> > I agree that a ld-wrapper seems to be needed.
> 
> I'll experiment with a wrapper and investigate how we could get it to
> cooperate with libtool to create shared libs.
> 
> Btw, are we waiting for Windows support of shared libs to enable/test on
> other platforms, or does shared work on Linux? Do we know these libraries
> can/do link as shared? I haven't tried building on Linux, and it's easier to
> ask :)

It seems I was not clear enough on that in the initial description... The libraries in question are already built as shared on all platforms. The libraries' native build mechanism (IOW, configure + make) is used on Linux and OS X. It is only Windows that needs special treatment (and the idea of this task is to try to change that).

> > But not with the rest.
> 
> You mean only with some projects? Not sure I get your point here.

I do not agree that there is any need to patch configure or Makefiles.
Comment 5 Ashod Nakashian 2015-07-11 15:31:21 UTC
> Do you speak about libwps itself here, or about its build inside libreoffice? 

Sorry, I wasn't clear. I think we're on the same page. I've, of course, applied your patch, and do recognize the distinction.

I am saying that the native libwps _had_ support for building with msvc[*] using configure/makefile. However that is no longer maintained. I had started with libwps, but also tried other projects since wasn't sure what was specific to it and what more generic.

Anyway, I think I know what to do. As I said, I need to wrap link.exe, which as you say is used by libtool. I also found (after reading your feedback,) where configure checks $LD to decide if shared linking is possible (line 13,918). This means the first test to pass is to make the wrapper respond correctly to configure's check.

Thanks again for your help. Will get back to you with progress (or issues).


[*] See line 15,766 in libwps's configure, for example, or search for "msvc" or "mvs".
Comment 6 Robinson Tryon (qubit) 2015-12-10 11:41:06 UTC Comment hidden (obsolete)
Comment 7 Robinson Tryon (qubit) 2016-02-18 14:52:20 UTC Comment hidden (obsolete)