Bug 70414 - create VSProject files for code editing in MSVS from gbuild
Summary: create VSProject files for code editing in MSVS from gbuild
Status: RESOLVED WORKSFORME
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
unspecified
Hardware: Other Windows (All)
: medium minor
Assignee: Honza Havlíček
URL:
Whiteboard: target:4.2.0 target:4.3.0 target:5.4.0
Keywords: difficultyInteresting, easyHack, skillScript
Depends on:
Blocks:
 
Reported: 2013-10-13 00:56 UTC by Björn Michaelsen
Modified: 2016-12-15 07:57 UTC (History)
8 users (show)

See Also:
Crash report or crash signature:


Attachments
definitions about a library (7.55 KB, text/plain)
2013-10-13 00:56 UTC, Björn Michaelsen
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Björn Michaelsen 2013-10-13 00:56:14 UTC
Created attachment 87541 [details]
definitions about a library

If you run e.g.:

 make cmd make -npf Makefile.gbuild all

you will get a huge output of gbuilds state (without actually building anything). It should also contain a piece of text like the attached one describing ~completely (include paths, compile flags, link flags, source files ...) how each library gets build.

This EasyHack is to create a script (or better a make target wrapping a script), that:
- finds that data in the output
- extract that data
- and generate a VS Project file, that allows to load the library as a project in VS with include paths etc. set so that InteliSense (code completing) works

In the first step it is NOT required to allow building directly from VS. Later, bonuspoints might get earned with extending the generated project files to trigger gbuild from VS with a Custom Action:

 http://msdn.microsoft.com/en-us/library/vstudio/d9k65z2d%28v=vs.100%29.aspx

So for starters, this is to allow Windows developers to continue using MSVS as an editor and IDE. Do NOT attempt a perfect 1:1 translation, ability to edit with code completion should be enough for starters!
Comment 1 Commit Notification 2013-11-19 11:48:39 UTC
Bjoern Michaelsen committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=ba99e296077e6bc6d6a153d01a45bd9ebe1a1d35

related fdo#70414 gbuild to ide: kdevelop



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 2 Michael Meeks 2013-11-19 12:09:17 UTC
Wow Bjoern; you're awesome =) hopefully MSVS / Eclipse are not -so- far away from KDevelop (conceptually ;-)

In the star-treck future I guess that could even become some generally useful tool for operating on the (harder and more awful ;-) autotoolsized projects too (?) Exciting !
Comment 3 Björn Michaelsen 2013-11-19 12:34:35 UTC
Well, I guess Eclipse/MSVS would likely be a bit more verbose as they are using this enterprisy newfangled "XML". But I dont think it would be much _harder_ per se.

As for extending this to autotoolsy projects Im not sure if that is so easy. It it really trivial to extract the build information from gbuild as it is exportable in a really easy to parse way, while autotools defies parsing even by most humans judging from a nonscientific survey in my environment(*). ;)


(*) It might work though if the project restricts itself to a welldefined
    subset of autotools (just like gbuild essentially restricts GNU make to
    a DSL for target definition).
Comment 4 Michael Meeks 2013-11-19 13:22:34 UTC
> As for extending this to autotoolsy projects Im not sure if that is so easy.
> It it really trivial to extract the build information from gbuild as it is
> exportable in a really easy to parse way, while autotools defies parsing
> even by most humans judging from a nonscientific survey in my environment

Heh - sure, I was thinking of parsing the generated gnumake rules and deps etc. from running a similar (but plunging) 'make -n -p' on an autotools project; all previous attempts I've seen to parse auto* ended in dismal failure ;-) but extending this approach would work nicely I think.
Comment 5 Björn Michaelsen 2013-11-19 14:12:05 UTC
(In reply to comment #4)
> Heh - sure, I was thinking of parsing the generated gnumake rules and deps
> etc. from running a similar (but plunging) 'make -n -p' on an autotools
> project; all previous attempts I've seen to parse auto* ended in dismal
> failure ;-) but extending this approach would work nicely I think.

Possible maybe. But since autotools uses a lot of recursive make, one would need to walk the dirs and then make sense of it by merging the pieces into one big view of the thing adding context here and there.
Comment 6 Honza Havlíček 2014-02-04 23:56:21 UTC
Hi, I will try to fix this bug. I have built LO on cygwin so far, generated the gstate output and read about .sln and .vcxproj structure.
Comment 7 Michael Meeks 2014-02-05 07:25:06 UTC
Honza - wonderful news; really looking forward to your work, and do ask for help in #libreoffice-dev on irc.freenode.net :-) mmeeks (is me) SweetShark is Bjoern.
Comment 8 Commit Notification 2014-02-10 19:45:55 UTC
Honza HavliÄek committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=2e2303a3e2904735cf7da416ec79e107785106f4

fdo#70414 Added generator of VS2012 project files



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 9 Honza Havlíček 2014-02-10 21:28:51 UTC
There is somewhat working version in master, so progress report:

What works:
* It is possible to generate solution (collection of projects) for each module that will contain C++ project for all Library_* and Executable_* in the the module.
* You can build (calls make {project makefile}), rebuild (calls make calls make {project makefile}.clean {project makefile}) and clean projects (calls make {project makefile}.clean). Use context menu of project or define keyboard shortcut (see below).
* There are multiple configurations, default one is for building projects, . You can switch between them from drop down menu in the toolbar (the one with text "Build" next to "Win32"). Other configuration are for unit testing and integration tests.
* You can run LO by from menu Debug->Start Without Debugging and attach to it (Debug-> Attach To Project, choose soffice.bin). Breakpoints work, you can change content of variables in debugger and all that good stuff.

What doesn't work:
* No projects for unit tests (parser doesn't currently support them)
* When running LO from VS, it always says that project is out of date and if it should be rebuild. How exactly does VS checks that is unclear, but it seems to check for some .pdb file in build directory.
* There might also be one solution for all projects, in addition to per-module solutions.
* Generation of projects only works when you run it in source directory, not for different directory.
* When you build solution all projects are build in parallel (= a lot of make module.build at once), that can probably lead to some trouble. Build only one project at time. VS2012 Express doesn't have shortcut for building only one project by default, I recomment defining it using Tools-> Options->Environment->Keyboard, Command "Build.BuildSelection" builds the project currently active file is part of 

Notes:
Since VS2010, MS has changed the project file structure to MSBuild (something like Ant). When VS builds project, it runs MSBuild file with target build. Except build, there are also rebuild, clean and publish (though probably only for some types of project). Instead of overriding targets in MSBuild, I used NMake type of project, that allows user to change the command performed by VS when target (build/rebuild/clean) is run in GUI: Project Properties -> Configuration Properties -> NMake.
Comment 10 Adam CloudOn 2014-02-11 09:18:49 UTC
@Honza Havlíček  - looks like an AWESOME AWESOME job !
Do you mind posting a short tutorial on how one can get up & running with this?
Simply pull latest master on Windows & one should find 'proj' files ? or some 'batch' file needs to be run?
Comment 11 Michael Meeks 2014-02-11 09:39:01 UTC
$ make vs2012-ide-integration

under Cygwin should generate the various sln files inside modules.

Honza - thanks for the great update; are we still using Cygwin 'make' underneath ? or are we starting to do native compiles ? [ if so, I guess some re-build performance testing is in order ;-].

Thanks !
Comment 12 Björn Michaelsen 2014-02-11 12:43:34 UTC
(In reply to comment #10)
> @Honza Havlíček  - looks like an AWESOME AWESOME job !
> Do you mind posting a short tutorial on how one can get up & running with
> this?

I agree with Adam on both counts, this is truly awesome and it would be great to have a tutorial like https://www.youtube.com/watch?v=-5hVXeHNt2M&hd=1 for it.

@Honza: Would you be willing to create something like that, or mentor someone else on it?
Comment 13 Honza Havlíček 2014-02-16 15:14:10 UTC
(In reply to comment #12)
> @Honza: Would you be willing to create something like that, or mentor
> someone else on it?

Yes, I will create something like that.
Comment 14 Commit Notification 2014-02-21 09:22:05 UTC
Honza HavliÄek committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=c6fd524295c0083c9a1e3f52a6abe6405cdc25a9

fdo#70414: Add dependencies for solution and create solution for all projects



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 15 Commit Notification 2014-02-23 03:37:12 UTC
Honza HavliÄek committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=b81ac16e65b311d6e43c05c22c65d2040c9d7e04

fdo#70414: Use correct target names for building Visual Studio projects.



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 16 vvort 2014-03-04 14:00:24 UTC
Hello all!

When I try to execute /opt/lo/bin/make vs2012-ide-integration
, I am getting strange error:

Vort@Vort-PC /cygdrive/e/_Projects/libreoffice-build
$ /opt/lo/bin/make vs2012-ide-integration
cd E:/_Projects/libreoffice-core && (LC_MESSAGES=C /opt/lo/bin/make cmd="/opt/lo/bin/make -npf Makefile.gbuild all" cmd || true) | E:/_Projects/libreoffice-core/bin/gbuild-to-ide --ide vs2012
make[1]: *** No rule to make target `cmd'.  Stop.
Solution LibreOffice:

What I need to do to fix this?
Comment 17 Honza Havlíček 2014-03-05 18:00:54 UTC
(In reply to comment #16)
> Hello all!
> 
> When I try to execute /opt/lo/bin/make vs2012-ide-integration
> , I am getting strange error:
> 
> Vort@Vort-PC /cygdrive/e/_Projects/libreoffice-build
> $ /opt/lo/bin/make vs2012-ide-integration
> cd E:/_Projects/libreoffice-core && (LC_MESSAGES=C /opt/lo/bin/make
> cmd="/opt/lo/bin/make -npf Makefile.gbuild all" cmd || true) |
> E:/_Projects/libreoffice-core/bin/gbuild-to-ide --ide vs2012
> make[1]: *** No rule to make target `cmd'.  Stop.
> Solution LibreOffice:
> 
> What I need to do to fix this?

Hi, 

thank you for trying out the generator. 

You are trying to generate solution in directory (/cygdrive/e/_Projects/libreoffice-build) other than the one with sources (E:/_Projects/libreoffice-core/).

I had some problems with that (IIRC the same message you got), try to 
* go to your directory with sources (/cygdrive/e/_Projects/libreoffice-core/) 
* run the autogen.sh with reasonable parameters (e.g. the ones from https://wiki.documentfoundation.org/Development/Windows_Build_Dependencies)
* run command "/opt/lo/bin/make vs2012-ide-integration" there.

If that fails, try to build the LO and then "/opt/lo/bin/make vs2012-ide-integration" (building shouldn't be necessary, but just to be sure).

I will try to investigate the issue more thoroughly.
Comment 18 vvort 2014-03-06 06:24:21 UTC
(In reply to comment #17)
> I had some problems with that (IIRC the same message you got), try to 
> * go to your directory with sources
> (/cygdrive/e/_Projects/libreoffice-core/) 
> * run the autogen.sh with reasonable parameters (e.g. the ones from
> https://wiki.documentfoundation.org/Development/Windows_Build_Dependencies)
> * run command "/opt/lo/bin/make vs2012-ide-integration" there.
Here is the results:

Vort@Vort-PC /cygdrive/e/_Projects/libreoffice-core
$ /opt/lo/bin/make vs2012-ide-integration                                       cd E:/_Projects/libreoffice-core && (LC_MESSAGES=C /opt/lo/bin/make cmd="/opt/lo/bin/make -npf Makefile.gbuild all" cmd || true) | E:/_Projects/libreoffice-core/bin/gbuild-to-ide --ide vs2012
cl : Command line warning D9025 : overriding '/Od' with '/O2'
/bin/sh: E:/_Projects/libreoffice-core/workdir/Package/ure_install.filelist.tmp: No such file or directory
/bin/sh: E:/_Projects/libreoffice-core/workdir/Package/ure_install.filelist.tmp: No such file or directory
/cygdrive/e/_Projects/libreoffice-core/external/libxml2/ExternalPackage_xml2.mk:22: *** file E:/_Projects/libreoffice-core/workdir/UnpackedTarball/xml2/win32/bin.msvc/libxml2.dll does not exist in the tarball.  Stop.
make[1]: *** [cmd] Error 2
Traceback (most recent call last):
  File "E:/_Projects/libreoffice-core/bin/gbuild-to-ide", line 833, in <module>
    VisualStudioIntegrationGenerator(gbuildparser).emit()
  File "E:/_Projects/libreoffice-core/bin/gbuild-to-ide", line 613, in emit
    project_guid = self.write_project(project_path, target)
  File "E:/_Projects/libreoffice-core/bin/gbuild-to-ide", line 675, in write_project
    os.makedirs(os.path.dirname(project_path), exist_ok = True)
  File "/usr/lib/python3.2/os.py", line 152, in makedirs
    mkdir(name, mode)
OSError: [Errno 17] File exists: './oox'
make: *** [vs2012-ide-integration] Error 1

> If that fails, try to build the LO and then "/opt/lo/bin/make
> vs2012-ide-integration" (building shouldn't be necessary, but just to be
> sure).
It usually takes 10 hours for me. It's too long.
If it's possible, please send to me project files for sdext module - I will try to test it separately.

> I will try to investigate the issue more thoroughly.
Thanks.
Comment 19 kienwt 2014-03-15 13:05:49 UTC
Any update on this?  I'm also getting a similar error.  Thanks.

 /cygdrive/c/sources/libo-core
$  /opt/lo/bin/make vs2012-ide-integration
cd C:/sources/libo-core && (LC_MESSAGES=C /opt/lo/bin/make cmd="/opt/lo/bin/make -npf Makefile.gbuild all" cmd || true) | C:/sources/libo-core/bin/gbuild-to-ide --ide vs2012
/bin/sh: C:/sources/libo-core/workdir/Package/ure_install.filelist.tmp: No such file or directory
/bin/sh: C:/sources/libo-core/workdir/Package/ure_install.filelist.tmp: No such file or directory
/cygdrive/c/sources/libo-core/external/libxml2/ExternalPackage_xml2.mk:21: *** file C:/sources/libo-core/workdir/UnpackedTarball/xml2/win32/bin.msvc/libxml2.dll does not exist in the tarball.  Stop.
make[1]: *** [cmd] Error 2
Traceback (most recent call last):
  File "C:/sources/libo-core/bin/gbuild-to-ide", line 833, in <module>
    VisualStudioIntegrationGenerator(gbuildparser).emit()
  File "C:/sources/libo-core/bin/gbuild-to-ide", line 613, in emit
    project_guid = self.write_project(project_path, target)
  File "C:/sources/libo-core/bin/gbuild-to-ide", line 675, in write_project
    os.makedirs(os.path.dirname(project_path), exist_ok = True)
  File "/usr/lib/python3.2/os.py", line 152, in makedirs
    mkdir(name, mode)
OSError: [Errno 17] File exists: './basic'
make: *** [vs2012-ide-integration] Error 1
Comment 20 kienwt 2014-03-15 13:11:36 UTC
Forgot to mention that I did a full build before running /opt/lo/bin/make vs2012-ide-integration
Comment 21 kienwt 2014-03-16 14:58:01 UTC
I ended up commented out the following line in gbuild-to-ide

#os.makedirs(os.path.dirname(project_path), exist_ok = True)

ran /opt/lo/bin/make vs2012-ide-integration several times, creating the following folders when it complained that they didn't exist:
neon
clucene
ucpp
Redland

After creating these folders and running /opt/lo/bin/make vs2012-ide-integration, it appears that the command now creates the VS2012 project files successfully.  Now to figure out how to build successfully...

I am using Windows 8 Pro and VS2012 Pro.
Comment 22 Björn Michaelsen 2014-04-27 21:39:00 UTC
Looks fixed to me?
Comment 23 Robinson Tryon (qubit) 2015-12-16 00:40:46 UTC Comment hidden (obsolete)
Comment 24 Robinson Tryon (qubit) 2016-02-18 16:37:29 UTC Comment hidden (obsolete)
Comment 25 Commit Notification 2016-12-15 07:57:29 UTC
Matúš Kukan committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=ee3f793a2d4e2977a3e952e1439708d8936953d7

tdf#70414: Fix VisualStudio ide integration

It will be available in 5.4.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.