Bjorn is correct about the ease of adding clang.  Making with clang is not an easy hack.

Suppose <args> are the arguments used in the absence of scan-build.  The steps are as follows:
(1) Hack core/autogen.sh so that scan-build './configure <args>' is run instead of './configure <args>'. First, change 
    my @args;
to
    my @args;
    push @args, "./configure";

Then, change 
    system ("./configure", @args) && die "Error running configure";
to
    system ("scan-build", @args) && die "Error running configure";
(2) cd core; sh ./autogen.sh
(3) Instead of running 'make' from the command line, run 'scan-build make'.

On my computer, I can get to step 
    (47/157) Building module berkeleydb
... and then my computer crashes.  I'm running with Ubuntu 11.10 on a 4-core machine with 8 GB of RAM.

Allan - sorry pardon, my fault. ... At least (I hope) that this has got easier since we did a chunk of fixing to make master compile with clang (though that may have bit-rotted over the last month of course).

Gosh - your computer crashes ? or the build crashes ?

Also - we could add a parameter --autogen-scan or something to enable that in autogen.sh - a patch for that would be great. Does: scan_build ./autogen.sh instead ? [ not sure how that tool works of course ] ?

Anyhow - sorry if you slogged away seemingly endlessly without getting anywhere here. More information on the crash would be great, and it'd be great to capture your autogen tweaks in the form of a patch.

Thanks.

(Just to clarify: The original description was also by Michael, I just copied that over from the wiki)

Created attachment 57120 [details]
Raw llvm logs

I'm trying to compile LO master with llvm.
Here are the raw logs with LO master from sources of 1 or 2 days.

Created attachment 57242 [details]
scan-build clang output until vcl

I attached logs from clang compilation (clang updated today on master updated today). Badfully, vcl didn't compile perhaps because of some elements missing after some cleaning.

Created attachment 57643 [details]
Complete compilation result

I attached the result of the complete compilation of sources from master updated today.
clang updated and compiled today.

I disabled only mozilla to be sure to have no pb.
Here's the autogen.lastrun
--disable-mozilla
--with-system-boost
--enable-symbols
--enable-ext-barcode
--enable-ext-diagram
--enable-ext-google-docs
--enable-ext-hunart
--enable-ext-languagetool
--enable-ext-nlpsolver
--enable-ext-ct2n
--enable-ext-numbertext
--enable-ext-oooblogger
--enable-ext-pdfimport
--enable-postgresql-sdbc
--enable-ext-presenter-console
--enable-ext-presenter-minimizer
--enable-ext-report-builder
--enable-ext-scripting-beanshell
--enable-ext-scripting-javascript
--enable-ext-typo
--enable-ext-validator
--enable-ext-watch-window
--enable-ext-wiki-publisher
--enable-dbus
--enable-graphite
--enable-evolution2
--enable-werror
--enable-debug
--enable-dbgutil
--enable-crashdump
--enable-kde4
--enable-dependency-tracking
--enable-online-update

java version "1.6.0_24"
OpenJDK Runtime Environment (IcedTea6 1.11pre) (6b24~pre2-1)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

boost : 1.48

Hope it'll bring some interesting things

Created attachment 57655 [details]
compilation with mozilla

I removed "disable-mozilla" in autogen and compiled with mozilla.
It worked too !
I attached the result of "make" (not "make mozilla" to have the whole result)

scan-build should produce a load of annotated html files as described at https://developer.mozilla.org/en/Clang_Static_Analysis
Do you get them?

(In reply to comment #8)
> scan-build should produce a load of annotated html files as described at
> https://developer.mozilla.org/en/Clang_Static_Analysis
> Do you get them?

Yep but it's very heavy so I had removed them thinking the output would be sufficient. (perhaps I was wrong :-( )

They are what's needed. Maybe you could ask on the mailing list about somewhere to host them?

Deteted "Easyhack" from summary

I have run and uploaded a full html report of LibreOffice master ~2012-07-28 at :

http://lbalbalba.x90x.net/clang-analyzer/libreoffice/

for the time being.

If there is enough interest, I can do this on a semi-regular basis, until there is a way to do the analysis and upload automatically on a regular basis and in a permanent location.

I have run the clang analyzer on LibreOffice master ~2012-08-02 with
'--with-system-libs' in order to exclude reporting of issues in 3rd party code. The only exceptions being:

1.) '--with-system-saxon=no'
Because the open source HE variant of saxon does not have all the
necessary features.

2.) '--with-system-libcmis'
Because I couldnt get LO building with Fedora's libcmis-0.2.3 and
libcmis-devel-0.2.3.

I have uploaded the generated report here:
http://lbalbalba.x90x.net/clang-analyzer/libreoffice-with-system-libs/

Please do note that the C++ *analyzer* (not the compiler) is still very much regarded as a 'work-in-progress' at this point in time (2012-08-01), so you are likely to run into false positives. And of course, if you do find a false positive, you can always consider assisting in improving the analyzer by submitting a test-case and bug report over here ;)  :

http://clang-analyzer.llvm.org/filing_bugs.html

I have run the analyzer on master (2012-08-15), but now with the clang compiler instead of using gcc for compilation (in order to prevent this error "error: use of undeclared identifier '__float128'").

The ./configure flags were :

--disable-ccache
--enable-debug
--with-system-libs
--with-system-hsqldb=no 
--with-system-saxon=no
--with-system-libcmis=no

The full report is here:
http://lbalbalba.x90x.net/clang-analyzer/libreoffice-with-clang/

The new, permanent home for the clang analyzer reports is :

http://dev-builds.libreoffice.org/clang_reports/

Added the URL to the location of the report(s).

Is it possible to update the folders automatically on a weekly basis? The same could apply to cppcheck stuff. I think this is a good entry point for new contributers.

Thomas: about cppcheck, I put fdo in see also.
I'm trying to upload new reports every 1 or 2 weeks (with cppcheck git updated too), see http://dev-builds.libreoffice.org/cppcheck_reports/master/.
For clang, it's more complicated (at least for me! :-)) because of building problems.

adding LibreOffice developer list as CC to unresolved EasyHacks for better visibility.

see e.g. http://nabble.documentfoundation.org/minutes-of-ESC-call-td4076214.html for details

Juan Picca committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=5573fd8cf991b62c405ec9a4b624ebaf28cb8894

fdo#39596: replace cstyle casts

It will be available in 4.5.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.

There is a wiki page describing how to generate these reports: https://wiki.documentfoundation.org/Development/Clang_Code_Analysis

I guess I could turn that into a shell script to periodically and automatically generate these reports. But is there still a desire for this, especially now that there also are cppcheck and coverity reports ?

I am building this as https://wiki.documentfoundation.org/Development/Clang_Code_Analysis
described now...
But I am not sure what should I look for. To see if clang can find any bugs? Can any body give me some more hints?

And I found some tricky things while building...is that helpful to others?

(In reply to danichocolate714 from comment #23)
> I am building this as
> https://wiki.documentfoundation.org/Development/Clang_Code_Analysis
> described now...
> But I am not sure what should I look for. To see if clang can find any bugs?
> Can any body give me some more hints?
> 
> And I found some tricky things while building...is that helpful to others?

Hi,


I think there are a few things one could choose to pick up (or not) here:


1.) Create the html reports
Perhaps building with clang requires updates to the code or libreoffice build system.

2.) Update / Improve the wiki page documentation
The wiki page that describes how to create the reports may not be as accurate or precise as one would like.

3.) Create an automated way to produce the clang reports 
Create a (shell, python, etc.) script that can then be scheduled somewhere to automaticallly create and update the reports, and upload them somewhere on a regular basis.

4.) Investigate the reports
Investigate the issues in the html report, and submit patches for the actual real issues found (instead of the false positives).


Hope this helps.

Hi Maarten,

1 I finished the build and checked all the comments to this issue again. Then did I realize as Julien Nabet mentioned: there are reports produced every one or two weeks (By now the latest one is Mar 20th 2015) at this link: http://dev-builds.libreoffice.org/cppcheck_reports/master/

2 I have edited the wiki page with account danichocolate_ https://wiki.documentfoundation.org/Development/Clang_Code_Analysis
adding some problems I faced during building. Hope it's helpful. 

3 There will be only three lines in the shell script. Thus I would think it's fine not to have a fully automatic shell script for now.

4 I am still looking into the bug reports. Most of them are style errors which wouldn't harm the project, I would like to report new bugs if I find any valuable ones. 

Based on Julien Nabet, libreoffice is built with clang frequently. Thus I would suggest to close this bug.

(In reply to danichocolate714 from comment #25)
> Hi Maarten,
> 
> 1 I finished the build and checked all the comments to this issue again.
> Then did I realize as Julien Nabet mentioned: there are reports produced
> every one or two weeks (By now the latest one is Mar 20th 2015) at this
> link: http://dev-builds.libreoffice.org/cppcheck_reports/master/
> ...
> Based on Julien Nabet, libreoffice is built with clang frequently. Thus I
> would suggest to close this bug.

Hi Danichocolate714,

cppcheck is a C/C++ static analyzer, clang is a front-end compiler.

I was building and put available quite regularly cppcheck reports but now it's indeed made automatically thanks to Maarten.

About clang, I'm not an expert at all, I can only say there's a Tinderbox which uses it and that clang can detect things that cppcheck can't detect (I don't think the contrary can happen but might be wrong).

So please, don't base on me to suggest to close this bug :-)

Hi,

I think you are confusing two different kinds of reports here.

You mention the cppcheck reports located at : http://dev-builds.libreoffice.org/cppcheck_reports/master/
These do indeed get generated automatically once every week now. (it has its own bug report in bugzilla #39440). And the method to generate those is described here: https://wiki.documentfoundation.org/Development/Cppcheck

But this bug report is about the clang reports. There are a few reports, but they are old: http://dev-builds.libreoffice.org/clang_reports/. The way to generate them is described here: https://wiki.documentfoundation.org/Development/Clang_Code_Analysis

Hi,

Yes you are right. My bad ><

A polite ping, still working on this patch ?

I am still trying to figure out, how to get it done without errors. Until then I am setting it to unassigned.

One of the problems with doing this, as far as I can see, is this: 

The developers of LibreOffice seem to be interested only/primarily in analyzing the core LibreOffice codebase, and not in the many dependencies/pre-requisites of that codebase (and understandably so). Unfortunately, the tool that does the actual analysis (scan-build) will and can only (largely due to the way the analyzer is interposing on the compiler) analyze *all* the code that gets build when you run 'make'. There is no way to tell 'scan-build' that you specifically want to include/exclude parts of the source(-tree) that gets build when you run 'make'. There's an enhancement request to add this functionality over at llvm bugzilla [1], but so far it has not been added.

So when you run 'autogen.sh/configure' with '--without-system-libs --without-system-headers', which would be the easiest way to ensure you get all the right dependencies/pre-requisites in their correct respective versions (included/pulled-in in the LibreOffice source tree), not only the LibreOffice codebase gets analyzed, but all the dependencies/pre-requisites as well, as all of these will be build during 'make'.

Which means that you are pretty much forced to use 'autogen.sh/configure' with '--with-system-libs --with-system-headers', as using the distribution provided binaries excludes them from being build during 'make' and therefore excluded from analysis. But this requires you to install all the dependencies/pre-requisites system-wide, and not all (or perhaps even none) Linux distributions offer the needed packages, or not in the right versions. Also, every time the LibreOffice source needs an updated version of a package or a new package, you're going to need a way to determine this new package/version in an automated way, and then install it (assuming the distribution has a pre-build package available in the right version to begin with).


[1]
https://bugs.llvm.org//show_bug.cgi?id=22594

Hii
I am Diya. I am starting work on this bug.

Please don't change assignee/status as this is not a one-shot thing.

I just noticed this option in the output of clang9 'scanbuild -h' :

 --exclude <path>

   Do not run static analyzer against files found in this
   directory (You can specify this option multiple times).
   Could be useful when project contains 3rd party libraries.

Although I have not actually tried it yet, it at least gives the impression that it might do what you are looking for. I'm assuming we need to exclude at least these :

$src/*/UnpackedTarball/*
$src/workdir/*
$build/workdir/*
$src/instdir/*
$src/external/*

I'm intrigued. I might take a look at this, but am not committing to anything yet. Also, Diya Mourya might already have something here already.

Well, I finally got around to testing this, and the '--exclude' option seems to work as expected. If you run 'scan-build' with (among others) these options :

--exclude $src/*/UnpackedTarball/* --exclude $src/workdir/* --exclude $src/instdir/* --exclude $src/external/

then the 3th party code does not end up in the final report.

So you can now run ./autogen with '--without-system-libs --without-system-headers' so you get the needed packages in the right versions during the build, and in combination with the mentioned '--exclude' options without them ending up in the final report. You do need clang9 (or newer) though, older versions do not have the '--exclude' option for scan-build.

Created attachment 166253 [details]
Full scan-build report of the LibreOffice codebase.

This attachment is a compressed tarfile, containing a full report of the LibreOffice codebase.

A first version of a shell script that can automate the generation of the reports has been put up for review in gerrit: https://gerrit.libreoffice.org/c/dev-tools/+/104372

Has all work in this hack been completed?
If not, can someone please provide me with a list of what all is yet to be done?

(In reply to adityapratapsingh51 from comment #40)
> Has all work in this hack been completed?
> If not, can someone please provide me with a list of what all is yet to be
> done?

Read the scan-build-report/README and run the script scan-build-report/scan-build-report.sh kindly created by Maarten.

So, does this EasyHack involve uploading updated tarfile of scan-build report or does it involve solving the bug reports in the scan-build report?

(In reply to adityapratapsingh51 from comment #42)
> So, does this EasyHack involve uploading updated tarfile of scan-build
> report or does it involve solving the bug reports in the scan-build report?

Hmm, you have a point (I didn't read the description earlier). This report has one commit (comment 21) for implementing a suggestion from the analyser, but that's it. So perhaps this is indeed solved in line with the original intent. Björn would need to comment.

Closing after discussion in the dev chat.