Bug 129632 - ccache does not get any hits on 'make sc' just after 'make sc.clean'
Summary: ccache does not get any hits on 'make sc' just after 'make sc.clean'
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
7.0.0.0.alpha0+
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: Dev-related
  Show dependency treegraph
 
Reported: 2019-12-26 16:50 UTC by Dennis Francis
Modified: 2021-08-13 23:18 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:
Regression By:


Attachments
ccache debug output comparison (26.36 KB, application/gzip)
2019-12-27 13:49 UTC, Dennis Francis
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Dennis Francis 2019-12-26 16:50:44 UTC
Description:
After a fresh build of master, and a 'make sc.clean' ccache does not get any hits on running 'make sc.build'.

$ cat autogen.input
--with-parallelism=2
--disable-skia
--without-junit
--enable-symbols
--without-doxygen
--without-help
--without-myspell-dicts
--enable-assert-always-abort

Steps to Reproduce:
1. Setup ccache if you don't have already.
2. Clear ccache cache if there are any.
3. Start a fresh build of core.git master branch (I am at head 56063d30628ee2497a7c559d7de42d26e9cfc8ae). To save time for the purposes of reproduction, do 'make build-nocheck'
4. run 'make sc.clean'
5. run 'make sc.build'

Actual Results:
The fifth step 'make sc.build' actually invokes the compiler every time because of ccache misses. It is pretty obvious due to the long time it takes to finish this step. 

Expected Results:
The fifth step 'make sc.build' should have finished under a minute. At least until a few weeks ago, with ccache this step used to finish under a minute.


Reproducible: Always


User Profile Reset: No



Additional Info:
uname -a
Linux dennis-laptop 4.19.0-6-amd64 #1 SMP Debian 4.19.67-2+deb10u2 (2019-11-11) x86_64 GNU/Linux

ccache -V
ccache version 3.6

I'm certain that there was no such issue in the master from late November. I'll try to get debug logs from ccache debug-mode runs, next time I build.
Comment 1 Dennis Francis 2019-12-27 13:49:39 UTC Comment hidden (no-value)
Comment 2 Dennis Francis 2019-12-27 13:50:41 UTC Comment hidden (no-value)
Comment 3 Dennis Francis 2019-12-29 11:04:59 UTC
Please ignore comment#2. Now I understand what the problems is. One thing I forgot to take into account was that the fresh build is done as a background process (make build-nocheck) and all other steps of incremental builds are done in the foreground (as there is no reason for me to run them in background). Ccache hash for a source file is different for foreground vs background builds due to conditional use of the compiler flag '-fdiagnostics-color' since commit 9ab2703d5c212746c961dcb238a4c3025d3a2587 "Improve -fdiagnostics-color handling".

CC'ing sberg as I'm not confident enough with gbuild code to implement a proper solution other than reverting the change. Thanks!
Comment 4 Miklos Vajna 2020-01-02 08:32:06 UTC
Ah, interesting. I didn't hit this, because I have 'export gb_COLOR=1' in my ~/.bash_login, so the gb_COLOR path is taken all the time. (Although I usually save the output of 'make' after a pull, and not otherwise, so I would also hit this.)
Comment 5 Luboš Luňák 2020-01-02 13:37:18 UTC
I assume you previously didn't get colored gcc output?

Your problem is not a bug, it's simply the way it is. Ccache does not share "identical" compilations if they differ in colorization, because, well, they do differ - there's a whole section on this in the ccache manual or in a github request or somewhere.

So if you want it to work the same way as before, unset gb_COLOR (with the price of not having any colors). Or always force gb_COLOR and accept that the redirected output will have escape sequences. Or do not do anything and accept that ccache does not share these. Or do not run your large build with redirected output. Or find the ccache request with details and improve ccache to be smart about sharing them. But whatever you do, there's a drawback.

And then there's the issue of gbuild trying to be (too) smart and doing a lot of stuff that it probably shouldn't be doing. But that's not limited to just this, there's a whole load of things that gbuild messes with and probably shouldn't (compiler colorization, make parallelism, icecream setup, whatnot).