I wish someone would pick up the zapcc project. Compilers do an insane amount of duplicated and redundant work and there's tremendous potential for speeding up C++ builds if you are willing to rethink how the compiler works a little bit. https://github.com/yrnkrn/zapcc
I work on the video game factorio, which is a c++ project. On a 9900k a rebuild takes about a minute, so it's pretty sizeable but not something ridiculous like oracle db or unreal engine. I tried using zapcc on it, and it was a complete failure. I don't have measurements to hand, but iirc it was actually slower than stock clang. I tested it on a threadripper 2950x with 64gb of ram, running Debian.
I just installed the game again to play the v1.0 release.
It looks and sounds very good! Congratulations on making it to the release milestone after many years of hard work.
As a developer, I must say that I'm deeply impressed by the simulation complexity and incredible scalability.
I keep telling people that Factorio is the most complex train simulator game that I've ever played. But it isn't a train simulator. It's a factory game. The trains are optional.
That's crazy to me, that a subset of a game can be more complex than entire games that "specialise" in the topic.
The first bit of track you place is always straight; but if you extend the rail (by clicking on the green arrow of the first piece of rail) you can place curves.
I've had good results on some of my own projects. The current implementation certainly has limitations, but it's really early and the potential is huge. For example, because it can cache at a sub-compilation-unit granularity it can make rebuilds of a single file faster, which no distributed build tool or cache can do.
We do use a unity build yeah. Without it would be way way longer, maybe 5 minutes+.
As for removing templates, there are a few places where we specifically opted to use custom macros instead of std::variant, for compile time reasons, but mostly we do use templates quite a lot.
A unity build gives some of the same build time benefits as zapcc for full rebuilds, but requires more work and compromises in code structure, and also increases your single-file incremental build time. If you're already using a unity build then I'd expect zapcc to not speed up full builds much if at all.
I'd be interested to see if zapcc could speed up your non-unity build to be close to your unity build, without requiring the same compromises. I'd also be interested to see if zapcc would speed up single-file incremental builds in your unity build.
Ultimately I can't recommend zapcc for general use as it's a fork of clang that's not being regularly rebased anymore, so it's getting out of date. That's why I wish someone would pick it up again.
That is what I normally use. ccache is great! With a unity build its usefulness is a bit diminished, but it's still great for switching back and forth between branches.
I remember watching the talk awhile ago, but haven't kept up with its progress. It's about using LLVM as a library to eliminate duplicate work, as far as I remember.
One reason why this might not be so simple is that what is often measured as a potential for speed improvement is I/O, as in "lines of code a compiler needs to read to process a translation unit".
However what happens is that C++ compilers are usually CPU-bound, not I/O bound.
What happens also is that C++, very differently from idiomatic C, Pascal, Modula, and their successors, has no clear separation between declaration and use of an interface, as not functions, but classes form the main interface, and what defines a class is the signature of its member functions and its members, and to know the size of the class, the size of its members need to be known, which has the consequence that the C++ compiler needs to read recursively everything which the class is defined with. This is a big difference to modula etc.
Also, of course the reading of data can be cached, but the question is how much the expansion of the read text can be cached - for example some will have different semantics if the DEBUG macro is set, or some WIN_xyz macro. As far as I understand, compilers supporting C++ modules will only cache the latest expansion.
Also, the build system with modules is likely to become more complex, with additional dependencies. I do not know whether this is still actual in every aspect (it could be that what is allowed to go into a module header has been narrowed down a bit), but this post : https://vector-of-bool.github.io/2019/01/27/modules-doa.html explains at least some of the topics involved, and not all of them are trivial.
Real-world usage of Clang modules (by Google) and preliminary/synthetic benchmarks of C++20 modules in GCC both show about 3x speedup. This might not be "much" for everyone, buy I will take it.
> benchmarks of C++20 modules in GCC both show about 3x speedup
compared to what? The worst possible case? Code in weakly coupled modules? Using PIMPL idiom? Unity builds? Using ccache in incremental builds?
Especially ccache is a relevant comparison, because what to a developer is of far more interest than a full build is an incremental build with a few files changed. Using modules requires to scan the source code in order to determine dependencies, so it can be slower than an incremental build using other tools. And in addition to this, C++ compilation with include headers is an embarrassingly parallel problem which means speed advantages from using multi-core systems will probably very relevant in the, say, ten years time in which C++ modules will be somewhat standardized in practice.
Not to say they can't help, but in any other large software project one would need to show that something is useful before adding a big change.
