Linus Torvalds (torvalds.delete@this.linux-foundation.org) on December 8, 2014 8:08 pm wrote:
> Maybe. And probably not. Given the choice between 16 cores and 4, I suspect most people
> will take 4, and prefer more cache, graphics, and integrated networking etc.

I'd take the 4-core option you proposed any day over the 16 cores one in spite of still spending a large portion of my time compiling code.

> Some things are cheap and easy. But those are done already. Building stuff in parallel? Trivial
> - except even there people tend to have build tools and Makefiles etc that make it not work that
> well in many cases. So even something really simple like a software workstation often doesn't
> scale that well (I'm happy to say that the kernel build scales exceptionally well, but most projects
> don't have tens of thousands of files and lots of effort on the build system.

I've got an interesting data-point to add that goes to show that a full parallel approach is not always the best choice. Mozilla's Gecko has a bizarre, complex build system it inherited from the good ole' days of Netscape and the new economy bubble. Last year quite a few developers sunk a significant portion of their time modernizing it and making sure it scaled properly by removing the typical scalar sections that plague almost all older codebases (recursive make invocations, non-parallel build steps, etc...). With well north of 20k source files, once the build system was properly parallelized, we could fully scale compilation on pretty much *any* existing hardware.

However compiling a file does have both fixed costs (e.g. bootstrapping today's large compilers) as well as totally redundant variable ones (e.g. re-parsing the same header files multiple times, which in a C++ codebase are usually both large and many). Enter unified builds: the next step after parallelizing the build system was to build sources in batches of 16 per compiler invocation instead of one at a time. This has a few drawbacks such as a significant increase in memory consumption as well as some nuisances if the sources have similarly named symbols or muck around with #defines. However the speedup was well worth it; I'll let a ML post from that period do the talking:

Recent build time improvements due to unified sources

In less than 4 months, clobber build time on my machine has decreased by ~44%,
objdir size by ~22%, and DWARF size (approximated by libxul-(text+data+bss))
by ~34%.

Now Gecko - as a large C++ codebase - is a special case; a C codebase would have benefited less thanks to not having to deal with the crazy C++ let's-stick-implementations-in-our-header-files plague. But this goes to show that parallelizing a workload to the extreme will not always guarantee you the best performance. Quite the contrary in many cases.