Reading through the recent discussion about the M1 and in particular the benchmarks that appeared on various review sites I kept thinking to myself: the M1 is a remarkably fast chip but who would like to work on macOS given how slow it is?

The thing is, I always had the feeling that macOS was slow but never really quantified it. I just loathed the idea of having to build Firefox on my Mac because it took forever. I also saw microbenchmarks were Linux was several times faster than macOS but those can't possibly give you a complete picture.

So I decided to go ahead and measure just how much slower it was on my daily workload: building Firefox. While I have a Mac I don't know how to run Linux on it, so to be able to do a more accurate comparison I ran the tests on macOS 10.15 and Debian 10 in two VMs with the same resources allocated to both (8 cores and 8GiB of memory) and paravirtualization disabled to give the kernels equal footing. In both cases I redirected the output to exclude CPU time spent rendering or in the terminal emulator.

The test consisted in building Firefox (hg revision ffeebb5c7d51) with debugging enabled and standard optimizations. All the tools required for building Firefox' are built from source and identical on the two platforms; and since you can cross-compile the inputs are also identical. Thus the differences are down to low-level libraries (libc, pthread), the kernel and a handful of system utilities such as tar, cp and friends.

The Firefox build system makes extensive use of Python, relies on GNU make for the actual build and does a bunch of assorted activities such as generating headers, calculating perfect hash functions, minifying JavaScript, etc... but the bulk of the time will be spent compiling C++ and Rust code. To speed-up repeated builds we often use sccache to cache the output of the C++ and Rust compilers. Sccache slots below the C++ preprocessor and replaces the compiler, acting as a server. It will fetch compiler output from a local zstd-compressed cache on a cache hit and invoke the actual compiler on a cache miss.

I did three tests: a plain build w/o sccache, a build w/ sccache and an empty cache, and a build w/ sccache with a fully populated cached. Here are the results:

Build         Debian    macOS
No sccache     20:41    34:32
sccache, cold  22:10    38:34
sccache, hot    3:20     7:12


The results speak for themselves. What you're seeing here is macOS being significantly slower than Linux running the exact same workload. The key here is that while this is a mostly CPU-intensive workload - with compiler invocations taking even dozens of seconds - it also spawns processes and threads, does I/O and IPC and generally exercise the underlying kernel. This becomes painfully obvious in the third test: with a hot cache the long C++ and Rust compiler invocations are gone, and sccache is just trying to respond as fast as possible to the build system, doing more I/O and IPC and spawning plenty of threads.