Article: MAQSIP-RT: An HPC Benchmark
By: Linus Torvalds (torvalds.delete@this.linux-foundation.org), June 28, 2010 7:48 am
Room: Moderated Discussions
? (0xe2.0x9a.0x9b@gmail.com) on 6/28/10 wrote:
>
>For example, I don't understand why don't you fully admit
>that current C/C++ compilers did not really embrace the
>idea of "precompiled header files". They started to support
>it long time ago - but then stopped in half-way of the
>process. So, what we have now is some *mediocre* (at best)
>implementation of this idea.
The thing is, precompiled header files really are very
hard to do right for C/C++. The semantics of a C header
include are not simply very amenable to doing it. It's
entirely possible - and quite common - for a header file
to totally change depending on some random preprocessor
symbol.
And even if you handle that issue (by versioning all the
precompiled headers by all the preprocessor symbols they
depend on), you still don't solve the problem: because
even in the absence of those kinds of big and obvious
textual context changes, you still end up having all the
subtle type issues where a type in a header file will
depend on types declared earlier in other places.
The fact is, the C header files are very much inherently
designed to be about textual expansion, not about some
notion of "interface" that you include. So the only simple
kind of precompilation you can do is to pre-parse the thing
(tokenization, comment removal etc), anything else is very
complicated indeed.
And don't get me wrong. "very complicated" doesn't mean
"impossible". It's been done, but almost every time it has
been done, it has resulted in either (a) nasty bugs where
the precompilation infrastructure means that the compiler
doesn't really follow C/C++ semantics or (b) not noticeably
faster compile times, because the added complexities add
enough overhead and the win isn't sufficiently large (which
is usually because the precompilation does mainly some
pre-tokenization, so the win simply isn't huge).
The thing is, you can get fast compile times, but
the way to do so is to keep your headers simple and clean.
And not have some spaghetti of headers including other
headers.
But that is hard, and requires insane amounts of effort.
It's hard even with plain C, and it tends to be harder
still in C++, if only because you end up having even more
complicated headers for the templates etc.
Which is why usually people resort to parallel builds. That
usually requires a fair amount of effort too, and seldom do
you see really well parallelized build environments. But
you can usually get a fairly good speedup reasonable easily
without spending an inordinate amount of time on it.
Still - even for "just" parallel builds, getting all the
dependencies right, and avoiding lots of serialization
points, can be quite hard. In a big project (and that's
when it matters most) you usually try to have subprojects
with independent makefiles etc. And when one of them is a
library that some other part depends on, it's usually not
all that trivial to build the two parts in parallel.
There is a reason why other languages try to not do the C
or C++ kind of preprocessor. They tend to try to specify
dependencies not as text replacement, but as "interface
definitions". That avoids a lot of the complexities with
precompiling the headers.
Linus
>
>For example, I don't understand why don't you fully admit
>that current C/C++ compilers did not really embrace the
>idea of "precompiled header files". They started to support
>it long time ago - but then stopped in half-way of the
>process. So, what we have now is some *mediocre* (at best)
>implementation of this idea.
The thing is, precompiled header files really are very
hard to do right for C/C++. The semantics of a C header
include are not simply very amenable to doing it. It's
entirely possible - and quite common - for a header file
to totally change depending on some random preprocessor
symbol.
And even if you handle that issue (by versioning all the
precompiled headers by all the preprocessor symbols they
depend on), you still don't solve the problem: because
even in the absence of those kinds of big and obvious
textual context changes, you still end up having all the
subtle type issues where a type in a header file will
depend on types declared earlier in other places.
The fact is, the C header files are very much inherently
designed to be about textual expansion, not about some
notion of "interface" that you include. So the only simple
kind of precompilation you can do is to pre-parse the thing
(tokenization, comment removal etc), anything else is very
complicated indeed.
And don't get me wrong. "very complicated" doesn't mean
"impossible". It's been done, but almost every time it has
been done, it has resulted in either (a) nasty bugs where
the precompilation infrastructure means that the compiler
doesn't really follow C/C++ semantics or (b) not noticeably
faster compile times, because the added complexities add
enough overhead and the win isn't sufficiently large (which
is usually because the precompilation does mainly some
pre-tokenization, so the win simply isn't huge).
The thing is, you can get fast compile times, but
the way to do so is to keep your headers simple and clean.
And not have some spaghetti of headers including other
headers.
But that is hard, and requires insane amounts of effort.
It's hard even with plain C, and it tends to be harder
still in C++, if only because you end up having even more
complicated headers for the templates etc.
Which is why usually people resort to parallel builds. That
usually requires a fair amount of effort too, and seldom do
you see really well parallelized build environments. But
you can usually get a fairly good speedup reasonable easily
without spending an inordinate amount of time on it.
Still - even for "just" parallel builds, getting all the
dependencies right, and avoiding lots of serialization
points, can be quite hard. In a big project (and that's
when it matters most) you usually try to have subprojects
with independent makefiles etc. And when one of them is a
library that some other part depends on, it's usually not
all that trivial to build the two parts in parallel.
There is a reason why other languages try to not do the C
or C++ kind of preprocessor. They tend to try to specify
dependencies not as text replacement, but as "interface
definitions". That avoids a lot of the complexities with
precompiling the headers.
Linus
Topic | Posted By | Date |
---|---|---|
New article online: MAQSIP RT | David Kanter | 2010/06/21 11:57 AM |
Why no GCC? | Rohit | 2010/06/22 09:25 PM |
Why no GCC? | David Kanter | 2010/06/23 12:45 AM |
sun 's cc better than GCC? | Rohit | 2010/06/23 05:04 AM |
sun 's cc better than GCC? | anon | 2010/06/23 07:49 AM |
Where is the GCC optimization effort directed? | Mark Roulo | 2010/06/23 10:42 AM |
GCC is very ugly bad everywhere in 64 bits | Vincent Diepeveen | 2010/06/23 02:49 PM |
even for 64-bit arch? | anon | 2010/06/23 02:59 PM |
GCC is very ugly bad everywhere in 64 bits | ajensen | 2010/06/23 11:03 PM |
GCC is very ugly bad everywhere in 64 bits | Gabriele Svelto | 2010/06/24 02:33 AM |
GCC is very ugly bad everywhere in 64 bits | ajensen | 2010/06/24 05:32 AM |
GCC is very ugly bad everywhere in 64 bits | Gabriele Svelto | 2010/06/24 07:18 AM |
GCC is very ugly bad everywhere in 64 bits | ajensen | 2010/06/24 09:50 AM |
Why GCC is big and complicated (my guess) | Mark Roulo | 2010/06/24 12:17 PM |
Why GCC is big and complicated (my guess) | Gabriele Svelto | 2010/06/28 04:00 AM |
GCC is very ugly bad everywhere in 64 bits | Bernd Schmidt | 2010/06/24 05:46 AM |
GCC is very ugly bad everywhere in 64 bits | ajensen | 2010/06/24 09:43 AM |
GCC is very ugly bad everywhere in 64 bits | Vincent Diepeveen | 2010/06/26 02:12 PM |
GCC is very ugly bad everywhere in 64 bits | Rob Thorpe | 2010/06/24 07:47 AM |
GCC is very ugly bad everywhere in 64 bits | Anon | 2010/06/24 05:23 PM |
Where is the GCC optimization effort directed? | Gabriele Svelto | 2010/06/23 10:45 PM |
Where is the GCC optimization effort directed? | ? | 2010/06/24 01:48 AM |
Where is the GCC optimization effort directed? | Gabriele Svelto | 2010/06/24 02:29 AM |
Where is the GCC optimization effort directed? | ? | 2010/06/24 03:13 AM |
Where is the GCC optimization effort directed? | Andi Kleen | 2010/06/24 03:15 AM |
Where is the GCC optimization effort directed? | ? | 2010/06/24 04:08 AM |
Where is the GCC optimization effort directed? | Gabriele Svelto | 2010/06/24 03:54 AM |
Where is the GCC optimization effort directed? | ? | 2010/06/24 04:15 AM |
Where is the GCC optimization effort directed? | Gabriele Svelto | 2010/06/24 07:22 AM |
Where is the GCC optimization effort directed? | Rohit | 2010/06/24 03:04 AM |
Placebo effect | ? | 2010/06/24 06:37 AM |
Placebo effect | Rohit | 2010/06/24 08:45 AM |
Placebo effect | Vincent Diepeveen | 2010/06/26 02:50 PM |
Compile time | Mark Roulo | 2010/06/26 05:28 PM |
Compile time | Richard Cownie | 2010/06/27 04:44 AM |
Compile time | Mark Roulo | 2010/06/27 10:12 AM |
Compile time | Mark Roulo | 2010/06/27 10:21 AM |
Compile time | EduardoS | 2010/06/27 11:37 AM |
Compile time | Richard Cownie | 2010/06/27 04:07 PM |
Compile time & efficiency | ? | 2010/06/28 12:03 AM |
Compile time & efficiency | Mark Christiansen | 2010/06/28 06:08 AM |
Compile time & efficiency | Linus Torvalds | 2010/06/28 07:48 AM |
kernel programming language | John Simon | 2010/06/29 06:46 PM |
Compile time & efficiency | Richard Cownie | 2010/06/28 09:29 AM |
Compile time & efficiency | Linus Torvalds | 2010/06/28 11:17 AM |
Compile time & efficiency | Richard Cownie | 2010/06/28 02:16 PM |
Compile time & efficiency | Richard Cownie | 2010/06/28 06:23 PM |
Compile time & efficiency | Mark Roulo | 2010/06/29 08:31 AM |
Compile time & efficiency | Richard Cownie | 2010/06/29 11:48 AM |
Compile time & efficiency | rwessel | 2010/06/29 12:28 PM |
C is a crappy | dev | 2010/06/29 07:12 PM |
C is a crappy, but only when you push it out of it's niche | Rohit | 2010/06/30 02:11 AM |
C is a crappy | anon | 2010/06/30 02:17 AM |
C is a crappy | dev | 2010/06/30 07:59 AM |
C is a crappy | Max | 2010/07/01 04:30 AM |
C is a crappy | Michael S | 2010/07/01 07:00 AM |
C is a crappy | Konrad Schwarz | 2010/07/01 08:02 AM |
C is a crappy | Michael S | 2010/07/01 08:50 AM |
C isn't so crappy | anon | 2010/07/01 10:11 AM |
C isn't so crappy | Mikael Tillenius | 2010/07/01 11:39 AM |
C is a crappy | Konrad Schwarz | 2010/07/01 11:22 AM |
C is a crappy | Max | 2010/07/02 08:44 AM |
C is a crappy | rwessel | 2010/07/02 12:33 PM |
C is a crappy | anon | 2010/07/02 01:17 PM |
C is a crappy | Max | 2010/07/02 02:56 PM |
C is a crappy | Max | 2010/07/02 03:13 PM |
C is a crappy | rwessel | 2010/07/02 03:32 PM |
C is a crappy | Max | 2010/07/02 04:19 PM |
C is a crappy | Gabriele Svelto | 2010/07/05 05:25 AM |
C is a crappy | gallier2 | 2010/07/02 12:14 AM |
C is a crappy | Ian Ollmann | 2010/07/06 03:07 PM |
Portability | Max | 2010/07/06 03:37 PM |
C is a crappy | hobold | 2010/07/07 02:31 AM |
C is a crappy | Ian Ollmann | 2010/07/07 05:18 PM |
failure to standardize types | Carlie Coats | 2010/07/07 04:11 AM |
C is a crappy | Konrad Schwarz | 2010/07/07 08:34 AM |
C is a crappy | Ian Ollmann | 2010/07/07 05:29 PM |
C is a crappy NOT | Konrad Schwarz | 2010/07/08 12:29 AM |
C is a crappy | anon | 2010/07/01 10:40 PM |
C type safety | ? | 2010/07/02 01:10 AM |
C type safety | anon | 2010/07/02 11:02 PM |
C is a crappy | dev | 2010/07/03 04:51 PM |
C is a crappy | anon | 2010/07/03 07:02 PM |
C is a crappy | dev | 2010/07/05 07:27 AM |
C is a crappy | ? | 2010/07/05 09:05 AM |
C is a crappy | anonymous | 2010/07/07 08:32 AM |
C is a crappy | ? | 2010/07/07 10:48 PM |
C is a crappy | Anon | 2010/07/08 12:53 AM |
C is a crappy and a crappie is a fish | anonymous | 2010/07/03 07:24 PM |
Compile time & efficiency | Michael S | 2010/06/29 03:18 AM |
Compile time & efficiency | rwessel | 2010/06/29 12:20 PM |
Compile time & efficiency | someone | 2010/06/30 11:03 AM |
Compile time & efficiency | Jouni Osmala | 2010/07/02 05:29 AM |
Compile time & efficiency | Max | 2010/06/28 05:05 PM |
Compile time & efficiency | EduardoS | 2010/06/28 05:11 PM |
Compile time & efficiency | Michael S | 2010/06/29 03:33 AM |
Compile time | Foo_ | 2010/06/28 09:03 AM |
sun 's cc better than GCC? | Silent | 2010/06/23 06:19 PM |
sun 's cc better than GCC? | Foo_ | 2010/06/23 07:06 PM |
sun 's cc better than GCC? | Andi Kleen | 2010/06/24 02:49 AM |
sun 's versus gcc | Vincent Diepeveen | 2010/06/23 03:07 PM |
Why no GCC? | Carlie Coats | 2010/06/23 05:11 AM |