Compile time & efficiency

Article: MAQSIP-RT: An HPC Benchmark
By: Linus Torvalds (torvalds.delete@this.linux-foundation.org), June 28, 2010 7:48 am
Room: Moderated Discussions
? (0xe2.0x9a.0x9b@gmail.com) on 6/28/10 wrote:
>
>For example, I don't understand why don't you fully admit
>that current C/C++ compilers did not really embrace the
>idea of "precompiled header files". They started to support
>it long time ago - but then stopped in half-way of the
>process. So, what we have now is some *mediocre* (at best)
>implementation of this idea.

The thing is, precompiled header files really are very
hard to do right for C/C++. The semantics of a C header
include are not simply very amenable to doing it. It's
entirely possible - and quite common - for a header file
to totally change depending on some random preprocessor
symbol.

And even if you handle that issue (by versioning all the
precompiled headers by all the preprocessor symbols they
depend on), you still don't solve the problem: because
even in the absence of those kinds of big and obvious
textual context changes, you still end up having all the
subtle type issues where a type in a header file will
depend on types declared earlier in other places.

The fact is, the C header files are very much inherently
designed to be about textual expansion, not about some
notion of "interface" that you include. So the only simple
kind of precompilation you can do is to pre-parse the thing
(tokenization, comment removal etc), anything else is very
complicated indeed.

And don't get me wrong. "very complicated" doesn't mean
"impossible". It's been done, but almost every time it has
been done, it has resulted in either (a) nasty bugs where
the precompilation infrastructure means that the compiler
doesn't really follow C/C++ semantics or (b) not noticeably
faster compile times, because the added complexities add
enough overhead and the win isn't sufficiently large (which
is usually because the precompilation does mainly some
pre-tokenization, so the win simply isn't huge).

The thing is, you can get fast compile times, but
the way to do so is to keep your headers simple and clean.
And not have some spaghetti of headers including other
headers.

But that is hard, and requires insane amounts of effort.
It's hard even with plain C, and it tends to be harder
still in C++, if only because you end up having even more
complicated headers for the templates etc.

Which is why usually people resort to parallel builds. That
usually requires a fair amount of effort too, and seldom do
you see really well parallelized build environments. But
you can usually get a fairly good speedup reasonable easily
without spending an inordinate amount of time on it.

Still - even for "just" parallel builds, getting all the
dependencies right, and avoiding lots of serialization
points, can be quite hard. In a big project (and that's
when it matters most) you usually try to have subprojects
with independent makefiles etc. And when one of them is a
library that some other part depends on, it's usually not
all that trivial to build the two parts in parallel.

There is a reason why other languages try to not do the C
or C++ kind of preprocessor. They tend to try to specify
dependencies not as text replacement, but as "interface
definitions". That avoids a lot of the complexities with
precompiling the headers.

Linus
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
New article online: MAQSIP RTDavid Kanter2010/06/21 11:57 AM
  Why no GCC?Rohit2010/06/22 09:25 PM
    Why no GCC?David Kanter2010/06/23 12:45 AM
      sun 's cc better than GCC?Rohit2010/06/23 05:04 AM
        sun 's cc better than GCC?anon2010/06/23 07:49 AM
          Where is the GCC optimization effort directed?Mark Roulo2010/06/23 10:42 AM
            GCC is very ugly bad everywhere in 64 bitsVincent Diepeveen2010/06/23 02:49 PM
              even for 64-bit arch?anon2010/06/23 02:59 PM
              GCC is very ugly bad everywhere in 64 bitsajensen2010/06/23 11:03 PM
                GCC is very ugly bad everywhere in 64 bitsGabriele Svelto2010/06/24 02:33 AM
                  GCC is very ugly bad everywhere in 64 bitsajensen2010/06/24 05:32 AM
                    GCC is very ugly bad everywhere in 64 bitsGabriele Svelto2010/06/24 07:18 AM
                      GCC is very ugly bad everywhere in 64 bitsajensen2010/06/24 09:50 AM
                        Why GCC is big and complicated (my guess)Mark Roulo2010/06/24 12:17 PM
                          Why GCC is big and complicated (my guess)Gabriele Svelto2010/06/28 04:00 AM
                GCC is very ugly bad everywhere in 64 bitsBernd Schmidt2010/06/24 05:46 AM
                  GCC is very ugly bad everywhere in 64 bitsajensen2010/06/24 09:43 AM
                  GCC is very ugly bad everywhere in 64 bitsVincent Diepeveen2010/06/26 02:12 PM
                GCC is very ugly bad everywhere in 64 bitsRob Thorpe2010/06/24 07:47 AM
                  GCC is very ugly bad everywhere in 64 bitsAnon2010/06/24 05:23 PM
            Where is the GCC optimization effort directed?Gabriele Svelto2010/06/23 10:45 PM
              Where is the GCC optimization effort directed??2010/06/24 01:48 AM
                Where is the GCC optimization effort directed?Gabriele Svelto2010/06/24 02:29 AM
                  Where is the GCC optimization effort directed??2010/06/24 03:13 AM
                    Where is the GCC optimization effort directed?Andi Kleen2010/06/24 03:15 AM
                      Where is the GCC optimization effort directed??2010/06/24 04:08 AM
                    Where is the GCC optimization effort directed?Gabriele Svelto2010/06/24 03:54 AM
                      Where is the GCC optimization effort directed??2010/06/24 04:15 AM
                        Where is the GCC optimization effort directed?Gabriele Svelto2010/06/24 07:22 AM
                Where is the GCC optimization effort directed?Rohit2010/06/24 03:04 AM
                  Placebo effect?2010/06/24 06:37 AM
                    Placebo effectRohit2010/06/24 08:45 AM
                    Placebo effectVincent Diepeveen2010/06/26 02:50 PM
                      Compile timeMark Roulo2010/06/26 05:28 PM
                        Compile timeRichard Cownie2010/06/27 04:44 AM
                          Compile timeMark Roulo2010/06/27 10:12 AM
                          Compile timeMark Roulo2010/06/27 10:21 AM
                            Compile timeEduardoS2010/06/27 11:37 AM
                            Compile timeRichard Cownie2010/06/27 04:07 PM
                              Compile time & efficiency?2010/06/28 12:03 AM
                                Compile time & efficiencyMark Christiansen2010/06/28 06:08 AM
                                Compile time & efficiencyLinus Torvalds2010/06/28 07:48 AM
                                  kernel programming languageJohn Simon2010/06/29 06:46 PM
                                Compile time & efficiencyRichard Cownie2010/06/28 09:29 AM
                                  Compile time & efficiencyLinus Torvalds2010/06/28 11:17 AM
                                    Compile time & efficiencyRichard Cownie2010/06/28 02:16 PM
                                      Compile time & efficiencyRichard Cownie2010/06/28 06:23 PM
                                        Compile time & efficiencyMark Roulo2010/06/29 08:31 AM
                                          Compile time & efficiencyRichard Cownie2010/06/29 11:48 AM
                                          Compile time & efficiencyrwessel2010/06/29 12:28 PM
                                            C is a crappydev2010/06/29 07:12 PM
                                              C is a crappy, but only when you push it out of it's nicheRohit2010/06/30 02:11 AM
                                              C is a crappyanon2010/06/30 02:17 AM
                                                C is a crappydev2010/06/30 07:59 AM
                                                  C is a crappyMax2010/07/01 04:30 AM
                                                    C is a crappyMichael S2010/07/01 07:00 AM
                                                      C is a crappyKonrad Schwarz2010/07/01 08:02 AM
                                                        C is a crappyMichael S2010/07/01 08:50 AM
                                                          C isn't so crappyanon2010/07/01 10:11 AM
                                                            C isn't so crappyMikael Tillenius2010/07/01 11:39 AM
                                                          C is a crappyKonrad Schwarz2010/07/01 11:22 AM
                                                          C is a crappyMax2010/07/02 08:44 AM
                                                            C is a crappyrwessel2010/07/02 12:33 PM
                                                              C is a crappyanon2010/07/02 01:17 PM
                                                              C is a crappyMax2010/07/02 02:56 PM
                                                                C is a crappyMax2010/07/02 03:13 PM
                                                                  C is a crappyrwessel2010/07/02 03:32 PM
                                                                    C is a crappyMax2010/07/02 04:19 PM
                                                                    C is a crappyGabriele Svelto2010/07/05 05:25 AM
                                                      C is a crappygallier22010/07/02 12:14 AM
                                                    C is a crappyIan Ollmann2010/07/06 03:07 PM
                                                      PortabilityMax2010/07/06 03:37 PM
                                                      C is a crappyhobold2010/07/07 02:31 AM
                                                        C is a crappyIan Ollmann2010/07/07 05:18 PM
                                                      failure to standardize typesCarlie Coats2010/07/07 04:11 AM
                                                      C is a crappyKonrad Schwarz2010/07/07 08:34 AM
                                                        C is a crappyIan Ollmann2010/07/07 05:29 PM
                                                          C is a crappy NOTKonrad Schwarz2010/07/08 12:29 AM
                                                  C is a crappyanon2010/07/01 10:40 PM
                                                    C type safety?2010/07/02 01:10 AM
                                                      C type safetyanon2010/07/02 11:02 PM
                                                    C is a crappydev2010/07/03 04:51 PM
                                                      C is a crappyanon2010/07/03 07:02 PM
                                                        C is a crappydev2010/07/05 07:27 AM
                                                          C is a crappy?2010/07/05 09:05 AM
                                                            C is a crappyanonymous2010/07/07 08:32 AM
                                                              C is a crappy?2010/07/07 10:48 PM
                                                              C is a crappyAnon2010/07/08 12:53 AM
                                                      C is a crappy and a crappie is a fishanonymous2010/07/03 07:24 PM
                                      Compile time & efficiencyMichael S2010/06/29 03:18 AM
                                        Compile time & efficiencyrwessel2010/06/29 12:20 PM
                                        Compile time & efficiencysomeone2010/06/30 11:03 AM
                                        Compile time & efficiencyJouni Osmala2010/07/02 05:29 AM
                                  Compile time & efficiencyMax2010/06/28 05:05 PM
                                    Compile time & efficiencyEduardoS2010/06/28 05:11 PM
                                    Compile time & efficiencyMichael S2010/06/29 03:33 AM
                              Compile timeFoo_2010/06/28 09:03 AM
          sun 's cc better than GCC?Silent2010/06/23 06:19 PM
            sun 's cc better than GCC?Foo_2010/06/23 07:06 PM
          sun 's cc better than GCC?Andi Kleen2010/06/24 02:49 AM
        sun 's versus gccVincent Diepeveen2010/06/23 03:07 PM
    Why no GCC?Carlie Coats2010/06/23 05:11 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell avocado?