Clang+LLVM

Article: Bulldozer Benchmarks, What Do They Mean?
By: Linus Torvalds (torvalds.delete@this.linux-foundation.org), April 17, 2011 2:00 pm
Room: Moderated Discussions
EduardoS (no@spam.com) on 4/17/11 wrote:
>
>How long this parser take? For the tokenize to be the slower part I presume the entire parsing process is fast...

Quite frankly, I was surprised at how things turned out.

I seriously expected tokenization to not even be an issue.
Especially since the project really does do some analysis
on the result too, although in the name of efficiency it
does that very lazily.

So for example, it obviously needs to lex and parse
everything, but after it has parsed types and inline
functions etc, it won't actually analyze them until
they are used. The argument was (and I think it is true)
that a lot of the "bulk" of C code tends to be about
headers, and that you don't want to analyze the parts
of the headers that aren't actively used.

So I ended up trying to be smart about avoiding work at
later phases, but that very fact probably ends up being
why sparse is so much more front-end heavy than most
similar projects.

IOW, sparse will create the parse tree for a structure
definition, but it won't actually bother to do the exact
layout or size of the structure unless that type is
actually needed by one of the functions that ends up being
built.

It can do a full "check" of my kernel build in about 40s,
which sounds pretty fast, but it's a fairly minimal config
and I can compile it from scratch in under 4 minutes. So
doing the lexing/parsing/analysis isn't even an order of
magnitude faster than really compiling it, it's a much
smaller factor.

In fact, if you use ccache (and I do), the compile phase
can be done in basically almost the same time as the
checking if you end up re-building the same tree a lot.

That "order of magnitude faster" was what I was aiming
for - and I was also thinking that I might be able to use
the project as a front-end to the compiler. One of my
idle thoughts was to add some syntactic preprocessing to
do more dynamic code generation.

The kernel currently does a lot of really ugly things with
the preprocessor and macros (using "typeof" and "sizeof" to
do some very crude forms of "generic programming") and so
one thing I was interested in was to see just if I could
do some better syntax-aware source code augmentation.

It was not to be. We ended up using sparse a lot for some
nice user-pointer safety analysis, and it was a great
help for some other details, but it's a pretty quiescent
project these days. And part of it (for me) was that I
decided that it would slow down the build too much if I
were ever to try to use it as a separate smarter pre-
processor.

And btw, I don't claim that sparse is all that great a
parser. I use it mostly as a "hey, I've done toy parsers
myself". We all have. I suspect it's why people talk so
much about parser generators etc - it's what the BSD
people call "bikeshedding": everybody can talk about it,
because everybody understands parsing. It's simple enough
that a lot of people have written parsers for at least
some half-way real situation.

It's also why I suspect not a lot of people talk about
code generator generators. Many CS people have written a
toy compiler or similar, but most people who did always
knew that their back-end was crap. Very few people do a
real optimizing compiler, and even fewer end up doing one
that can handle multiple different target architectures.

People can talk about "ambiguous grammars" all they like,
but it's still a walk in the park compared to the real-
life ambiguities of trying to pick good register allocation
and the instruction mix to get the best end results.

That's where you _really_ don't know what the correct
answer is, and it's not about how many tokens you have to
look ahead or not.

Linus
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
New article: Bulldozer benchmark analysisDavid Kanter2011/03/30 02:27 AM
  New article: Bulldozer benchmark analysisNaN2011/03/30 07:47 AM
    New article: Bulldozer benchmark analysisDavid Kanter2011/03/30 09:25 AM
      New article: Bulldozer benchmark analysisdc2011/03/30 12:07 PM
      New article: Bulldozer benchmark analysisFoo_2011/03/30 12:17 PM
        New article: Bulldozer benchmark analysisDavid Kanter2011/03/30 01:34 PM
          BD won't "kick in" until 2.5GHzBaronMAtrix2011/03/31 08:37 AM
            BD won't "kick in" until 2.5GHzDavid Kanter2011/03/31 09:21 AM
              He is an AMDZone lunaticNIKOLAS2011/04/01 06:40 AM
                See how he behaves hereDavid Kanter2011/04/01 08:08 AM
                He is an AMDZone lunatichorsefly2011/04/02 05:46 AM
                  This is not relevantDavid Kanter2011/04/02 09:16 AM
                  Post removedDavid Kanter2011/04/02 12:39 PM
          New article: Bulldozer benchmark analysisanonymous2011/04/04 05:08 AM
      New article: Bulldozer benchmark analysisinterested2011/04/01 03:28 PM
        New article: Bulldozer benchmark analysisDavid Kanter2011/04/01 03:53 PM
          New article: Bulldozer benchmark analysisNaN2011/04/02 07:12 AM
            New article: Bulldozer benchmark analysisDavid Kanter2011/04/02 09:05 AM
              New article: Bulldozer benchmark analysisNaN2011/04/02 09:24 AM
                New article: Bulldozer benchmark analysisDavid Kanter2011/04/02 12:36 PM
                  AMD's Open64 compiler ?Alex2011/04/04 02:23 AM
                    AMD's Open64 compiler ?David Kanter2011/04/04 08:19 AM
                      AMD's Open64 compiler ?Richard Cownie2011/04/04 09:52 AM
                        AMD's Open64 compiler ?David Kanter2011/04/04 01:20 PM
                          AMD's Open64 compiler ?none2011/04/04 02:25 PM
                        AMD's Open64 compiler ?gallier22011/04/04 11:06 PM
                          thanks, very interesting (NT)Richard Cownie2011/04/05 03:44 AM
                          AMD's Open64 compiler ?Andreas Stiller2011/04/05 08:23 AM
                        AMD's Open64 compiler ?Vincent Diepeveen2011/04/06 01:11 AM
                      AMD's Open64 compiler ?Vincent Diepeveen2011/04/06 01:01 AM
                        AMD's Open64 compiler ?none2011/04/06 01:41 AM
                          AMD's Open64 compiler ?Vincent Diepeveen2011/04/07 02:02 PM
                            AMD's Open64 compiler ?Brett2011/04/07 04:55 PM
                              AMD's Open64 compiler ?Megol2011/04/09 11:03 AM
                                Clang+LLVMBrett2011/04/11 10:51 PM
                                  Clang+LLVMRob Thorpe2011/04/12 12:52 AM
                                    Clang+LLVMBrett2011/04/12 08:27 PM
                                      I prefer my dollars exponential, not cubic (NT)David Kanter2011/04/12 11:34 PM
                                        I prefer cubic meters of euros over cubic feet of dollarsJouni Osmala2011/04/13 01:13 AM
                                          Life is not about money, but about power (and informational asymetry)?2011/04/18 01:14 AM
                                      Clang+LLVMRob Thorpe2011/04/13 10:37 AM
                                        Clang+LLVMLinus Torvalds2011/04/13 11:47 AM
                                          Clang+LLVMRob Thorpe2011/04/13 01:46 PM
                                            Clang+LLVMLinus Torvalds2011/04/13 04:40 PM
                                              Clang+LLVManon2011/04/13 07:31 PM
                                                Clang+LLVMLinus Torvalds2011/04/14 06:11 AM
                                              Clang+LLVMRob Thorpe2011/04/14 12:33 PM
                                                Clang+LLVMLinus Torvalds2011/04/14 12:48 PM
                                                  Clang+LLVMAndi Kleen2011/04/14 06:38 PM
                                                    Clang+LLVMJouni Osmala2011/04/14 09:54 PM
                                                  Clang+LLVM?2011/04/14 11:56 PM
                                                    Clang+LLVMLinus Torvalds2011/04/15 06:23 AM
                                                      Clang+LLVM?2011/04/15 10:11 PM
                                                        Clang+LLVMLinus Torvalds2011/04/16 08:10 AM
                                                          Clang+LLVMLinus Torvalds2011/04/16 08:32 AM
                                                            Competing for speed?2011/04/16 11:47 PM
                                                            Clang+LLVMEduardoS2011/04/17 12:38 PM
                                                              Clang+LLVMLinus Torvalds2011/04/17 02:00 PM
                                                                Competing for speed?2011/04/17 10:57 PM
                                                                  Competing for speedLinus Torvalds2011/04/18 09:07 AM
                                                                    Competing for speedanon2011/04/18 05:38 PM
                                                                      Competing for speedLinus Torvalds2011/04/18 07:24 PM
                                                                        Competing for speedanon2011/04/19 05:05 AM
                                                                          Competing for speedanon2011/04/21 01:35 PM
                                                                            Competing for speedLinus Torvalds2011/04/22 11:09 AM
                                                                    Competing for speed?2011/04/18 11:45 PM
                                                                      Competing for speedLinus Torvalds2011/04/19 06:41 AM
                                                                        Competing for speedLinus Torvalds2011/04/19 07:46 AM
                                                                        Oh ... the irony?2011/04/19 10:36 AM
                                                                          Oh ... the ironyLinus Torvalds2011/04/19 01:49 PM
                                                                            What's the speedup?iz2011/04/19 10:54 PM
                                                                              What's the speedup?Linus Torvalds2011/04/20 06:48 AM
                                                                            Oh ... the irony?2011/04/23 05:52 AM
                                                                          Oh ... the ironyDougal2011/04/20 06:23 AM
                                                                            Oh ... the irony?2011/04/25 12:05 PM
                                                                              Oh ... the ironyDougal2011/04/26 05:22 AM
                                                                                Oh ... the irony?2011/04/27 12:14 AM
                                                  Clang+LLVMRob Thorpe2011/04/16 01:23 PM
                                                    Clang+LLVManon2011/04/16 03:01 PM
                                                      Clang+LLVMRob Thorpe2011/04/18 04:53 AM
                                                        Clang+LLVManon2011/04/18 05:28 PM
                                        Clang+LLVMPaul2011/04/14 10:43 AM
                                  Clang+LLVMFoo_2011/04/12 05:36 AM
                              AMD's Open64 compiler ?Paul2011/04/10 02:50 AM
                                C++ front-endsDavid Kanter2011/04/10 06:29 AM
                                  C++ front-endsPaul2011/04/10 07:19 AM
                                    C++ front-endsanonymous2011/04/10 07:28 AM
                                      C++ front-endsnone2011/04/10 07:36 AM
                                      C++ front-endsPaul2011/04/10 07:44 AM
                                    C++ front-endsMichael S2011/04/10 07:47 AM
                                      C++ front-endsnone2011/04/10 07:51 AM
                                      C++ front-endsRob Thorpe2011/04/10 11:13 AM
                                        C++ front-endsPaul2011/04/11 10:07 AM
                                AMD's Open64 compiler ?Konrad Schwarz2011/04/14 01:06 AM
                              Testing Clang + LLVMVincent Diepeveen2011/04/20 06:31 AM
                                Does your program crash when compiled with -O0? (NT)Michael S2011/04/20 10:11 AM
                                  Does your program crash when compiled with -O0? (NT)Vincent Diepeveen2011/04/20 11:33 AM
                                Testing Clang + LLVMMegol2011/04/20 11:05 AM
                                  Testing Clang + LLVMVincent Diepeveen2011/04/20 11:43 AM
                                    Testing Clang + LLVMstubar2011/04/20 12:23 PM
                                      Testing Clang + LLVMVincent Diepeveen2011/04/20 12:42 PM
                                        Testing Clang + LLVMRob Thorpe2011/04/20 02:30 PM
                                          Testing Clang + LLVMahnoun2011/04/20 03:13 PM
                                            Testing Clang + LLVMMark Roulo2011/04/20 04:05 PM
                                              Testing Clang + LLVMMegol2011/04/22 04:30 AM
                                    Testing Clang + LLVMahnoun2011/04/20 02:31 PM
                                      Testing Clang + LLVMVincent Diepeveen2011/04/21 02:17 AM
                                        Testing Clang + LLVMnone2011/04/21 02:36 AM
                                        Testing Clang + LLVMahnoun2011/04/21 11:02 AM
                                          Testing Clang + LLVMMichael S2011/04/21 12:01 PM
                                            Testing Clang + LLVMMichael S2011/04/21 03:29 PM
                                              Testing Clang + LLVMBrett2011/04/21 04:49 PM
                                              Testing Clang + LLVMEduardoS2011/04/22 02:22 PM
                                                Testing Clang + LLVMMichael S2011/04/23 09:59 AM
                                            Testing Clang + LLVMDavid Ball2011/04/21 06:43 PM
                                Testing Clang + LLVMBrett2011/04/20 12:54 PM
                            AMD's Open64 compiler ?Megol2011/04/09 10:49 AM
                              Imperative compilers (and the lack of it)?2011/04/10 12:36 AM
                                Imperative compilers (and the lack of it)?2011/04/10 12:43 AM
                                Imperative compilers (and the lack of it)Paul2011/04/10 02:37 AM
                                  Imperative compilers (and the lack of it)?2011/04/10 05:30 AM
                                  Imperative compilers (and the lack of it)Vincent Diepeveen2011/04/10 06:20 AM
                                  Imperative compilers (and the lack of it)Gabriele Svelto2011/04/11 08:24 AM
                                  Imperative compilers (and the lack of it)Michael S2011/04/12 12:12 AM
                                    Imperative compilers (and the lack of it)?2011/04/12 01:41 AM
                                      Imperative compilers (and the lack of it)EduardoS2011/04/12 03:13 PM
                                        U+269BMichael S2011/04/13 12:45 AM
                                          U+269B?2011/04/13 03:52 AM
                                            U+269BBrendan2011/04/20 07:18 AM
                                              U+269BMichael S2011/04/20 09:11 AM
                                                U+269BHeath Provost2011/04/20 10:26 AM
                                                  You are right, Heath. Sorry, Brannon.Michael S2011/04/20 10:42 AM
                                                    Sorry again, Brendan (NT)Michael S2011/04/20 10:43 AM
                                                    Testing western non-us letters.Jouni Osmala2011/04/20 09:41 PM
                                                      Those show up fine to me (NT)David Kanter2011/04/21 07:48 AM
                                                      Testing western non-us letters.Heath Provost2011/04/21 11:15 AM
                                                        Testing western non-us letters.EduardoS2011/04/22 02:14 PM
                                                          Testing western non-us letters.Heath Provost2011/04/22 03:21 PM
                                                            Testing western non-us letters.EduardoS2011/04/22 04:24 PM
                                                            We actually upgraded to 2008 R2David Kanter2011/04/22 10:23 PM
                                                              We actually upgraded to 2008 R2EduardoS2011/04/22 10:33 PM
                                                                We actually upgraded to 2008 R2rwessel2011/04/22 11:11 PM
                                                                  We actually upgraded to 2008 R2David Kanter2011/04/22 11:28 PM
                                                                    We actually upgraded to 2008 R2Ricardo B2011/04/23 09:43 AM
                                                                      We actually upgraded to 2008 R2David Kanter2011/04/23 10:31 AM
                                                                    We actually upgraded to 2008 R2sparkplug2011/04/23 08:25 PM
                                                                      We actually upgraded to 2008 R2David Kanter2011/04/24 03:36 PM
                                                                        We actually upgraded to 2008 R2anon2011/04/24 06:32 PM
                                                                          We actually upgraded to 2008 R2David Kanter2011/04/24 11:12 PM
                                                                        We actually upgraded to 2008 R2EduardoS2011/04/24 08:10 PM
                                                                          We actually upgraded to 2008 R2David Kanter2011/04/24 11:04 PM
                                                                        We actually upgraded to 2008 R2anonymous2011/04/24 08:44 PM
                                                                          We actually upgraded to 2008 R2David Kanter2011/04/24 10:57 PM
                                                                        File size optimizationsjrl2011/04/25 02:03 AM
                                                                          File size optimizationsRicardo B2011/04/25 07:14 AM
                                                                            File size optimizationsjrl2011/04/25 05:52 PM
                                                                          File size optimizationsDavid Kanter2011/04/25 07:49 AM
                                                                            File size optimizationsRicardo B2011/04/25 08:22 AM
                                                                              ContinuationRicardo B2011/04/25 08:25 AM
                                                                              File size optimizationsChristian Packmann2011/04/25 09:56 AM
                                                                                File size optimizationsEduardoS2011/04/25 11:57 AM
                                                                                  File size optimizationsRicardo B2011/04/25 01:01 PM
                                                                            File size optimizationsEduardoS2011/04/25 12:01 PM
                                                                            Javascript is at the root of the latency problemjrl2011/04/25 06:03 PM
                                                                          File size optimizationsEduardoS2011/04/25 12:14 PM
                                                                            File size optimizationsrwessel2011/04/25 12:49 PM
                                                                              File size optimizationsEduardoS2011/04/25 12:57 PM
                                                                            File size optimizationsjrl2011/04/25 05:54 PM
                                                                              A simple approachSilent2011/04/25 09:24 PM
                                                                                Show Thread exists but the default is Yes (NT)Silent2011/04/25 09:26 PM
                                                                          File size optimizationsHeath Provost2011/04/25 02:02 PM
                                    Imperative compilers (and the lack of it)Azazel2011/04/12 08:23 PM
                                Imperative compilers (and the lack of it)none2011/04/10 06:00 AM
                                  Imperative compilers (and the lack of it)dmsc2011/04/10 06:39 AM
                                  Imperative compilers (and the lack of it)EduardoS2011/04/10 12:55 PM
                                compiler failure rateshobold2011/04/10 10:55 AM
                                  compiler failure rates?2011/04/10 11:13 PM
                                    compiler failure rateshobold2011/04/11 05:07 AM
                                Imperative compilers (and the lack of it)EduardoS2011/04/10 12:55 PM
                                  Imperative compilers (and the lack of it)?2011/04/10 11:13 PM
                                    Imperative compilers (and the lack of it)Konrad Schwarz2011/04/14 01:28 AM
                                      Imperative compilers (and the lack of it)gallier22011/04/14 04:47 AM
                                      Imperative compilers (and the lack of it)EduardoS2011/04/14 02:04 PM
                                        Imperative compilers (and the lack of it)Konrad Schwarz2011/04/15 03:43 AM
                                Hard register problemsRob Thorpe2011/04/11 12:14 AM
                                  Hard register problemsMax2011/04/11 03:05 PM
                                    Hard register problemsRob Thorpe2011/04/11 03:28 PM
                                Imperative compilers (and the lack of it)gallier22011/04/11 09:26 AM
                                  I wonder why.j2011/04/11 10:39 PM
                                    I wonder why.?2011/04/11 11:44 PM
                                Imperative compilers (and the lack of it)nksingh2011/04/12 11:00 PM
                                  Imperative compilers (and the lack of it)?2011/04/12 11:23 PM
                              AMD's Open64 compiler ?Vincent Diepeveen2011/04/10 06:06 AM
                                AMD's Open64 compiler ?none2011/04/10 07:23 AM
                                  AMD's Open64 compiler ?Eugene Nalimov2011/04/11 03:57 PM
                                    AMD's Open64 compiler ?EduardoS2011/04/12 02:55 PM
                                AMD's Open64 compiler ?Megol2011/04/10 11:05 PM
                                  AMD's Open64 compiler ?Vincent Diepeveen2011/04/20 12:52 PM
                                    Yap transform? What is it? (NT)Michael S2011/04/21 09:04 AM
                                      Yap transform? What is it? (NT)Vincent Diepeveen2011/04/22 02:56 AM
                    AMD's Open64 compiler ?Vincent Diepeveen2011/04/05 12:02 PM
    New article: Bulldozer benchmark analysisNaN2011/03/30 05:24 PM
      New article: Bulldozer benchmark analysisChris2011/04/05 03:33 AM
  New article: Bulldozer benchmark analysisdc2011/03/30 08:09 AM
    New article: Bulldozer benchmark analysisMS2011/03/30 08:53 AM
      New article: Bulldozer benchmark analysisDaniel Bizo2011/03/31 12:20 AM
        New article: Bulldozer benchmark analysisanun2011/03/31 02:48 AM
    New article: Bulldozer benchmark analysisAlex2011/03/30 09:18 AM
      New article: Bulldozer benchmark analysisIntelUser20002011/03/30 01:19 PM
    New article: Bulldozer benchmark analysisGroo2011/03/31 06:14 AM
      New article: Bulldozer benchmark analysisanon2011/03/31 08:29 AM
        AMD's memory controllers are fineDavid Kanter2011/03/31 09:27 AM
          AMD's memory controllers are fineanon2011/04/05 11:24 AM
            AMD's memory controllers are fineanon2011/04/05 04:52 PM
              AMD's memory controllers are fineVincent Diepeveen2011/04/06 01:36 AM
                AMD's memory controllers are fineanon2011/04/06 06:59 AM
                  AMD's memory controllers are fineVincent Diepeveen2011/04/07 02:15 PM
                    AMD's memory controllers are fineVincent Diepeveen2011/04/07 02:27 PM
                      AMD's memory controllers are fineanon2011/04/07 02:57 PM
                    AMD's memory controllers are fineanon2011/04/07 02:54 PM
        New article: Bulldozer benchmark analysisAzazel2011/03/31 09:54 AM
        New article: Bulldozer benchmark analysisAlex2011/03/31 11:38 PM
      New article: Bulldozer benchmark analysisanon2011/04/01 03:23 AM
    Streaming conflicts can lead to performance degradationsgc2011/04/08 06:35 AM
  AMD's probe filterAndreas Stiller2011/04/05 08:37 AM
  a waste of timeMoritz2011/04/09 07:01 AM
    a waste of timehorsefly2011/04/09 03:34 PM
      a waste of timeDavid Kanter2011/04/09 06:56 PM
        a waste of timehorsefly2011/04/10 05:13 AM
          InvestorsDavid Kanter2011/04/10 06:27 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell tangerine? ūüćä