Cache to cache comparison

Article: Parallelism at HotPar 2010
By: David Kanter (dkanter.delete@this.realworldtech.com), August 3, 2010 1:21 pm
Room: Moderated Discussions
Richard Cownie (tich@pobox.com) on 8/3/10 wrote:
---------------------------
>David Kanter (dkanter@realworldtech.com) on 8/3/10 wrote:
>---------------------------
>
>>According to the paper from Hackenberg, using a copy test on 8 cores (two Nehalem
>>sockets), they can 623GB/s, 278GB/s and 76GB/s for the caches. Cutting that in
>>half, yields 311, 139 and 38GB/s respectively for a 4 core Nehalem. Westmere should
>>hit 466GB/s, 208.5GB/s. I don't know if the L3 cache bandwidth increased by 50% though...
>
>Thanks, those figures sound more plausible. But maybe
>there's still an extra factor of 2x if we only want read
>bandwidth, not read bandwidth + write bandwidth ?

That's true for Nehalem, although not any AMD designs, nor for Sandy Bridge.

>>Cache sizes are radically different though. Westmere has >32KB/core L1, 256KB/core L2 and 2MB/core L3.
>
>Not so radically different, a Radeon 5870 has 20 L1 caches
>each with 8KB for a total of 160KB. That's bigger than
>4 x 32KB L1's = 128KB, but smaller than 4 x 256KB L2's.
>Of course the larger number of caches is less effective
>if much of the data has to be replicated.

Westmere is 6 cores, so cumulative 192KB L1D, and 1.5MB L2.

>>So realistically, you're talking about a difference of ~2X for cache bandwidth
>>(just comparing L1 to L1 and L2 to L2).
>> I could believe that in practice, the advantage
>>is more like 3X...given the different optimizations in the cache hierarchy, but
>>I don't see well written code getting much higher than that.
>
>Sounds plausible. So on this analysis, the GPU cache
>hierarchy offered some quite big advantages over the
>Core2 hierarchy (for certain apps); but Nehalem's
>multiple L2 cache's helped it catch up; and then Westmere's
>higher clock speeds and other tweaks will have closed the
>gap again.

I'm not quite sure I'd phrase it that way...in the sense that we're talking about a *very* modern GPU. Cypress is newer than Nehalem and more contemporary to Westmere.

IIRC RV770 was more contemporary to Nehalem and not the Core2.

>With SandyBridge offering AVX and a bunch of architectural
>tweaks, we can probably expect the CPU:GPU ratio to move
>further towards CPUs over the next year. Though there
>may still be some niches where the GPU looks attractive
>for its performance-per-dollar or performance-per-watt -
>Intel charges a fortune for the big cpu's with lots
>of cores and lots of cache ...

For read bandwidth, Sandy Bridge is key because they have two independent AGUs...rather than 1 LD AGU + 1 ST AGU. So now you can really fire off two loads per cycle. Although the cache is limited to 48B/cycle.

In some ways, it may be even more interesting to compare Bulldozer to a GPU...since I get the sense that they made a deliberate decision to optimize for throughput at the cost of per-core performance (whereas Intel has not made that choice for mainstream parts).

>At the least, anyone expecting comparisons from 2007-2008
>to give a guide to results in 2011 is probably going to
>have some surprises.

Yeah, I'd say that's the biggest take away. A multi-core CPU can take advantage of die area just as much as a multi-core device targeted at graphics (GPU). There's a difference of about 2-4X due to architecture and optimization points for the vast majority of GPGPU workloads. For workloads that can use HW that is unique to the GPU, the difference is bigger...but that's an exceptionally rare case.

And of course for non-data parallel workloads, the CPU will probably blow away the GPU by a huge margin.

The one tricky part is that Intel's manufacturing is 12-18 months ahead of TSMC and GF...so in some cases an Intel multi-core will have a fair advantage due to power/area improvements.

David
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Tarek's Hotpar 2010 article onlineDavid Kanter2010/07/27 08:29 AM
  Tarek's Hotpar 2010 article onlineAnon2010/07/27 02:57 PM
    Tarek's Hotpar 2010 article onlineDavid Kanter2010/07/27 11:48 PM
      Tarek's Hotpar 2010 article onlineAnon2010/07/28 03:44 PM
        Tarek's Hotpar 2010 article onlineanon2010/07/28 05:10 PM
        CPU+GPUDavid Kanter2010/07/29 12:38 PM
          CPU+GPUMark Christiansen2010/07/30 07:36 AM
            CPU+GPUhobold2010/07/30 07:54 AM
              CPU+GPUGabriele Svelto2010/07/30 08:18 AM
                CPU+GPUhobold2010/07/30 03:46 PM
              CPU+GPUAaron Spink2010/08/02 06:32 PM
                CPU+GPUhobold2010/08/03 04:33 AM
                  CPU+GPURicardo B2010/08/03 05:19 AM
            CPU+GPU - the software problemVincent Diepeveen2010/07/31 05:50 PM
              CPU+GPU - the software problemmpx2010/08/02 12:19 PM
                CPU+GPU - the software problemRichard Cownie2010/08/02 01:48 PM
                  CPU+GPU - the software problemGabriele Svelto2010/08/03 12:10 AM
                    CPU+GPU - the software problemhobold2010/08/03 04:41 AM
                      CPU+GPU - the software problemRichard Cownie2010/08/03 06:17 AM
                        CPU+GPU - the software problemhobold2010/08/04 07:45 AM
                          NVidia in a holeRichard Cownie2010/08/04 08:23 AM
                            NVidia in a holeRohit2010/08/04 08:33 AM
                              NVidia in a holeRichard Cownie2010/08/04 08:50 AM
                                NVidia in a holeRohit2010/08/05 07:31 AM
                                  NVidia in a holeGroo2010/08/05 02:08 PM
                                NVidia in a holeGroo2010/08/05 02:07 PM
                              NVidia in a holeMark Roulo2010/08/04 09:26 AM
                                NVidia in a holeLaughabee2010/08/18 08:28 AM
                            Can nVidia survive without a sub-$100 GPU market?Mark Roulo2010/08/04 09:29 AM
                              Can nVidia survive without a sub-$100 GPU market?Richard Cownie2010/08/04 09:42 AM
                                Can nVidia survive without a sub-$100 GPU market?Mark Roulo2010/08/04 09:55 AM
                                  Can nVidia survive without a sub-$100 GPU market?Linus Torvalds2010/08/04 11:35 AM
                                    "profit" is subjectiveRichard Cownie2010/08/04 11:49 AM
                                      "profit" is subjectivea reader2010/08/06 04:54 AM
                                        no price warRichard Cownie2010/08/06 07:18 AM
                                  Can nVidia survive without a sub-$100 GPU market?Aaron Spink2010/08/04 12:19 PM
                                  Can nVidia survive without a sub-$100 GPU market?Konrad Schwarz2010/08/06 08:13 AM
                                CPU lineup != GPU lineupRohit2010/08/05 07:43 AM
                                  CPU lineup != GPU lineupRichard Cownie2010/08/05 08:12 AM
                                    CPU lineup != GPU lineupRohit2010/08/05 08:37 AM
                                      CPU lineup != GPU lineupRichard Cownie2010/08/05 08:56 AM
                                        CPU lineup != GPU lineupRohit2010/08/05 11:19 AM
                                          CPU lineup != GPU lineuphobold2010/08/06 04:08 AM
                                            CPU lineup != GPU lineupRohit2010/08/06 07:24 AM
                                              CPU lineup != GPU lineuphobold2010/08/06 07:41 AM
                              Can nVidia survive without a sub-$100 GPU market?David Hess2010/08/04 11:31 AM
                              Can nVidia survive without a sub-$100 GPU market?Ricardo B2010/08/04 01:16 PM
                              Semiconductor economicsDavid Kanter2010/08/05 09:13 AM
                                Semiconductor economicsRichard Cownie2010/08/05 09:51 AM
                                  NV buying ViA could shake up thingsRohit2010/08/05 11:09 AM
                                    NV buying ViA could shake up thingsRichard Cownie2010/08/05 11:25 AM
                                      Not likelyDavid Kanter2010/08/05 02:39 PM
                                        Not likelyrandom2010/08/05 02:49 PM
                                          Not likelyAaron Spink2010/08/05 03:18 PM
                                        Not likelyGroo2010/08/05 10:05 PM
                                        Not likelyRohit2010/08/06 07:25 AM
                                    NV buying ViA could shake up thingsGroo2010/08/05 10:02 PM
                                    NV buying ViA could shake up thingsehud2010/08/06 01:53 AM
                                      NV buying ViA could shake up thingsRohit2010/08/06 07:30 AM
                                        NV buying ViA could shake up thingsKevin G2010/08/06 05:54 PM
                                          ViA buying NV could shake up thingsBrendan2010/08/19 06:23 AM
                                        NV buying ViA could shake up thingsGroo2010/08/06 08:24 PM
                            NVidia in a holempx2010/08/04 12:01 PM
                              NVidia in a holeRichard Cownie2010/08/04 12:31 PM
                                NVidia in a holercf2010/08/04 03:40 PM
                              NVidia in a holeRohit2010/08/05 07:51 AM
                              NVidia in a holeGroo2010/08/05 09:50 PM
                              NVidia in a holeAaron Spink2010/08/05 11:52 PM
                          Anyone else wish Intel would buy Nvidia ? (NT)Silent2010/08/07 01:30 AM
                            Hell noa2010/08/07 08:37 PM
                        CPU+GPU - the software problemno thanks2010/08/04 07:54 PM
                          CPU+GPU - the software problemJouni Osmala2010/08/05 01:41 AM
                            CPU+GPU - the software problemIntelUser20002010/08/05 07:54 PM
                              CPU+GPU - the software problemJouni Osmala2010/08/05 09:36 PM
                            CPU+GPU - the software problemLinus Torvalds2010/08/06 08:08 AM
                              CPU+GPU - the software problemanonymous2010/08/06 09:39 AM
                                CPU+GPU - the software problemLinus Torvalds2010/08/06 10:56 AM
                                  non-existent Via+NV vs real OntarioRichard Cownie2010/08/06 12:12 PM
                              CPU+GPU - the software problemkoby m.2010/08/07 02:06 AM
                  CPU+GPU - the software problemhobold2010/08/03 04:48 AM
                CPU+GPU - the software problemhobold2010/08/03 04:45 AM
                  CPU+GPU - the software problemRichard Cownie2010/08/03 06:22 AM
          CPU+GPUAnon2010/07/30 09:11 PM
            CPU+GPUanon2010/07/31 03:19 AM
            CPU+GPUAaron Spink2010/08/02 06:45 PM
          100x speedups -- here we go againAM2010/08/02 01:37 AM
            100x speedups -- here we go againanon2010/08/02 03:32 AM
              100x speedups -- here we go againanon2010/08/02 11:14 PM
            GPU raw H/W only has about a 10x advantage over a CPUMark Roulo2010/08/02 09:41 AM
              GPU raw H/W only has about a 10x advantage over a CPUMichael S2010/08/02 03:31 PM
                GPU raw H/W only has about a 10x advantage over a CPUanon2010/08/02 09:36 PM
                  GPU raw H/W only has about a 10x advantage over a CPUMichael S2010/08/03 12:41 AM
                    GPU raw H/W only has about a 10x advantage over a CPUMark Roulo2010/08/03 08:22 AM
                      GPU raw H/W only has about a 10x advantage over a CPURohit2010/08/04 07:08 AM
                      There is reasonable explanation...Jouni Osmala2010/08/04 10:10 PM
                  GPU raw H/W only has about a 10x advantage over a CPUMark Roulo2010/08/03 08:19 AM
              simple reality check for youAM2010/08/03 01:45 AM
                simple reality check for younone2010/08/03 07:27 AM
                  simple reality check for youAM2010/08/03 11:50 PM
                    simple reality check for youanon2010/08/04 12:06 AM
                      simple reality check for youAM2010/08/05 03:51 AM
                        catch the same boat as everyone else next time (NT)anon2010/08/05 04:44 AM
                Simple reality checkDavid Kanter2010/08/03 08:24 AM
                  Simple reality checkGabriele Svelto2010/08/03 10:51 PM
                  Simple reality checkAM2010/08/03 11:57 PM
                    Simple reality checkanon2010/08/04 12:12 AM
                      Simple reality checkAM2010/08/05 03:59 AM
                        Simple reality checkanon2010/08/05 04:47 AM
                          Study the papers, troll (NT)AM2010/08/06 12:01 AM
                    Simple reality checkhobold2010/08/04 05:05 AM
                    Simple reality checkDean Kent2010/08/04 09:30 AM
                      Simple reality checkMark Roulo2010/08/04 09:59 AM
                      Suggestion for David Kanter and Mark RouloAM2010/08/05 03:57 AM
                        Suggestion for David Kanter and Mark Roulonone2010/08/05 04:22 AM
                          Suggestion for David Kanter and Mark RouloAM2010/08/06 12:04 AM
                        Suggestion for David Kanter and Mark Roulonone2010/08/05 04:38 AM
                          Suggestion for David Kanter and Mark Rouloanon2010/08/05 04:43 AM
                            Suggestion for David Kanter and Mark RouloAM2010/08/06 12:11 AM
                              Suggestion for David Kanter and Mark Rouloanon2010/08/06 07:52 AM
                          Suggestion for David Kanter and Mark RouloAM2010/08/06 12:10 AM
                            Suggestion for David Kanter and Mark Roulonone2010/08/06 01:35 AM
                              price doesn't matter only in theories (NT)AM2010/08/09 02:13 AM
                            Not apple4appleMichael S2010/08/06 05:13 AM
                              Not apple4appleAM2010/08/09 02:09 AM
                                Not apple4appleMichael S2010/08/09 04:35 AM
                                  Not apple4appleAM2010/08/10 01:05 AM
                                    Not apple4applenone2010/08/10 01:38 AM
                                      Back to topicAM2010/08/11 12:03 AM
                                      Final note (and some remarks on Intel's paper)AM2010/08/13 04:28 AM
                                        Final note (and some remarks on Intel's paper)none2010/08/13 09:10 AM
                                          Final note (and some remarks on Intel's paper)AM2010/08/16 01:14 AM
                                            Final note (and some remarks on Intel's paper)none2010/08/16 06:29 AM
                                              One example of 1000X shown to be wrong.sea2010/08/16 07:55 PM
                                                You're short of some factsAM2010/08/17 01:13 AM
                                                  Maybe you need to read beyond the PR statementSteve Underwood2010/08/17 02:39 AM
                                                    Maybe you need to read beyond the PR statementGroo2010/08/17 08:18 PM
                                                    check the hw they usedAM2010/08/18 03:33 AM
                                                Please realize what Monte Carlo meansVincent Diepeveen2010/08/18 03:28 AM
                                              Final note (and some remarks on Intel's paper)AM2010/08/17 01:18 AM
                                        Final note (and some remarks on Intel's paper)anon2010/08/14 01:22 AM
                                          Final note (and some remarks on Intel's paper)AM2010/08/16 01:15 AM
                                            Final note (and some remarks on Intel's paper)anon2010/08/16 02:10 PM
                                              Final note (and some remarks on Intel's paper)AM2010/08/17 01:15 AM
                                                Final note (and some remarks on Intel's paper)Michael S2010/08/17 02:58 AM
                                                  Final note (and some remarks on Intel's paper)AM2010/08/18 03:17 AM
                                                    Final note (and some remarks on Intel's paper)gallier22010/08/18 04:52 AM
                                                    Final note (and some remarks on Intel's paper)Michael S2010/08/18 05:33 AM
                                                      Final note (and some remarks on Intel's paper)Gabriele Svelto2010/08/18 06:11 AM
                                                      Final note (and some remarks on Intel's paper)Steve Underwood2010/08/18 04:03 PM
                                                        Intel might be moving from ISA to platformhobold2010/08/19 03:58 AM
                                                          Intel might be moving from ISA to platformSteve Underwood2010/08/22 07:00 PM
                                                            Intel might be moving from ISA to platformAnon2010/08/22 10:43 PM
                                                              Intel might be moving from ISA to platformajensen2010/08/23 12:37 AM
                                                                Intel might be moving from ISA to platformMichael S2010/08/23 02:13 AM
                                                                  Intel might be moving from ISA to platformSteve Underwood2010/08/23 02:35 AM
                                                                  Intel might be moving from ISA to platformhobold2010/08/26 04:37 AM
                                                                Intel might be moving from ISA to platformAnon2010/08/23 03:47 PM
                                                              Intel might be moving from ISA to platformSteve Underwood2010/08/23 02:25 AM
                                                                Intel might be moving from ISA to platformhobold2010/08/23 03:03 AM
                                                                  Intel might be moving from ISA to platformSteve Underwood2010/08/23 04:26 AM
                                                                Intel might be moving from ISA to platformAnon2010/08/23 03:55 PM
                                                                Intel might be moving from ISA to platformrwessel2010/08/23 06:41 PM
                                                                  Intel might be moving from ISA to platformSteve Underwood2010/08/23 07:30 PM
                                                                    Intel might be moving from ISA to platformrwessel2010/08/23 09:50 PM
                                                                      Intel might be moving from ISA to platformSteve Underwood2010/08/23 10:34 PM
                                                                        Intel might be moving from ISA to platformrwessel2010/08/24 01:03 AM
                                                              Intel might be moving from ISA to platformIan Ollmann2010/08/23 09:21 PM
                                                          Intel might be moving from ISA to platformajensen2010/08/22 10:36 PM
                                                            Intel might be moving from ISA to platformIan Ollmann2010/08/23 09:45 PM
                                                      Final note (and some remarks on Intel's paper)AM2010/08/19 01:29 AM
                                                        Final note (and some remarks on Intel's paper)Richard2010/08/19 04:51 AM
                                                          I stand corrected.Michael S2010/08/19 05:30 AM
                simple reality check for youanonymous2010/08/03 10:25 AM
                  simple reality check for youAM2010/08/03 11:47 PM
                    simple reality check for youanonymous2010/08/04 10:17 AM
                      simple reality check for youanon2010/08/04 08:33 PM
                      SPECint/FPAM2010/08/05 03:49 AM
                  simple reality check for youRohit2010/08/13 08:31 PM
                    simple reality check for youhobold2010/08/16 06:29 AM
              maybe texture caches ?Richard Cownie2010/08/03 09:27 AM
                maybe texture caches ?none2010/08/03 09:59 AM
                  maybe texture caches ?Richard Cownie2010/08/03 10:15 AM
                    maybe texture caches ?Mark Roulo2010/08/03 10:23 AM
                      maybe texture caches ?Richard Cownie2010/08/03 10:33 AM
                        maybe texture caches ?Mark Roulo2010/08/03 10:37 AM
                          the Hess paperRichard Cownie2010/08/03 11:16 AM
                    maybe texture caches ?Aaron Spink2010/08/03 01:23 PM
                      maybe texture caches ?Mark Roulo2010/08/03 04:13 PM
                        maybe texture caches ?Michael S2010/08/04 01:47 AM
                  maybe texture caches ?Michael S2010/08/03 10:23 AM
                maybe texture caches ?Mark Roulo2010/08/03 10:19 AM
                  maybe texture caches ?Richard Cownie2010/08/03 10:42 AM
                Cache to cache comparisonDavid Kanter2010/08/03 12:28 PM
                  Cache to cache comparisonRichard Cownie2010/08/03 12:52 PM
                    Cache to cache comparisonDavid Kanter2010/08/03 01:21 PM
                      Cache to cache comparisonRichard Cownie2010/08/03 03:55 PM
                      Cache to cache comparisonGabriele Svelto2010/08/03 11:23 PM
                        Nehalem vs Core2Richard Cownie2010/08/04 04:24 PM
                          Nehalem vs Core2IntelUser20002010/08/04 07:26 PM
                          Nehalem vs Core2Gabriele Svelto2010/08/04 10:51 PM
                            Nehalem vs Core2Richard Cownie2010/08/05 03:27 AM
                              Nehalem vs Core2Gabriele Svelto2010/08/05 06:00 AM
                                Nehalem vs Core2Richard Cownie2010/08/05 07:10 AM
                                  Nehalem vs Core2Gabriele Svelto2010/08/05 07:41 AM
                                    Nehalem vs Core2Michael S2010/08/05 10:02 AM
                              Nehalem vs Core2Michael S2010/08/05 10:12 AM
                          Nehalem vs Core2Michael S2010/08/05 12:36 AM
                            Nehalem vs Core2Richard Cownie2010/08/05 08:33 AM
                              Nehalem vs Core2Michael S2010/08/05 09:44 AM
                                Nehalem vs Core2Richard Cownie2010/08/05 10:03 AM
                                  Nehalem vs Core2Michael S2010/08/05 04:21 PM
                                    Nehalem vs Core2Richard Cownie2010/08/05 07:15 PM
                                      I'm wrong, you're right - i's 2 dieRichard Cownie2010/08/05 07:21 PM
                                    Nehalem vs Core2Richard Cownie2010/08/06 08:12 AM
                                      Nehalem vs Core2Michael S2010/08/06 08:33 AM
                                      Nehalem L1 cache latency a hedge for higher clock speeds?Mark Roulo2010/08/06 08:36 AM
                                        Nehalem L1 cache latency a hedge for higher clock speeds?Kevin G2010/08/07 10:14 AM
                                          Nehalem L1 cache latency a hedge for higher clock speeds?IntelUser20002010/08/07 02:32 PM
                                        Nehalem L1 cache latency a hedge for higher clock speeds?someone2010/08/07 02:35 PM
                          difference between C2D/C2Q chipsetsMichael S2010/08/05 12:57 AM
                          Nehalem core goals ... my takeMark Roulo2010/08/06 07:53 AM
                            Bulldozer single-threadRichard Cownie2010/08/06 04:05 PM
                          Nehalem vs Core2Carlie Coats2010/08/07 08:41 AM
                            scalingMichael S2010/08/07 11:12 AM
                            why X5460?Michael S2010/08/07 11:12 AM
                              why X5460?Carlie Coats2010/08/08 04:34 AM
                                why X5460?Michael S2010/08/08 04:42 AM
                                  Polling/PIO based COMM libraries?Michael S2010/08/08 05:14 AM
                    Cache to cache comparisonAnts Aasma2010/08/04 10:00 AM
                      Cache to cache comparisonRichard Cownie2010/08/04 10:08 AM
                        Cache to cache comparisonAnts Aasma2010/08/04 11:28 AM
                      Cache to cache comparisonMichael S2010/08/04 11:33 AM
                        Cache to cache comparisonMark Roulo2010/08/04 12:29 PM
                          Cache to cache comparisonAnts Aasma2010/08/04 01:10 PM
                            Fermi and G200 instruction latencyMark Roulo2010/08/04 02:10 PM
                              Fermi and G200 instruction latencyAnts Aasma2010/08/04 03:01 PM
                        Cache to cache comparisonAnts Aasma2010/08/04 01:00 PM
                        Register read comparisonDavid Kanter2010/08/05 09:16 AM
                          Register read comparisonGabriele Svelto2010/08/05 11:42 PM
                maybe texture caches ?Aaron Spink2010/08/03 01:17 PM
              GPU raw H/W only has about a 10x advantage over a CPURichard2010/08/09 04:58 AM
                GPU raw H/W only has about a 10x advantage over a CPUnone2010/08/09 05:19 AM
                  GPU raw H/W only has about a 10x advantage over a CPUVincent Diepeveen2010/08/24 07:18 AM
                    Forget it, they won't hear itanon2010/08/24 04:37 PM
                    GPU raw H/W only has about a 10x advantage over a CPURichard2010/08/25 01:03 PM
                This forum supports pre tagMichael S2010/08/09 06:03 AM
    Tarek's Hotpar 2010 article onlineRich Vuduc2010/07/28 10:24 AM
      Tarek's Hotpar 2010 article onlineAnon2010/07/28 04:18 PM
  Low hanging fruitMoritz2011/02/03 12:20 PM
    Low hanging fruitJukka Larja2011/02/06 05:01 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?