number of sockets is wrong metric (was: New article: 8 socket commodity servers)

Article: 8-Socket Commodity Servers: Flourish or Perish?
By: Michael S (already5chosen.delete@this.yahoo.com), March 15, 2010 6:05 am
Room: Moderated Discussions
slacker (s@lack.er) on 3/14/10 wrote:
---------------------------
>longtimelurker (rwt@nospam.maibaums.net) on 3/14/10 wrote:
>---------------------------
>>Which brings me to the actual point I'm trying to make: Isn't the number of sockets
>>an antiquated metric? What you are really interested in with those big boxes is
>>the total number and throughput of the memory channels and that they're reasonably low latency from any core.
>
>I agree, in a limited sense, that number of sockets is an antiquated metric. This
>is because, in The Old Way™, multiple sockets was the only way to have multiple
>cores in general purpose machines. Thus, performance scaling across many sockets
>was an important metric. In The New Way™, you get multiple cores in a single socket,
>so why care about multi-socket scaling?
>
>All of the problems which had useful parallelism across 2 - 64 cores will eventually
>be shoe-horned on to a single chip. For these problems, socket count will eventually
>be totally irrelevant (because they'll run on single-socket, many-core processors).
>
>However, you can only get so many pins on a regular-sized package. Since you need
>to scale memory bandwidth with the number of cores, it will become increasingly
>difficult to maintain some sort of balance between bandwidth:FLOPs. It's not very
>useful to integrate a large number of cores on a single chip if you can't get enough
>data in and out of the chip. Thus, multi-socket systems will be the primary method
>of increasing aggregate memory bandwidth in many-core systems.
>

Thinking about it, aggregate memory bandwidth delivered as lot of independent streams is something that money+power+latency could buy, even within physical limitations of a single socket. Imagine a true serial solution, not glorified parallel, like HT3/QPI/FBD, not multilane serial, like IB/PCIe/XAUI, but a true serial like 10GBASE-T just, say 25% slower and reaching 100cm instead of 100m. With a lot of engineering effort it should be possible to build such phy under 0.5W. 256 channels like that should comfortably fit in 1500-pin package. With 256B packets each channel would deliver 900 MB/s with 90% efficiency.
At 64 cores there are 4 channels per core = 3.2 GB/s as measured by Stream. That's close to what we have today in Nehalem-EP, better than Shanghai and significantly better than Nehalem-EX and Istanbul.

So, bandwidth, by itself, is solable. High bandwidth of individual channel and/or low latency are much harder nuts to crack. But, then again, for non-NUMA-friendly access pattern in 16-socket system those characteristics are not particularly great either, although 2-4 times better than the crazy single socket from my above thought experiment.
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
New article: 8 socket commodity serversDavid Kanter2010/03/09 11:27 AM
  New article: 8 socket commodity serversVincent Diepeveen2010/03/09 02:46 PM
    number of sockets is wrong metric (was: New article: 8 socket commodity servers)longtimelurker2010/03/14 06:13 AM
      number of sockets is wrong metric (was: New article: 8 socket commodity servers)EduardoS2010/03/14 06:34 AM
      number of sockets is wrong metric (was: New article: 8 socket commodity servers)Wes Felter2010/03/14 11:33 AM
        Magny-CoursMax2010/03/14 05:56 PM
          Magny-Coursanonymous2010/03/14 07:33 PM
            Magny-Courslongtimelurker2010/03/15 03:54 AM
      number of sockets is wrong metric (was: New article: 8 socket commodity servers)Vincent Diepeveen2010/03/14 12:31 PM
        number of sockets is wrong metric (was: New article: 8 socket commodity servers)longtimelurker2010/03/14 02:37 PM
          number of sockets is wrong metric (was: New article: 8 socket commodity servers)Vincent Diepeveen2010/03/15 12:36 PM
      number of sockets is wrong metric (was: New article: 8 socket commodity servers)David Kanter2010/03/14 12:56 PM
        Bad mathDavid Kanter2010/04/01 02:24 AM
      number of sockets is wrong metric (was: New article: 8 socket commodity servers)slacker2010/03/14 03:51 PM
        number of sockets is wrong metric (was: New article: 8 socket commodity servers)Michael S2010/03/15 06:05 AM
          number of sockets is wrong metric (was: New article: 8 socket commodity servers)slacker2010/03/15 02:02 PM
            Memory interfacesDavid Kanter2010/03/15 02:17 PM
              Memory interfacesslacker2010/03/15 10:08 PM
                Patents on tiny components vs. large, complex thingsmpx2010/03/16 12:41 AM
                  Patents on tiny components vs. large, complex thingsRichard Cownie2010/03/16 06:58 AM
                    Patents on tiny components vs. large, complex thingsMS2010/03/17 06:42 PM
                      Patents on tiny components vs. large, complex thingsa reader2010/03/18 09:45 PM
          Serial Port Memory TechnologyDavid Hess2010/03/21 04:32 AM
  New article: 8 socket commodity serversMichael S2010/03/09 04:13 PM
    New article: 8 socket commodity serverstheluketaylor2010/03/09 06:32 PM
    New article: 8 socket commodity serversJesper Frimann2010/03/09 11:35 PM
    New article: 8 socket commodity serversDavid Kanter2010/03/10 01:38 AM
      New article: 8 socket commodity serversTim2010/03/16 09:44 AM
  New article: 8 socket commodity serversanon2010/03/09 07:59 PM
    New article: 8 socket commodity serversDavid Kanter2010/03/10 12:06 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?