Latency and HPC Workloads

Article: Intel's Near-Threshold Voltage Computing and Applications
By: forestlaughing (forestlaughing.delete@this.yahoo.com), October 19, 2012 11:43 am
Room: Moderated Discussions
anon (anon.delete@this.anon.com) on October 16, 2012 6:48 pm wrote:
> > Just as soon as the players I have mentioned, plus
> anyone else who plays the same game, stops advertising "the n-th fastest
> computer in the world" based on a single benchmark (Linpack), I'll back off on
> > the snarling insults about the practice.

top 500 one-upsmanship is for the newspapers only, and everyone participating in it knows that it is a silly game. It's no different from the sort of bluster you hear from lots of other industries, or segments of the IT industry. Roll your eyes, move along, nothing to see here.

> I some knowledge of
> supercomputer procurement. Not in the top 10, but in the top 100. The clients
> were very specific about their workloads, and gave a dozen or so which were run
> by their users for their acceptance suite. Their top500 submission was fun
> because it gave an "Nth fastest supercomputer" tag, but it was at the bottom of
> the priority list (and I don't think it was required for acceptance).
>
> Do
> other HPC sites really just start out by wanting to reach #1 (or some top500
> goal), and not have any real ideas about how the machine will be used? I highly
> doubt it.

They don't. They learned that linpack != real performance decades ago. Depending on the site it may not be the bottom of the priority list, but at no site is it the top of the list.


> length. I have already argued at length about why gigantic computer
> centers with lousy interconnect are in the interest neither of science nor of
> > the national purse. They do serve the interest of some of the players I

For your application. Which is not representative of all applications, or all science. If these machines are so useless for science, why do so many scientist want to get time on them to run their applications?

> have
> > already mentioned. Someone has responded, and probably the person
> who
> > identified himself here as forestlaughing, that these gigantic
> machines are
> > actually throughput machines that are rarely employed in
> their actual
> > giganticness, except to deal with many users under a single

That's sort of what I have said. The very large machines at DOE, NSF, DOD, etc run a mix of applications. Some fill the entire machine, some fill substantial fractions of the machine, and some use only a handful of nodes. This has little to do with bureaucracy, and everything to do with the fact that you don't purchase a supercomputer for a single user. You buy one (or a couple) and users share the machine. You can use a capability machine for capacity workloads, but not the other way around.



When IBM
> > bailed on Blue Waters and I seemed to be the only
> one saying that the Emperor
> > was plainly walking naked, I gave up.

I happen to know Robert was not the only one who was upset that Percs/Blue Waters didn't happen. However, a lot of great ideas don't happen, when reality interferes.

> Blue
> Waters was not revolutionary. It was a "commodity" non-vector POWER7 CPU with a
> custom interconnect. What was good about Blue Waters that is no good with BG/Q
> or K supercomputer?

IBM's Blue waters (PERCS) was not revolutionary. It did have a higher ratio of interconnect bandwidth to compute capability. (Bytes/flop) But it was only a modestly better ratio. One of the reasons it was better, is the topology of the network.
Percs was supposed to use a direct connect topology. (Cray has a similar architecture in their cascade machine, which they call a dragonfly topology.) In this topology, the bisectional bandwidth scales at O(n) where n is the number of nodes in the cluster. By contrast, the torus topology employed by Blue Gene, K, tianhe-1, and Cray-XE scales at O(the 2/3 root of n). DC/Dragonfly is more expensive than a torus, but provides better bandwidth, particularly as the machine gets very large. Fat trees also scale O(n), but are even more expensive because you need twice as many links as you do in a DC topology, as well as external router cabinets. Fat trees are rarely used in very large clusters, due to cost.
The second reason the percs network was supposed to be so fast, is that there were just a lot of cables involved. Lots of optical fibre, lots of copper traces in the backplane. Percs was an expensive machine, not just because it was custom. Reliable sources in Illinois suggest that IBM screwed something up in their router design, and would need to respin the router in order to get the performance they had promised, and that the router may have gotten too large to fit on a single chip. Fixing that would add a lot of cost, and put the machine way behind schedule. NCSA decided they didn't want to pay more money for a machine that was become less and less competitive, as its delivery date slipped. That is all rumor though.
Percs would undoubtedly be better, for Robert's FFT algorithms, than blue gene, or the XE6 that Blue waters turned into. However, not hugely so. Even on Percs, the interconnect bandwidth was still a tiny fraction of the backplane bandwidth of the large SMPs of the world. I still don't see those algorithms scaling to several thousand nodes. I'm curious why Robert thinks Percs would have been the magic bullet for his problems, given the modest improvement it represents, compared to other solutions.

< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
New article: Intel's Near-Threshold ComputingDavid Kanter09/18/12 12:26 PM
  Higher SRAM voltage and shared L1Paul A. Clayton09/18/12 02:38 PM
    Higher SRAM voltage and shared L1David Kanter09/18/12 05:20 PM
      Higher SRAM voltage and shared L1Eric09/20/12 10:44 AM
        Higher SRAM voltage and shared L1David Kanter09/20/12 12:24 PM
      Yes, that kind of asynchronousPaul A. Clayton09/20/12 02:53 PM
    Higher SRAM voltage and shared L1somebody09/19/12 09:27 AM
      So micro-turboboost is doubly impracticalPaul A. Clayton09/20/12 02:53 PM
  Big littleDoug S09/18/12 03:04 PM
    Big littleDavid Kanter09/18/12 04:05 PM
    Big littleRicardo B09/19/12 04:06 AM
  New article: Intel's Near-Threshold Computingdefderdar09/18/12 09:39 PM
    New article: Intel's Near-Threshold Computingtarlinian09/19/12 08:32 AM
      New article: Intel's Near-Threshold ComputingDavid Kanter09/19/12 10:44 AM
  New article: Intel's Near-Threshold ComputingMark Christiansen09/19/12 11:31 AM
    New article: Intel's Near-Threshold ComputingChris Brodersen09/19/12 12:54 PM
  New article: Intel's Near-Threshold ComputingEric09/20/12 10:47 AM
  Latency and HPC WorkloadsRobert Myers10/03/12 10:52 AM
    Latency and HPC Workloadsanon10/03/12 06:50 PM
      Latency and HPC WorkloadsRobert Myers10/04/12 10:24 AM
        Latency and HPC WorkloadsSHK10/08/12 05:42 AM
          Latency and HPC WorkloadsMichael S10/08/12 01:59 PM
            Latency and HPC WorkloadsSHK10/08/12 02:42 PM
              Latency and HPC WorkloadsMichael S10/08/12 05:12 PM
                Latency and HPC Workloadsforestlaughing10/15/12 08:41 AM
                  The original context was Micron RLDRAM (NT)Michael S10/15/12 08:55 AM
                    The original context was Micron RLDRAMforestlaughing10/15/12 10:21 AM
              Latency and HPC Workloads - Why not SRAM?Kevin G10/09/12 09:48 AM
                Latency and HPC Workloads - Why not SRAM?Michael S10/09/12 10:33 AM
                  Latency and HPC Workloads - Why not SRAM?SHK10/09/12 12:55 PM
                    Why not SRAM? - CapacityRohit10/09/12 09:13 PM
                  Latency and HPC Workloads - Why not SRAM?Kevin G10/09/12 03:04 PM
                    Latency and HPC Workloads - Why not SRAM?Michael S10/09/12 04:52 PM
                      Latency and HPC Workloads - Why not SRAM?Robert Myers10/10/12 10:11 AM
                        Latency and HPC Workloads - Why not SRAM?forestlaughing10/15/12 08:02 AM
                          Latency and HPC Workloads - Why not SRAM?Robert Myers10/15/12 09:04 AM
                            Latency and HPC Workloads - Why not SRAM?forestlaughing10/16/12 09:13 AM
                          Latency and HPC Workloads - Why not SRAM?SHK10/16/12 08:12 AM
                    Latency and HPC Workloads - Why not SRAM?slacker10/11/12 01:35 PM
                      SRAM leakageDavid Kanter10/11/12 03:00 PM
          Latency and HPC Workloadsforestlaughing10/15/12 08:57 AM
            Latency and HPC WorkloadsRobert Myers10/16/12 07:28 AM
              Latency and HPC WorkloadsMichael S10/16/12 07:35 AM
              Latency and HPC Workloadsanon10/16/12 08:17 AM
                Latency and HPC WorkloadsRobert Myers10/16/12 09:56 AM
                  Supercomputer variant of Kahan quotePaul A. Clayton10/16/12 11:09 AM
                    Supercomputer variant of Kahan quoteanon10/17/12 01:17 AM
                      Supercomputer variant of Kahan quoteRobert Myers10/17/12 04:34 AM
                        Supercomputer variant of Kahan quoteanon10/17/12 05:12 AM
                          Supercomputer variant of Kahan quoteRobert Myers10/17/12 02:38 PM
                            Supercomputer variant of Kahan quoteanon10/17/12 05:24 PM
                              Supercomputer variant of Kahan quoteRobert Myers10/17/12 05:45 PM
                                Supercomputer variant of Kahan quoteanon10/17/12 05:58 PM
                                Supercomputer variant of Kahan quoteanon10/17/12 05:58 PM
                                  Supercomputer variant of Kahan quoteRobert Myers10/17/12 07:14 PM
                                    Supercomputer variant of Kahan quoteanon10/17/12 08:36 PM
                                      Supercomputer variant of Kahan quoteRobert Myers10/18/12 09:47 AM
                                        Supercomputer variant of Kahan quoteanon10/19/12 02:34 AM
                                          Supercomputer variant of Kahan quoteanon10/19/12 04:47 AM
                                          Supercomputer variant of Kahan quoteRobert Myers10/19/12 03:14 PM
                        Supercomputer variant of Kahan quoteMichael S10/17/12 06:56 PM
                          Supercomputer variant of Kahan quoteanon10/17/12 09:02 PM
                            Supercomputer variant of Kahan quoteRobert Myers10/18/12 01:29 PM
                              Supercomputer variant of Kahan quoteanon10/19/12 02:27 AM
                                Supercomputer variant of Kahan quoteRobert Myers10/19/12 07:24 AM
                                  Supercomputer variant of Kahan quoteanon10/19/12 08:00 AM
                                    Supercomputer variant of Kahan quoteRobert Myers10/19/12 09:28 AM
                                      Supercomputer variant of Kahan quoteanon10/19/12 10:27 AM
                              Supercomputer variant of Kahan quoteforestlaughing10/19/12 10:26 AM
                                Supercomputer variant of Kahan quoteRobert Myers10/19/12 07:04 PM
                                  Supercomputer variant of Kahan quoteEmil Briggs10/20/12 04:52 AM
                                    Supercomputer variant of Kahan quoteRobert Myers10/20/12 07:51 AM
                                      Supercomputer variant of Kahan quoteEmil Briggs10/20/12 08:33 AM
                                        Supercomputer variant of Kahan quoteEmil Briggs10/20/12 08:34 AM
                                          Supercomputer variant of Kahan quoteRobert Myers10/20/12 09:35 AM
                                            Supercomputer variant of Kahan quoteEmil Briggs10/20/12 10:04 AM
                                              Supercomputer variant of Kahan quoteRobert Myers10/20/12 11:23 AM
                  Latency and HPC Workloadsanon10/16/12 06:48 PM
                    Latency and HPC Workloadsforestlaughing10/19/12 11:43 AM
              Latency and HPC Workloadsforestlaughing10/19/12 09:38 AM
                Latency and HPC WorkloadsRobert Myers10/19/12 11:40 AM
                Potential false economics in researchPaul A. Clayton10/19/12 12:54 PM
                  Potential false economics in researchVincent Diepeveen10/20/12 08:59 AM
                  Potential false economics in researchforestlaughing10/23/12 10:56 AM
                    Potential false economics in researchRobert Myers10/23/12 07:16 PM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell blue?