Memory power and bandwidth?

Article: Computational Efficiency for CPUs and GPUs in 2012
By: Iain McClatchie (iain-rwt.delete@this.mcclatchie.com), August 6, 2012 2:09 pm
Room: Moderated Discussions
Micron has a very nice spreadsheet online for figuring out DDR3L power dissipation. Maybe it seems unreasonable, but DDR3 is no longer interesting for new designs, as the voltage is just too high to get good performance.

You need to get specific power numbers off a data sheet, such as the DDR3L 4Gb one here:
http://media.digikey.com/PDF/Data%20Sheets/Micron%20Technology%20Inc%20PDFs/MT41K_1G4,512M8.pdf

Plug those numbers into the spreadsheet (why they don't give you accurate numbers in the first place I don't know). Then tweak the DDR3 Config (4 Gb, x4, -125, Fast PD exit) and System Config numbers (1.35 V, 800 MHz, burst 8, etc).

My first point: two-rank SDRAM systems are crazy. Take a look over on the right hand side of the System Config tab. It's got a nice diagram for termination power in the read and write cases.

In the case of reading from the DRAM, the responding DRAM burns 3.1 mW/pin, of which 1.4 mW gets burned in the termination resistor (not sure if you add those two). 8.3 mW/pin gets burned in the CPU termination. But 23.6 mW gets burned in the "passive" DRAM's termination.

In the case of writing to the DRAM, the CPU burns 5.9 mW/pin driving the data, the receiving DRAM burns 16.5 mW/pin in it's termination, and the "passive" DRAM still burns 23.6 mW/pin.

Clearly, the "passive" DRAM in a two-rank system is a power hog. If we keep bandwidth and capacity the same, and compare a single rank and two rank system, we see that the single rank system is way better:
576 bit wide = 144 packages in 2 ranks * 8bits, 4 Gb/package = 64 GB, 38.0 watts
576 bit wide = 144 packages in 1 rank * 4 bits, 4 Gb/package = 64 GB, 31.3 watts

The advantage of the first system is only configurability: you can install 32 GB at first, with the same bandwidth, and then add another 32 GB later.

Second question: how to deal with server configs.
I'll note that people don't put the same CPU SKUs into personal machines that they put into server machines. Intel and AMD would like to differentiate these SKUs so they can get server folks to pay more. So they can arrange for the server SKUs to come with more memory per CPU package, and be priced higher.

I'm not really sure what the market is like for low #core, high memory systems. I'd argue that if I want 256 GB in a system, it seems quite reasonable to put four processor packages in there. So you'd still have motherboards with 1, 2, 4, and 8 processor slots, it's just that each slot would now populate CPU and memory together. No difference logically, and the electrical environment would be quite similar, just with longer traces between the CPU pins and the socket/slot impedance ripple.

Still, if folks want to put 1 TB into a four-socket system, then the CPU will have to be packaged onto it's board with buffered memory, just as is done now. It will not be reasonable to put 576 DRAM packages onto a single board with the CPU, so the CPU thing will have to be a sandwich of 2 or 4 boards, only one of which carries the CPU and plugs into the socket. This is going to drive up the capacitance on the CPU's pins and will drive power dissipation up and performance down, just as it does today. I think it would probably be more useful for AMD/Intel to put these CPUs into a 4000 pin FBGA and stick with unbuffered memory and 2000-pin busses. Note that the CPU vendor gets to make that decision without affecting the socket interface. And users will decide if they'd rather have that configuration, or just go with more CPU sockets in the system.

Third question: does GDDR5 burn more power in termination in order to go faster?

Probably a bit more, but not vastly more. Transmission lines on PC boards tend to be around 50 ohms. There's no huge advantage in getting away from this impedance (40 Gb/s electrical links run on 50 ohm lines, or rather, between coupled pairs of such lines). You want to terminate the line with a 50 ohm termination, so that the wave just disappears into the termination and does not come back to haunt future bits. That 50 ohm termination can be as simple as 100 ohms to Vdd and 100 ohms to Vss, which then acts like a 200 ohm dead short and dissipates 9 milliwatts per pin. Or you can do something more clever, and have an actively regulated Vdd/2 rail and a 50 ohm resistor to that. On average, the highs and lows on the bus should average out, and you should end up with a tradeoff between the capacitance on this rail and the regulation current required. In practice, these regulators tend to burn a disconcertingly large amount of current, and I'm really not sure why. I'm sure they'll get better over time, and maybe there are already good ones with which I'm not yet familiar.

The bottom line is that termination power should scale with the number of pins and not pin speed.

-Iain
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
New Article: Compute Efficiency 2012David Kanter07/25/12 01:37 AM
  New Article: Compute Efficiency 2012SHK07/25/12 02:31 AM
    New Article: Compute Efficiency 2012David Kanter07/25/12 02:42 AM
  New Article: Compute Efficiency 2012none07/25/12 03:18 AM
    New Article: Compute Efficiency 2012David Kanter07/25/12 11:25 AM
  GCN (NT)EBFE07/25/12 03:25 AM
    GCN - TFLOP DPjp08/09/12 01:58 PM
      GCN - TFLOP DPDavid Kanter08/09/12 03:32 PM
        GCN - TFLOP DPKevin G08/11/12 05:22 PM
      GCN - TFLOP DPEric08/09/12 05:12 PM
        GCN - TFLOP DPjp08/10/12 01:23 AM
          GCN - TFLOP DPEBFE08/12/12 08:27 PM
            GCN - TFLOP DPjp08/13/12 02:02 AM
              GCN - TFLOP DPEBFE08/13/12 07:45 PM
                GCN - TFLOP DPjp08/14/12 01:21 AM
  New Article: Compute Efficiency 2012Adrian07/25/12 04:39 AM
    New Article: Compute Efficiency 2012EBFE07/25/12 09:33 AM
    New Article: Compute Efficiency 2012David Kanter07/25/12 11:11 AM
  New Article: Compute Efficiency 2012sf07/25/12 06:46 AM
    New Article: Compute Efficiency 2012aaron spink07/25/12 09:08 AM
      New Article: Compute Efficiency 2012someone07/25/12 10:06 AM
    New Article: Compute Efficiency 2012David Kanter07/25/12 11:14 AM
      New Article: Compute Efficiency 2012EBFE07/26/12 02:27 AM
        BG/QDavid Kanter07/26/12 09:31 AM
          VR-ZONE KNC B0 leak, poor number?EBFE08/03/12 01:57 AM
            VR-ZONE KNC B0 leak, poor number?Eric08/03/12 07:59 AM
              VR-ZONE KNC B0 leak, poor number?EBFE08/04/12 06:37 AM
                VR-ZONE KNC B0 leak, poor number?aaron spink08/04/12 06:51 PM
                Leaks != productsDavid Kanter08/05/12 03:19 AM
                  Leaks != productsEBFE08/06/12 02:49 AM
                VR-ZONE KNC B0 leak, poor number?Eric08/05/12 10:37 AM
                  VR-ZONE KNC B0 leak, poor number?EBFE08/06/12 03:09 AM
                    VR-ZONE KNC B0 leak, poor number?aaron spink08/06/12 04:33 AM
                      VR-ZONE KNC B0 leak, poor number?jp08/07/12 03:08 AM
                        VR-ZONE KNC B0 leak, poor number?Eric08/07/12 04:58 AM
                          VR-ZONE KNC B0 leak, poor number?jp08/07/12 05:17 AM
                            VR-ZONE KNC B0 leak, poor number?Eric08/07/12 05:22 AM
                              VR-ZONE KNC B0 leak, poor number?anonymou508/07/12 09:43 AM
                            VR-ZONE KNC B0 leak, poor number?jp08/07/12 05:23 AM
                              VR-ZONE KNC B0 leak, poor number?aaron spink08/07/12 07:24 AM
                        VR-ZONE KNC B0 leak, poor number?aaron spink08/07/12 07:20 AM
                          VR-ZONE KNC B0 leak, poor number?jp08/07/12 11:22 AM
                            VR-ZONE KNC B0 leak, poor number?EduardoS08/07/12 03:15 PM
                        KNC has FMADavid Kanter08/07/12 09:17 AM
  New Article: Compute Efficiency 2012forestlaughing07/25/12 08:51 AM
    New Article: Compute Efficiency 2012Eric07/27/12 05:12 AM
      New Article: Compute Efficiency 2012hobold07/27/12 11:53 AM
        New Article: Compute Efficiency 2012Eric07/27/12 12:51 PM
          New Article: Compute Efficiency 2012hobold07/27/12 02:48 PM
            New Article: Compute Efficiency 2012Eric07/27/12 03:29 PM
        New Article: Compute Efficiency 2012anon07/29/12 02:25 AM
          New Article: Compute Efficiency 2012hobold07/29/12 11:53 AM
  Efficiency? No, lack of highly useful featuressomeone07/25/12 09:58 AM
    Best case for GPUsDavid Kanter07/25/12 11:28 AM
      Best case for GPUsfranzliszt07/25/12 01:39 PM
      Best case for GPUsChuck07/25/12 08:13 PM
        Best case for GPUsDavid Kanter07/25/12 09:45 PM
        Best case for GPUsEric07/27/12 05:51 AM
  Silverthorn data pointMichael S07/25/12 02:45 PM
    Silverthorn data pointDavid Kanter07/25/12 04:06 PM
  New Article: Compute Efficiency 2012Unununium07/25/12 05:55 PM
    New Article: Compute Efficiency 2012EduardoS07/25/12 08:12 PM
      Ops... I'm wrong...EduardoS07/25/12 08:14 PM
  New Article: Compute Efficiency 2012TacoBell07/25/12 08:36 PM
    New Article: Compute Efficiency 2012David Kanter07/25/12 09:49 PM
    New Article: Compute Efficiency 2012Michael S07/26/12 02:33 AM
  Line and factorMoritz07/26/12 01:34 AM
    Line and factorPeter Boyle07/27/12 07:57 AM
      not entirelyMoritz07/27/12 12:22 PM
      Line and factorEduardoS07/27/12 05:24 PM
        Line and factorMoritz07/28/12 12:52 PM
  tables Michael S07/26/12 02:39 AM
  Interlagos L2+L3Rana07/26/12 03:13 AM
    Interlagos L2+L3Rana07/26/12 03:13 AM
    Interlagos L2+L3David Kanter07/26/12 09:21 AM
      SP vs DP & performance metricsjp07/27/12 07:08 AM
        SP vs DP & performance metricsEric07/27/12 07:57 AM
          SP vs DP & performance metricsjp07/27/12 09:18 AM
            SP vs DP & performance metricsaaron spink07/27/12 09:36 AM
              SP vs DP & performance metricsjp07/27/12 09:47 AM
                "Global" --> systemPaul A. Clayton07/27/12 10:31 AM
                  "Global" --> systemjp07/27/12 03:55 PM
                    "Global" --> systemaaron spink07/27/12 07:33 PM
                      "Global" --> systemjp07/28/12 02:00 AM
                        "Global" --> systemaaron spink07/28/12 06:54 AM
                          "Global" --> systemjp07/29/12 02:12 AM
                            "Global" --> systemaaron spink07/29/12 05:03 AM
                              "Global" --> systemnone07/29/12 09:05 AM
                                "Global" --> systemEduardoS07/29/12 10:26 AM
                                "Global" --> systemjp07/30/12 02:24 AM
                                  "Global" --> systemaaron spink07/30/12 03:05 AM
                                "Global" --> systemaaron spink07/30/12 03:03 AM
                                  daxpy is STREAM TRIADPaul A. Clayton07/30/12 06:10 AM
                SP vs DP & performance metricsaaron spink07/27/12 07:25 PM
                  SP vs DP & performance metricsEmil Briggs07/28/12 06:40 AM
                    SP vs DP & performance metricsaaron spink07/28/12 07:05 AM
                      SP vs DP & performance metricsjp07/28/12 11:04 AM
                        SP vs DP & performance metricsBrett07/28/12 03:32 PM
                      SP vs DP & performance metricsEmil Briggs07/28/12 06:11 PM
                        SP vs DP & performance metricsanon07/29/12 02:53 AM
                        SP vs DP & performance metricsaaron spink07/29/12 05:39 AM
                          Coherency for discretesRohit07/29/12 09:24 AM
                          SP vs DP & performance metricsanon07/29/12 11:09 AM
                          SP vs DP & performance metricsEric07/29/12 01:08 PM
        SP vs DP & performance metricsaaron spink07/27/12 09:25 AM
  Regular updates?Joe07/27/12 09:35 AM
  New Article: Compute Efficiency 201230907/27/12 10:34 PM
  New Article: Compute Efficiency 2012Ingeneer07/30/12 09:01 AM
    New Article: Compute Efficiency 2012David Kanter07/30/12 01:11 PM
      New Article: Compute Efficiency 2012Ingeneer07/30/12 08:04 PM
        New Article: Compute Efficiency 2012David Kanter07/30/12 09:32 PM
          Memory power and bandwidth?Iain McClatchie08/03/12 04:35 PM
            Memory power and bandwidth?David Kanter08/04/12 11:22 AM
              Memory power and bandwidth?Michael S08/04/12 02:36 PM
              Memory power and bandwidth?Iain McClatchie08/06/12 02:09 PM
              Memory power and bandwidth?Eric08/07/12 06:28 PM
                WorkloadsDavid Kanter08/08/12 10:49 AM
                  WorkloadsEric08/09/12 05:21 PM
                Latency and bandwidth bottlenecks Paul A. Clayton08/08/12 04:02 PM
                  Latency and bandwidth bottlenecks Eric08/09/12 05:32 PM
                    Latency and bandwidth bottlenecks none08/10/12 06:06 AM
                  Latency and bandwidth bottlenecks -> BDPajensen08/11/12 03:21 PM
            Memory power and bandwidth?Ingeneer08/06/12 11:26 AM
  NV aims for 1.8+ TFLOPS DP ?jp08/11/12 01:21 PM
    NV aims for 1.8+ TFLOPS DP ?David Kanter08/11/12 09:25 PM
      NV aims for 1.8+ TFLOPS DP ?jp08/12/12 02:45 AM
      NV aims for 1.8+ TFLOPS DP ?EBFE08/12/12 10:02 PM
        NV aims for 1.8+ TFLOPS DP ?jp08/13/12 01:54 AM
          NV aims for 1.8+ TFLOPS DP ?Gabriele Svelto08/13/12 09:16 AM
            NV aims for 1.8+ TFLOPS DP ?Vincent Diepeveen08/14/12 03:04 AM
          NV aims for 1.8+ TFLOPS DP ?David Kanter08/13/12 09:50 AM
            NV aims for 1.8+ TFLOPS DP ?jp08/13/12 11:17 AM
        NV aims for 1.8+ TFLOPS DP ?EduardoS08/13/12 06:45 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell blue?