Nicolas Capens (nicolas.capens@gmail.com) on 2/15/11 wrote:
---------------------------
>Hi David,
>
>David Kanter (dkanter@realworldtech.com) on 2/11/11 wrote:
>---------------------------
>>>That's exactly my point. The latency doesn't make or break >being thoughput-oriented.
>>>Unless you want to imply that AMD's GPUs are less of a >throughput-oriented architecture than NVIDIA's?
>>
>>It's just one of many things.
>
>Then please sum up the "many things" that make a CPU not >capable of high *effective*
>throughput for graphics and HPC workloads. Keep in mind >that a 6-core Sandy Bridge
>with FMA would be capable of delivering 650 GFLOPS, while >the 12 EU IGP can only
>do 130 GFLOPS. What sort of magic makes the latter, which >has lower theoretical
>throughput and little or no latency optimizations, achieve >significantly better effective throughput?

CPUs are designed for good throughput, but that's simply not their primary design target. If you don't believe me, let's look at some hard facts for single precision performance.

SNB gets about 2.3GFLOP/W and 21.3GB/s memory bandwidth
GF104 gets about 6GFLOP/mm2 and 115GB/s memory bandwidth
Cayman gets about 10GFLOP/mm2 and 176GB/s memory bandwidth
Cypress gets about 14GFLOP/mm2 and 160GB/s memory bandwidth

That's about an order of magnitude difference between a best of breed CPU and GPU. You simply cannot look at that data and conclude that CPUs are equally good at throughput. Moreover, the reality is that the CPU in question has a pretty big power (and area) advantage due to using 32nm HK/MG vs. 40nm polysilicon.

Many of the design choices in CPUs will reduce perf/area, for example:

1. 8T SRAMs for L1 cache
2. High frequency operation
3. Forwarding and bypass networks
4. OOOE
5. Microcode
6. Branch prediction
7. uop caches
8. Coherency
9. Large TLBs
10. Larger SRAM cells for higher frequency
11. Multi-ported (rather than banked) RFs
12. Load-to-use latency from L1

The list goes on...


>Also note once more that a GeForce GT 420 has a TDP of 50 Watt, at practically
>the same clock frequency as Sandy Bridge's IGP, while also only achieving 130 GFLOPS.
>Mobile Sandy Bridge CPUs have a TDP of 45 Watt. Note that it can be further reduced
>by clocking them lower, while still offering more GFLOPS >than the IGP.

There are 4 cores in SNB, and 1 GPU. What's the area/power efficiency? The GPU doesn't use all 45W, even when running full throttle.

>Obviously FMA increases the power consumption of the CPU >as well, but the control
>logic remains practically the same.

Not really. It's a 3 or 4 operand instruction and requires substantially more work. But the area increase is less than the FLOP/s increase.

>So to power consumed by the control logic goes
>down with every process node, while more of the power >budget is spent on actual
>floating-point operations. At the same time GPUs have to >do the reverse. There's
>no FMA to add and simply doubling the number of shader >cores does not double the
>*effective* throughput. Instead they need to rely on >smarter scheduling or larger
>register files and caches to compensate for the latency.

Well, if you want to talk about effective throughput, then we need to come up with a mutually agreed upon workload that measures effective throughput. Theoretical FLOP/s are nice and easy to work with, albeit imperfect.

[ranting snipped]

>I believe you're greatly overestimating the power >consumption of the CPUs control
>logic, compared to the GPUs control logic. It takes more >area to achieve ILP, that's
>certainly true, but when you're only executing NOPs >there's practically no switching
>activity. The power consumption is proportional to the >number of uops being processed.

No it's not. Power consumption = C*F*V^2. C has to do with transistors switching which is only loosely correlated to uops. Some uops switch more than others.

>>And honestly, latency between NV and ATI is pretty
>>similar. If you look at it quantitatively - comparing the latency of dependent
>>operations, you'll see they are vastly more similar to each other, than a CPU.

[snip]

>Dependent instruction latency is irrelevant to being >throughput oriented or not.

You are 100% wrong here. The dependent integer operation latency is roughly 8X worse on a GPU than a CPU. That has a big cost in terms of area and power for CPUs, and it enables them to actually run general purpose workloads.

>>>>Anyway, long latency instructions are the best case for >>GPUs, the reason CPUs are
>>>>clocked at 3GHz+ and hav deep OoO buffers is the >>integer operations where GPUs are, what? 30 times slower?
>>>
>>>Still not proving this is relevant to being throughput-oriented or not.
>>
>>Actually it does. CPUs are simply the best for latency sensitive workloads. GPUs
>>are the best for some throughput sensitive workloads.
>
>It's clear by now that this "30 times slower" was yet >another prejudice. So calling
>things "the best" based on this misinformation is >worthless.

Actually, it's not too far off. As you said, the latency for an Nvidia integer operation is about 24 cycles. So the latency of a pair of dependent ALU ops is roughly 48 cycles @ 1.5GHz (32nm). On a CPU, that would be 2 cycles @ 3.4GHz (~0.6ns). That's actually higher than 30X.

Even ATI is ~20ns for dependent ALU ops.

Anyway, you just need to look at the data. The conclusions are obvious.


David