EduardoS (no@spam.com) on 1/28/11 wrote:
---------------------------
>Nicolas Capens (nicolas.capens@gmail.com) on 1/28/11 wrote:
>---------------------------
>>Now why would a GPU manufacturer risk not being able to schedule two instructions
>>simultaneously and invest die space into a more complex scheduler? That's simply
>>because hiding latency with DLP takes resources too. Storing intermediate results
>>takes lots of register space. At the same time, GPUs need more registers per strand
>>because shaders still get longer and more branchy. Fermi features a stack but it's
>>still tiny as you want to avoid spilling to RAM. So apparently we've reached the
>>point where it's cheaper to use CPU technology to deal with latency than to hide it the GPU way.
>
>Mostly true but still not the fat I'm talking about, Fermi is superscalar (like
>R600 was)

R600 isn't superscalar, it's VLIW. Superscalar architectures like GF104 dynamically schedule independent instructions, while with VLIW they are statically scheduled. Superscalar adds more "fat".

>and yet it's instruction latency is an order magnitude higher than CPUs,
>a lot (insane if talking about througput) amount of resources are spent just to
>make instruction latency lower, resources in the form of voltage, density and transistors,
>about everything on CPU is targeting low latency, in GPU it's througput, even when
>they increase single-threaded performance to save register space is to allow more
>executing units and more througput

Sounds to me like we're talking about the same kind of fat. But anyway, I'm not denying there's still a significant difference betweem CPUs and GPUs. All I'm saying is that there has already been a tremendous convergence, and it's not stopping.

Before multi-core, the throughput of CPUs scaled sub-linearly. The Prescott core had three times more transistors than Willamette, but less than twice the (theoretical) throughput. To be exact, it had 62% throuhput/transistor of Willamette. With Sandy Bridge versus Willamette, they more than compensated it by bringing it back to 115%. And that's including the IGP transistors, without counting its throughput. If we assume a 6-core SB without IGP is about the same size, the compute density has increased by 172%.

If we compare G92b versus GF110, the throughput/transistor dropped by a factor of 56%.

Get the picture? CPUs are on an upward slope, while GPUs are on a downward slope. The compute density for CPUs will double again with the addition of FMA, and AVX can be extended further. At the same time, GPUs need to invest transistors into superscalar execution, larger register files, larger caches, generic instructions, smaller SIMD width, concurrent kernel execution, etc., further reducing compute density. Note that this is not a bad thing. The computing capacity is still increasing, just more slowly, and we're getting higher efficiency at complex tasks in return.

CPUs are already very efficient at complex tasks, so they are investing transistors mainly in compute density. It may take another decade or so, but they're clearly on a collision course. It's why NVIDIA desperately wanted an x86 license. Not to expand its market, but to stay in the market in the next decade! They've shifted all hope to the ARM architecture now, and given that Microsoft will also support ARM in the future, I have high hopes for this move.

To stay competitive, Intel needs to complete the AVX instruction set, by adding integer instructions and gather/scatter, sooner rather than later. The initial implementation doesn't have to be optimal yet, but to ensure optimal adoption by the software world the instructions need to be available.

>nothing "latency at all costs" like what is done on CPUs.

I'm still not sure what you mean. Latency is mainly dictated by physical properties like switching speed and wire length. CPUs nowadays focus on optimizing power efficiency instead (32 nm easily allows 10+ GHz when using larger gate widths and more pipeline stages). I assume you mean single-threaded performance instead; frequency times ILP? Yes, that's still important to CPUs, but they no longer focus on it at all costs. If they were, we wouldn't have multi-core, Hyper-Threading, AVX, and clock frequencies below 4 GHz. As I've proven above, even while the transistor count increases exponentially, a larger percentage of the total transistor count goes to increasing throughput.

Again, I think you should let go of the prejudice that CPUs are fat. They definitely were before, but that's changing, and GPUs are now the ones that are getting fatter. The main thing that's required to bridge the larger portion of the remaining gap, is gather/scatter.