>That's exactly my point. The latency doesn't make or break >being thoughput-oriented.
>Unless you want to imply that AMD's GPUs are less of a >throughput-oriented architecture than NVIDIA's?

It's just one of many things. And honestly, latency between NV and ATI is pretty similar. If you look at it quantitatively - comparing the latency of dependent operations, you'll see they are vastly more similar to each other, than a CPU.

>>Anyway, long latency instructions are the best case for GPUs, the reason CPUs are
>>clocked at 3GHz+ and hav deep OoO buffers is the integer operations where GPUs are, what? 30 times slower?
>
>Still not proving this is relevant to being throughput-oriented or not.

Actually it does. CPUs are simply the best for latency sensitive workloads. GPUs are the best for some throughput sensitive workloads.

>>>Note though that on GT200 it was 24 cycles, so there's some convergence taking place.
>>
>>In GT200 it was pathetically slow, not being so slow doesn't prove anything, R600 still have lower latency than Fermi.
>
>Again proving latency optimizations don't make it any less >of a throughput-oriented
>architecture.

Yes it does. Unlike software, hardware is a 0-sum game where spending power or area for an optimization means it isn't available for another. Sometimes you can save power with an optimization, but you spend area. Sometimes you save area, but spend power. Regardless, you have finite engineering resources, which means you have to choose whether to spend more on latency optimization or throughput optimization.

If you are spending a lot of power/area to reduce latency, you are not by definition NOT spending that power/area to improve throughput. Reducing latency can improve throughput, but it's not the optimal strategy. It improves flexibility, but again, is not optimal for streaming type workloads.

>The
>real reason they differ is because a higher clock frequency requires more pipeline
>stages thus more latching latency. And the reason CPUs >have lower latency despite
>higher clock frequency is because forwarding makes the >execution pipeline shorter.

No, CPUs have lower latency because the engineers spend more power and transistors to achieve that. The ALUs in a CPU are simply more power hungry and less area efficient than the ones in GPUs.

The reason why CPUs are so much lower latency in both absolute and cycle terms is that their ALUs and FPUs are designed differently. They spend more area and power to reduce latency. GPUs use higher latency ALU/FPU blocks so they can cram more of them in.

But a lot of the latency doesn't have to do with the ALU/FPU, which are in most respects the simplest part of the system. What's the latency of a chain of dependent load operations? e.g. traversing a linked list?

>With that taken into account the latency of CPUs is still >somewhat lower, but not
>very sigificantly so, and as pointed out there's still >convergence taking place.

The latency of CPUs is vastly lower. Again, look at something like N dependent ALU ops, or N dependent loads.

>>>Furthermore, the latency of the CPU is based on making use of argument forwarding,
>>to bypass the register file. On a GPU the latency includes accessing the register
>>file, which for AMD's architecture takes half of the total latency (base on what you wrote about AMD Cayman here:
>>
>>I didn't read the article neither care about it (sorry DK), the important thing
>>is, Cayman DOES forwarding, it's explicity but it does, it does not access the register
>>file (I mean, the big SRAM banks) for dependent instructions.
>
>According to David's article, there are "virtual" registers (let's call the other
>ones stored in SRAM "context" registers), but there's no mention these virtual registers
>halve the lantecy for dependent instructions (or >equivalently; increase the available context registers per >wavefront).

You didn't read my article. Those are used for an entirely different purpose. Results from the ALU take extra cycles to write back, so they aren't immediately available for the next operation. The virtual registers (which are common in VLIW) are used to decrease the latency of operand availability by 1 cycle. They do not halve the latency, since we're talking about ~20 cycles or so. You remove 1/20, that's ~5%.

>Possibly the forwarding you talk about is the one explained on the next page of
>David's article (http://www.realworldtech.com/page.cfm?ArticleID=RWT121410213827&p=6).
>This is forwarding between pipelines, not back to the top of the pipeline for the
>next instrution, but for the same instruction. It allows "horizontal" operations
>like dot products, which takes higher latency than multiply-add. Sandy Bridge supports
>dot products too, and it takes 12 cycles (for comparison, >add takes 3 and mul takes 5).

This doesn't matter. What is the latency of a chain of N dependent adds? What is the latency of a chain of N dependent loads? Those are the things I'm asking, and you don't seem to want to answer and try to compare to a CPU.

>Once more this proves that at the ALU level the latency >difference is nowhere near
>as big as David and you would like to believe.

Prove it. I've asked you about the latency of dependent ALU ops, and dependent loads in a CPU and how it compares to a GPU. I haven't heard a peep. There's your quantitative comparison, and it will be readily apparent how different the two are.

>Why would this be "too" many transistors? Reducing latency >improves effective throughput
>for complex workloads. Effective throughput is the only >thing that matters. Focussing
>too aggressively on increasing the ALU count leads to >being latency bound, meaning
>"too" many transistors are spent on it.

That depends on the workloads you are optimizing for. GPUs are not running GCC, they only work on a limited set of stuff, and even within that realm, it is a more limited set of workloads that they are good at. Those restrictions mean it is not really general purpose, but where the workload fits to the hardware, it will be faster.

>Regardless of whether it's arithmetic latency or memory >latency, GPUs continue
>to have to invest transistors into latency reduction. At >the same time, CPUs slow
>cache/core growth, widen the vectors, add FMA, add SMT, >etc. The word.

What's the memory latency in ns for a GPU again? What's the memory latency for a CPU?

[snip]

>>>Unless of course you want to imply that NVIDIA's GPUs are any less of a GPU due to their higher clock frequency?
>>
>>Clock frequency != latency, got it?
>
>Yes, I got that all along. Chickens are not eggs either, >but that doesn't mean they're unrelated.
>
>Besides, it's David who first mentioned both frequency and latency as arguments
>for being throughput oriented or not. The fact of the matter is that today's GPUs
>have different frequencies and different latencies but they're all considered throughput-oriented.

Yes, but those differences are fairly minor. Quantitatively compare them to a CPU. You'll be surprised.

[snip]

>>>The MHz-race is over.
>>
>>Clock frequency != latency, pushing clock frequency too high can actually increase
>>your latency and CPUs are designed to reduce the latency!
>
>Certainly, but you're not proving that the clock frequency >of future CPUs is "too"
>high to achieve a good effective throughput.
>
>That's why I said the MHz-race is over. GPUs clock >frequencies keep increasing at a faster pace than CPUs.

Really? How fast have GPU frequencies increased over the last 2 years? The last 5 years?

>VLIW isn't the limiting factor. NVIDIA's architecture also has low ALU utilization
>when you stuff the code with SFU operations or branches. >But while NVIDIA's utilization
>isn't 100%, AMD's overall utilization is only half of it!

That's not clear at all. Nvidia and AMD run different code, so it's impossible to say without using performance counters.

>Clearly something else is at play which makes the fifth >ALU not worthwhile. So
>it must have been intentionally removed to improve VLIW >scheduling efficiency, or
>you're implying the AMD engineers are stupid.

Perhaps it's not efficient for 28nm and below, but it was efficient for 55nm+?

>Note that despite a high demand for floating-point performance in HPC computing,
>both Intel and AMD never had any CPU with more than two >floating-point execution
>ports (not even Itanium). This should tell you something >about the scheduling difficulties
>of VLIW5, and the direction AMD is trying to take (complex >HPC workloads). The word.

Actually it has nothing to do with that at all. It has to do with the ratio of load ports on the cache (and memory bandwidth) to the FPUs.

Take Intel's CPUs, which up until recently had 1 load port. There was no reason to ever have more than 2 FPUs, because you cannot feed 4 FPUs with a single load per cycle.

Itanium has 2 FMAC pipelines, which is twice the performance of an x86. No surprise then that it has 2X the load/store units and cache ports.

DK