Nicolas Capens (nicolas.capens@gmail.com) on 2/11/11 wrote:
---------------------------
>That's exactly my point. The latency doesn't make or break being thoughput-oriented.
>Unless you want to imply that AMD's GPUs are less of a throughput-oriented architecture than NVIDIA's?
>
>...
>
>Still not proving this is relevant to being throughput-oriented or not.
>
>...
>
>Again proving latency optimizations don't make it any less of a throughput-oriented
>architecture. And when looking at the GPU generations from one vendor, the latencies
>are going down while for CPUs the latency remained nearly constant, proving there's
>convergence taking place. Comparing the latencies for different GPUs (with significantly
>different clock frequencies) isn't valid at all so it doesn't disprove this. The
>real reason they differ is because a higher clock frequency requires more pipeline
>stages thus more latching latency. And the reason CPUs have lower latency despite
>higher clock frequency is because forwarding makes the execution pipeline shorter.
>
>With that taken into account the latency of CPUs is still somewhat lower, but not
>very sigificantly so, and as pointed out there's still convergence taking place.

No convergence, GF114 improved latency over previous generation, but previous generation was very slow, going from very slow to slow doesn't prove they intent to go from slow to fast.

On the other hand, single threaded performance and per core performance of CPUs are improving, right? Alot from P4 to Core 2, a bit from Core 2 to Nehalem and another bit from Nehalem to Sandy Bridge, the latency for CPUs is low, they are fast, it's hard to be faster when you are already fast but both companies are pushing it hard, AMD bulldozer will bring a decent improve on single thread performance.

>According to David's article, there are "virtual" registers (let's call the other
>ones stored in SRAM "context" registers), but there's no mention these virtual registers
>halve the lantecy for dependent instructions (or equivalently; increase the available context registers per wavefront).

Accessing SRAM is slow, accessing those "virtual registers" is fast, hell, there are buffers very similiar between each pipeline stage! Anyway, they work for forwarding.

>Possibly the forwarding you talk about is the one explained on the next page of
>David's article (http://www.realworldtech.com/page.cfm?ArticleID=RWT121410213827&p=6).
>This is forwarding between pipelines, not back to the top of the pipeline for the
>next instrution, but for the same instruction. It allows "horizontal" operations
>like dot products,

Not related to horizontal operations, not between pipelines, it's between one insruction and the next instruction.

>which takes higher latency than multiply-add.

Dot-products doesn't needs to have a significantly higher latency than multiplies, it's the same case of multiply-add.

>Sandy Bridge supports
>dot products too, and it takes 12 cycles (for comparison, add takes 3 and mul takes 5).

Intel implementation of dot product (micro-coded?) sucks too much, an ugly hack doesn't prove anything.

Maybe future microprocessors will have a decent and actually usefull implementation of dot product, maybe Bulldozer will do that.

>This "ugly failure" hasn't stopped the PlayStation 3 from selling on average a
>million units a month. And despite the aging the sales numbers are still on an upward
>slope. So much for not meaning anything today.

PS3 would sell even if it had shit inside of it, P4 sold a lot but most will agree it wasn't any better than Athlon 64.

>And in case you consider it an "ugly failure" because of the heterogenous programming
>model, that too proves one of my points. Developers prefer homogenous architectures.

Sure developers prefer homogenous architectures, it's much easier, also, we prefer one single core fast enough than a thousand slow cores, as we prefer to not care about SIMD, unfortunally life isn't easy.

>Besides, it's David who first mentioned both frequency and latency as arguments
>for being throughput oriented or not.

I'm not David and may not agree with him, don't use his arguments as mine.

>That's why I said the MHz-race is over. GPUs clock frequencies keep increasing at a faster pace than CPUs. The word.

Did you forgot 9800GTX?

>Next to cost, effective performance/Watt is everything. Pick a price (read: die
>size), a TDP, and performance/Watt is what sets architectures apart.
>
>You can also do the reverse: set a performance goal and see how cheap you can make
>it to conquer a bigger market share than the competition. You can use a smaller
>die by clocking it higher, till you get close to the maximum TDP acceptable for
>the system you're creating. Again performance/Watt is the key factor.

AMD did that with Magny cours, Intel does that all the time with Itanium, nVidia did that too! That's true, if you increase die size and lower your voltage at the same TDP you will have better throughput, to some extend but it works.

With Magny Cours AMD sacrified 1/3 single threaded performance to remain competitive at throughput, for some workloads it's a nice trade-off and keeping 2/3 of single threaded performance isn't a show stoper.

But GPUs doesn't have 2/3 of single threaded performance, they have 1/10, 1/100 depending on the workload, GPUs doesn't target latency at all, CPUs sacrify a lot for latency, GPUs have core voltage at 1V, CPUs are at 1.3V, and more, keep reading.

>So with all due respect, your example of shrinking an 8086 is pretty silly (and
>I mean silly aside from the fact that it's meant to be a silly example). If you
>take cost and acceptable TDP into account, you need another architecture. One with the best performance/Watt.

An 8086 shrunk to 32nm will likely have higher performance/Watt than a Core i7, wants a more modern processor? Ok, ARM M4 will beat i7 to death in performance/Watt comparisons, Bobcat have better performance/Watt than future Bulldozer chips, Atom have better performance/Watt than i7, simple chips (even old chips shrunk to new process) tend to have better performance/Watt than new chips because the amount of energy efficiency sacrified by the new chips in name of absolute performance, doesn't the patetically high voltage of CPUs already proves that?

>Taking the SFU away is clearly not an option. So you need to ensure that the fat
>doesn't weigh down the architecture too much. The solution they found was to unify
>it with three slim ALUs, which has the added effect of better VLIW scheduling. If
>the latter wasn't a reason and just a coincidental benefit, then why didn't they keep the fifth ALU?

They have instructions using from one to four ALUs, why the fuck they would have a fifth ALU that isn't able to do anything special?

They could even have 8 ALUs, or 16 ALUs, schedulling? Hell... It's a f***ing wide SIMD, to improve schedulling they could just execute two or more workitems per lane, reduce SIMD, regroup wavefront if branch granularity impose any issue, increasing VLIW width at the cost of SIMD width cost a bit more hardware, a bit energy efficiency but is more flexible, trade offs, AMD engineers didn't think it worths.

It could improve single threaded performance by more than the extra cost of hardware, if it was a CPU sure they would do, but since they don't care about single threaded performance at all...

>Clearly something else is at play which makes the fifth ALU not worthwhile. So
>it must have been intentionally removed to improve VLIW scheduling efficiency, or
>you're implying the AMD engineers are stupid.

AMD GPUs are able to do many nice thing like to dependent multiplies per clock and so, it's true they have a bit too much shadding power compared to everything else, I would say too much that there is little incentive to improve the compiler.

But if the hardware engineers re modifying the hardware because the compiler team can't do a reasonable work than in fact someone there is stupid.

>Note that despite a high demand for floating-point performance in HPC computing,
>both Intel and AMD never had any CPU with more than two floating-point execution
>ports (not even Itanium). This should tell you something about the scheduling difficulties
>of VLIW5, and the direction AMD is trying to take (complex HPC workloads). The word.

Both Intel and AMD never had any CPU with single cycle floating point latency (not even the low clocked Itanium), in AMD case the floating point scheduler is a bit bigger than the integer one to compensate the extra latency, to fully utilize it's units AMD has to have 8 non-dependent floating instructions.

And nether Intel or AMD ever had the luxury of thinking about executing two "threas" in one thread to improve schedulling...

>Obviously when moving to 28 nm they'll increase ALU count. But it will take a major
>amount of the extra transistor budget, to fight Amdahl's Law and keep these ALUs fed with data.

I mean, they will improve ALU count more than a simple die shrink would allow, they will increase their throughput advantage over CPUs.