Nicolas Capens (nicolas.capens.delete@this.gmail.com) on January 6, 2014 9:15 pm wrote:
> How does that not apply to Knights Landing? And by the way that 1 byte/FLOP is the GPU's cache
> bandwidth. Kepler is even limited to 0.33 bytes/FLOP. That's not because there's too little bandwidth,
> but rather an 'excess' of FLOPS. They're there to burst process arithmetic heavy portions of code.
> Kind of like beating Amdahl's Law by going even wider, because it's so damn cheap. Knights Landing
> won't be any different. I don't think having 2 bytes/FLOP is going to make it capable of sustaining
> a higher peak performance than 1 byte/FLOP, for the same power envelope.

Have you looked at the actual microarchitectures? If you have you might have noticed that the GPU register files are the size of KNL L1 or larger. Essentialy, KNL register file is L0 compared to GPU register files. So if you compare bandwidth to GPUs you must be careful to look at comparable intersection in cache hierarchy. Going up a level, the GPU L1+local memory+shared memory are very small compared to proposed KNL L2, and KNL will offer far bigger L2/FPU.

> NVIDIA and ATI have been designing throughput architectures for ages. You think Intel can swoop in with a "superior
> cache hierarchy" to bend the laws of computing and physics? Intel's best hope for Knights Landing is to keep
> x86 relevant in the HPC market. Their process technology might give them an advantage, as well as the reasonable
> single-threaded scalar performance, but if providing more cache bandwidth was the key to success you'd expect
> NVIDIA and ATI to put more focus on that. By the way, their cache hierarchies are substantially different, but
> neither is taking a a big overall lead over the other. So I wouldn't expect a third player to suddenly beat
> them on all fronts by providing a higher L1 bandwidth than what is typically necessary.

While I do believe NVIDIA and ATI do have accumulated experience in optimizing graphics worksets that Intel cannot match I also do believe what I have written. More bandwidth cannot hurt. The key to overall performance has always been removing bottlenecks. I expect GPUs to keep throwing more FPUs at the problem and Intel to try to squeeze better performance from mm2 of silicon.

> To recap: a huge L1 bandwidth of one load operation per arithmetic instruction doesn't help
> if L2 can't sustain it. You're either accessing the same data multiple times, in which case
> an L0 cache would be more power efficient, or you're accessing lots of unique data, in which
> case you're L2 bandwidth limited and a second L1 read port doesn't help. Cases in between are
> rare, and due to other things that cause the IPC to dip below 2, I believe having two L1 load
> ports doesn't make that much of a difference and may not outweigh the power and area cost.

In reality there are lots of codes inbetween where work can be blocked to maximize usage of both L1 and L2 bandwidth.