NoSpammer (no.delete@this.spam.com) on January 7, 2014 3:58 am wrote:
> Nicolas Capens (nicolas.capens.delete@this.gmail.com) on January 6, 2014 9:15 pm wrote:
> > How does that not apply to Knights Landing? And by the way that 1 byte/FLOP is the GPU's cache
> > bandwidth. Kepler is even limited to 0.33 bytes/FLOP. That's not because there's too little bandwidth,
> > but rather an 'excess' of FLOPS. They're there to burst process arithmetic heavy portions of code.
> > Kind of like beating Amdahl's Law by going even wider, because it's so damn cheap. Knights Landing
> > won't be any different. I don't think having 2 bytes/FLOP is going to make it capable of sustaining
> > a higher peak performance than 1 byte/FLOP, for the same power envelope.
>
> Have you looked at the actual microarchitectures? If you have you might have noticed that the
> GPU register files are the size of KNL L1 or larger. Essentialy, KNL register file is L0 compared
> to GPU register files. So if you compare bandwidth to GPUs you must be careful to look at comparable
> intersection in cache hierarchy. Going up a level, the GPU L1+local memory+shared memory are
> very small compared to proposed KNL L2, and KNL will offer far bigger L2/FPU.

The GPU's register files are not a substitute for L1 cache, and furthermore, the number per thread isn't that impressive. Kepler has 31 registers per thread at full occupancy. To use more you have to lower the occupancy, which will degrade performance.

So the bandwidths can be compared. It would be different if AVX-512 had significantly fewer registers and thus had to use stack memory for local variables and thus access L1 more often. In that case a higher L1 bandwidth compared to GPUs would be warranted.

> > NVIDIA and ATI have been designing throughput architectures
> > for ages. You think Intel can swoop in with a "superior
> > cache hierarchy" to bend the laws of computing and physics?
> > Intel's best hope for Knights Landing is to keep
> > x86 relevant in the HPC market. Their process technology
> > might give them an advantage, as well as the reasonable
> > single-threaded scalar performance, but if providing more
> > cache bandwidth was the key to success you'd expect
> > NVIDIA and ATI to put more focus on that. By the way, their
> > cache hierarchies are substantially different, but
> > neither is taking a a big overall lead over the other. So I wouldn't expect a third player to suddenly beat
> > them on all fronts by providing a higher L1 bandwidth than what is typically necessary.
>
> While I do believe NVIDIA and ATI do have accumulated experience in optimizing graphics worksets
> that Intel cannot match I also do believe what I have written. More bandwidth cannot hurt. The
> key to overall performance has always been removing bottlenecks. I expect GPUs to keep throwing
> more FPUs at the problem and Intel to try to squeeze better performance from mm2 of silicon.

More bandwidth can definitely hurt. If you have two load ports while one suffices, you're consuming more power and thus you lose performance by having to clock lower or have fewer cores.

> > To recap: a huge L1 bandwidth of one load operation per arithmetic instruction doesn't help
> > if L2 can't sustain it. You're either accessing the same data multiple times, in which case
> > an L0 cache would be more power efficient, or you're accessing lots of unique data, in which
> > case you're L2 bandwidth limited and a second L1 read port doesn't help. Cases in between are
> > rare, and due to other things that cause the IPC to dip below 2, I believe having two L1 load
> > ports doesn't make that much of a difference and may not outweigh the power and area cost.
>
> In reality there are lots of codes inbetween where work can
> be blocked to maximize usage of both L1 and L2 bandwidth.

Maximizing usage of both the L1 and L2 bandwidth doesn't necessarily mean it's a case in between, when both bandwidths are equal. Also, the L2 bandwidth is typically half-duplex and also feeds L1I, so maximizing that bandwidth can mean anything and you may need blocking for optimal performance even with one L1D read port.