Michael S (already5chosen.delete@this.yahoo.com) on January 20, 2014 4:37 pm wrote:
> Nicolas Capens (nicolas.capens.delete@this.gmail.com) on January 20, 2014 3:48 pm wrote:
> >
> > One load per FMA is not necessary. NVIDIA's GK110 architecture can sustain its theoretical DPFLOPS
> > rating in DGEMM, with two DPFMA units and just one load unit. It even achieves 45% of its peak performance
> > in SGEMM with six SPFMA units and one load unit. So Haswell's two load units suffice to supply all
> > four arithmetic ports with memory operands. You need Hyper-Threading to help hide the latency of
> > Haswell's 5-cycle FMA, but Xeon Phi has a 4-cycle latency, twice the registers, and 4-way SMT. So
> > one load port would suffice for KNL to sustain a peak IPC of 2 instructions.
> >
>
> IPC is an irrelevant metric. DFLOPs/clock is what matters.

Actually performance/Watt is all that matters (which equates to effective DFLOPS/Watt for DGEMM). Trying to sustain the highest DFLOPS/clock at all time does not lead to the best performance/Watt.

> Assuming that David is correct about dual-FMA (which I personally am not going to believe
> before it officially confirmed by Intel) show me how you get 24 DFLOPs/clock (=75% utilization)
> in the inner DGEMM loop without second load pipeline. If 24 DFLOPs/clock are too hard
> then do at least 21 DFLOPs/clock, 66% utilization is also fine with me.

What I'm proposing is a single-read-port L1 cache with a dual-ported L0 cache. Hence more than half of the instructions can have a load operand, as long as some of them reference the same memory. This lowers register pressure, which reduces the number of cycles wasted on spill/restore instructions, and leaves more registers available for intermediate results.

So getting back to performance/Watt, I believe the L0 cache would save enough power by reducing the number of L1 accesses and reducing the register file complexity, that you can achieve better performance/Watt overall even if in a few cases you do want two unique L1 accesses in the same cycle and instead you have to spread it over two cycles. Stalls can and do happen due to other cache misses as well, so it doesn't make much of a difference to stall due to a second L0 miss every now and then either. Stalls are not a complete loss either. The CPU can use them to prefetch more data and prevent a stall at a later point.

So overall I don't think an L0 cache would lower performance significantly over a dual-ported L1 cache, while the power savings could be more significant.