Michael S (already5chosen.delete@this.yahoo.com) on January 22, 2014 8:17 am wrote:
> Nicolas Capens (nicolas.capens.delete@this.gmail.com) on January 21, 2014 10:07 pm wrote:
> > Michael S (already5chosen.delete@this.yahoo.com) on January 20, 2014 4:37 pm wrote:
> > > Nicolas Capens (nicolas.capens.delete@this.gmail.com) on January 20, 2014 3:48 pm wrote:
> > > > one load port would suffice for KNL to sustain a peak IPC of 2 instructions.
> > >
> > > IPC is an irrelevant metric. DFLOPs/clock is what matters.
> >
> > Actually performance/Watt is all that matters (which equates to effective DFLOPS/Watt for DGEMM). Trying
> > to sustain the highest DFLOPS/clock at all time does not lead to the best performance/Watt.
>
> Bullshit.
> If anything, DFLOPS/yearly_TCO is good metric, not DFLOPS/Watt, but even that not always true.

I thought you said DFLOPS/clock is what matters... ?

Seriously now, it's obvious that TCO and absolute performance matter too, but both are increasingly dominated by power consumption. It's relatively easy to crank up absolute performance by adding more cores. What will stop you more than anything else is power consumption. Likewise to decrease TCO you have to lower power consumption so you can have more FLOPS relatively cheaply. The best HPC products on the market today have the best or close to the best performance/Watt.

But let's not lose track of the IPC discussion that started this. To achieve good FLOPS/clock or FLOPS/TCO or FLOPS/Watt, you need an architecture that has a reasonable chance of sustaining its peak IPC. Obviously having two vector units would be senseless if there's a marginal chance of dual-issue. But reversely you also shouldn't take every possible measure to ensure the IPC is maximized, because that becomes more expensive than what you gain. The best result, for all the metrics you might care about, lies somewhere in the middle.

Given that GPUs have a 2:1 ratio of ALU:MEM at best, I think it's very reasonable for KNL to have just one L1 load port, and some sort of L0 structure with two load ports to allow operand reuse. I'm not saying this is what KNL must look like. I'm saying there's a chance David is wrong about KNL having two L1 load ports.

> More importantly, L1
> cache access and register read access are not that big parts of the total power consumption.

Indeed they are not, but when every other component is optimized for power, they start to stand out. So you shouldn't forego an easy optimization of the L1 and RF power consumption. GPU manufacturers have made substantial changes to the register file and caches with every generation. Project Echelon even goes as far as adding a 16-entry operand register file, shared between 4 threads. So its reuse rate is quite low, but they still consider it important enough.

> > > Assuming that David is correct about dual-FMA (which I personally am not going to believe
> > > before it officially confirmed by Intel) show me how you get 24 DFLOPs/clock (=75% utilization)
> > > in the inner DGEMM loop without second load pipeline. If 24 DFLOPs/clock are too hard
> > > then do at least 21 DFLOPs/clock, 66% utilization is also fine with me.
> >
> > What I'm proposing is a single-read-port L1 cache with a dual-ported L0 cache. Hence more
> > than half of the instructions can have a load operand, as long as some of them reference the
> > same memory. This lowers register pressure, which reduces the number of cycles wasted on spill/restore
> > instructions, and leaves more registers available for intermediate results.
> >
>
> I asked for your variant of DGEMM inner loop.

With all due respect I don't have the time to concern myself with writing fine-tuned code just because you "asked" for it. If you have what you believe to be already near-optimal code then I might take a look at it to see if it can be optimized specifically for a 2:1 load ratio. Alternatively we could also look at how GPUs achieve a high DGEMM score with a 2:1 ratio and see where KNL may have difficulties/opportunities. Either way it seems a very reasonable assumption that with an L0 cache you can take advantage of not having to store reused source operands in registers.

Also, there's more to computing than DGEMM. GPU manufacturers are even shifting away from the 2:1 ratio because apparently that offers a better cost/gain ratio across all workloads.