David Kanter (dkanter.delete@this.realworldtech.com) on January 8, 2014 4:53 pm wrote:
> Gabriele Svelto (gabriele.svelto.delete@this.gmail.com) on January 8, 2014 2:51 pm wrote:
> > Nicolas Capens (nicolas.capens.delete@this.gmail.com) on January 8, 2014 2:33 pm wrote:
> > > Occam's razor is about making the fewest assumptions. You're making two assumptions:
> > > 1) The L0 cache is very complex.
> > > 2) The Intel compiler is stupid.
> > >
> > > Firstly an L0 cache isn't complex. Only a few entries are needed and there's only ~20 bits worth of address
> > > encoding to test for a match (http://www.realworldtech.com/forum/?threadid=138897&curpostid=139072).
> >
> > An extra cache layer which must be coherent with another with 72 cores isn't complex and requires
> > only ~20 bits worth of address? Seriously? With no other processor ever implementing anything
> > even remotely similar? It's existence sounds like a HUGE assumption to make considering there's
> > not even the slightest hint of it anywhere (Care to point to some patent? A diagram? Another processor
> > doing something similar? Anything at all that's not pulled out of thin air?).
> >
> > > Secondly it is one of the easiest optimizations to avoid reading memory operands multiple times. It has
> > > to be a deliberate choice (http://www.realworldtech.com/forum/?threadid=138897&curpostid=138966).
> >
> > Right, because compilers never have bugs and never emit
> > sub-optimal code for unreleased architectures. Never.
>
> It's clear that this L0 cache is a half baked idea predicated on some particularly terrible assumptions.
>
> First, register files are lower power than caches...

That is not universally true. Caches have fewer transistors per bit can operate at a longer latency, so a sufficiently small cache is more power efficient than a register file. Also, an L1 access requires additional power for the address generation/translation, but the L0 could use symbolic indexing.

KNC consumes only 40% more power for an instruction which acceses L1, versus an instruction which only uses registers: http://www.eecs.harvard.edu/~shao/papers/shao2013-islped.pdf Thus it's easy to see that an L0 cache with only a few entries would be more power efficient than both the L1 cache and the register file. Keeping a single L1 load port for KNL and even reducing the power consumption over register accesses with an L0 cache would be a very significant win Intel is not likely to overlook.

> somebody already pointed out that tagged vs.
> directly addressed structures suck. Nicolas is assuming otherwise, so that's a bad sign.

KNC's L1 is 8-way set associative, which consumes at least as much power as an 8-entry fully associative cache, just for the tag matching. The L0 cache I'm suggesting would need 2 entries per thread, maybe 4.

> Second, virtually addressed caches are a stupid idea. This should be evident from the fact that almost
> all high performance CPU caches are VIPT or PIPT. Also, how will this interact with threads?

The L0 cache I'm proposing would not be virtually addressed. It would use the instruction's encoding for the memory operand (i.e. the SIB byte, its EVEX extension bits, and part of the displacement bits). So you're 'symbolically' matching a memory operand that has been loaded before (e.g. [rax+r14*4] in Eric's example code). The entry would be invalidated if rax or r14 is written to, or when the entry's corresponding L1 cache line is modified/invalidated.

> Third, as Michael S pointed out - this is insanely brittle.

No more brittle than anything else happening in the L1 cache and LSU. It's basically a symbolically indexed line buffer with multiple entries and two read ports (http://caps.cs.binghamton.edu/papers/ghose_islped_1999.pdf). Shouldn't be too much of a challenge for Intel to design something like this.

> Fourth, the coherency implications are not considered.

That's because there are no coherency implications worth mentioning. It can be kept coherent with L1 at all time. Everything the LSU already does to keep L1 coherent, it can apply to L0 as well.

> Fifth, why would you want to create ANOTHER structure that stores must check?

The L0 cache would be read-only.

> I 100% agree with Gabriele here - the most logical explanation is that KNL
> isn't going to be released for a whole year, so the compiler is immature.

It's the same compiler since Larrabee. It's an utterly trivial optimization to use a register that already contains the data you need, instead of loading it from memory again. So this is a deliberate change for KNL and an indication that Intel *wants* to use memory operands instead of registers and thus it is highly likely that they are more power efficient. Which requires something like this L0 cache.

> I'll further add that I'm willing to wager that I'm correct about the 2 load
> pipelines, and that Nicolas is wrong about a virtually addressed L0 cache.

What exactly do you mean by 2 load pipelines? If you mean that both vector units would be able to use a memory operand in the same cycle, then yes, based on Eric's code that seems to be a necessity. However due to the duplicate loading of the same memory this can be provided by a dual-ported L0 cache, while the L1 cache itself only requires a single load port.