Ricardo B (ricardo.b.delete@this.xxxxx.xx) on January 8, 2014 2:13 am wrote:
> Nicolas Capens (nicolas.capens.delete@this.gmail.com) on January 7, 2014 10:28 pm wrote:
>
> > For Eric's example it would be ample to have just two entries to eliminate enough
> > L1 accesses to only require it to have a single load port. Sure, technically it's
> > still a CAM with just two entries, but I don't think that's expensive at all.
> >
> > You need 5 + 5 bits for the registers since AVX-512 has 32 registers. But no need to go with physical
> > register addresses (which adds 2 bits for each register), just use the 2 bit thread index for both.
> > Then 2 bits for the scale, and you can use a limited number of bits for the displacement. There's
> > no need to support a full 32-bit displacement. Something like 12 bits may suffice, and anything with
> > a larger displacement just wouldn't use the L0 cache. I'd even make it support only multiple-of-64
> > displacements, saving 6 bits. Thus only 20 bits in total would have to be compared.
>
> It's still a considerable amount of logic.
> Logic you're activating for every (vector) load, despite most of the cases
> it's useless, as a few entries cache would be constantly thrashed.

You have to compare it to the alternatives. Either you need a second L1 load port and activate a second AGU, which adds more logic and consumes more power than the L0 cache, or you need a register file with more port, again requiring more logic and power.

The L0 cache would be teensy. But every hit would avoid an expensive L1 access or register read. Best case you can even avoid two of those per cycle. And when the compiler is instructed to liberally use memory accesses like in Eric's examples, there will be lots of hits.

> Also, in Eric's example above[1] there's a lot of accesses that would not be cached by a 2 entry L0.
> [1] http://www.realworldtech.com/forum/?threadid=138897&curpostid=138933

You're having the wrong expectations. The L0's purpose is to provide enough of the repeated memory operands to only require one L1 load port. With just 2 entries for the L0, that goal is achieved and the L0 itself is tiny enough to have negligible power consumption. With more entries it would service a few more of the repeated memory operands, but it could start consuming more power than it saves. So it's a balancing act, and a fairly low hit rate (compared to expectations for an L1 cache) is perfectly fine. For this particular example an L0 hit rate of just 9% would suffice to save the complexity of a second L1 port and more register file ports and theoretically sustain an IPC of 2. But with just two entries the hit rate is actually 35%, saving having to access the L1 cache in that many cases.

So this tiny little structure would create a threefold saving in power consumption.

> I think the code is a compiler fluke.

It's the 'worst' code I've seen in a decade. Seriously, it only takes one glance at the output to realize something is very wrong. It would be very shameful for Intel to ship a compiler like this, even if it's an early version (AVX-512 isn't that different from LRBni so it wouldn't count as an early version anyway). So it would make a lot more sense if this wasn't bad code at all, but instead taking advantage of a hardware feature which makes things faster and more power efficient.

> I really can't see how you can make a cache that's faster and cheaper, energy wise, than a register access.

You're probably thinking in terms of a typical 32 kB L1 cache access which requires address generation/translation and all that and a small scalar register file. Instead the L0 cache would be perhaps 512 bytes and two read ports versus an 8 kB vector register files with many read and write ports. I don't think it's a big stretch that the L0 would be more power efficient.