anon (anon.delete@this.anon.com) on January 16, 2014 10:01 pm wrote:
> Nicolas Capens (nicolas.capens.delete@this.gmail.com) on January 16, 2014 2:05 pm wrote:
> > The advantage of the symbolically indexed L0 cache over a line buffer would be that you don't have
> > to compute a virtual and physical address.
>
> You actually *do* have to compute them to fill it. You have to compare to test it. And it can't hold stores.

That has nothing to do with the L0 cache. When it's filled, it means a miss happened and you need to access L1 and thus the address is needed anyway. You do not need the address for an L0 hit. Saving power on L0 hits is the whole point.

> And with a poor hit rate, where are your calculations to show that the extra power cost is worthwhile?

The AVX-512 code that has been posted here would have lots of L0 hits with only two entries. Each of those hits saves all the power required to calculate the virtual and physical addresses and access the L1 cache and fetch and compare tags.

Here are some papers with power calculations for similar caches below the L1 cache:



> You also ignored the question about a cache in front of the register file for frequently accessed registers.

I'm sure that can be useful, as some GPUs have register file caches or similar structures already. However, this discussion is about the number of L1 read ports and finding an explanation for why the AVX-512 code accesses the same memory multiple times.

> > basically just a few narrow comparators.
>
> Yes, that is the basic part. The complicated part is intercepting all changes
> to any cached registers and invalidating; intercepting all TLB and L1 evictions
> and invalidating; and intercepting all aliasing stores and invalidating.

How is that complicated? The indexes for destination registers are 4-bit each. With two L0 entries you only need four 4-bit comparators to invalidate them if the base or index register is written to. For L1 evictions you have to compare the 9-bit line index of each entry. Aliasing stores can be treated the same as a line eviction so don't need any extra work. So all this is really very cheap.

> > I find it
> > hard to believe an even better alternative would exist.
> > But feel free to present them.
>
> The traditional register file + L1 cache, and none of the complexity of managing the symbolic
> cache, is a good place to start looking. Then add a register file cache if the main regfile becomes
> too large to be efficient. So now feel free to present why this L0 cache is better.

For the cost of a few very narrow comparators, the L0 saves: register file accesses for the base and index register, computing the full virtual address, translating to the physical address through a TLB lookup, accessing the L1 array, accessing the tag array, and comparing the tag against the physical address.

> If this is still not enough, I would say something which is much better thought-out in terms of
> indexing and invalidation would be superior to this L0 idea, e.g., Intel's "signature buffer".
>
> http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.78.601&rep=rep1&type=pdf

How is that superior? It's the exact same idea!

> I would suspect that mid-size vector code will look quite
> a bit different, particularly when you have enough registers to hold repeatedly used vectors.

Too bad, because Intel's compiler isn't reusing the registers even though there are enough. It indicates they've done something to the memory hierarchy, not the register file. If anything the L0 cache just lowers the register file pressure and thus it can be simplified.

> > > There is really no good evidence for this L0 cache. Any handwaved advantage
> > > of it can easily be balanced with a handwaved disadvantage. I can't see why
> > > there is any point in continuing to talk about it without more data.
> >
> > Of course there's no evidence of an L0 cache. We're speculating about a brand new architecture.
>
> That's fine, but when it gets to the point of arguing between one another with unevidenced claims of better efficency,
> and asserting that the complexity will cause no implementation issues, then it becomes pointless.

Why would that be pointless? You have shown no indication that the proposed L0 cache has any complexity that would cause issues. Or do you really need to see "evidence" that a handful of narrow comparators are really cheap compared to all the logic required for an L1 access? If there's anything I've missed to make this work and keep it power efficient, let me know and we can discuss that further. But don't tell me the current argument is pointless without any counter-arguments.