Gabriele Svelto (gabriele.svelto.delete@this.gmail.com) on January 9, 2014 2:41 am wrote:
> Nicolas Capens (nicolas.capens.delete@this.gmail.com) on January 9, 2014 12:40 am wrote:
> > Despite me calling it an L0 cache, it's not like any of the other cache "layers". It is read-only,
> > and keeping it coherent with L1 is easy since you can store the corresponding L1 cache line index
> > with each entry, and invalidate them in the same cycle the L1 cache line is invalidated.
>
> So more logic and more wires to move the data too since you're store-path now goes both
> to the L1 cache and this L0 in parallel. You probably need to hook it up to your store-queue
> too for effective load-to-store forwarding. Do you consider this simple?

The L0 cache would save requiring a second load port on L1, reduce the port count on the register file, and is tiny itself. So I think overall you could save more logic and wires than what you need for the L0 cache. Also, it would basically be part of the LSU itself, and it could potentially share things with the store queue and/or be part of a gather unit which can access multiple cache lines per cycle.

Do I consider this simple? In terms of saving power, yes, I believe this could be simpler than requiring two L1 load ports and many register load ports. I don't think it's simple in terms of design effort though, but that has never stopped Intel from implementing the most power efficient option.

> > Sure. Having multiple line buffers is fairly similar, and it saves
> > power: http://caps.cs.binghamton.edu/papers/ghose_islped_1999.pdf
>
> From your link:
>
> "The hardware augmentations to the basic set–associative cache are as follows. First, we need four latches
> to hold the number of the four most recently accessed sets. Second, four more comparators are needed to compare
> the set number field of the current access against the set numbers of the lines sitting in the four line buffers.
> Third, if no line buffer hit occur, the data from the set selected by the current address applied to the cache
> must be retrieved into one of the four line buffers. Consequently, on a line buffer miss, a victim must be
> selected from one of the four line buffers. The need to write back an updated line buffer’s content into
> the tag and data arrays, when the line buffer gets selected as a victim, is avoided by writing through updates
> to the line buffer into the corresponding data array on write accesses that result in a line buffer hit. Along
> with these, a multiplexing facility is needed to direct the outputs of the tag and data arrays into the victim
> line buffer. In this case, the four line buffers effectively make up a 4–entry fully–associative write–through
> cache, which is accessed in parallel with the normal cache without any prolongation in the cycle time or without
> any impact on the normal cache access pipeline."
>
> Maybe this sounds simple to you, it doesn't sound simple to me

Whether it's simple to build it irrelevant. The paper concludes it saves power.

> and considering nobody implemented
> anything even remotely similar also tells me it's probably not a good idea power-wise.

Single line buffers appear to be a very common thing, and store-to-load forwarding is also a sort of fully associative caching which avoids L1 accesses, as is write combining. Intel's L0 instruction cache can also be considered similar. And all of these examples save power. So I don't see why an L0 read-only data cache with symbolic fully associative indexing would be much of a challenge for Intel.

> > I don't think you're grasping just how much this code is
> > deviating from the norm. It is not reusing any data
> > that has already been loaded into a register, anywhere, in
> > either example! That is usually one of the fundamental
> > tasks of register allocation. Any course in compiler design covers this as a basic optimization.
> >
> > So this isn't some heuristic making the wrong decision in a corner case. It requires no advanced
> > analysis or deeper knowledge of the source code to perform this trivial and unambiguous optimization.
>
> I've worked on a C/C++ compiler, a .NET JIT compiler and a Java JIT compiler. I'm not the most experienced
> compiler writer around but I tend to know my stuff and I think you're missing an important point.
>
> The pattern you're seeing can be quite common for C/C++ code when mixing pointers to the same types: the
> compiler must assume all those pointers alias and hence will not promote loads into registers as the value
> of those loads might be made stale by preceding stores. As others in this thread have figured out already
> adding the 'restrict' keyword in the example function changes code generation significantly:
>
> http://www.realworldtech.com/forum/?threadid=138897&curpostid=139110

Except in the old (second) example there are no stores which would cause the compiler to have to load from memory again! Yet there are redundant loads. And even in that latest example the number of load operations is the same both with and without the restrict keyword, they're just ordered differently. So that doesn't make the existence of an L0 cache any less likely.