Maynard Handley (name99.delete@this.name99.org) on August 18, 2018 4:22 pm wrote:
[snip]
> Oops left out one more idea. Once you have a machinery to handle remote atomics AND you own
> the entire stack, you can do some crazy things. For example consider the problem of zeroing
> an entire page. Obviously doing this on the main CPU sucks, but doing it on a secondary thread
> (and thus the machinery of callbacks and waits) also sucks. But what about this?
> - you extend the cache protocol to include invalidate messages that describe not just a single line but an address
> plus a count of lines (say up to 256 or so, whatever is necessary to cover a page seems about right)
> - you offer an atomic remote zero (or remote fill) instruction that is given a line address
> plus count. The first thing it does is flush the line from all other caches, then it marks
> the set of lines locally as in some sort of intermediate state, then, fast as possible, it
> starts zeroing each line and, once zeroed, it goes back to a standard "valid" state.

All zero cache lines (and pages) compress extremely well. I think it
makes more sense to use metadata to indicate such compression (at page
granularity one could steal a state in PTEs, which further avoids excess
cost in metadata fetching) and avoid memory read entirely. The lines can
be zero filled around the first write.

There are many microarchitectural possibilities for improving performance or
energy efficiency, especially when one controls the system software (compiler,
standard libraries, OS). It is difficult for an outsider (especially one without
hardware design experience) to know what promising academic proposals are
rejected because they have weaknesses with some existing software or general
software practice (or otherwise have variable effectiveness, potentially
penalizing some workloads), because they are fundamentally flawed, because they
require significant design changes/commitments (or otherwise encounter friction
in established organizations), or for some other reasons.

Skewed associativity in a physically addressed L2 cache seems like a no-brainer
(the aliasing and latency from indexing complexity would seem not to be
significant; the area and power cost from multiple indexing for tags may be
an issue as might the inability to partially pre-index the data array speculating
on a hit), but skewed associativity is not generally used (possibly not even
for TLBs where aliasing is not an issue).