Cache associativity

David Kanter (dkanter@realworldtech.com) on 8/16/05 wrote:
>
>It sounds like you really prefer highly associative caches.

I definitely do. With some nice FP loops you may be able
to work around direct-mapped caches, and argue that you
can make the cache sufficiently faster that it's worth
the hoops the simpler/faster hardware makes you jump
through. I suspect that embedded people might have the
same argument.

With general-purpose programming, the pain is just too
big. You get a lot of cache misses due to way contention.

There's tons of data on this. If you want spec D$ miss
rates, see for example

http://www.cs.wisc.edu/multifacet/misc/spec2000cache-data/new_tables/specint_64-amean.tab

which says that for D$ there's about 25% more misses
for a direct-mapped cache than for a two-way one in the
L1 cache size range (the difference is even bigger for I$,
but the miss numbers there are smaller, of course).

And that's ignoring the worst case - that's just average.

So I personally would want at least 4-way in the L1, and
as much as possible in the L2. And if full associativity
doesn't work out, then some mixing in of other bits to hash
the thing around to avoid common alignment-induced "hot
ways", that sounds like a good idea to me (people seem to
call it "pseudo-associative").

One of the things I personally like about highly associative
caches is the graceful degradation. I'd much rather
have a system that tends to slow down more gracefully than
fall of a steep cliff ("glass jaw") when something bad
happens. A direct-mapped cache basically is asking for
trouble - it may perform fine "on average", but then it has
nasty situations where it really sucks.

Me, I'll take "consistently good" over "really really good
if all the planets align correctly" any day. When it comes
to caches, that means that I'd much rather take a two-
cycle L1 that is big and has high associativity over a
single-cycle small one. Even if the single-cycle one then
runs like a bat out of hell when things go the rigt way.

I guess this is all the same argument that make me prefer
a P-M over a P4. "Plodding and dependable workhorse" is
better than a sprinter that hits a brick wall every once in
a while.

Linus