sr (nobody.delete@this.nowhere.com) on September 8, 2021 11:35 am wrote:
> Hugo Décharnes (hdecharn.delete@this.outlook.fr) on September 8, 2021 10:46 am wrote:
> > You just repeated what I said, that is, VIVT is not the way to go.
> >
> > BTW, L2 access time is critical, and TLB coverage (not accuracy) does not compensate, in that most
> > workload are bounded by the cache latency, not the TLB coverage. Yes, VIVPT enables to move TLB away
> > from the time-critical load-return path, enabling bigger TLBs. But you must reduce the L2 hit latency
> > as much as possible. We are talking about percents of performance for a typical big core.
>
> Point for VIVT L1 is that there's zero need to burn power for TLB translation for L1 hit.
> Data already in cache can accessed and verified without need to translate address. As CPU
> core can do many L1-operations per cycle at least for L1-data TLB needs to be multiported,
> where instead L2 can service only one request per clock so single-ported TLB will be fine.

The claim that a highly performant TLB must be multi-ported is a myth.
The theory is explained here: https://eprints.soton.ac.uk/347147/1/__userfiles.soton.ac.uk_Users_spd _mydesktop _MALEC.pdf

The essential insight is that
- most memory requests are strongly clustered to a single page or a few pages
- to the extent that requests are spread over many pages, that same code is probably not exactly latency essential to the single cycle level

----------------------

It will (of course) come as no surprise that this is exactly what Apple implements.
The Apple D TLB is single-ported, supporting 4 piggyback ports. Hence a single TLB request can service up to four memory requests (read and write mix) in a single cycle.

The TLB is (as best my experimental probing can tell) preceded by four queues, defined by the lowest two bits of the virtual page, and holding up to three or four entries.
Hence the flow is that memory requests (for M1 there can be up to 4 per cycle) are sorted into these queues and, as a practical matter, this pretty much gathers together multiple requests targeting the same page. Servicing queues appears to be either by oldest queue or round robin (my guess is round robin is easier and no practical difference in performance).

Clearly you can create a pathological stream that will behave badly given this design, running at 1 lookup per cycle (I did so as part of my tests). But with almost any realistic pattern, you get three or four lookups per cycle, at a worst case of one to three cycle latency as the four different queues are serviced, and with up to three elements waiting in a queue, all serviced together. I could definitely hit cases where the average number of items serviced was around 2.8 per cycle (ie a mix of pages each cycle, so three same-page references being sorted to the same queue and then being serviced together).

(The same sort of insights [requests cluster to individual cache lines] make naive cache banking work much less well than you might expect, because multiple requests in a given cycle land up routing to the same line.
Apple use the same sort of design for the D-cache, split into two banks by the lowest bit of the cache lineID, with piggyback ports allowing a single line lookup to service multiple requests, and with the two banks providing effectively a dual-porting.)

---------------

For the I-cache, if you care, the design is rather different, with both the physical page number and the cache linedID are preserved from one cycle to the next in temporary register storage, so that most cycles both their timing hit and energy hit can be avoided.
Once again, of course, it works because of strong locality (even more so with 16kiB pages).

Finally I cannot figure out how Apple bypasses D- way prediction, but they don't seem to need it -- at least even the most random stream of addresses (in terms of hitting different ways) that I could generate never seemed to cause even a hiccup in terms of invalid way prediction. I have a few hypotheses, but no strong leads, and no patents suggest an explanation.
On the I- side it's easier -- way info is stored in various appropriate places, like, for example, the Next Fetch Predictor -- and this is described in patents.