bandwidth*delay product

By: Michael S (already5chosen.delete@this.yahoo.com), December 8, 2022 8:06 am
Room: Moderated Discussions
Simon Farnsworth (simon.delete@this.farnz.org.uk) on December 8, 2022 7:22 am wrote:
> Etienne (etienne_lorrain.delete@this.yahoo.fr) on December 8, 2022 6:20 am wrote:
> > Looks like my AMD Ryzen 9 7950x has a L3 cache bandwidth of 63.9 GB/s, my current DRAM DDR5
> > has either 49.6 GB/s (Jedec) or 52.5 GB/s (AMD Expo) measured by memtest86 UEFI.
> > It seems some companies are increasing DRAM bandwidth: 8Gbps DDR5.
> >
> > I assume latency to L3 cache is still probably better than latency to
> > DRAM, but in simple terms, do we still need L3 cache in processors?
>
> A critical difference between peak DRAM throughput, and L3 throughput, is that L3 throughput is independent
> of access pattern (as long as you never leave L3, of course) - you get the same throughput from L3 whether
> you read cachelines sequentially, or whether you read cachelines in a random order, and you get the same
> throughput when writing whether you write sequentially, or whether you write in a random order. There's
> also no penalty for mixing writes and reads - the timings are the same for read then read another line,
> as for read then write and for write then read and for write then write another line.
>
> DDR5 doesn't offer that - your throughput is lower if you read or write 64 byte chunks at
> random throughput the chip than if you arrange to stay in the same bank group as much as
> possible. There's also a small penalty for mixing reads and writes, so you benefit from L3
> if it lets you do more writes in sequence before switching back to reads or vice-versa.

That's one thing.
Another important thing is that even when your access pattern doesn't suffer from many DRAM page misses/conflict and even when accesses distributed ideally accross four logical channels, you still can achieved near-full bandwidth of dual-channel DDR5 only in special [for client workload] scenarios. Specifically, these scenarios are:
Scenario 1: Several cores access DRAM simultaneously but has luck to not interfere with one another in terms of DRAM page misses.
Scenario 2: One core accesses DRAM in a way that is perfectly predictable by L3 HW prefetch engines.

In all other scenarios you are kept far away from theoretical bandwidth limits because bandwidth*delay product of DDR5 exceeds a number of L2 miss buffers available to any individual core.
5.2 GT/s * 16 B/T * 80 ns / 64B/buffer = 104 buffers
Each Zen3 (and likely Zen4) L2 cache supports 64 outstanding misses at most.

This problem is new for dual-channel desktop/laptop CPUs, but it is well known for almost 2 decades in bigger servers and workstations where both bandwidth and latency are significantly higher.
I think, John McCalpin a.k.a. Dr. Bandwidth wrote an article about it back in 2007 or 2008.


< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
What happens when DRAM has more bandwidth than Layer 3 cache?Etienne2022/12/08 06:20 AM
  What happens when DRAM has more bandwidth than Layer 3 cache?Simon Farnsworth2022/12/08 07:22 AM
    bandwidth*delay productMichael S2022/12/08 08:06 AM
    What happens when DRAM has more bandwidth than Layer 3 cache?---2022/12/08 10:44 AM
      What happens when DRAM has more bandwidth than Layer 3 cache?blaine2022/12/08 05:07 PM
  What happens when DRAM has more bandwidth than Layer 3 cache?Michael S2022/12/08 07:32 AM
    What happens when DRAM has more bandwidth than Layer 3 cache?Etienne2022/12/08 08:05 AM
      What happens when DRAM has more bandwidth than Layer 3 cache?Michael S2022/12/08 08:13 AM
        What happens when DRAM has more bandwidth than Layer 3 cache?Etienne2022/12/08 01:56 PM
  What happens when DRAM has more bandwidth than Layer 3 cache?Peter E. fry2022/12/08 08:20 AM
  Programs do not see bandwidth. Programs only see latency. Heikki Kultala2022/12/08 08:26 AM
    Programs do not see bandwidth. Programs only see latency. Chester2022/12/08 11:07 AM
  What happens when DRAM has more bandwidth than Layer 3 cache?Doug S2022/12/08 09:31 AM
  What happens when DRAM has more bandwidth than Layer 3 cache?---2022/12/08 10:32 AM
    What happens when DRAM has more bandwidth than Layer 3 cache?Michael S2022/12/08 02:42 PM
      What happens when DRAM has more bandwidth than Layer 3 cache?---2022/12/08 03:54 PM
        What happens when DRAM has more bandwidth than Layer 3 cache?Anon2022/12/20 06:46 AM
  What happens when DRAM has more bandwidth than Layer 3 cache?Andrey2022/12/08 03:10 PM
    What happens when DRAM has more bandwidth than Layer 3 cache?Etienne2022/12/14 03:20 AM
  What happens when DRAM has more bandwidth than Layer 3 cache?Gionatan Danti2022/12/09 12:31 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell tangerine? 🍊