Programs do not see bandwidth. Programs only see latency.

By: Chester (lamchester.delete@this.gmail.com), December 8, 2022 11:07 am
Room: Moderated Discussions
Heikki Kultala (heikki.kultala.delete@this.gmail.com) on December 8, 2022 8:26 am wrote:
> Programs perform memory accesses.
> What matters is when a load gets the data it needs.
>
> This is all about latency, not bandwidth.
>
> Bandwidth only matters when it runs low and it makes the latency grow.
>
> CPUs do not have caches to improve bandwidth. They have caches to improve latency.

This is untrue. With increasing core counts you easily run into memory bandwidth issues if your caching strategy is not good. That's especially an issue with well vectorized applications like video encoding. With some specialized applications (y-cruncher, stable diffusion) you can run into DRAM bottlenecks even with a decent L3 hitrate.

The memtest86 number is also completely incorrect. Using AVX loads, you can get over 90 GB/s from L3 with a single core on Zen 2. Zen 4 can pull over 150 GB/s from L3, again with a single core. The single core number really isn't as important as the all-core number, where the 3950X and 7950X can both get over 1 TB/s from L3. Otherwise if L3 bandwidth isn't higher than DRAM bandwidth, you'd struggle to get good scaling with 16 cores because you'd run out of bandwidth.

Latency is always important, yes. But the more threads you load (explicit parallelism), the more bandwidth tends to become important.

> Also, you are probably measuring the bandwidth of all the memory channels, but only half
> of the L3 cache. There are two totally separate 32 MiB L3 caches in Ryzen 7960X.
>
> If you are performing your accesses from single thread, or few
> threads of same die, you are only using half of the L3 cache.

< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
What happens when DRAM has more bandwidth than Layer 3 cache?Etienne2022/12/08 06:20 AM
  What happens when DRAM has more bandwidth than Layer 3 cache?Simon Farnsworth2022/12/08 07:22 AM
    bandwidth*delay productMichael S2022/12/08 08:06 AM
    What happens when DRAM has more bandwidth than Layer 3 cache?---2022/12/08 10:44 AM
      What happens when DRAM has more bandwidth than Layer 3 cache?blaine2022/12/08 05:07 PM
  What happens when DRAM has more bandwidth than Layer 3 cache?Michael S2022/12/08 07:32 AM
    What happens when DRAM has more bandwidth than Layer 3 cache?Etienne2022/12/08 08:05 AM
      What happens when DRAM has more bandwidth than Layer 3 cache?Michael S2022/12/08 08:13 AM
        What happens when DRAM has more bandwidth than Layer 3 cache?Etienne2022/12/08 01:56 PM
  What happens when DRAM has more bandwidth than Layer 3 cache?Peter E. fry2022/12/08 08:20 AM
  Programs do not see bandwidth. Programs only see latency. Heikki Kultala2022/12/08 08:26 AM
    Programs do not see bandwidth. Programs only see latency. Chester2022/12/08 11:07 AM
  What happens when DRAM has more bandwidth than Layer 3 cache?Doug S2022/12/08 09:31 AM
  What happens when DRAM has more bandwidth than Layer 3 cache?---2022/12/08 10:32 AM
    What happens when DRAM has more bandwidth than Layer 3 cache?Michael S2022/12/08 02:42 PM
      What happens when DRAM has more bandwidth than Layer 3 cache?---2022/12/08 03:54 PM
        What happens when DRAM has more bandwidth than Layer 3 cache?Anon2022/12/20 06:46 AM
  What happens when DRAM has more bandwidth than Layer 3 cache?Andrey2022/12/08 03:10 PM
    What happens when DRAM has more bandwidth than Layer 3 cache?Etienne2022/12/14 03:20 AM
  What happens when DRAM has more bandwidth than Layer 3 cache?Gionatan Danti2022/12/09 12:31 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell tangerine? 🍊