lmbench is horribly broken

By: Wilco (Wilco.Dijkstra.delete@this.ntlworld.com), March 18, 2017 5:40 pm
Room: Moderated Discussions
Exophase (exophase.delete@this.gmail.com) on March 18, 2017 4:26 pm wrote:

> Sorry but I'm still having a hard time with this. And I might be missing something still.
> But as far as I understand it with lmbench's thrash_initialize the order in which memory
> accesses and page table accesses are made is random. The stated intention of the test is
> to thrash caching of both data and the page table. It says so in the comments:
>
> * Access a different page each time. This will eventually
> * cause a tlb miss each page. It will also cause maximal
> * thrashing in the cache between the user data stream and
> * the page table entries.
>
> Now if your LLC is large enough to cache the process's entire page table eventually you'll always hit
> regardless of how random the accesses are, but this should only be true if you're not also filling the
> cache with other stuff. And that's only really the case if the data you're loading is hitting the same
> cache lines from different virtual addresses like in your example. That isn't how lmbench is setup.
>
> I really don't think anyone's arguing that memory latency tests should try to strip out the cost
> of cache misses along the way because that's unavoidable. I do actually think that having to go
> to L3 to fill the TLB could be a significant extra cost, far more than the 10 cycles you gave earlier,
> but that's still not nearly as bad as adding an entire second dependent memory access.

Actually once you start missing the LLC you end up doing 2 DRAM accesses just to get the TLB entry plus a third for the actual access. Since every memory access will need a line for the TLB as well as one for the data, a typical LLC can only cache a small number of random accesses: a 2MByte LLC with a 64-byte line size can contain only 16384 random accesses before it misses (both for the TLB entry and data). How large your main TLB is doesn't matter at all, it's always way too small when you randomly access a large memory region.

Basically it doesn't matter how advanced your TLB is, you quickly end up getting the worst case of 2-3 DRAM accesses per load. And given there is no parallelism possible when pointer chasing a single list, other advanced features like hit-under-miss and dual TLB walkers don't make any difference at all.

So it's fine to use lmbench if you wanted to show that no matter how advanced your TLBs and caches are, they totally suck doing completely random accesses. However in the real world applications exhibit a significant amount of spatial locality (even if having lots of TLB misses) and the cache and TLBs are highly optimized to make use of that.

Wilco
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
ARM A73 benchmarksSymmetry2017/03/14 06:24 AM
  ARM A73 benchmarksPer Hesselgren2017/03/14 07:18 AM
    ARM A73 benchmarks-latencyPer Hesselgren2017/03/14 08:58 AM
      ARM A73 benchmarks-latencySymmetry2017/03/14 10:12 AM
        ARM A73 benchmarks-latencyPer Hesselgren2017/03/14 03:54 PM
          ARM A73 benchmarks-latencyWilco2017/03/15 01:45 AM
            ARM A73 benchmarks-latencyPer Hesselgren2017/03/15 02:57 AM
              ARM A73 benchmarks-latencyPer Hesselgren2017/03/15 03:00 AM
                ARM A73 benchmarks-latencyPer Hesselgren2017/03/15 03:01 AM
                  clickable linkMichael S2017/03/15 04:05 AM
            ARM A73 benchmarks-latencyLinus Torvalds2017/03/15 10:05 AM
              ARM A73 benchmarks-latencyIreland2017/03/15 05:02 PM
              ARM A73 benchmarks-latencyGabriele Svelto2017/03/16 03:45 AM
                ARM A73 benchmarks-latencyLinus Torvalds2017/03/16 02:01 PM
                  lmbench is horribly brokenWilco2017/03/16 04:57 PM
                    lmbench is horribly brokenLinus Torvalds2017/03/16 06:49 PM
                      lmbench is horribly brokenLinus Torvalds2017/03/17 01:10 PM
                        lmbench is horribly brokenLinus Torvalds2017/03/17 01:52 PM
                        lmbench is horribly brokenExophase2017/03/17 02:31 PM
                          lmbench is horribly brokenGabriele Svelto2017/03/17 03:20 PM
                          lmbench is horribly brokenLinus Torvalds2017/03/17 05:56 PM
                            lmbench is horribly brokenExophase2017/03/17 06:21 PM
                              lmbench is horribly brokenLinus Torvalds2017/03/17 06:43 PM
                                lmbench is horribly brokenIreland2017/03/17 07:37 PM
                                  lmbench is horribly brokenbakaneko2017/03/18 11:17 AM
                                    lmbench is horribly brokenIreland2017/03/18 12:23 PM
                                      lmbench is horribly brokenanon2017/03/18 07:35 PM
                                      lmbench is horribly brokenbakaneko2017/03/21 08:08 AM
                                        lmbench is horribly brokenIreland2017/03/21 03:14 PM
                                lmbench is horribly brokenGabriele Svelto2017/03/18 04:01 PM
                                  accessing dram RichardC2017/03/18 06:33 PM
                                lmbench is horribly brokenExophase2017/03/18 04:26 PM
                                  lmbench is horribly brokenWilco2017/03/18 05:40 PM
                                    benchmarking reality?Anon2017/03/19 02:29 PM
                                    lmbench is horribly brokenLinus Torvalds2017/03/19 04:25 PM
                                      mea culpa (lmbench is horribly broken)Linus Torvalds2017/03/19 06:05 PM
                                        mea culpa (lmbench is horribly broken)Bill Broadley2017/03/21 01:41 AM
                                          mea culpa (lmbench is horribly broken)Linus Torvalds2017/03/21 09:01 AM
                                            mea culpa (lmbench is horribly broken)Linus Torvalds2017/03/21 11:14 AM
                                            mea culpa (lmbench is horribly broken)Linus Torvalds2017/03/21 05:03 PM
                                              mea culpa (lmbench is horribly broken)Etienne2017/03/22 04:37 AM
                                              mea culpa (lmbench is horribly broken)Tim McCaffrey2017/03/22 08:54 AM
                                                mea culpa (lmbench is horribly broken)Tim McCaffrey2017/03/22 09:34 AM
                                                mea culpa (lmbench is horribly broken)Linus Torvalds2017/03/22 10:35 AM
                                                  mea culpa (lmbench is horribly broken)Ireland2017/03/22 12:11 PM
                                                    mea culpa (lmbench is horribly broken)Ireland2017/03/22 12:26 PM
                                                    mea culpa (lmbench is horribly broken)rwessel2017/03/22 03:03 PM
                                                      mea culpa (lmbench is horribly broken)Ireland2017/03/22 03:35 PM
                                                  mea culpa (lmbench is horribly broken)Linus Torvalds2017/03/22 01:35 PM
                                                    mea culpa (lmbench is horribly broken)Gabriele Svelto2017/03/23 08:05 AM
                                                      mea culpa (lmbench is horribly broken)Linus Torvalds2017/03/23 10:43 AM
                                                        mea culpa (lmbench is horribly broken)Gabriele Svelto2017/03/23 01:56 PM
                                                          mea culpa (lmbench is horribly broken)Ireland2017/03/23 02:36 PM
                                                  mea culpa (lmbench is horribly broken)Travis2017/03/22 01:38 PM
                                              mea culpa (lmbench is horribly broken)anon2017/03/22 07:22 PM
                                                mea culpa (lmbench is horribly broken)Travis2017/03/22 08:57 PM
                                                  mea culpa (lmbench is horribly broken)anon2017/03/23 12:44 AM
                                                    mea culpa (lmbench is horribly broken)Michael S2017/03/23 05:59 PM
                                                      mea culpa (lmbench is horribly broken)Travis2017/03/23 09:03 PM
                                                    power8 numbersoctoploid2017/03/24 11:47 PM
                                                      power8 numbers stride=128octoploid2017/03/25 04:36 AM
                                                        power8 numbers stride=128Linus Torvalds2017/03/25 10:50 AM
                                                          power8 numbers stride=128Gabriele Svelto2017/03/25 11:27 PM
                                              mea culpa (lmbench is horribly broken)anon2017/03/23 01:14 AM
                                                mea culpa (lmbench is horribly broken)Linus Torvalds2017/03/23 11:22 AM
                                                  Thank you. Associativity misses explain it.anon2017/03/23 10:48 PM
                                                    Thank you. Associativity misses explain it.Linus Torvalds2017/03/24 01:26 PM
                                                      Thank you. Associativity misses explain it.Travis2017/03/24 10:01 PM
                                                        thanks should read "but if it is any TYPE of mix" (NT)Travis2017/03/24 10:02 PM
                                                        Thank you. Associativity misses explain it.Linus Torvalds2017/03/25 12:10 PM
                                                          Thank you. Associativity misses explain it.Travis2017/03/25 04:08 PM
                                                            Thank you. Associativity misses explain it.Linus Torvalds2017/03/26 10:27 AM
                                  lmbench is horribly brokenLinus Torvalds2017/03/19 03:51 PM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?