lmbench is horribly broken

By: Wilco (Wilco.Dijkstra.delete@this.ntlworld.com), March 18, 2017 5:40 pm
Room: Moderated Discussions
Exophase (exophase.delete@this.gmail.com) on March 18, 2017 4:26 pm wrote:

> Sorry but I'm still having a hard time with this. And I might be missing something still.
> But as far as I understand it with lmbench's thrash_initialize the order in which memory
> accesses and page table accesses are made is random. The stated intention of the test is
> to thrash caching of both data and the page table. It says so in the comments:
>
> * Access a different page each time. This will eventually
> * cause a tlb miss each page. It will also cause maximal
> * thrashing in the cache between the user data stream and
> * the page table entries.
>
> Now if your LLC is large enough to cache the process's entire page table eventually you'll always hit
> regardless of how random the accesses are, but this should only be true if you're not also filling the
> cache with other stuff. And that's only really the case if the data you're loading is hitting the same
> cache lines from different virtual addresses like in your example. That isn't how lmbench is setup.
>
> I really don't think anyone's arguing that memory latency tests should try to strip out the cost
> of cache misses along the way because that's unavoidable. I do actually think that having to go
> to L3 to fill the TLB could be a significant extra cost, far more than the 10 cycles you gave earlier,
> but that's still not nearly as bad as adding an entire second dependent memory access.

Actually once you start missing the LLC you end up doing 2 DRAM accesses just to get the TLB entry plus a third for the actual access. Since every memory access will need a line for the TLB as well as one for the data, a typical LLC can only cache a small number of random accesses: a 2MByte LLC with a 64-byte line size can contain only 16384 random accesses before it misses (both for the TLB entry and data). How large your main TLB is doesn't matter at all, it's always way too small when you randomly access a large memory region.

Basically it doesn't matter how advanced your TLB is, you quickly end up getting the worst case of 2-3 DRAM accesses per load. And given there is no parallelism possible when pointer chasing a single list, other advanced features like hit-under-miss and dual TLB walkers don't make any difference at all.

So it's fine to use lmbench if you wanted to show that no matter how advanced your TLBs and caches are, they totally suck doing completely random accesses. However in the real world applications exhibit a significant amount of spatial locality (even if having lots of TLB misses) and the cache and TLBs are highly optimized to make use of that.

Wilco
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
ARM A73 benchmarksSymmetry03/14/17 06:24 AM
  ARM A73 benchmarksPer Hesselgren03/14/17 07:18 AM
    ARM A73 benchmarks-latencyPer Hesselgren03/14/17 08:58 AM
      ARM A73 benchmarks-latencySymmetry03/14/17 10:12 AM
        ARM A73 benchmarks-latencyPer Hesselgren03/14/17 03:54 PM
          ARM A73 benchmarks-latencyWilco03/15/17 01:45 AM
            ARM A73 benchmarks-latencyPer Hesselgren03/15/17 02:57 AM
              ARM A73 benchmarks-latencyPer Hesselgren03/15/17 03:00 AM
                ARM A73 benchmarks-latencyPer Hesselgren03/15/17 03:01 AM
                  clickable linkMichael S03/15/17 04:05 AM
            ARM A73 benchmarks-latencyLinus Torvalds03/15/17 10:05 AM
              ARM A73 benchmarks-latencyIreland03/15/17 05:02 PM
              ARM A73 benchmarks-latencyGabriele Svelto03/16/17 03:45 AM
                ARM A73 benchmarks-latencyLinus Torvalds03/16/17 02:01 PM
                  lmbench is horribly brokenWilco03/16/17 04:57 PM
                    lmbench is horribly brokenLinus Torvalds03/16/17 06:49 PM
                      lmbench is horribly brokenLinus Torvalds03/17/17 01:10 PM
                        lmbench is horribly brokenLinus Torvalds03/17/17 01:52 PM
                        lmbench is horribly brokenExophase03/17/17 02:31 PM
                          lmbench is horribly brokenGabriele Svelto03/17/17 03:20 PM
                          lmbench is horribly brokenLinus Torvalds03/17/17 05:56 PM
                            lmbench is horribly brokenExophase03/17/17 06:21 PM
                              lmbench is horribly brokenLinus Torvalds03/17/17 06:43 PM
                                lmbench is horribly brokenIreland03/17/17 07:37 PM
                                  lmbench is horribly brokenbakaneko03/18/17 11:17 AM
                                    lmbench is horribly brokenIreland03/18/17 12:23 PM
                                      lmbench is horribly brokenanon03/18/17 07:35 PM
                                      lmbench is horribly brokenbakaneko03/21/17 08:08 AM
                                        lmbench is horribly brokenIreland03/21/17 03:14 PM
                                lmbench is horribly brokenGabriele Svelto03/18/17 04:01 PM
                                  accessing dram RichardC03/18/17 06:33 PM
                                lmbench is horribly brokenExophase03/18/17 04:26 PM
                                  lmbench is horribly brokenWilco03/18/17 05:40 PM
                                    benchmarking reality?Anon03/19/17 02:29 PM
                                    lmbench is horribly brokenLinus Torvalds03/19/17 04:25 PM
                                      mea culpa (lmbench is horribly broken)Linus Torvalds03/19/17 06:05 PM
                                        mea culpa (lmbench is horribly broken)Bill Broadley03/21/17 01:41 AM
                                          mea culpa (lmbench is horribly broken)Linus Torvalds03/21/17 09:01 AM
                                            mea culpa (lmbench is horribly broken)Linus Torvalds03/21/17 11:14 AM
                                            mea culpa (lmbench is horribly broken)Linus Torvalds03/21/17 05:03 PM
                                              mea culpa (lmbench is horribly broken)Etienne03/22/17 04:37 AM
                                              mea culpa (lmbench is horribly broken)Tim McCaffrey03/22/17 08:54 AM
                                                mea culpa (lmbench is horribly broken)Tim McCaffrey03/22/17 09:34 AM
                                                mea culpa (lmbench is horribly broken)Linus Torvalds03/22/17 10:35 AM
                                                  mea culpa (lmbench is horribly broken)Ireland03/22/17 12:11 PM
                                                    mea culpa (lmbench is horribly broken)Ireland03/22/17 12:26 PM
                                                    mea culpa (lmbench is horribly broken)rwessel03/22/17 03:03 PM
                                                      mea culpa (lmbench is horribly broken)Ireland03/22/17 03:35 PM
                                                  mea culpa (lmbench is horribly broken)Linus Torvalds03/22/17 01:35 PM
                                                    mea culpa (lmbench is horribly broken)Gabriele Svelto03/23/17 08:05 AM
                                                      mea culpa (lmbench is horribly broken)Linus Torvalds03/23/17 10:43 AM
                                                        mea culpa (lmbench is horribly broken)Gabriele Svelto03/23/17 01:56 PM
                                                          mea culpa (lmbench is horribly broken)Ireland03/23/17 02:36 PM
                                                  mea culpa (lmbench is horribly broken)Travis03/22/17 01:38 PM
                                              mea culpa (lmbench is horribly broken)anon03/22/17 07:22 PM
                                                mea culpa (lmbench is horribly broken)Travis03/22/17 08:57 PM
                                                  mea culpa (lmbench is horribly broken)anon03/23/17 12:44 AM
                                                    mea culpa (lmbench is horribly broken)Michael S03/23/17 05:59 PM
                                                      mea culpa (lmbench is horribly broken)Travis03/23/17 09:03 PM
                                                    power8 numbersoctoploid03/24/17 11:47 PM
                                                      power8 numbers stride=128octoploid03/25/17 04:36 AM
                                                        power8 numbers stride=128Linus Torvalds03/25/17 10:50 AM
                                                          power8 numbers stride=128Gabriele Svelto03/25/17 11:27 PM
                                              mea culpa (lmbench is horribly broken)anon03/23/17 01:14 AM
                                                mea culpa (lmbench is horribly broken)Linus Torvalds03/23/17 11:22 AM
                                                  Thank you. Associativity misses explain it.anon03/23/17 10:48 PM
                                                    Thank you. Associativity misses explain it.Linus Torvalds03/24/17 01:26 PM
                                                      Thank you. Associativity misses explain it.Travis03/24/17 10:01 PM
                                                        thanks should read "but if it is any TYPE of mix" (NT)Travis03/24/17 10:02 PM
                                                        Thank you. Associativity misses explain it.Linus Torvalds03/25/17 12:10 PM
                                                          Thank you. Associativity misses explain it.Travis03/25/17 04:08 PM
                                                            Thank you. Associativity misses explain it.Linus Torvalds03/26/17 10:27 AM
                                  lmbench is horribly brokenLinus Torvalds03/19/17 03:51 PM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?