lmbench is horribly broken

By: Wilco (Wilco.Dijkstra.delete@this.ntlworld.com), March 16, 2017 4:57 pm
Room: Moderated Discussions
Linus Torvalds (torvalds.delete@this.linux-foundation.org) on March 16, 2017 2:01 pm wrote:
> Gabriele Svelto (gabriele.svelto.delete@this.gmail.com) on March 16, 2017 3:45 am wrote:
> >
> > If TLB performance wasn't important then why so much resources were spent in improving them in the last few
> > years? We've got improvements in the speed of TLBs (dual walkers, faster reload and invalidation), in their
> > size (larger first level TLBs, second-level TLBs) and effectiveness (larger pages, tagging, nested paging).
>
> There's a lot of other subtle things too, like whether the CPU caches the TLB lookups
> in the regular data caches, and how well it handles TLB misses in the pipeline.
>
> For example, maybe you have an fairly out-of-order memory model, and you can reorder loads
> and stores - but your memory pipeline might need the physical address to be able to do that.
> So a TLB miss may be more than just having to look things up in page tables: it may end up
> basically serializing your memory operations, and causing your pipeline to stall entirely,
> and turn what should be an out-of-order machine effectively into a slow in-order one.
>
> And in-order is particularly bad when it means that you don't get to overlap the memory accesses.
>
> It's fairly easy to see a memory pipeline that works well when things are nice and in the TLB, and
> just falls entirely flat on its face when even just some of the memory accesses miss in the TLB.
>
> Software TLB filling shows this problem very very clearly, but honestly, hardware walking in no way guarantees
> it doesn't happen. An overly simple hardware walker will have the exact same issues that a SW walker has.

These are all good reasons why lmbench is such a horrible broken benchmark as it doesn't allow any parallelism - it uses a single chain of memory references which are guaranteed to miss the TLB as well as the cache. And because the accesses are spread randomly across all of memory they easily swamp the intermediate page table caches too. That means you end up with 2 DRAM accesses for the table walk plus the memory access, all completely serialized. This has absolutely nothing to do with average memory latency in the real world.

As a result an average or even poor TLB implementation gets the same "latency" scores as one that can handle multiple TLB misses in parallel with hit under miss. I've seen in-order cores score better than out-of-order cores with much more advanced TLBs.

So yes, Geekbench 4 absolutely deserves credit for avoiding these obvious bugs.

Wilco
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
ARM A73 benchmarksSymmetry2017/03/14 06:24 AM
  ARM A73 benchmarksPer Hesselgren2017/03/14 07:18 AM
    ARM A73 benchmarks-latencyPer Hesselgren2017/03/14 08:58 AM
      ARM A73 benchmarks-latencySymmetry2017/03/14 10:12 AM
        ARM A73 benchmarks-latencyPer Hesselgren2017/03/14 03:54 PM
          ARM A73 benchmarks-latencyWilco2017/03/15 01:45 AM
            ARM A73 benchmarks-latencyPer Hesselgren2017/03/15 02:57 AM
              ARM A73 benchmarks-latencyPer Hesselgren2017/03/15 03:00 AM
                ARM A73 benchmarks-latencyPer Hesselgren2017/03/15 03:01 AM
                  clickable linkMichael S2017/03/15 04:05 AM
            ARM A73 benchmarks-latencyLinus Torvalds2017/03/15 10:05 AM
              ARM A73 benchmarks-latencyIreland2017/03/15 05:02 PM
              ARM A73 benchmarks-latencyGabriele Svelto2017/03/16 03:45 AM
                ARM A73 benchmarks-latencyLinus Torvalds2017/03/16 02:01 PM
                  lmbench is horribly brokenWilco2017/03/16 04:57 PM
                    lmbench is horribly brokenLinus Torvalds2017/03/16 06:49 PM
                      lmbench is horribly brokenLinus Torvalds2017/03/17 01:10 PM
                        lmbench is horribly brokenLinus Torvalds2017/03/17 01:52 PM
                        lmbench is horribly brokenExophase2017/03/17 02:31 PM
                          lmbench is horribly brokenGabriele Svelto2017/03/17 03:20 PM
                          lmbench is horribly brokenLinus Torvalds2017/03/17 05:56 PM
                            lmbench is horribly brokenExophase2017/03/17 06:21 PM
                              lmbench is horribly brokenLinus Torvalds2017/03/17 06:43 PM
                                lmbench is horribly brokenIreland2017/03/17 07:37 PM
                                  lmbench is horribly brokenbakaneko2017/03/18 11:17 AM
                                    lmbench is horribly brokenIreland2017/03/18 12:23 PM
                                      lmbench is horribly brokenanon2017/03/18 07:35 PM
                                      lmbench is horribly brokenbakaneko2017/03/21 08:08 AM
                                        lmbench is horribly brokenIreland2017/03/21 03:14 PM
                                lmbench is horribly brokenGabriele Svelto2017/03/18 04:01 PM
                                  accessing dram RichardC2017/03/18 06:33 PM
                                lmbench is horribly brokenExophase2017/03/18 04:26 PM
                                  lmbench is horribly brokenWilco2017/03/18 05:40 PM
                                    benchmarking reality?Anon2017/03/19 02:29 PM
                                    lmbench is horribly brokenLinus Torvalds2017/03/19 04:25 PM
                                      mea culpa (lmbench is horribly broken)Linus Torvalds2017/03/19 06:05 PM
                                        mea culpa (lmbench is horribly broken)Bill Broadley2017/03/21 01:41 AM
                                          mea culpa (lmbench is horribly broken)Linus Torvalds2017/03/21 09:01 AM
                                            mea culpa (lmbench is horribly broken)Linus Torvalds2017/03/21 11:14 AM
                                            mea culpa (lmbench is horribly broken)Linus Torvalds2017/03/21 05:03 PM
                                              mea culpa (lmbench is horribly broken)Etienne2017/03/22 04:37 AM
                                              mea culpa (lmbench is horribly broken)Tim McCaffrey2017/03/22 08:54 AM
                                                mea culpa (lmbench is horribly broken)Tim McCaffrey2017/03/22 09:34 AM
                                                mea culpa (lmbench is horribly broken)Linus Torvalds2017/03/22 10:35 AM
                                                  mea culpa (lmbench is horribly broken)Ireland2017/03/22 12:11 PM
                                                    mea culpa (lmbench is horribly broken)Ireland2017/03/22 12:26 PM
                                                    mea culpa (lmbench is horribly broken)rwessel2017/03/22 03:03 PM
                                                      mea culpa (lmbench is horribly broken)Ireland2017/03/22 03:35 PM
                                                  mea culpa (lmbench is horribly broken)Linus Torvalds2017/03/22 01:35 PM
                                                    mea culpa (lmbench is horribly broken)Gabriele Svelto2017/03/23 08:05 AM
                                                      mea culpa (lmbench is horribly broken)Linus Torvalds2017/03/23 10:43 AM
                                                        mea culpa (lmbench is horribly broken)Gabriele Svelto2017/03/23 01:56 PM
                                                          mea culpa (lmbench is horribly broken)Ireland2017/03/23 02:36 PM
                                                  mea culpa (lmbench is horribly broken)Travis2017/03/22 01:38 PM
                                              mea culpa (lmbench is horribly broken)anon2017/03/22 07:22 PM
                                                mea culpa (lmbench is horribly broken)Travis2017/03/22 08:57 PM
                                                  mea culpa (lmbench is horribly broken)anon2017/03/23 12:44 AM
                                                    mea culpa (lmbench is horribly broken)Michael S2017/03/23 05:59 PM
                                                      mea culpa (lmbench is horribly broken)Travis2017/03/23 09:03 PM
                                                    power8 numbersoctoploid2017/03/24 11:47 PM
                                                      power8 numbers stride=128octoploid2017/03/25 04:36 AM
                                                        power8 numbers stride=128Linus Torvalds2017/03/25 10:50 AM
                                                          power8 numbers stride=128Gabriele Svelto2017/03/25 11:27 PM
                                              mea culpa (lmbench is horribly broken)anon2017/03/23 01:14 AM
                                                mea culpa (lmbench is horribly broken)Linus Torvalds2017/03/23 11:22 AM
                                                  Thank you. Associativity misses explain it.anon2017/03/23 10:48 PM
                                                    Thank you. Associativity misses explain it.Linus Torvalds2017/03/24 01:26 PM
                                                      Thank you. Associativity misses explain it.Travis2017/03/24 10:01 PM
                                                        thanks should read "but if it is any TYPE of mix" (NT)Travis2017/03/24 10:02 PM
                                                        Thank you. Associativity misses explain it.Linus Torvalds2017/03/25 12:10 PM
                                                          Thank you. Associativity misses explain it.Travis2017/03/25 04:08 PM
                                                            Thank you. Associativity misses explain it.Linus Torvalds2017/03/26 10:27 AM
                                  lmbench is horribly brokenLinus Torvalds2017/03/19 03:51 PM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?