SPEC is NOT a representation of average workloads

By: Chester (lamchester.delete@this.gmil.com), November 18, 2020 3:06 pm
Room: Moderated Discussions
Dummond D. Slow (mental.delete@this.protozoa.us) on November 18, 2020 7:17 am wrote:
> Chester (lamchester.delete@this.gmail.com) on November 18, 2020 7:02 am wrote:
> > > 30% figure is an outlier figure for a few workloads. Across
> > > the board it's not even close to that. Can we stop
> > > declaring SMT an absolute must just because Cinebench has long dependency chains and scales well off of it?
> > >
> > > https://i.imgur.com/nSi11Gl.png
> > >
> > > Saying stuff like "It's simple: SMT. Had Apple implemented it, it would run away in
> > > Cinebench." is just idiotic in the face of the argument you want to make about SMT
> > > MT scaling and Apple's reasons and internal decisions about not implementing it.
> >
> > Sure, Cinebench isn't the best representation of average workloads. But SPEC is far
> > worse. No consumer cares about SPEC. The subtests are mostly based off applications
> > no one uses, or very specific scientific simulations. It's even useless as a benchmark
> > to see whether your system is working properly, because it's so overpriced.
> >
> > Even when they base a subtest off something a consumer might
> > do, the results are hilariously off. For example:
> > Encoding a 4K video using ffmpeg libx264 slow preset, on Haswell locked to 2.2 GHz
> > - affinity set to 4 threads: 6.6 fps
> > - no affinity set: 7.6 fps (1.15x scaling)
>
> You should manually set lower number threads, because if you do this via afinity, the encoder might still
> spawn the same number of threads. Or perhaps, the best way is actualy switching HT on and off.
> x264's thread number is complex, it is not = CPU's threads. It is CPU threads*1.5 because *1 didn't
> saturate cores/threads due to the way the frame-threading works as opposed to slice threading.
> So if you have 4c/4t CPU, 6 threads are spawned. If you are on 4c/8t, 12 threads are spawned.
> And besides that, input decoding and lookahead are done separately (and the lookahead might
> be actually slice-threaded too now, not sure, x265 does that, x264 does not).
> If you still used 12 threads pinned to 4 logical cores, you might have
> got a bit different FPS, possibly (not sure if for worse, who knows).

Ok, I'll try this sometime.

>
> >
> > 525.x264_r test: 1.0315x scaling
> >
>
> You know I am not completely sure about this as I can't check, but I heard that SPEC uses x264's
> code WITHOUT SIMD assembly compiled (whether disabling ASM intentionally or not compiling with yasm
> or whatever, no idea). If that is true it probably grossly changes character of the workload because
> compiler autovectorization tends to fails on vectorizing multimedia integer algorithms. Normally,
> x264 spends like 55% (figure given by devs 8-9 years back) in hand-tuned x86 SIMD.
>
> (Interestingly you would think that should push 1 thread to exploit the
> execution units more fully but it still has gains from SMT/HT...)
>
>
> > A quick look at the benchmark description shows them using -bitrate 1000.
> > If that's 1 kbps (or 1 mbps) bitrate, it's hilariously unrealistic.
>
> it has been long time since I have not used CRF but the bitrate is in kbps because that is the reasonalbe
> unit, so that is not a problem I think, 1000 kbps is fine (depends on resolution naturaly)

1000 kbps is way too low for any res today. Youtube isn't exactly the champion of quality. They recommend 5 mbps for 720p30, and 8 mbps for 1080p30. You can push a bit lower if you need to save every last bit, but I've never liked the results for say, 3.5 mbps w/720p30. 1000 kbps is ridiculous.

> > Now take AT Bench's POV-Ray scores for the i5-6600K (1741, 3.6 GHz all core turbo)
> > and i7-6700K (2419, 4.2 GHz all core turbo). Scaling down the 6700K's score to account
> > for clock speed difference gives 2073. SMT scaling would be roughly 1.19x
> >
> > 511.povray_r: 1.0291x scaling.
> >
> > What's going on?
> >
> > Also, some SPEC numbers make it seem like negative SMT scaling is common. It's not. I've
> > personally never seen an application that can use all available threads do worse when
> > SMT is enabled. Can we stop looking at the irrelevant pile of garbage that is SPEC?
> >
> > And what makes you claim "Cinebench has long dependency chains"? How do you know SMT scaling is from that
> > rather than hiding cache misses better? Because Cinebench (R20, have not tested R23) does suffer from L1/L2
> > cache misses. In ST mode, execution spends 9% of cycles stalled with a L1D miss pending. You get about
> > 16.4 L1D MPKI if you count loads hitting the fill buffer as misses. And L2 hitrate is around 50%.
>
>

< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
1st round of M1 GPU benchmarksMaynard Handley2020/11/16 01:46 PM
  1st round of M1 GPU benchmarksAdrian2020/11/16 02:02 PM
    1st round of M1 GPU benchmarksI_vs2020/11/17 02:18 AM
  1st round of M1 GPU benchmarksRType2020/11/16 02:06 PM
    1st round of M1 GPU benchmarksMaynard Handley2020/11/16 02:45 PM
      And the NZ'ers are first to deliver cinebench scoresMaynard Handley2020/11/16 03:58 PM
        And the NZ'ers are first to deliver cinebench scoresWill2020/11/16 04:30 PM
          re: "scaling"Dummond D. Slow2020/11/16 05:06 PM
            re: "scaling"Maynard Handley2020/11/16 05:29 PM
              re: "scaling"Dummond D. Slow2020/11/16 05:51 PM
                re: "scaling"Maynard Handley2020/11/16 06:51 PM
                  re: "scaling"Dummond D. Slow2020/11/16 07:22 PM
                    re: "scaling"Maynard Handley2020/11/16 07:23 PM
                      re: "scaling"Maynard Handley2020/11/16 07:27 PM
                        re: "scaling"Dummond D. Slow2020/11/16 07:37 PM
                          re: "scaling"Chester2020/11/16 09:25 PM
                          Actual power consumption values.Dummond D. Slow2020/11/17 08:05 AM
                    re: "scaling"Doug S2020/11/17 11:25 AM
                      Apple loses a lot by not having turbo and SMTDummond D. Slow2020/11/17 12:18 PM
                        Apple loses a lot by not having turbo and SMTDoug S2020/11/17 01:21 PM
                          Apple loses a lot by not having turbo and SMTblaine2020/11/19 06:18 PM
                            Apple loses a lot by not having turbo and SMTDummond D. Slow2020/11/19 06:31 PM
                        Apple loses a lot by not having turbo and SMTTJ2020/11/18 02:10 AM
                          Cinebench is NOT a representation of average workloadsAndrei F2020/11/18 06:21 AM
                            SPEC is NOT a representation of average workloadsChester2020/11/18 08:02 AM
                              SPEC is NOT a representation of average workloadsDummond D. Slow2020/11/18 08:17 AM
                                SPEC is NOT a representation of average workloadsnone2020/11/18 09:18 AM
                                  x264 without SIMD is nonsenseDummond D. Slow2020/11/18 09:31 AM
                                  SPEC is NOT a representation of average workloadsMaynard Handley2020/11/18 10:13 AM
                                    SPEC is NOT a representation of average workloadsDummond D. Slow2020/11/18 10:21 AM
                                      SPEC is NOT a representation of average workloadsWilco2020/11/18 10:42 AM
                                        role of hand-optimized assembly in multimedia encodersDummond D. Slow2020/11/18 12:06 PM
                                          role of hand-optimized assembly in multimedia encodersdifferent anon2020/11/18 02:27 PM
                                            role of hand-optimized assembly in multimedia encodersDummond D. Slow2020/11/18 03:47 PM
                                              role of hand-optimized assembly in multimedia encodersDummond D. Slow2021/05/18 01:29 PM
                                          role of hand-optimized assembly in multimedia encodersRichardC2020/11/18 04:42 PM
                                          role of hand-optimized assembly in multimedia encodersnone2020/11/19 08:17 AM
                                      SPEC is NOT a representation of average workloadsnone2020/11/18 11:14 AM
                                        SPEC is NOT a representation of average workloadsDummond D. Slow2020/11/18 12:17 PM
                                      SPEC is NOT a representation of average workloadsMaynard Handley2020/11/18 12:03 PM
                                        SPEC is NOT a representation of average workloadsDummond D. Slow2020/11/18 12:25 PM
                                          SPEC is NOT a representation of average workloadsMaynard Handley2020/11/18 01:03 PM
                                            SPEC is NOT a representation of average workloadsDummond D. Slow2020/11/18 03:57 PM
                                              SPEC is NOT a representation of average workloadsMaynard Handley2020/11/18 05:36 PM
                                                SPEC is NOT a representation of average workloadsDummond D. Slow2020/11/18 05:50 PM
                                                  SPEC is NOT a representation of average workloadsDoug S2020/11/19 10:13 AM
                                                    SPEC is NOT a representation of average workloads-.-2020/11/19 03:34 PM
                                                    SPEC is NOT a representation of average workloadsDummond D. Slow2020/11/19 03:35 PM
                                              SPEC is NOT a representation of average workloadsEtienne Lorrain2020/11/19 05:04 AM
                                                SPEC is NOT a representation of average workloadsDummond D. Slow2020/11/19 06:58 AM
                                SPEC is NOT a representation of average workloadsChester2020/11/18 03:06 PM
                                  about bitratesDummond D. Slow2020/11/18 04:04 PM
                                    about bitratesChester2020/11/19 03:45 PM
                                  SPEC is NOT a representation of average workloadsOok2020/11/18 04:05 PM
                                    SPEC is NOT a representation of average workloadsDummond D. Slow2020/11/18 04:21 PM
                                  SPEC is NOT a representation of average workloadsNoSpammer2020/11/19 07:48 AM
                              SPEC is NOT a representation of average workloadsAndrei F2020/11/18 08:44 AM
                                SPEC is NOT a representation of average workloadsDummond D. Slow2020/11/18 08:48 AM
                                  SPEC is NOT a representation of average workloadsAdrian2020/11/18 09:15 AM
                                    SPEC is NOT a representation of average workloadsDummond D. Slow2020/11/18 09:41 AM
                                      SPEC is NOT a representation of average workloadsAdrian2020/11/18 10:04 AM
                                        SPEC is NOT a representation of average workloadsDummond D. Slow2020/11/18 10:32 AM
                                          SPEC is NOT a representation of average workloadsAdrian2020/11/18 11:26 AM
                                    SPEC is NOT a representation of average workloadsMaynard Handley2020/11/18 10:22 AM
                                      SPEC is NOT a representation of average workloadsAdrian2020/11/18 11:28 AM
                                  SPEC is NOT a representation of average workloadsAndrei F2020/11/18 09:34 AM
                                    SPEC is NOT a representation of average workloadsDummond D. Slow2020/11/18 10:00 AM
                                    SPEC is NOT a representation of average workloadsChester2020/11/18 03:18 PM
                                    SPEC is NOT a representation of average workloadsJukka Larja2020/11/20 09:03 AM
                                SPEC is NOT a representation of average workloadsChester2020/11/18 03:13 PM
                              SPEC is NOT a representation of average workloadsnone2020/11/18 09:07 AM
                                SPEC is NOT a representation of average workloadsMichael S2020/11/18 02:33 PM
                                  Interesting, thanks! (NT)none2020/11/19 08:03 AM
                            Cinebench is NOT a representation of average workloadsrwessel2020/11/18 08:03 AM
                              Cinebench is NOT a representation of average workloadsDoug S2020/11/18 08:44 AM
                                Cinebench is NOT a representation of average workloadsDummond D. Slow2020/11/18 08:52 AM
                                  Cinebench is NOT a representation of average workloadsDoug S2020/11/18 03:59 PM
                                    Cinebench is NOT a representation of average workloadsDummond D. Slow2020/11/18 04:12 PM
                                      Cinebench is NOT a representation of average workloadsDoug S2020/11/19 10:19 AM
                                        Cinebench is NOT a representation of average workloadsDummond D. Slow2020/11/19 04:03 PM
                                          Cinebench is NOT a representation of average workloadsUngo2020/11/19 09:12 PM
                                            Cinebench is NOT a representation of average workloadsDummond D. Slow2020/11/20 09:45 AM
                                It's not really SMT or adding small coresDummond D. Slow2020/11/18 08:58 AM
                                  It's not really SMT or adding small coresDoug S2020/11/18 04:02 PM
                                    It's not really SMT or adding small coresDummond D. Slow2020/11/18 04:27 PM
                                Cinebench is NOT a representation of average workloadsxyz2020/11/18 01:14 PM
                                  Cinebench is NOT a representation of average workloadsMaynard Handley2020/11/18 01:54 PM
                                    Cinebench is NOT a representation of average workloadsGabriele Svelto2020/11/18 02:41 PM
                                      Cinebench is NOT a representation of average workloadsMaynard Handley2020/11/18 02:52 PM
                                        Cinebench is NOT a representation of average workloadsGabriele Svelto2020/11/18 03:05 PM
                            Cinebench is NOT a representation of average workloadsTJ2020/11/18 12:46 PM
                            SMT shines in IO heavy & commercial DB workloadsxyz2020/11/18 01:05 PM
                            "Average workloads" don't exist (NT)Foo_2020/11/19 03:55 AM
                          Apple loses a lot by not having turbo and SMTDummond D. Slow2020/11/18 08:02 AM
                            Apple loses a lot by not having turbo and SMTMaynard Handley2020/11/18 09:37 AM
                              Apple loses a lot by not having turbo and SMTDummond D. Slow2020/11/18 09:55 AM
                              Apple loses a lot by not having turbo and SMTDavid Hess2020/11/18 01:12 PM
                                Apple loses a lot by not having turbo and SMTMaynard Handley2020/11/18 02:12 PM
                                  Apple loses a lot by not having turbo and SMTJörn Engel2020/11/18 05:04 PM
                                    Apple loses a lot by not having turbo and SMTanonymou52020/11/18 05:37 PM
                                      Apple loses a lot by not having turbo and SMTJörn Engel2020/11/19 04:28 AM
                                  Apple loses a lot by not having turbo and SMTDavid Hess'2020/11/19 07:25 PM
                          Apple loses a lot by not having turbo and SMTAdrian2020/11/18 08:23 AM
                            Apple loses a lot by not having turbo and SMTgallier22020/11/19 12:21 AM
                              Apple loses a lot by not having turbo and SMTJames2020/11/19 02:22 AM
                                Apple loses a lot by not having turbo and SMTWilco2020/11/19 05:06 AM
                              Apple loses a lot by not having turbo and SMTanonymou52020/11/19 02:34 AM
                                Apple loses a lot by not having turbo and SMTgallier22020/11/19 09:04 AM
                                  big.LITTLE with different ISAsJames2020/11/19 09:41 AM
                                    big.LITTLE with different ISAsgallier22020/11/20 03:36 AM
                                    big.LITTLE with different ISAsanon2020/11/20 01:31 PM
                              Apple loses a lot by not having turbo and SMTAdrian2020/11/19 02:56 AM
                        Apple loses a lot by not having turbo and SMTJon Masters2020/11/18 12:46 PM
                          Apple loses a lot by not having turbo and SMTanonymou52020/11/18 03:41 PM
                          Apple loses a lot by not having turbo and SMTDummond D. Slow2020/11/18 04:17 PM
                            Apple loses a lot by not having turbo and SMTMaynard Handley2020/11/18 05:46 PM
                              Apple loses a lot by not having turbo and SMTJon Masters2020/11/19 12:31 AM
                                Apple loses a lot by not having turbo and SMTDummond D. Slow2020/11/19 07:01 AM
                              Apple loses a lot by not having turbo and SMTAdrian2020/11/19 02:50 AM
                                Apple loses a lot by not having turbo and SMTAdrian2020/11/19 03:10 AM
                                Apple loses a lot by not having turbo and SMTMaynard Handley2020/11/19 10:13 AM
                                  Apple loses a lot by not having turbo and SMTDummond D. Slow2020/11/19 04:10 PM
                        Apple loses a lot by not having turbo and SMTjuanrga2020/11/19 02:49 AM
                    re: "scaling"sr2020/11/17 12:39 PM
                      re: "scaling"Adrian2020/11/17 01:41 PM
                        re: "scaling"Adrian2020/11/17 01:52 PM
                          re: "scaling"Chester2020/11/17 06:29 PM
                  re: "scaling"anonymou52020/11/16 08:18 PM
                    re: "scaling"Maynard Handley2020/11/16 08:44 PM
        And the NZ'ers are first to deliver cinebench scoresDummond D. Slow2020/11/16 05:12 PM
          And the NZ'ers are first to deliver cinebench scoresMaynard Handley2020/11/16 05:31 PM
            And the NZ'ers are first to deliver cinebench scoresDummond D. Slow2020/11/16 05:46 PM
        And the NZ'ers are first to deliver cinebench scoresMaynard Handley2020/11/16 07:04 PM
          And the NZ'ers are first to deliver cinebench scoresDummond D. Slow2020/11/16 07:28 PM
  1st round of M1 GPU benchmarksRayla2020/11/16 04:16 PM
    1st round of M1 GPU benchmarksChester2020/11/16 09:17 PM
  Phoronix Mac mini M1 benchanon52020/11/20 11:51 AM
    Phoronix Mac mini M1 benchPer Hesselgren2020/11/24 10:04 AM
      Phoronix Mac mini M1 benchPaul2020/11/25 03:08 AM
        Phoronix Mac mini M1 benchPaul2020/11/25 03:12 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell avocado?