who is the champ right now?

By: Adrian (a.delete@this.acm.org), May 26, 2022 7:10 am
Room: Moderated Discussions
Andrei F (no-longer-public.delete@this.gmail.com) on May 26, 2022 6:16 am wrote:
> Adrian (a.delete@this.acm.org) on May 26, 2022 6:09 am wrote:
> > -.- (blarg.delete@this.mailinator.com) on May 26, 2022 5:23 am wrote:
> > > Michael S (already5chosen.delete@this.yahoo.com) on May 26, 2022 3:36 am wrote:
> > > > Sorry for may be naive question, in recent couple of years I was not following.
> > > > How the ST field at the fastest stock frequency (no overcloking) looks by now?
> > > > Say, Apple M1 (Pro/Max/Ultra) vs Zen3 with big cache vs regular
> > > > Zen3 vs Alder Lake vs Rocket/Tiger Lake vs Comet Lake?
> > > > Non-FP scores are of primary interest.
> > >
> > > Not sure why you bring up Intel's older processors, but again, depends on what you measure exactly.
> > >
> > > SPEC2006 is somewhat an industry standard, and you can find a ranking here:
> > > https://www.anandtech.com/bench/CPU-2020/2797 - ranking does include SPECfp,
> > > so you'd have to go to individual scores if you just want SPECInt results.
> > >
> > > The AMD 5800X3D and Intel 12900KS are missing from the list, though
> > > I'd still expect Alder Lake to come out on top for SPEC.
> >
> >
> > SPEC2006 might be an industry standard, but I do not trust at all that list of results from Anandtech.
> >
> > For benchmarks written in high-level languages, the compiler used and the compilation options
> > can be the cause of much higher differences in benchmark results, than the differences in
> > actual achievable CPU performance, which are very small for the top competitors.
> >
> > Only for a few of the CPUs listed by Anandtech it is possible to discover, by diving into the corresponding
> > Anandtech articles, the compiler versions and the compiler options. Even in those cases, the values used do
> > not appear to be optimal or uniform across the reviews separated by long times, of many months or years.
> >
> > It is pretty certain that Apple M1 has a performance that is overestimated on that list, by
> > using a clang version appropriate for it, while the x86 CPUs use also clang, with less appropriate
> > versions and options, instead of using a new enough gcc with the best options.
> >
> >
> >
> >
> >
>
> This complete utter nonsense. The compiler flags were set as optimally as possible on
> the x86 binaries, and they're all as close as possible to apples to apples. Result sets
> within given articles or database are all with the same versions and settings.


For meaningful results, I would hope that the same compiler versions had been used.

However, before writing the previous message, I have tried to follow the links from the Anandtech graph and search for the compiler versions and options, and more often than not, I could not find them, to verify if this assertion is true.

The links pointed the the reviews published by Anandtech at the launch of those CPUs.

At the time of the older of those reviews there was no compiler version yet, able to tune the optimization for the CPUs of the newest of those reviews.

So if an old clang version has been used for all x86 CPUs, the newer must have been disadvantaged.

I am pretty certain that the Apple clang version that has been used for M1 cannot have been the same as that used for the old x86 CPUs, as it could not have been available at that time.

Maybe the flags have been set "as optimally as possible on the x86 binaries", but I am not convinced. On one example that I have found following the links, the compilation was made for a Haswell, even if the target was an Ice Lake. I do not have much experience with optimization using clang, so maybe optimizing for another CPU does not make any difference in this case, but on gcc specifying the correct CPU would have had effects.


Moreover, the Apple M1 clang cannot be said to be more similar to the x86 clang than to the x86 gcc. For the fairest comparison between x86 and Appple, the best available compiler should have been chosen on each platform, and on x86 I doubt that clang is that.


I understand very well that there is a lot of work to produce such benchmark results, and that it would not be worthwhile to retest all the older systems together with the newest system, in order to use for all of them the newest compiler, which is appropriate for testing the new CPU, while also updating all the old results, to be more accurately comparable.

The Anandtech database serves a useful purpose as it is, but it must be understood that when there are results which are separated by only a few percent, then a retesting with a different compiler or options is very likely to change the order of the CPUs, so the Anandtech order cannot be said to represent with any certainty the SPEC CPU2006 order that would be obtained when applying the benchmark rules, i.e. using the best compiler with the best options, for each CPU.


< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Zen 4 is really badKara2022/05/26 01:52 AM
  Zen 4 is really bad-.-2022/05/26 02:13 AM
    who is the champ right now?Michael S2022/05/26 03:36 AM
      who is the champ right now?Kara2022/05/26 04:16 AM
        Alder lake better than zen4!Kara2022/05/26 04:24 AM
          Alder lake better than zen4!Adrian2022/05/26 05:40 AM
            Alder lake better than zen4!anon22022/05/26 06:14 PM
              Alder lake better than zen4!Adrian2022/05/27 04:18 AM
                Alder lake better than zen4!Matt Hughes2022/05/28 11:15 PM
      who is the champ right now?-.-2022/05/26 05:23 AM
        who is the champ right now?Adrian2022/05/26 06:09 AM
          who is the champ right now?Andrei F2022/05/26 06:16 AM
            who is the champ right now?Adrian2022/05/26 07:10 AM
              who is the champ right now?Andrei F2022/05/26 07:37 AM
                who is the champ right now?Andrei F2022/05/26 07:59 AM
              who is the champ right now?Eric Fink2022/05/27 12:17 AM
                who is the champ right now?Adrian2022/05/27 05:41 AM
            Optimally?Anon2022/05/26 09:43 AM
              Optimally?Andrei F2022/05/26 12:10 PM
          who is the champ right now?-.-2022/05/26 06:43 PM
            who is the champ right now?Adrian2022/05/27 05:24 AM
          Nothing wrong with anads spec2006 results - see the breakdownHeikki Kultala2022/05/27 03:23 PM
            Nothing wrong with anads spec2006 results - see the breakdownDoug S2022/05/27 09:59 PM
              Nothing wrong with anads spec2006 results - see the breakdownMatt Hughes2022/05/28 06:32 PM
                Nothing wrong with anads spec2006 results - see the breakdownMichael S2022/05/29 01:44 AM
                  Nothing wrong with anads spec2006 results - see the breakdownAdrian2022/05/29 03:53 AM
    Zen 4 is really badAdrian2022/05/26 05:16 AM
      Zen 4 is really bad-.-2022/05/26 05:58 AM
        Zen 4 test conditionsAdrian2022/05/26 06:14 AM
          Zen 4 test conditionsGroo2022/05/26 06:01 PM
        Zen 4 ECCAdrian2022/05/26 06:33 AM
          Zen 4 ECC-.-2022/05/26 06:38 PM
            Zen 4 ECCAdrian2022/05/27 05:11 AM
              Zen 4 ECC-.-2022/05/27 08:30 PM
                Zen 4 ECCMichael S2022/05/28 11:19 AM
          Zen 4 ECCdrAgonear2022/05/26 11:58 PM
            Zen 4 ECCDavid Hess2022/05/30 07:29 PM
      Zen 4 is really badgoose2022/05/26 08:14 PM
        Zen 4 is really badAdrian2022/05/27 04:44 AM
          Zen 4 is really badJames2022/05/27 06:13 AM
  Zen 4 is really badJohn H2022/05/26 05:45 AM
  Zen 4 is really badDoug S2022/05/26 10:50 AM
    Zen 4 is really badDummond D. Slow2022/05/28 09:14 PM
      Zen 4 is really badDoug S2022/05/29 09:49 AM
  Please check the faxts instead of posting crap like thisHeikki Kultala2022/05/26 11:05 AM
    Zen 4 IPC2022/05/26 11:24 AM
      Zen 4 IPC and AVX-512Per Hesselgren2022/05/27 01:07 AM
        Zen 4 IPC and AVX-512Per Hesselgren2022/05/27 03:44 AM
        Zen 4 IPC and AVX-512Adrian2022/05/27 04:57 AM
          Zen 4 IPC and AVX-512anon2022/05/27 08:22 AM
  Zen 4 is really badParsnip2022/05/29 04:20 AM
    What do you mean by 3-wide desing?Heikki Kultala2022/05/29 01:05 PM
      What do you mean by 3-wide desing?Doug S2022/05/29 02:32 PM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell tangerine? 🍊