Interesting Zen IPC benchmarks

By: Chester (lamchester.delete@this.gmail.com), November 29, 2020 7:16 am
Room: Moderated Discussions
Groo (charlie.delete@this.semiaccurate.com) on November 27, 2020 9:27 am wrote:
> Chester (lamchester.delete@this.gmail.com) on November 26, 2020 6:32 pm wrote:
>
> > https://www.spec.org/cpu2017/Docs/runrules.html#rule_4.4.2 isn't very clear.
> >
> > If you've over or under clocked, you're supposed to put that in nominal or max frequency fields.
> >
> > It's not clear whether MCE counts as overclocking though. A 9900K official result
> > lists 3600 MHz as the nominal frequency, even though at stock a 9900K will run all
> > cores at 4.7 GHz. So the 'nominal frequency' field is pretty much meaningless.
> >
> > Another thing board vendors do is raise or remove time limits on PL2 power, allowing Intel
> > chips to sustain higher power targets (and higher clocks) for longer. Technically this isn't
> > stock either, but it's also questionable whether it's considered overclocking.
> >
> > There's a lot of debate about how reviewers should treat this stuff. But it
> > doesn't seem like SPEC cares - they only care that the system vendor supports
> > the configuration (which is irrelevant to people buying their own parts).
>
> Good points. That said SPEC isn't really a benchmark for consumer systems, or wasn't meant to be.
> The way a lot of people I talked to explained it was that no serious company running spec for the
> right reasons, IE as a sales/comparison tool, would do OC and related tricks. Why? Because if customers
> bought the systems and ran the bench, they would get significantly lower results and then hilarity
> would follow. By hilarity I mean lawyers, nasty phone calls from CxOs, etc.

Yeah, that's the problem with using SPEC as a benchmark for consumer systems.

On the OC note, say some vendor binned chips like siliconlottery, built systems with CPUs overclocked out of the box, and submitted OC-ed SPEC scores. That would be fine as long as they put the OC clocks in the results and supported their product. And that might be ok if you're looking to purchase whole systems.

Of course it wouldn't mean much to consumers looking to select parts without such a guaranteed overclock.

> So for the intended market, the rules as they stand seem pretty OK but probably need an update. For the consumer
> market we are already well past the point of honesty on the part of the vendors, many of them have pushed cheating
> down into firmware and even hardware in a few credible accusations. It will only get worse.

If everyone is 'cheating' by ignoring turbo time limits and/or using MCE, and consumers are likely to leave those options on or turn them on themselves if not enabled by default, what should reviewers do?

Should they test the config customers will most likely be using? Or turn off those options to get a 'proper' stock result?

> In the end it will be as it always is, look at the source who is publishing the benchmark. If it is Dell,
> Fujitsu, or a similar tier-1, it is probable safe especially if they sell many different CPUs to the same
> market. Silicon vendors can be less honest, and anything lower than that, big grain of salt.
>
> This is why I go off on Twitter and in my articles about disclosure. At the moment, Intel is
> the only one doing right, or very close to it. AMD bitterly complained for years about how
> bad Intel was, then stopped even trying when they took the lead. Now they appear to be inching
> closer to tolerable. The rest are pretty worthless but I will keep on pointing it out.
>
> What it comes down to is look a little deeper than the scores themselves, and
> point it out when people do wrong. If you don't, they will keep doing it.
>
> -Charlie

Yeah, keep pointing it out. But most people by now know to at least wait for independent reviews, and to take initial manufacturer benchmarks with a grain of salt.
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Interesting Zen IPC benchmarksAdrian2020/11/21 08:14 AM
  Interesting Zen IPC benchmarksjuanrga2020/11/21 10:22 AM
    Interesting Zen IPC benchmarksChester2020/11/21 03:49 PM
      Interesting Zen IPC benchmarksAndrei F2020/11/22 05:08 AM
        Interesting Zen IPC benchmarksChester2020/11/22 09:33 PM
          Interesting Zen IPC benchmarksnone2020/11/23 01:59 AM
            Interesting Zen IPC benchmarksnone2020/11/23 02:01 AM
              Interesting Zen IPC benchmarksnone2020/11/23 02:01 AM
          No...David Kanter2020/11/23 08:16 AM
            No...Chester2020/11/23 03:15 PM
              No...Foo_2020/11/24 03:34 AM
                No...Chester2020/11/25 12:45 AM
                  No...Per Hesselgren2020/11/25 01:57 AM
                  No...Adrian2020/11/25 04:21 AM
          Interesting Zen IPC benchmarksDoug S2020/11/23 01:03 PM
            Interesting Zen IPC benchmarksChester2020/11/23 03:26 PM
              Programs people useFoo_2020/11/24 03:43 AM
                Programs people useJukka Larja2020/11/24 07:07 AM
                Cinebench is the new Dhrystone :) (NT)none2020/11/25 01:10 AM
              Interesting Zen IPC benchmarksjuanrga2020/11/24 08:38 AM
          Interesting Zen IPC benchmarksAndrei F2020/11/24 05:47 AM
            Interesting Zen IPC benchmarksChester2020/11/25 12:32 AM
              Questionable thoughtsbenchmark critic2020/11/25 08:41 AM
                Questionable thoughtsChester2020/11/25 03:14 PM
                Questionable thoughtsnone2020/11/26 01:14 AM
                  Links?benchmark critic2020/11/26 09:48 AM
              Interesting Zen IPC benchmarksGroo2020/11/25 01:48 PM
                Interesting Zen IPC benchmarksChester2020/11/25 04:36 PM
                  Interesting Zen IPC benchmarksGroo2020/11/26 02:46 PM
                    Interesting Zen IPC benchmarksChester2020/11/26 07:32 PM
                      Interesting Zen IPC benchmarksGroo2020/11/27 10:27 AM
                        Interesting Zen IPC benchmarksChester2020/11/29 07:16 AM
                          Interesting Zen IPC benchmarksGroo2020/11/29 09:56 AM
                            Interesting Zen IPC benchmarksChester2020/11/29 04:41 PM
                Interesting Zen IPC benchmarksRayla2020/11/26 07:46 AM
                  BenchmarksDavid Kanter2020/11/26 10:05 AM
                    Benchmarksblaine2020/11/26 01:04 PM
          Interesting Zen IPC benchmarksPer Hesselgren2020/11/24 10:11 AM
            Interesting Zen IPC benchmarksChester2020/11/25 12:42 AM
      Interesting Zen IPC benchmarksjuanrga2020/11/22 07:09 AM
        Interesting Zen IPC benchmarksChester2020/11/22 09:53 PM
          Interesting Zen IPC benchmarksjuanrga2020/11/23 01:16 PM
            Interesting Zen IPC benchmarksChester2020/11/23 02:27 PM
              Interesting Zen IPC benchmarksjuanrga2020/11/24 08:25 AM
            Interesting Zen IPC benchmarksAdrian2020/11/24 11:51 AM
              Interesting Zen IPC benchmarksjuanrga2020/11/26 04:52 AM
  The Stilt's Zen 3 IPC benchmarksDummond D. Slow2020/11/25 09:29 AM
    The Stilt's Zen 3 IPC benchmarksChester2020/11/25 04:49 PM
      The Stilt's Zen 3 IPC benchmarksDummond D. Slow2020/11/25 05:58 PM
        The Stilt's Zen 3 IPC benchmarksDoug S2020/11/26 09:19 AM
      The Stilt's Zen 3 IPC benchmarksDummond D. Slow2020/11/25 06:13 PM
        The Stilt's Zen 3 IPC benchmarksChester2020/11/26 11:24 AM
          The Stilt's Zen 3 IPC benchmarksitsmydamnation2020/11/26 03:06 PM
            The Stilt's Zen 3 IPC benchmarksChester2020/11/26 07:10 PM
            The Stilt's Zen 3 IPC benchmarksDoug S2020/11/27 04:17 PM
      The Stilt's Zen 3 IPC benchmarksjuanrga2020/11/26 05:10 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell avocado?