huh??

By: Linus Torvalds (torvalds.delete@this.linux-foundation.org), July 4, 2015 1:37 pm
Room: Moderated Discussions
Wilco (Wilco.Dijkstra.delete@this.ntlworld.com) on July 4, 2015 11:41 am wrote:
>
> Many results don't make sense indeed. I wonder if the benchmarks were forced to run on
> a specific CPU at a fixed frequency - without that the results will be totally bogus.

I don't think they actually ran the benchmarks at all.

The numbers for some of the oddest ones are suspicious. Look at the 7420 arm64 numbers for gcc, eon and perlbmk: 2000, 2500 and 4000 respectively. Yeah, round numbers like that happen, but three ones like that that just happen to be that round?

So I wonder what "The scores we publish are only estimates" really means. It could mean that they want to make it clear that it's not an official Spec submission, and they kind of try to imply that.

But it could mean that they are just marketing estimates from some company, and have never seen any actual real benchmark run, or at best were run in some simulation with a perfect memory subsystem. They even say that they haven't been able to run 64-bit benchmarks due to lack of software availability, but then they quote specint2000 numbers anyway? Where did they come from? That's very unclear.

And gcc getting the same nice round score on a53 and a57? Yeah, not likely. And perlbmk on a53 has another suspiciously round score.

Or maybe it's real, and they just happen to be rounded to even thousands (or halves), and the fact that they seem to make no sense is just "that's life, deal with it".

I don't believe it for a second.

Linus
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
Exynos 7420(CortexA57) SPECint2000eha2015/07/02 03:45 PM
  huh??anonymous92015/07/02 06:31 PM
    huh??none2015/07/03 12:04 AM
      huh??Wilco2015/07/04 10:41 AM
        huh??Linus Torvalds2015/07/04 01:37 PM
          huh??Doug S2015/07/04 09:46 PM
            huh??Ungo2015/07/04 11:57 PM
          huh??Andrei F.2015/07/05 02:43 AM
            huh??Linus Torvalds2015/07/05 10:59 AM
              huh??Gabriele Svelto2015/07/05 11:49 AM
                huh??anonymous92015/07/05 01:19 PM
                  huh??anon2015/07/06 06:26 PM
                huh??Linus Torvalds2015/07/05 01:34 PM
                huh??Doug S2015/07/05 04:41 PM
                  huh??none2015/07/05 10:58 PM
            These tests are measuring something elseDavid Kanter2015/07/06 12:30 AM
              These tests are measuring something elseMaynard Handley2015/07/06 09:38 AM
                These tests are measuring something elseLinus Torvalds2015/07/06 12:00 PM
                These tests are measuring something elseMichael S2015/07/07 01:17 AM
                  These tests are measuring something elsenone2015/07/07 03:48 AM
                  These tests are measuring something elseLinus Torvalds2015/07/07 08:51 AM
                    These tests are measuring something elseEduardoS2015/07/07 01:41 PM
                      These tests are measuring something elseLinus Torvalds2015/07/08 09:32 AM
                  These tests are measuring something elseDoug S2015/07/07 10:11 AM
                    These tests are measuring something elsenone2015/07/07 10:41 PM
                      These tests are measuring something elseEduardoS2015/07/08 03:10 AM
                      These tests are measuring something elseLinus Torvalds2015/07/08 09:44 AM
                        These tests are measuring something elseGabriele Svelto2015/07/09 12:08 AM
                  These tests are measuring something elseMaynard Handley2015/07/07 11:18 AM
                    These tests are measuring something elseEduardoS2015/07/07 01:40 PM
                    These tests are measuring something elseDoug S2015/07/07 09:11 PM
                    These tests are measuring something elseMichael S2015/07/08 12:28 AM
                      These tests are measuring something elseDavid Kanter2015/07/08 04:53 PM
            huh??someotherdude2015/07/06 12:53 AM
            huh??Wilco2015/07/07 10:32 AM
              huh??Exophase2015/07/07 10:48 AM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell green?