Intel is cheating AnTuTu

By: (0xe2.0x9a.0x9b.delete@this.gmail.com), July 16, 2013 3:16 am
Room: Moderated Discussions
Linus Torvalds (torvalds.delete@this.linux-foundation.org) on July 15, 2013 8:45 pm wrote:
> bakaneko (nyan.delete@this.hyan.wan) on July 15, 2013 7:47 pm wrote:
> >
> > As someone who works for ARM as compiler writer,
> > why don't you tell us in more detail how Intel
> > cheated?
>
> Exophase's post to anandtech was quoted here earlier, I think. It has the relevant details:
>
> http://forums.anandtech.com/showthread.php?t=2330027
>
> and quite frankly, while optimizing multiple bit operations into a word is a very
> valid optimization, the code icc generates there seems a fair bit past that.
>
> Sure, it could in theory happen with a really smart compiler and lots of generic optimizations.
> In practice? It really smells like the compiler actively targeting a very particular code-sequence.
> IOW, compiler cheating. The timing that Exophase points out makes it look worse.
>
> And Wilco is right that it smells pretty bad when AnTuTu seems to be so close to
> intel, and seem to have bent over backwards using recent versions of icc etc.
>
> It's all "explainable". But it doesn't pass the smell test.
>
> Linus

The optimization is clearly doable by a machine. In my opinion this means there is no reason to criticize the compiler or the compiler team for adding the optimization to the compiler. The blame should go towards people who published the benchmark results without making it clear that the results are a mixture of raw CPU performance and compiler optimizations in the context of a particular benchmark.

It isn't compiler cheating. It is misattribution of benchmark results. The improved benchmark numbers should have been attributed to both the CPU and the compiler, rather than just to the CPU alone.

I think it would be best for benchmarks to take into account the number of executed instructions. Seeing the numbers of executed instructions when comparing benchmarks would make it easier to distinguish CPU performance from compiler performance (and from other stuff). It would be nice for this to become the standard way of reporting benchmarks by benchmarking sites. It would be interesting to see similar numbers in GPU benchmarks.

-Atom
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
ARM at Computexanon2013/06/03 08:54 AM
  ARM at ComputexExophase2013/06/03 09:31 AM
    ARM at ComputexWilco2013/06/03 10:52 AM
      ARM at ComputexExophase2013/06/03 12:02 PM
        ARM at Computexnone2013/06/03 01:28 PM
          ARM at ComputexAlberto2013/06/04 10:41 AM
            ARM at Computexanon2013/06/04 01:36 PM
              ARM at ComputexWilco2013/06/04 03:56 PM
                ARM at ComputexDavid Kanter2013/06/04 04:51 PM
                  ARM at ComputexNobodyatAll2013/06/04 05:53 PM
                  ARM at Computexanon2013/06/04 06:53 PM
        ARM at ComputexWilco2013/06/03 01:47 PM
    ARM at Computexanon2013/06/04 04:03 AM
      ARM at ComputexDavid Kanter2013/06/04 08:16 AM
      ARM at ComputexExophase2013/06/04 10:36 AM
        ARM at ComputexWilco2013/06/04 04:09 PM
          ARM at ComputexNobodyatAll2013/06/04 04:23 PM
          ARM at ComputexDavid Kanter2013/06/04 05:00 PM
            ARM at Computexnone2013/06/05 05:26 AM
              ARM at Computexanon2013/06/05 07:19 AM
              ARM at Computexaaron spink2013/06/05 03:04 PM
                Saltwell Outperforming A15...With Less Power!!!Jason2013/06/07 01:56 PM
                  Saltwell Outperforming A15...With Less Power!!!John2013/06/08 06:28 AM
                    Saltwell Outperforming A15...With Less Power!!!Kevin G2013/06/08 02:18 PM
                      Saltwell Outperforming A15...With Less Power!!!David Kanter2013/06/08 08:43 PM
                      Saltwell Outperforming A15...With Less Power!!!tarlinian2013/06/09 02:00 PM
                        Saltwell Outperforming A15...With Less Power!!!none2013/06/09 03:11 PM
                          Saltwell Outperforming A15...With Less Power!!!none2013/07/10 11:46 AM
                            Saltwell Outperforming A15...With Less Power!!!Wilco2013/07/10 07:59 PM
                              Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/10 10:03 PM
                                Saltwell Outperforming A15...With Less Power!!!anon2013/07/10 10:29 PM
                                  Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/10 11:30 PM
                                    Saltwell Outperforming A15...With Less Power!!!Wilco2013/07/11 01:04 AM
                                      Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/11 03:10 AM
                                        Saltwell Outperforming A15...With Less Power!!!anon2013/07/11 04:10 AM
                                        Saltwell Outperforming A15...With Less Power!!!Wilco2013/07/11 04:43 AM
                                        Saltwell Outperforming A15...With Less Power!!!Steve2013/07/14 08:10 PM
                                          Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/14 10:55 PM
                                          Saltwell Outperforming A15...With Less Power!!!bakaneko2013/07/15 02:03 AM
                                            Saltwell Outperforming A15...With Less Power!!!anon2013/07/16 01:07 AM
                                              Saltwell Outperforming A15...With Less Power!!!bakaneko2013/07/16 06:25 AM
                                                Saltwell Outperforming A15...With Less Power!!!none2013/07/16 06:34 AM
                                                Saltwell Outperforming A15...With Less Power!!!Exophase2013/07/16 10:34 AM
                                                  Saltwell Outperforming A15...With Less Power!!!bakaneko2013/07/18 02:24 PM
                                                    Saltwell Outperforming A15...With Less Power!!!none2013/07/18 03:26 PM
                                    Saltwell Outperforming A15...With Less Power!!!anon2013/07/11 02:50 AM
                                      Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/11 11:25 PM
                                        Saltwell Outperforming A15...With Less Power!!!anon2013/07/11 11:45 PM
                                          Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/12 12:10 AM
                                            Saltwell Outperforming A15...With Less Power!!!anon2013/07/12 01:52 AM
                                    Saltwell Outperforming A15...With Less Power!!!Gabriele Svelto2013/07/11 04:20 AM
                                      Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/11 11:51 PM
                                Saltwell Outperforming A15...With Less Power!!!bakaneko2013/07/11 12:02 AM
                                  Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/11 12:28 AM
                                    Saltwell Outperforming A15...With Less Power!!!none2013/07/11 01:49 AM
                                      Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/11 03:00 AM
                                        Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/11 03:02 AM
                                        Saltwell Outperforming A15...With Less Power!!!none2013/07/11 04:44 AM
                                          Saltwell Outperforming A15...With Less Power!!!Michael S2013/07/11 05:33 AM
                                            Saltwell Outperforming A15...With Less Power!!!none2013/07/11 05:58 AM
                                              Saltwell Outperforming A15...With Less Power!!!none2013/07/11 05:59 AM
                                              Saltwell Outperforming A15...With Less Power!!!Wilco2013/07/11 07:40 AM
                                                Saltwell Outperforming A15...With Less Power!!!Michael S2013/07/11 08:34 AM
                                                  Saltwell Outperforming A15...With Less Power!!!none2013/07/11 08:50 AM
                                              Saltwell Outperforming A15...With Less Power!!!Michael S2013/07/11 09:02 AM
                                          Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/11 10:11 PM
                                      Saltwell Outperforming A15...With Less Power!!!Michael S2013/07/11 04:12 AM
                                        Saltwell Outperforming A15...With Less Power!!!none2013/07/11 04:51 AM
                                          Saltwell Outperforming A15...With Less Power!!!Wilco2013/07/11 05:03 AM
                                            Saltwell Outperforming A15...With Less Power!!!none2013/07/11 05:12 AM
                                              Saltwell Outperforming A15...With Less Power!!!Wilco2013/07/11 07:26 AM
                                                Saltwell Outperforming A15...With Less Power!!!none2013/07/11 08:29 AM
                                                Saltwell Outperforming A15...With Less Power!!!Wilco2013/07/12 04:08 AM
                                                  Saltwell Outperforming A15...With Less Power!!!Michael S2013/07/12 04:25 AM
                                              Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/11 11:55 PM
                                                Saltwell Outperforming A15...With Less Power!!!Megol2013/07/13 07:20 AM
                                    Java/Dalvik benchmarksbakaneko2013/07/11 02:26 AM
                                  Saltwell Outperforming A15...With Less Power!!!Linus Torvalds2013/07/11 10:57 AM
                                    Saltwell Outperforming A15...With Less Power!!!Linus Torvalds2013/07/11 11:20 AM
                                      Saltwell Outperforming A15...With Less Power!!!Ian Ameline2013/07/11 03:20 PM
                                      Saltwell Outperforming A15...With Less Power!!!Sylvain Collange2013/07/12 02:29 AM
                                        Saltwell Outperforming A15...With Less Power!!!Wilco2013/07/12 03:51 AM
                                          Saltwell Outperforming A15...With Less Power!!!Sylvain Collange2013/07/12 06:00 AM
                                      Saltwell Outperforming A15...With Less Power!!!bakaneko2013/07/12 03:28 AM
                                        Saltwell Outperforming A15...With Less Power!!!Linus Torvalds2013/07/12 09:11 AM
                                          Saltwell Outperforming A15...With Less Power!!!bakaneko2013/07/12 10:12 AM
                                          Saltwell Outperforming A15...With Less Power!!!2013/07/12 10:48 AM
                                            Saltwell Outperforming A15...With Less Power!!!Linus Torvalds2013/07/12 11:14 AM
                                              JIT2013/07/13 05:50 AM
                                                How'd you get this flower symbol as your user name ?anon2013/07/13 09:20 AM
                                                  How'd you get this flower symbol as your user name ?Doug S Ꭳ❂〄☢⚓♜☎♬⚄☠2013/07/13 10:12 AM
                                                JITx2013/07/15 06:43 AM
                                    Not reallyAnon2013/07/11 11:21 AM
                                    consistency, pleaseMichael S2013/07/11 11:22 AM
                                      consistency, pleaseLinus Torvalds2013/07/11 11:52 AM
                                        consistency, pleaseMichael S2013/07/11 02:43 PM
                                          consistency, pleaseLinus Torvalds2013/07/11 03:12 PM
                                            consistency, pleaseMark Roulo2013/07/11 03:54 PM
                                              the discussion was about absence of AVX in Silvermont (NT)Michael S2013/07/11 03:59 PM
                                                the discussion was about absence of AVX in SilvermontMark Roulo2013/07/11 04:19 PM
                                                  the discussion was about absence of AVX in SilvermontLinus Torvalds2013/07/12 09:16 AM
                                                    Thank youMark Roulo2013/07/12 09:58 AM
                                                    the discussion was about absence of AVX in Silvermontanon2013/07/12 10:10 AM
                                                      Not really (again)Anon2013/07/12 10:26 AM
                                                        Not really (again)anon2013/07/12 05:40 PM
                                                    the discussion was about absence of AVX in SilvermontLinus Torvalds2013/07/12 10:31 AM
                                                      you are wrong at about 100% of you saidMichael S2013/07/13 11:22 AM
                                                        you are wrong at about 100% of you saidLinus Torvalds2013/07/13 12:08 PM
                                                          you are wrong at about 100% of you saidMichael S2013/07/13 01:13 PM
                                                            you are wrong at about 100% of you saidLinus Torvalds2013/07/13 02:11 PM
                                                            you are wrong at about 100% of you saidvvid2013/07/13 03:20 PM
                                                              you are wrong at about 100% of you saidMichael S2013/07/14 01:38 AM
                                    Saltwell Outperforming A15...With Less Power!!!anon2013/07/11 06:41 PM
                                    Saltwell Outperforming A15...With Less Power!!!bakaneko2013/07/12 03:22 AM
                  Round 2: Baytrail benchmark (yes Antutu!) crushes Snapdragon 800.sammy2013/07/12 11:28 AM
                    Round 2: AnTuTu scores drop dramatically for AtomWilco2013/07/12 11:59 AM
                      Round 2: AnTuTu scores drop dramatically for AtomSteve2013/07/14 08:16 PM
                        Round 2: AnTuTu scores drop dramatically for AtomKlimax2013/07/14 10:59 PM
                        Round 2: AnTuTu scores drop dramatically for AtomDavid Kanter2013/07/15 01:57 PM
                          Intel is cheating AnTuTuWilco2013/07/15 03:43 PM
                            Intel is cheating AnTuTubakaneko2013/07/15 07:47 PM
                              Intel is cheating AnTuTuLinus Torvalds2013/07/15 08:45 PM
                                Intel is cheating AnTuTubakaneko2013/07/16 02:42 AM
                                  Intel is cheating AnTuTuWilco2013/07/16 03:47 AM
                                    Intel is cheating AnTuTubakaneko2013/07/16 05:50 AM
                                      Intel is cheating AnTuTuWilco2013/07/17 03:33 PM
                                        Intel is cheating AnTuTuKlimax2013/07/17 11:29 PM
                                          Intel is cheating AnTuTunone2013/07/17 11:35 PM
                                            Intel is cheating AnTuTuKlimax2013/07/18 01:41 AM
                                              Intel is cheating AnTuTuanon2013/07/18 02:10 AM
                                                Intel is cheating AnTuTuMichael S2013/07/18 03:39 AM
                                                Intel is cheating AnTuTuWilco2013/07/18 04:07 AM
                                                  Intel is cheating AnTuTunone2013/07/18 04:15 AM
                                                    Intel is cheating AnTuTuWilco2013/07/18 05:18 AM
                                                    Intel is cheating AnTuTuMichael S2013/07/18 06:44 AM
                                                      Intel is cheating AnTuTunone2013/07/18 07:47 AM
                                                        strict aliasingMichael S2013/07/18 08:10 AM
                                                          strict aliasing rules have existed since C89 (NT)Ricardo B2013/07/18 05:28 PM
                                                  Intel is cheating AnTuTuKlimax2013/07/18 11:30 AM
                                                Intel is cheating AnTuTuKlimax2013/07/18 11:39 AM
                                                  Intel is cheating AnTuTuanon2013/07/18 12:05 PM
                                                    Intel is cheating AnTuTuKlimax2013/07/20 12:05 AM
                                                      Intel is cheating AnTuTunone2013/07/20 02:50 AM
                                                  Intel is cheating AnTuTunone2013/07/18 12:38 PM
                                                    Intel is cheating AnTuTuKlimax2013/07/19 11:57 PM
                                  Intel is cheating AnTuTuLinus Torvalds2013/07/16 12:32 PM
                                Intel is cheating AnTuTu2013/07/16 03:16 AM
                                  Intel is cheating AnTuTubakaneko2013/07/16 04:55 AM
                                    Intel is cheating AnTuTuWilco2013/07/17 03:46 PM
                                      Intel is cheating AnTuTubakaneko2013/07/18 02:05 PM
                                      Intel is cheating AnTuTuDavid Kanter2013/07/19 02:31 AM
                          Round 2: AnTuTu scores drop dramatically for Atomanon2013/07/15 04:43 PM
                            Round 2: AnTuTu scores drop dramatically for Atomnone2013/07/16 12:15 AM
                          Round 2: AnTuTu scores drop dramatically for AtomGroo2013/07/16 07:35 AM
        ARM at Computexanon2013/06/04 10:43 PM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell avocado?