Intel is cheating AnTuTu

By: anon (anon.delete@this.anon.com), July 18, 2013 12:05 pm
Room: Moderated Discussions
Klimax (danklima.delete@this.gmail.com) on July 18, 2013 11:39 am wrote:
> anon (anon.delete@this.anon.com) on July 18, 2013 2:10 am wrote:
> > Klimax (danklima.delete@this.gmail.com) on July 18, 2013 1:41 am wrote:
> > > none (none.delete@this.none.com) on July 17, 2013 11:35 pm wrote:
> > > > Klimax (danklima.delete@this.gmail.com) on July 17, 2013 11:29 pm wrote:
> > > > > Wilco (Wilco.Dijkstra.delete@this.ntlworld.com) on July 17, 2013 3:33 pm wrote:
> > > > > > bakaneko (nyan.delete@this.hyan.wan) on July 16, 2013 5:50 am wrote:
> > > > > > > Wilco (Wilco.Dijkstra.delete@this.ntlworld.com) on July 16, 2013 3:47 am wrote:
> > > > > > > > bakaneko (nyan.delete@this.hyan.wan) on July 16, 2013 2:42 am wrote:
> > > > > > > > > Linus Torvalds (torvalds.delete@this.linux-foundation.org) on July 15, 2013 8:45 pm wrote:
> > > > > > > > > > bakaneko (nyan.delete@this.hyan.wan) on July 15, 2013 7:47 pm wrote:
> > > > > > > > > > >
> > > > > > > > > > > As someone who works for ARM as compiler writer,
> > > > > > > > > > > why don't you tell us in more detail how Intel
> > > > > > > > > > > cheated?
> > > > > > > > > >
> > > > > > > > > > Exophase's post to anandtech was quoted here earlier, I think. It has the relevant details:
> > > > > > > > > >
> > > > > > > > > > http://forums.anandtech.com/showthread.php?t=2330027
> > > > > > > > > >
> > > > > > > > > > and quite frankly, while optimizing multiple bit operations into a word is a very
> > > > > > > > > > valid optimization, the code icc generates there seems a fair bit past that.
> > > > > > > > > >
> > > > > > > > > > Sure, it could in theory happen with a really smart compiler and lots of generic optimizations.
> > > > > > > > > > In practice? It really smells like the compiler actively targeting a very particular code-sequence.
> > > > > > > > > > IOW, compiler cheating. The timing that Exophase points out makes it look worse.
> > > > > > > > > >
> > > > > > > > > > And Wilco is right that it smells pretty bad when AnTuTu seems to be so close to
> > > > > > > > > > intel, and seem to have bent over backwards using recent versions of icc etc.
> > > > > > > > > >
> > > > > > > > > > It's all "explainable". But it doesn't pass the smell test.
> > > > > > > > >
> > > > > > > > > It's also all besides the point. I would expect a better
> > > > > > > > > explanation from someone who claims to know their shit about
> > > > > > > > > compilers than a logical fallacy ("Intel improved their
> > > > > > > > > compiler recently", "Intel is cheating because it hits one
> > > > > > > > > function in a certain benchmark"). Instead he nags like his
> > > > > > > > > marriage has gone bad.
> > > > > > > >
> > > > > > > > If you had actually read the whole thread including the links to various articles I posted
> > > > > > > > then you would have found the detailed explanation that makes it obvious that Intel has been
> > > > > > > > cheating AnTuTu. Why should I explain all the details again in every post I make?
> > > > > > >
> > > > > > > I read the whole thread, but for some reason don't remember
> > > > > > > anything noteworthy.
> > > > > > >
> > > > > > > > > And no, the ICC results aren't that far fetched. Intel
> > > > > > > > > actually recommends -O3 -xSSSE3_ATOM* with the NDK. Which
> > > > > > > > > could also explain why Exophase saw optimizations which
> > > > > > > > > would be counterproductive for bigger programs. (If the
> > > > > > > > > flags have similar meaning to gcc, where inlining and loop
> > > > > > > > > unrolling have similar problems.)
> > > > > > > >
> > > > > > > > Given the ICC results already dropped 20% after minor changes in AnTuTu it seems this
> > > > > > > > optimization is no longer effective. And that proves it was very specific to the actual
> > > > > > > > source code rather than a generic optimization that any compiler does.
> > > > > > >
> > > > > > > How is setting/clearing a range of bits not a useful
> > > > > > > optimization?
> > > > > >
> > > > > > Nobody sets multiple adjacent bits in memory one at a time. Even if one did so it would
> > > > > > typically be a few bits, certainly not hundreds or thousands. Having seen and written
> > > > > > various implementations of bit-sets I know how the typical ones look like.
> > > > > >
> > > > > > > And how does this - just because this is not an optimization
> > > > > > > any compiler does (Or at least gcc with -Os) - suddenly make
> > > > > > > this an benchmark busting trick? Oh, yes because the time
> > > > > > > frame fits.
> > > > > >
> > > > > > Nobody writes code like this, so no compiler implements this optimization. So yes, ICC gaining
> > > > > > this optimization just before AnTuTu switched to ICC is proof that the optimization was added
> > > > > > to break AnTuTu. If ICC had implemented this particular optimization for many years then it
> > > > > > would be a different matter of course (although that still doesn't explain exactly why AnTuTu
> > > > > > switched to a non-standard compiler with options tuned for AnTuTu just for x86).
> > > > > So far no proof yet. Do you have any? Any evidence at all?
> > > >
> > > > Show us code that's from real code that benefits from this optimization.
> > >
> > > No one at hand, but Flash files have a lot of variable bitfields. Also same kind of code
> > > prompting std::vector... (So why such unrealistic bit manipulation in a benchmark?)
> > >
> > > Anyway, claimants that Intel cheats MUST prove it, no shifting
> > > of burden of proof. So any evidence of cheating yet?
> >
> > This isn't a criminal court case. It's speculation, and yes there is circumstantial evidence, if you
> > had followed the thread. From the evidence, I would say there is a pretty good chance that Intel made
> > this optimization for this particular benchmark. There is also some non-zero chance it was a coincidence.
> So far there is no such thing as evidence of such cheat. Only baseless
> speculation and looking for timings (without any facts for that).
>
> Like nobody posted loop similar to that used by benchmark (like
> reconstructed from disassembly) being left intact by ICC.
>
> Also, when there are accusations of cheating you should post proper evidence.

Evidence was posted. Look up what circumstantial means. Several people who know more than you about the matter have agreed that it is likely to be an ICC special case.

>
> > Clearly you'll never get 100% indisputable proof (without seeing ICC source code).
> >
> > What we do know is that it was not a good benchmark of hardware capabilities,
> > and it was clearly rigged in favor of the Intel chip.
> >
> > I think we have to just leave it at that.
>
> No. It was broken benchmark. Nothing more. (So far)
>
> That's that.

No, when I say it was rigged in favor of Intel, I mean they used non supported compiler and tuned options. They claim "it was to get the best performance from the chip", but conveniently failed to do the same for the ARM compilation.

I'm talking about 2 issues: one with ICC optimizer, the other with benchmark setup.
< Previous Post in ThreadNext Post in Thread >
TopicPosted ByDate
ARM at Computexanon2013/06/03 08:54 AM
  ARM at ComputexExophase2013/06/03 09:31 AM
    ARM at ComputexWilco2013/06/03 10:52 AM
      ARM at ComputexExophase2013/06/03 12:02 PM
        ARM at Computexnone2013/06/03 01:28 PM
          ARM at ComputexAlberto2013/06/04 10:41 AM
            ARM at Computexanon2013/06/04 01:36 PM
              ARM at ComputexWilco2013/06/04 03:56 PM
                ARM at ComputexDavid Kanter2013/06/04 04:51 PM
                  ARM at ComputexNobodyatAll2013/06/04 05:53 PM
                  ARM at Computexanon2013/06/04 06:53 PM
        ARM at ComputexWilco2013/06/03 01:47 PM
    ARM at Computexanon2013/06/04 04:03 AM
      ARM at ComputexDavid Kanter2013/06/04 08:16 AM
      ARM at ComputexExophase2013/06/04 10:36 AM
        ARM at ComputexWilco2013/06/04 04:09 PM
          ARM at ComputexNobodyatAll2013/06/04 04:23 PM
          ARM at ComputexDavid Kanter2013/06/04 05:00 PM
            ARM at Computexnone2013/06/05 05:26 AM
              ARM at Computexanon2013/06/05 07:19 AM
              ARM at Computexaaron spink2013/06/05 03:04 PM
                Saltwell Outperforming A15...With Less Power!!!Jason2013/06/07 01:56 PM
                  Saltwell Outperforming A15...With Less Power!!!John2013/06/08 06:28 AM
                    Saltwell Outperforming A15...With Less Power!!!Kevin G2013/06/08 02:18 PM
                      Saltwell Outperforming A15...With Less Power!!!David Kanter2013/06/08 08:43 PM
                      Saltwell Outperforming A15...With Less Power!!!tarlinian2013/06/09 02:00 PM
                        Saltwell Outperforming A15...With Less Power!!!none2013/06/09 03:11 PM
                          Saltwell Outperforming A15...With Less Power!!!none2013/07/10 11:46 AM
                            Saltwell Outperforming A15...With Less Power!!!Wilco2013/07/10 07:59 PM
                              Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/10 10:03 PM
                                Saltwell Outperforming A15...With Less Power!!!anon2013/07/10 10:29 PM
                                  Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/10 11:30 PM
                                    Saltwell Outperforming A15...With Less Power!!!Wilco2013/07/11 01:04 AM
                                      Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/11 03:10 AM
                                        Saltwell Outperforming A15...With Less Power!!!anon2013/07/11 04:10 AM
                                        Saltwell Outperforming A15...With Less Power!!!Wilco2013/07/11 04:43 AM
                                        Saltwell Outperforming A15...With Less Power!!!Steve2013/07/14 08:10 PM
                                          Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/14 10:55 PM
                                          Saltwell Outperforming A15...With Less Power!!!bakaneko2013/07/15 02:03 AM
                                            Saltwell Outperforming A15...With Less Power!!!anon2013/07/16 01:07 AM
                                              Saltwell Outperforming A15...With Less Power!!!bakaneko2013/07/16 06:25 AM
                                                Saltwell Outperforming A15...With Less Power!!!none2013/07/16 06:34 AM
                                                Saltwell Outperforming A15...With Less Power!!!Exophase2013/07/16 10:34 AM
                                                  Saltwell Outperforming A15...With Less Power!!!bakaneko2013/07/18 02:24 PM
                                                    Saltwell Outperforming A15...With Less Power!!!none2013/07/18 03:26 PM
                                    Saltwell Outperforming A15...With Less Power!!!anon2013/07/11 02:50 AM
                                      Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/11 11:25 PM
                                        Saltwell Outperforming A15...With Less Power!!!anon2013/07/11 11:45 PM
                                          Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/12 12:10 AM
                                            Saltwell Outperforming A15...With Less Power!!!anon2013/07/12 01:52 AM
                                    Saltwell Outperforming A15...With Less Power!!!Gabriele Svelto2013/07/11 04:20 AM
                                      Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/11 11:51 PM
                                Saltwell Outperforming A15...With Less Power!!!bakaneko2013/07/11 12:02 AM
                                  Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/11 12:28 AM
                                    Saltwell Outperforming A15...With Less Power!!!none2013/07/11 01:49 AM
                                      Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/11 03:00 AM
                                        Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/11 03:02 AM
                                        Saltwell Outperforming A15...With Less Power!!!none2013/07/11 04:44 AM
                                          Saltwell Outperforming A15...With Less Power!!!Michael S2013/07/11 05:33 AM
                                            Saltwell Outperforming A15...With Less Power!!!none2013/07/11 05:58 AM
                                              Saltwell Outperforming A15...With Less Power!!!none2013/07/11 05:59 AM
                                              Saltwell Outperforming A15...With Less Power!!!Wilco2013/07/11 07:40 AM
                                                Saltwell Outperforming A15...With Less Power!!!Michael S2013/07/11 08:34 AM
                                                  Saltwell Outperforming A15...With Less Power!!!none2013/07/11 08:50 AM
                                              Saltwell Outperforming A15...With Less Power!!!Michael S2013/07/11 09:02 AM
                                          Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/11 10:11 PM
                                      Saltwell Outperforming A15...With Less Power!!!Michael S2013/07/11 04:12 AM
                                        Saltwell Outperforming A15...With Less Power!!!none2013/07/11 04:51 AM
                                          Saltwell Outperforming A15...With Less Power!!!Wilco2013/07/11 05:03 AM
                                            Saltwell Outperforming A15...With Less Power!!!none2013/07/11 05:12 AM
                                              Saltwell Outperforming A15...With Less Power!!!Wilco2013/07/11 07:26 AM
                                                Saltwell Outperforming A15...With Less Power!!!none2013/07/11 08:29 AM
                                                Saltwell Outperforming A15...With Less Power!!!Wilco2013/07/12 04:08 AM
                                                  Saltwell Outperforming A15...With Less Power!!!Michael S2013/07/12 04:25 AM
                                              Saltwell Outperforming A15...With Less Power!!!Klimax2013/07/11 11:55 PM
                                                Saltwell Outperforming A15...With Less Power!!!Megol2013/07/13 07:20 AM
                                    Java/Dalvik benchmarksbakaneko2013/07/11 02:26 AM
                                  Saltwell Outperforming A15...With Less Power!!!Linus Torvalds2013/07/11 10:57 AM
                                    Saltwell Outperforming A15...With Less Power!!!Linus Torvalds2013/07/11 11:20 AM
                                      Saltwell Outperforming A15...With Less Power!!!Ian Ameline2013/07/11 03:20 PM
                                      Saltwell Outperforming A15...With Less Power!!!Sylvain Collange2013/07/12 02:29 AM
                                        Saltwell Outperforming A15...With Less Power!!!Wilco2013/07/12 03:51 AM
                                          Saltwell Outperforming A15...With Less Power!!!Sylvain Collange2013/07/12 06:00 AM
                                      Saltwell Outperforming A15...With Less Power!!!bakaneko2013/07/12 03:28 AM
                                        Saltwell Outperforming A15...With Less Power!!!Linus Torvalds2013/07/12 09:11 AM
                                          Saltwell Outperforming A15...With Less Power!!!bakaneko2013/07/12 10:12 AM
                                          Saltwell Outperforming A15...With Less Power!!!2013/07/12 10:48 AM
                                            Saltwell Outperforming A15...With Less Power!!!Linus Torvalds2013/07/12 11:14 AM
                                              JIT2013/07/13 05:50 AM
                                                How'd you get this flower symbol as your user name ?anon2013/07/13 09:20 AM
                                                  How'd you get this flower symbol as your user name ?Doug S Ꭳ❂〄☢⚓♜☎♬⚄☠2013/07/13 10:12 AM
                                                JITx2013/07/15 06:43 AM
                                    Not reallyAnon2013/07/11 11:21 AM
                                    consistency, pleaseMichael S2013/07/11 11:22 AM
                                      consistency, pleaseLinus Torvalds2013/07/11 11:52 AM
                                        consistency, pleaseMichael S2013/07/11 02:43 PM
                                          consistency, pleaseLinus Torvalds2013/07/11 03:12 PM
                                            consistency, pleaseMark Roulo2013/07/11 03:54 PM
                                              the discussion was about absence of AVX in Silvermont (NT)Michael S2013/07/11 03:59 PM
                                                the discussion was about absence of AVX in SilvermontMark Roulo2013/07/11 04:19 PM
                                                  the discussion was about absence of AVX in SilvermontLinus Torvalds2013/07/12 09:16 AM
                                                    Thank youMark Roulo2013/07/12 09:58 AM
                                                    the discussion was about absence of AVX in Silvermontanon2013/07/12 10:10 AM
                                                      Not really (again)Anon2013/07/12 10:26 AM
                                                        Not really (again)anon2013/07/12 05:40 PM
                                                    the discussion was about absence of AVX in SilvermontLinus Torvalds2013/07/12 10:31 AM
                                                      you are wrong at about 100% of you saidMichael S2013/07/13 11:22 AM
                                                        you are wrong at about 100% of you saidLinus Torvalds2013/07/13 12:08 PM
                                                          you are wrong at about 100% of you saidMichael S2013/07/13 01:13 PM
                                                            you are wrong at about 100% of you saidLinus Torvalds2013/07/13 02:11 PM
                                                            you are wrong at about 100% of you saidvvid2013/07/13 03:20 PM
                                                              you are wrong at about 100% of you saidMichael S2013/07/14 01:38 AM
                                    Saltwell Outperforming A15...With Less Power!!!anon2013/07/11 06:41 PM
                                    Saltwell Outperforming A15...With Less Power!!!bakaneko2013/07/12 03:22 AM
                  Round 2: Baytrail benchmark (yes Antutu!) crushes Snapdragon 800.sammy2013/07/12 11:28 AM
                    Round 2: AnTuTu scores drop dramatically for AtomWilco2013/07/12 11:59 AM
                      Round 2: AnTuTu scores drop dramatically for AtomSteve2013/07/14 08:16 PM
                        Round 2: AnTuTu scores drop dramatically for AtomKlimax2013/07/14 10:59 PM
                        Round 2: AnTuTu scores drop dramatically for AtomDavid Kanter2013/07/15 01:57 PM
                          Intel is cheating AnTuTuWilco2013/07/15 03:43 PM
                            Intel is cheating AnTuTubakaneko2013/07/15 07:47 PM
                              Intel is cheating AnTuTuLinus Torvalds2013/07/15 08:45 PM
                                Intel is cheating AnTuTubakaneko2013/07/16 02:42 AM
                                  Intel is cheating AnTuTuWilco2013/07/16 03:47 AM
                                    Intel is cheating AnTuTubakaneko2013/07/16 05:50 AM
                                      Intel is cheating AnTuTuWilco2013/07/17 03:33 PM
                                        Intel is cheating AnTuTuKlimax2013/07/17 11:29 PM
                                          Intel is cheating AnTuTunone2013/07/17 11:35 PM
                                            Intel is cheating AnTuTuKlimax2013/07/18 01:41 AM
                                              Intel is cheating AnTuTuanon2013/07/18 02:10 AM
                                                Intel is cheating AnTuTuMichael S2013/07/18 03:39 AM
                                                Intel is cheating AnTuTuWilco2013/07/18 04:07 AM
                                                  Intel is cheating AnTuTunone2013/07/18 04:15 AM
                                                    Intel is cheating AnTuTuWilco2013/07/18 05:18 AM
                                                    Intel is cheating AnTuTuMichael S2013/07/18 06:44 AM
                                                      Intel is cheating AnTuTunone2013/07/18 07:47 AM
                                                        strict aliasingMichael S2013/07/18 08:10 AM
                                                          strict aliasing rules have existed since C89 (NT)Ricardo B2013/07/18 05:28 PM
                                                  Intel is cheating AnTuTuKlimax2013/07/18 11:30 AM
                                                Intel is cheating AnTuTuKlimax2013/07/18 11:39 AM
                                                  Intel is cheating AnTuTuanon2013/07/18 12:05 PM
                                                    Intel is cheating AnTuTuKlimax2013/07/20 12:05 AM
                                                      Intel is cheating AnTuTunone2013/07/20 02:50 AM
                                                  Intel is cheating AnTuTunone2013/07/18 12:38 PM
                                                    Intel is cheating AnTuTuKlimax2013/07/19 11:57 PM
                                  Intel is cheating AnTuTuLinus Torvalds2013/07/16 12:32 PM
                                Intel is cheating AnTuTu2013/07/16 03:16 AM
                                  Intel is cheating AnTuTubakaneko2013/07/16 04:55 AM
                                    Intel is cheating AnTuTuWilco2013/07/17 03:46 PM
                                      Intel is cheating AnTuTubakaneko2013/07/18 02:05 PM
                                      Intel is cheating AnTuTuDavid Kanter2013/07/19 02:31 AM
                          Round 2: AnTuTu scores drop dramatically for Atomanon2013/07/15 04:43 PM
                            Round 2: AnTuTu scores drop dramatically for Atomnone2013/07/16 12:15 AM
                          Round 2: AnTuTu scores drop dramatically for AtomGroo2013/07/16 07:35 AM
        ARM at Computexanon2013/06/04 10:43 PM
Reply to this Topic
Name:
Email:
Topic:
Body: No Text
How do you spell avocado?