Wilco (Wilco.Dijkstra.delete@this.ntlworld.com) on July 11, 2013 1:04 am wrote:
> Klimax (danklima.delete@this.gmail.com) on July 10, 2013 11:30 pm wrote:
> > anon (anon.delete@this.anon.com) on July 10, 2013 10:29 pm wrote:
> > > Klimax (danklima.delete@this.gmail.com) on July 10, 2013 10:03 pm wrote:
> > > > Wilco (Wilco.Dijkstra.delete@this.ntlworld.com) on July 10, 2013 7:59 pm wrote:
> > > > > none (none.delete@this.none.com) on July 10, 2013 11:46 am wrote:
> > > > > > none (none.delete@this.none.com) on June 9, 2013 3:11 pm wrote:
> > > > > > > tarlinian (tarlinian.delete@this.gmail.com) on June 9, 2013 2:00 pm wrote:
> > > > > > > > Kevin G (kevin.delete@this.cubitdesigns.com) on June 8, 2013 2:18 pm wrote:
> > > > > > > > > John (Jngu14.delete@this.gmail.com) on June 8, 2013 6:28 am wrote:
> > > > > > > > > > Jason (oxkct.delete@this.7tags.com) on June 7, 2013 1:56 pm wrote:
> > > > > > > > > > > I guess this explains why Samsung went with Intel over their own Exynos on the Galaxy
> > > > > > > > > > > Tab. Look at how the Lenovo K800 (Atom) compared to Galaxy S4 (Exynos/SnapDragon) :
> > > > > > > >
> > > > > > > > > A bit of Google-fu points toward any relevant information being behind a paywall at
> > > > > > > > > ABI Research. Everywhere else seems to be parroting the same Business Wire story.
> > > > > > > >
> > > > > > > > If anyone's interested, the ABI post acting as advertising for this work can
> > > > > > > > be found here. It has some interesting comment I'm not really sure why they
> > > > > > > > talk about current draw instead of power...it's a rather odd story.
> > > > > > >
> > > > > > > And I bet most of their benchmarks are from AnTuTu which seems to be the only CPU benchmark favoring Intel.
> > > > > >
> > > > > > It seems that indeed AnTuTu has a heavy bias towards Intel and that Abi Research used it:
> > > > > >
> > > > > > http://www.eetimes.com/author.asp?section_id=36&itc=eetimes_sitedefault&doc_id=1318857
> > > > >
> > > > > And Exophase figured out how Intel cheated AnTuTu.
> > > > >
> > > > > Wilco
> > > >
> > > > Proper optimization. If a loop or similar construct can be replaced, then problem is in benchmark
> > > > and its code and not in compiler. Same type of optimization as code elimination, because it
> > > > will never execute or is never used. Or memcpy/memset, which are often inlined for smaller
> > > > variable/array. (Interestingly, similar problems but the other way can be found in Geekbench
> > > > and likely other benchmarks penalizing Atom. See later post in your own link.)
>
> Claiming that Geekbench is favouring ARM without any hard evidence is a bit
> rich coming from Intel given they've just been exposed gaming AnTuTu.
>
> > > > Intel can't be held responsible that ARM's compilers suck at optimizations.
> > >
> > > Totally incorrect.
> > >
> > > Firstly, they used GCC, not ARM's compiler.
> > >
> > > Secondly, they used specific optimization for the x86 core, and
> > > a generic optimization (and not even a very good one) for ARM.
> > >
> > > Finally, the use of ICC is not common. It is very true that in the real world, you
> > > cannot separate the CPU from the compiler. But if you go by the real world, you
> > > also have to accept that ICC is probably not the compiler that will be used.
> > >
> > > It's pretty obvious that the numbers being generated were way too optimistic.
> > > Sadly, the people paying for the test should really have picked up on this and
> > > got them to improve their methodology and make expectations more realistic.
> > >
> > > The fact they did not allow ARM on an equal footing I hope is not the same
> > > old story of over promising and under delivering from Atom cores.
> > >
> >
> > GCC is not known to be good optimization compiler. Maybe ARM camp should help GCC.
>
> ARM has contributed to GCC for many years, and GCC has improved significantly recently.
Looks like nothing to do with optimizations.

> > As for ICC, it is expensive, but not for students (Intel C++ Studio
> > XE is free for us) and some open source projects support it.
> >
> > Anyway, as I said if a loop is written as such that good optimizing compiler can replace it,
> > then benchmark is broken. (And I suspect it is only matter of time when GCC will gain that)
>
> I agree any benchmark where a few compiler tricks make such a huge difference to the score is utterly
> broken. I'm sure it would be possible to implement the same trick(s) in GCC given enough effort.
> However even if AnTuTu wasn't already irrelevant as a CPU benchmark, breaking it certainly has made
> it completely useless, so it seems unlikely it is worth the effort. What do you prefer, 5-10% better
> performance across most applications or doubling the scores on a few benchmarks?

No argument about breaking. On the other hand, impact on real applications is much harder to measure. (Closest is IIRC PCMark and then question of weighting of points has to be answered)

In any case it doesn't mean ARM have shown they sped up real world use cases...

> > Iin the end, it's story about GCC vs. ICC, which is known for
> > a long time to be very unequal and badly written benchmark.
> >
> > So the only incorrect thing was about ARM compiler and rest stands.
>
> No - the story is about Intel's underhanded tactics of optimizing a closed source benchmark. The
> source code is not available, so even upgrading to a newer GCC or changing the options isn't possible.
> So how did Intel manage to replace GCC altogether? It seems likely they paid off AnTuTu.
>
> Wilco
>

No. It shows that Intel's optimization is still superior to others. (Closest one is VC not GCC/CLANG) One part of issue is, that nobody demonstrated that this particular optimization breaks upon small almost irrelevant change to code like it was done to Sunspider benchmark.

(With such observed loop in disassembly it shouldn't be hard to replicate in C/C++ and do mods and observe ASM/binary...)