anon (anon.delete@this.anon.com) on July 11, 2013 11:45 pm wrote:
> Klimax (danklima.delete@this.gmail.com) on July 11, 2013 11:25 pm wrote:
> > anon (anon.delete@this.anon.com) on July 11, 2013 2:50 am wrote:
> > > Klimax (danklima.delete@this.gmail.com) on July 10, 2013 11:30 pm wrote:
> > > > anon (anon.delete@this.anon.com) on July 10, 2013 10:29 pm wrote:
> > > > > Klimax (danklima.delete@this.gmail.com) on July 10, 2013 10:03 pm wrote:
> > > > > > Wilco (Wilco.Dijkstra.delete@this.ntlworld.com) on July 10, 2013 7:59 pm wrote:
> > > > > > > none (none.delete@this.none.com) on July 10, 2013 11:46 am wrote:
> > > > > > > > none (none.delete@this.none.com) on June 9, 2013 3:11 pm wrote:
> > > > > > > > > tarlinian (tarlinian.delete@this.gmail.com) on June 9, 2013 2:00 pm wrote:
> > > > > > > > > > Kevin G (kevin.delete@this.cubitdesigns.com) on June 8, 2013 2:18 pm wrote:
> > > > > > > > > > > John (Jngu14.delete@this.gmail.com) on June 8, 2013 6:28 am wrote:
> > > > > > > > > > > > Jason (oxkct.delete@this.7tags.com) on June 7, 2013 1:56 pm wrote:
> > > > > > > > > > > > > I guess this explains why Samsung went with Intel over their own Exynos on the Galaxy
> > > > > > > > > > > > > Tab. Look at how the Lenovo K800 (Atom) compared to Galaxy S4 (Exynos/SnapDragon) :
> > > > > > > > > >
> > > > > > > > > > > A bit of Google-fu points toward any relevant information being behind a paywall at
> > > > > > > > > > > ABI Research. Everywhere else seems to be parroting the same Business Wire story.
> > > > > > > > > >
> > > > > > > > > > If anyone's interested, the ABI post acting as advertising for this work can
> > > > > > > > > > be found here. It has some interesting comment I'm not really sure why they
> > > > > > > > > > talk about current draw instead of power...it's a rather odd story.
> > > > > > > > >
> > > > > > > > > And I bet most of their benchmarks are from AnTuTu which seems to be the only CPU benchmark favoring Intel.
> > > > > > > >
> > > > > > > > It seems that indeed AnTuTu has a heavy bias towards Intel and that Abi Research used it:
> > > > > > > >
> > > > > > > > http://www.eetimes.com/author.asp?section_id=36&itc=eetimes_sitedefault&doc_id=1318857
> > > > > > >
> > > > > > > And Exophase figured out how Intel cheated AnTuTu.
> > > > > > >
> > > > > > > Wilco
> > > > > >
> > > > > > Proper optimization. If a loop or similar construct can be replaced, then problem is in benchmark
> > > > > > and its code and not in compiler. Same type of optimization as code elimination, because it
> > > > > > will never execute or is never used. Or memcpy/memset, which are often inlined for smaller
> > > > > > variable/array. (Interestingly, similar problems but the other way can be found in Geekbench
> > > > > > and likely other benchmarks penalizing Atom. See later post in your own link.)
> > > > > >
> > > > > > Intel can't be held responsible that ARM's compilers suck at optimizations.
> > > > >
> > > > > Totally incorrect.
> > > > >
> > > > > Firstly, they used GCC, not ARM's compiler.
> > > > >
> > > > > Secondly, they used specific optimization for the x86 core, and
> > > > > a generic optimization (and not even a very good one) for ARM.
> > > > >
> > > > > Finally, the use of ICC is not common. It is very true that in the real world, you
> > > > > cannot separate the CPU from the compiler. But if you go by the real world, you
> > > > > also have to accept that ICC is probably not the compiler that will be used.
> > > > >
> > > > > It's pretty obvious that the numbers being generated were way too optimistic.
> > > > > Sadly, the people paying for the test should really have picked up on this and
> > > > > got them to improve their methodology and make expectations more realistic.
> > > > >
> > > > > The fact they did not allow ARM on an equal footing I hope is not the same
> > > > > old story of over promising and under delivering from Atom cores.
> > > > >
> > > >
> > > > GCC is not known to be good optimization compiler. Maybe ARM camp should help GCC.
> > > > As for ICC, it is expensive, but not for students (Intel C++ Studio
> > > > XE is free for us) and some open source projects support it.
> > >
> > > None of that is correct. As Wilco said, GCC isn't too bad, and it is being improved.
> >
> > Better counterpoint would be to say what is focus of GCC.
>
> We know what is the focus of GCC. It is certainly not to win benchmarks for one particular architecture.

No. (Also irrelevant, because so far nobody showed it is bench-only!)
All language features. (In contrast VS compiler team goes for middle, where reasonable number of things get implemented while having code generated optimal.

> >
> > > At first glance, GCC looks pretty horrible, but look closer.
> > >
> > > http://www.spec.org/cpu2006/results/res2013q1/cpu2006-20130204-25469.html
> > > http://www.spec.org/cpu2006/results/res2013q1/cpu2006-20130204-25471.html
> > >
> > > Now if you look at the base results, GCC is within about 5% of the IBM compiler
> > > on POWER7+. In some of the benchmarks GCC is significantly faster.
> >
> > And?
>
> And it shows GCC performs very well on integer for POWERPC.
>
> > We need comparison against ICC or VS (2012 preferably as it is current version
> > before 2013 ships) These results are bit orthogonal to our discussion.
> >
> > > As for ICC not being expensive, that's not the point. The issue is simply the reality of what is being
> > > used. Intel even had to implement ARM->x86 JIT because of the gap between theory and reality.
> > >
> > > It's not that GCC is a bad compiler so much (although it's probably at a small disadvantage
> > > compared to Intel or ARM compilers). It is that the compiler was changed to non standard,
> > > and options highly tuned. Whereas base options used for GCC.
> >
> > Frankly, options should be largely irrelevant otherwise benchmark is
> > broken. (The ones important should relate to instructions selection)
>
> No. Not if it can be vectorized, or unrolled, or use different
> instructions. These things can make very large differences.

Note that largely. Anyway not contesting, but same problem for all black box benchmarks.
(Geekbench)
> >
> > > >
> > > > Anyway, as I said if a loop is written as such that good optimizing compiler can replace it,
> > > > then benchmark is broken. (And I suspect it is only matter of time when GCC will gain that)
> > >
> > > I don't think that's necessarily completely true. If the transforms are realistic, then fine. But
> > > as you see in the analysis, GCC is using a generic target and not using NEON instructions or pipeline
> > > specific tuning. There is also suggestion that the transformation that Intel's compiler made is not
> > > generic but a special-cased one to help the benchmark. In that case, GCC may never gain such transformation
> > > because they don't tend to do such special case optimization, and doing it generically may *never*
> > > become a worthwhile cost/benefit (if the optimization is expensive to do).
> >
> > A suggestion is irrelelvant. It needs to be proven. See Sunspider js benchmark how it could be done.
>
> The burden of proof is on the people with the results and
> claiming the results are valid. Not the other way around.


Wrong. Original claim needs to be proven. (Existence of modification to original code that would break it)

You won't get to shift the burden of proof. There's original claim, prove it.

See Sunspider js controversy for good example.

> >
> > > >
> > > > Iin the end, it's story about GCC vs. ICC, which is known for
> > > > a long time to be very unequal and badly written benchmark.
> > > >
> > > > So the only incorrect thing was about ARM compiler and rest stands.
> > >
> > > If, by "stands", you mean "is utterly useless as any measure of comparison", then I agree.
> > >
> > > But we already knew that didn't we? It was obvious, as soon as I simply read the
> > > headlines, that the fantastical claims of Intel's core were too good to be true.
> > >
> >
> > Broken benchmarks are broken benchmarks. Both sides love them...
>
> What does that even mean? Irrelevant. We're discussing this particular benchmark,
> which was very dishonestly stacked in favor of the Intel CPU. Whether or not
> any other companies do something similar, has no bearing on that fact.

Not so.