Klimax (danklima.delete@this.gmail.com) on July 12, 2013 12:10 am wrote:
> anon (anon.delete@this.anon.com) on July 11, 2013 11:45 pm wrote:
> > Klimax (danklima.delete@this.gmail.com) on July 11, 2013 11:25 pm wrote:
> > > anon (anon.delete@this.anon.com) on July 11, 2013 2:50 am wrote:
> > > > Klimax (danklima.delete@this.gmail.com) on July 10, 2013 11:30 pm wrote:
> > > > > anon (anon.delete@this.anon.com) on July 10, 2013 10:29 pm wrote:
> > > > > > Klimax (danklima.delete@this.gmail.com) on July 10, 2013 10:03 pm wrote:
> > > > > > > Wilco (Wilco.Dijkstra.delete@this.ntlworld.com) on July 10, 2013 7:59 pm wrote:
> > > > > > > > none (none.delete@this.none.com) on July 10, 2013 11:46 am wrote:
> > > > > > > > > none (none.delete@this.none.com) on June 9, 2013 3:11 pm wrote:
> > > > > > > > > > tarlinian (tarlinian.delete@this.gmail.com) on June 9, 2013 2:00 pm wrote:
> > > > > > > > > > > Kevin G (kevin.delete@this.cubitdesigns.com) on June 8, 2013 2:18 pm wrote:
> > > > > > > > > > > > John (Jngu14.delete@this.gmail.com) on June 8, 2013 6:28 am wrote:
> > > > > > > > > > > > > Jason (oxkct.delete@this.7tags.com) on June 7, 2013 1:56 pm wrote:
> > > > > > > > > > > > > > I guess this explains why Samsung went with Intel over their own Exynos on the Galaxy
> > > > > > > > > > > > > > Tab. Look at how the Lenovo K800 (Atom) compared to Galaxy S4 (Exynos/SnapDragon) :
> > > > > > > > > > >
> > > > > > > > > > > > A bit of Google-fu points toward any relevant information being behind a paywall at
> > > > > > > > > > > > ABI Research. Everywhere else seems to be parroting the same Business Wire story.
> > > > > > > > > > >
> > > > > > > > > > > If anyone's interested, the ABI post acting as advertising for this work can
> > > > > > > > > > > be found here. It has some interesting comment I'm not really sure why they
> > > > > > > > > > > talk about current draw instead of power...it's a rather odd story.
> > > > > > > > > >
> > > > > > > > > > And I bet most of their benchmarks are from AnTuTu which seems to be the only CPU benchmark favoring Intel.
> > > > > > > > >
> > > > > > > > > It seems that indeed AnTuTu has a heavy bias towards Intel and that Abi Research used it:
> > > > > > > > >
> > > > > > > > > http://www.eetimes.com/author.asp?section_id=36&itc=eetimes_sitedefault&doc_id=1318857
> > > > > > > >
> > > > > > > > And Exophase figured out how Intel cheated AnTuTu.
> > > > > > > >
> > > > > > > > Wilco
> > > > > > >
> > > > > > > Proper optimization. If a loop or similar construct can be replaced, then problem is in benchmark
> > > > > > > and its code and not in compiler. Same type of optimization as code elimination, because it
> > > > > > > will never execute or is never used. Or memcpy/memset, which are often inlined for smaller
> > > > > > > variable/array. (Interestingly, similar problems but the other way can be found in Geekbench
> > > > > > > and likely other benchmarks penalizing Atom. See later post in your own link.)
> > > > > > >
> > > > > > > Intel can't be held responsible that ARM's compilers suck at optimizations.
> > > > > >
> > > > > > Totally incorrect.
> > > > > >
> > > > > > Firstly, they used GCC, not ARM's compiler.
> > > > > >
> > > > > > Secondly, they used specific optimization for the x86 core, and
> > > > > > a generic optimization (and not even a very good one) for ARM.
> > > > > >
> > > > > > Finally, the use of ICC is not common. It is very true that in the real world, you
> > > > > > cannot separate the CPU from the compiler. But if you go by the real world, you
> > > > > > also have to accept that ICC is probably not the compiler that will be used.
> > > > > >
> > > > > > It's pretty obvious that the numbers being generated were way too optimistic.
> > > > > > Sadly, the people paying for the test should really have picked up on this and
> > > > > > got them to improve their methodology and make expectations more realistic.
> > > > > >
> > > > > > The fact they did not allow ARM on an equal footing I hope is not the same
> > > > > > old story of over promising and under delivering from Atom cores.
> > > > > >
> > > > >
> > > > > GCC is not known to be good optimization compiler. Maybe ARM camp should help GCC.
> > > > > As for ICC, it is expensive, but not for students (Intel C++ Studio
> > > > > XE is free for us) and some open source projects support it.
> > > >
> > > > None of that is correct. As Wilco said, GCC isn't too bad, and it is being improved.
> > >
> > > Better counterpoint would be to say what is focus of GCC.
> >
> > We know what is the focus of GCC. It is certainly not to win benchmarks for one particular architecture.
>
> No. (Also irrelevant, because so far nobody showed it is bench-only!)
> All language features. (In contrast VS compiler team goes for middle, where reasonable
> number of things get implemented while having code generated optimal.
>
> > >
> > > > At first glance, GCC looks pretty horrible, but look closer.
> > > >
> > > > http://www.spec.org/cpu2006/results/res2013q1/cpu2006-20130204-25469.html
> > > > http://www.spec.org/cpu2006/results/res2013q1/cpu2006-20130204-25471.html
> > > >
> > > > Now if you look at the base results, GCC is within about 5% of the IBM compiler
> > > > on POWER7+. In some of the benchmarks GCC is significantly faster.
> > >
> > > And?
> >
> > And it shows GCC performs very well on integer for POWERPC.
> >
> > > We need comparison against ICC or VS (2012 preferably as it is current version
> > > before 2013 ships) These results are bit orthogonal to our discussion.
> > >
> > > > As for ICC not being expensive, that's not the point. The issue is simply the reality of what is being
> > > > used. Intel even had to implement ARM->x86 JIT because of the gap between theory and reality.
> > > >
> > > > It's not that GCC is a bad compiler so much (although it's probably at a small disadvantage
> > > > compared to Intel or ARM compilers). It is that the compiler was changed to non standard,
> > > > and options highly tuned. Whereas base options used for GCC.
> > >
> > > Frankly, options should be largely irrelevant otherwise benchmark is
> > > broken. (The ones important should relate to instructions selection)
> >
> > No. Not if it can be vectorized, or unrolled, or use different
> > instructions. These things can make very large differences.
>
> Note that largely. Anyway not contesting, but same problem for all black box benchmarks.
> (Geekbench)
> > >
> > > > >
> > > > > Anyway, as I said if a loop is written as such that good optimizing compiler can replace it,
> > > > > then benchmark is broken. (And I suspect it is only matter of time when GCC will gain that)
> > > >
> > > > I don't think that's necessarily completely true. If the transforms are realistic, then fine. But
> > > > as you see in the analysis, GCC is using a generic target and not using NEON instructions or pipeline
> > > > specific tuning. There is also suggestion that the transformation that Intel's compiler made is not
> > > > generic but a special-cased one to help the benchmark. In that case, GCC may never gain such transformation
> > > > because they don't tend to do such special case optimization, and doing it generically may *never*
> > > > become a worthwhile cost/benefit (if the optimization is expensive to do).
> > >
> > > A suggestion is irrelelvant. It needs to be proven. See Sunspider js benchmark how it could be done.
> >
> > The burden of proof is on the people with the results and
> > claiming the results are valid. Not the other way around.
>
>
> Wrong. Original claim needs to be proven. (Existence of modification to original code that would break it)
>
> You won't get to shift the burden of proof. There's original claim, prove it.
>
> See Sunspider js controversy for good example.
>
> > >
> > > > >
> > > > > Iin the end, it's story about GCC vs. ICC, which is known for
> > > > > a long time to be very unequal and badly written benchmark.
> > > > >
> > > > > So the only incorrect thing was about ARM compiler and rest stands.
> > > >
> > > > If, by "stands", you mean "is utterly useless as any measure of comparison", then I agree.
> > > >
> > > > But we already knew that didn't we? It was obvious, as soon as I simply read the
> > > > headlines, that the fantastical claims of Intel's core were too good to be true.
> > > >
> > >
> > > Broken benchmarks are broken benchmarks. Both sides love them...
> >
> > What does that even mean? Irrelevant. We're discussing this particular benchmark,
> > which was very dishonestly stacked in favor of the Intel CPU. Whether or not
> > any other companies do something similar, has no bearing on that fact.
>
> Not so.

I really don't know what you're talking about then. Your initial post was clearly so far off the mark that you hadn't even read the linked post. This is the claim I was referring to.

I don't know why all this other stuff has come up, but I don't think anybody claims sunspider is particularly good benchmark either. I certainly wasn't talking about other things. I don't really care to handwave about them without some link to actual numbers or proper analysis.