Ian Ollmann (iano@apple.com) on 7/19/10 wrote:
---------------------------
>David Kanter (dkanter@realworldtech.com) on 7/9/10 wrote:
>---------------------------
>>Aaron Spink (aaronspink@notearthlink.net) on 7/8/10 wrote:
>>---------------------------
>>>anon (anon@anon.anon) on 7/8/10 wrote:
>>>---------------------------
>>>>so how do you get the level of efficiency? sse4 (?) and lots of hand-tuned code?
>>>>
>>>good compilers, good libraries, easy to optimize target >architecture. ICC, MKL/IPP/TBB, x86.
>>
>>It's also about having a microarchitecture that specifically invests a lot of area/power
>>into techniques that improve utilization. Out-of-order execution comes to mind
>>here, and techniques like branch prediction, etc. that let a CPU execute latency
>>bound code (e.g. things that are not totally parallel) with good performance.
>
>I personally find the OoO engine to be a bit of a hinderance to performance. Its
>great for the massively overwhelming amount of code out there written with a C/C++
>compiler with not too much thought to performance. (read: 99.99%+ of code) There,
>you might get 10% of peak instead of 3% of peak. However, when you are trying to
>get the last drop out of the architecture and are willing to go to assembly, something
>with a simpler execution engine, typically an in order machine with non-destructive
>ISA, is far easier to take to close to peak performance. In my experience, the
>OoO engine starts to trip over itself somewhere in the 50-80% peak performance range,
>and you start having to do a lot of "nonsense" optimizations (that might work only
>in your benchmark app on that particular core with that cache config and that code
>alignment on the page, etc) to get over that. These things usually break in the
>next gen, so often aren't worth it, unless you have a lot of time and money on your hands.

I don't see how an OOOE engine can possibly "trip over itself" simply due to being OOOE. Most heuristics or things that can go wrong such as caches, prefetching, and branch prediction are equally applicable and desirable on any performance oriented in-order architecture too.

Are you worried that it may have the ability to follow further down speculative branch chains, increasing the likeliness it will do incorrect speculative calculations and memory loads?

Profiling can be a lot harder because you can see cycles accumulate where the engine runs out of work to do rather than where it caused the longest latency event, and at mispredicts. Haven't profiled on in-order architectures for a long time, but IIRC non blocking caches and branch prediction gives similar problems.