hobold (hobold@vectorizer.org) on 7/19/10 wrote:
---------------------------
>anon (anon@anon.com) on 7/19/10 wrote:
>---------------------------
>[...]
>>I don't see how an OOOE engine can possibly "trip over itself" simply due to being
>>OOOE.
>
>In that strictness, your statement is probably correct. I guess the more fundamental
>issue behind the sentiment - that complex OoOE engines tend to make optimizing for
>theoretical peak throughput harder - lies in the more general idea that those machines
>are optimized for the "average case".
>
>Unfortunately, that average case is never even remotely close to maximum theoretical
>throughput. As a result, some of those complex OoOE machines cannot actually achieve
>peak theoretical throughput in practice, because a queue here or a buffer there
>does not have the capacity to keep up with the amount of in-flight instructions
>in that most extreme case. (An example of this are the loop epicycles in PPro and
>Netburst, where successive iterations of simple loops go through a deterministic
>but weird sequence of varying timing, depending on the current watermarks in the various queues and buffers.)
>
>I personally think this is one of the very few failures of the quantitative approach
>to processor design. On the face of it, optimizing these queues and those buffers
>for smaller size will in fact save a bit of the silicon and power budgets. But a
>consequence is that "no one achieves peak throughput anyway" becomes a self fulfilling prophecy.
>
>In my not so humble opinion, the fact that such highly >tuned code makes up only
>a tiny percentage of all existing software does not imply >an only tiny importance of such codes.

Obviously not. But I think the issue is that pretty much everyone in the mobile and PC space (including phones) has opted to spend transistors, power and design effort on improving the average case. For very specialize purpose applications, that trade-off may not make sense (e.g. game consoles).

But philosophically speaking, I think it makes a ton of sense to try and improve the average case, as long as you are not making things horrible for skilled users.

While only 1% of code may be tuned, it is probably widely replicated across systems. e.g. GPU drivers are quite tuned, and very common. The tricky part is that unfortunately, most PCs are bought for non-optimal code. So the balance between the two is very tricky.

>As Ian Ollmann mentioned, the real reason why "perfectly" >tuned code is so rare
>is that it tends to lose its optimality with every small >change of the hardware.

Which is probably why this only happens for consoles and deeply embedded applications.

>This would not be the case if the hardware could more >reliably be persuaded to run
>at peak throughput. If tuned code didn't break so easily, >there would be a larger
>gamut of it at any given time, and there would be more >business sense in taking the time to tune and maintain it.
>
>As it stands now, the self-fulfilling prophecy takes care that only code poets
>and desperados ever try to reach "light speed". From personal experience, I can
>tell you that it is very frustrating to see your code slow down on new hardware,
>considering the fact that you lovingly tuned it over the >course of quite some time.

I agree this is an issue, and it's one of the areas that I think CPUs need to step up a bit. In an ideal world, the CPU would only require optimizations that are forward compatible (e.g. use AVX or SSE, not x87), rather than one-time hacks. One of the nice things about the architecture of TMTA's chips (or a GPU) is that there is enough flexibility in the interface that the driver/JIT can be the one responsible for microarchitectural tweaks. This leaves the programmer/compiler free to focus on more generally useful optimizations.

David