Jouni Osmala (josmala@cc.hut.fi) on 9/13/09 wrote:
---------------------------
>>>>Actually you never mentioned mispredict penalty until now (probably because I asked
>>>>if you mix it up with pipeline length).
>>>
>>>Its the primary negative effect that comes with pipeline length, its direct RESULT of the increased pipeline length.
>>>Also term critical loop while being more general term, but when talking with main
>>>pipeline length equals misprediction penalty. Other critical loops are not important in this discussion.
>>
>>I think one crucial point you're still missing (or just don't want to admit) is
>>that pipeline and mispr. penalty can grow not *just* because of OoOE, it comes along
>>as a natural by-product of pretty much ANY attempt to beef up a core with larger
>>and thus slower structures, more ex. units with larger and thus slower bypass networks
>>driving more loads, etc. etc. -- completely regardless of whether machine is static
>>or not. Plus, larger core has direct impact on the time to propagate signals across.
>
>Branch missprediction grows based on L1I cache and beefing up of FRONT-END. I'm
>for simpler front-end more raw power in back-end. The Signal propagation problem
>for larger core depends on WHERE the larger structures are...

Your point keeps evolving to something new with every single post. :) But at least finally something you want, not how OoOE is a killer of clock rate etc.

I hope you know that raw power in back-end (lots of EUs, if I get you right) is not free for in-orders. At all. Logic is still required to check data availability, and in addition to it and regardless of iO/OoO, the problem of data movement remains: you can't have lots of EUs and enjoy as fast data movement as between 2 of them w/o going for multicycle bypasses. Interesting that I have to explain it to you. ;)

>>Anyway. Here is one more datapoint for you to consider:
>>
>>Consider R4400: in-order, 8-stage monolithic pipeline with 3-cycle branch delay, 250 MHz on 0.35 um.
>
>http://www.cs.umass.edu/~weems/CmpSci635A/635lecture7.html
>
>Same applies here.
>2 stages for instruction selection 1 for register file and first execution stage.
>Everything after that is cache access...
>
>Now you are counting the L1D cache access time as cost for being inorder CPU ;)

Funny how it takes 8 cycles to complete ALU insn then. :) Actually it's true -- scroll to the place where the professor shows when ALU insn writes back its result.

>>And compare it with MIPS R10K: OoO, 5 stages int. pipeline, 200 MHz in 0.35um,
>
>http://www.stanford.edu/class/ee282h/handouts/Handout36.pdf
>
>2.5 stages for getting instruction, 0.5 for register file and 1 for first execution stage.
>
>Equal number of stages, in the frontend, 20% lower clockfrequency. -> TIME difference
>is equal to extra stage in front end.

Int. EX is stage 4 in both pipelines, R10K doing reg mapping, insn Qing and RF reading in stages 2 and 3, all of it. Nice try, dude. And let me remind you about PPC603 which executes ALU insns in stage 3 of its 4-stage pipeline, also doing reg renaming, of course.

And funny that you talk about 20% lower clock rate considering that your own reference cites 250 MHz figure on the next slide after citing 0.35 um process and other details. Explains a lot about the way you argue. ;)

>>2-3 cycle misprediction penalty (+ possible delays).
>>
>>Do the math, really.
>
>I have done, and you just don't even know the operands in the math. The math truly
>needs to be done two to three levels more detailed level than what you know. And
>having low number of simpler designs as examples from which could be found lower level details helps.
>
>Just having A number of all pipeline stages is VERY high level number that has
>millions of things unrelated to discussion in hand, multiple ways of what to count
>in the pipeline, and you haven't even started to get the front-end back-end difference
>;) You always count front-end+back-end when the costs in discussion are in the front
>end. There's the structures that increase the branchmisprediction penalty.

Do two things: a) scroll up where branch delay was mentioned in my post and b) check the figure where professor shows you in which stage ALU insn completes in this monolithic pipeline. :)

>People can add unlimited number of structures to a design to inorder pipeline,
>to make it more complex, including structures for OoO logic. I'd rather have somewhat simpler pipeline.

Fine. Here is your homework, Jouni Osmala: show an example of in-order CPU with a *shorter* pipeline than 4-stage OoO PPC603, while clocking as high -- 200 MHz in 0.35 and 300 MHz in 0.28 um. Needless to say, it should be a commercially available part.

>>>>>If you go read what each pipeline stage DOES then you understand that yes, there
>>>>>is logical reason why EV6 has another pipeline stage,
>>>>>
>>>>>I cut most of discussion to make my point simpler for you to understand by just
>>keeping it on single simplest example
>>>>
>>>>Actually some of the things you said in the previous post would be much more interesting
>>>>to discuss (e.g. that in-order machines are a better option from performance point
>>>of view). I'm *very* curious why. :)
>>>
>>>Less power per executable instruction -> more cores per given power budget -> faster if running hand optimized code.
>>
>>Very broken logic. The "less power per insn" is very much arguable. Do you want
>>examples? Consider in-order machine like Itanium vs same-company-made Core2. Both
>
>And newer manufacturing process x86 gets gives it better power efficiency in given
>time, especially now when they had quite a leap from 0.65u to 0.45u process in power
>efficiency. Newest x86 desktop processor that is manufactured on same process as current itanium is Prescott.

I have to correct you. 90-nm P-M was released a few months later.

>I have already established, that from hardware point view itanium has the structures
>of an OoO core but they are in different use than in normal OoO core.
>It has quite complex register renamer to handle the rotating registers, uhh. Thats
>one of the power costs of normal OoO processors over in-order. 2nd it has quite complex OoO Memory pipeline.

I seriously doubt that anything of what you mention consumes really lots of power and space and acts as a serious clock-rate limiter in Itanium cores (remembering your OoOE-related arguments), much easier for me to believe that it's limited by the 1-cycle 6-unit ALU network. Maybe L1D, or maybe both paths are similar. I wonder if anyone here can say for sure.

>>The "hand-optimized" part only works when you not only have lots of ILP to take
>>advantage of (presuming you mean very wide in-orders, not single- or dual-issue
>>ones), but insn latencies don't vary much during execution (so that static schedule
>>has little to gain from insn reordering), which is a *double* special case, limited
>>pretty much to array/stream processing tasks, and by far not all of them.
>
>My bias is that when I code, I find enough times a streaming solution although
>average coder would give you spaghetti solution. Thats one of my primary optimization methods.

Good for you, my point was simply that no manual optimization for in-order machine can give you what OoOE machine can do *when latency is not always what you expect it to be* in your branchless well-scheduled code.

>>I don't know what pipeline diagrams you used, EV5 pipeline which you can find in
>>original DEC papers shows memory pipe separately. So maybe you should start using
>>good references? Take a look at DTJ paper, it has a detailed pipeline diagram and
>>a very detailed description of the pipeline (vol. 7 no. 1 issue).
>
>The only DTJ version I could find on HP pages doesn't have any images in it. It
>seems HP is trying to save some bandwith costs ;(
>
>
>>>>Mispr. penalty grew from 5 cycle in EV5 to 7 cycles in EV6, 1 extra cycle is likely
>>>>due to 64k I$, the other extra cycle might well result from something unrelated to OoOE as well.
>>>
>>>Its exacly because OoO it has dependencies that either limit the clockfrequency
>>>(to less than half of alpha) or takes 3 stages. Its partially because OoO takes
>>>more area->Longer distance to transfer things AND more complex logic-> longer logic path.
>>>
>>>>And, btw, why don't you take into account that EV5 also increased mispr. penalty
>>>>by a cycle from EV4 while staying in-order (and I would suggest you check a few
>>>
>>>EV5 has slightly more complex instruction selection than EV4 & buffering stage
>>>to separate execution from instruction fetch which is something thats always seen
>>>in OoO pipelines but rarely in inorder pipelines.
>>>But that buffering&selection is still my point, there is extra work there that
>>>costs latency. What made EV5 longer than EV4 was the extra work it needed to do
>>>in those stages, checking and buffering instructions, just like OoO is extra work
>>>in selecting & buffering instructions. It clearly was worth it but it had costs...
>>>Of course they COULD of done it without adding extra stage, but then it would of
>>>cost clockspeed which would of been counter productive...
>>
>>Your point seems to make good progress towards becoming totally meaningless. Nothing
>>goes without costs. The more EUs you have (regardless of in-order or OoO core),
>>the larger and slower are your bypass networks; the larger the caches, the slower
>>they are, etc. etc. So is there any sense left in your point anymore, really?
>
>What everyone thinks as OoO execution is just a large bunch of buffers and logic
>gates designed to handle data dependencies and selecting instrucions. Those buffers
>are just larger and logic is more complex than in inorder pipeline, that costs time&power&area.

... just like, say, McK and derivative cores, which are very, very far from being power- and die-efficient despite the lack of dynamic scheduling.

>My standard point but not in this discussion is always reminding programmers that
>what ever features they want from processors always costs something.

Maybe that's why programmers shouldn't really design ISAs, at least alone (unless they're so brilliant as to design it with as good consideration of implementation impacts as hw people would do).

>>>>sources rather than just trust me)? Do you admit a possibility that one more extra
>>>>cycle of mispr. penalty in EV6 is also a result of physically larger, more capable
>>>>core, in which some circuits/units have higher latency as a direct consequence of
>>>>getting bigger, and the distances signals have to travel are longer? After all,
>>>
>>>OoO Logic costs AREA between instruction fetch and instruction execution -> that
>>>area increases distances -> OoO logic cost time between instruction execution&instruction fetch stages.
>>
>>Judging from facts, the losses are very small. Certainly small enough to justify
>>OoOE when performance and power efficiency are important.
>
>Performance yes, power efficiency is not always.
>
>>>>I hope you wouldn't characterize EV6's slower L1 caches as the "price" designers
>>>>had to pay for going OoOE. :) Or would you?
>>>
>>>I wouldn't, but one thing about that large cache is that it still has single cycle
>>>access to its array, so the effect of that is LESS than a single clock cycle, since
>>>it takes some time access even the 8kb array. D-cache latency has lot of logic before
>>>and after the array access thats why it is longer than 1 cycle.
>>
>>Another angle to look at it is that there was no easier way to design caches for
>>EV6 than to inherit them from EV5. At the same cycle time, it would still be 2-cycle
>>cache. The point is, apparently larger but 1-cycle-slower caches were judged to
>>be a better option. See? You can't have your cake AND eat it; a better design is
>>one that makes right tradeoffs in the course of design for given objective. Hopefully
>>now things are more clear for you and you can rethink some of the things discussed here armed with good references.
>
>Some of that everything is trade off you seem to have learned during this conversation
>so this probably wasn't complete waste of time.
>I'm just starting to think that I've stumbled at some anglo-cultural standard argumentation,
>rhetoric competion where people try to "WIN" arguments, when I'm just trying to
>fix programmers missunderstandings of hardware. Anyway I don't really have time
>for these arguments, and if I'd known before hand how much time I would have spend
>in writing here AND reading material everywhere on internet, I wouldn't of posted my original post.
>
>Ps. trying to win arguments agains non-english speakers on english rhetoric should
>be for like shooting fish in barrel for natives.

LOL, your English is okay for tech discussions (except for one or two cases when I really didn't understand you), and I wouldn't count myself in rhetoric camp at all, for we had a very fact-based argument (thanks to me) and have clarified a lot, haven't we (beginning from pipeline length, through which process Klamath was fabbed on, to what limited NetBurst uarches :))? Yeah, yeah, must have been done tens of times even here at RWT and by people with much deeper insights, but I don't regret, as the more I live, the more I'm convinced this is the best way to learn.

Don't forget about the homework, Jouni Osmala, I'm very curious if you can accomplish the task to back at least one of your opinions. ;)