Nicolas Capens (nicolas.capens@gmail.com) on 2/23/11 wrote:
---------------------------
>Hi Michael,
>
>Michael S (already5chosen@yahoo.com) on 2/23/11 wrote:
>---------------------------
>>>>>Itanium supports FMA, but it's actually limited by the ability to schedule other
>>>>>instructions. There are specific scheduling limitations, and it can only load 2
>>>>>x 64-bit per cycle, while it has 4 FMA units.
>>>>
>>>>No, Itanium2 family CPUs have 2 FMA units.
>>>>
>>>>Or are you thinking about fpma? On all production (i.e. past-Merced) Itanium CPUs
>>>>it is effectively split in two "uOPs" and issued simultaneously to both F-type slots.
>>>>So no, on production Itaniums SIMD FP has the same 2-FMAs per clock throughput as
>>>>a scalar FP. That's why nobody uses it.
>>>
>>>Yes I'm thinking of fpma. According to the manual (http://download.intel.com/design/Itanium2/manuals/25110901.pdf),
>>>Itanium 2 has two extended-precision floating-point units, and two additional single-precision
>>>floating-point units, all of which support multiply-add. The performance monitor
>>>also counts fpma as 4 floating-point operations.
>>>
>>>Maybe the assembler allows writing both fpma.s and fpma.d for symmetry, in which
>>>case the latter would actually be two fma.d's? Else I completely missed something.
>>>
>>
>>You are reading very old and incomplete manual.
>>Here is a (slightly) newer version:
>>http://download.intel.com/design/Itanium2/manuals/25111003.pdf
>>
>>Read SIMD FP paragraph on p.31. Also on p.17.
>>
>>Funny thing, this information doesn't appear in direct form in the newest edition of Itanium documentation.
>
>Those are different manuals (hardware versus software), not old and new versions
>of the same manual. But yeah, the documentation doesn't appear to be fully consistent or complete.
>
>>>>>Note that Tim didn't take into account that FMA reduces register pressure. Itanium
>>>>>has plenty of registers, but x86 does not so FMA is a bigger win.
>>>>
>>>>Come on, how exactly FMA affects register pressure on OoO machine with big renaming
>>>>registers pool where software don't have to schedule for latency of multiplier?
>>>
>>>To hide the latency of long dependency chains you need software pipelining, but
>>>this increases register pressure (software pipelining uses architectural registers,
>>>so renaming doesn't help here). FMA reduces the latency of the critical path a bit
>>>so less software pipelining is needed, keeping register pressure low.
>>>
>>>Granted, the effect is small, but FMA is a bigger win on register-impaired architectures like x86.
>>>
>>
>>Multiplicand-to-result is not on the critical path in majority of multiply-accumulate
>>heavy algorithms. So OoO does a fine job of removing false dependencies and software don't have to pipeline this part.
>
>Horner scheme polynomial evaluation puts both the multiplications and accumulations
>on the critical path. Estrin's scheme has a shorter critical path, but at a higher
>total operation count so that's not an option for high throughput. So polynomial
>evalution does potentially require software pipelining to hide the latency, and
>FMA can lessen that need and save registers.

Yes, for Horner scheme FMA is a clear win. And yes, I think that, except for special cases, programmers should prefer Horner over Estrin or similar artificial tricks esp. when working in single-precision. If for now other things then due to Horner's numerical properties being easier to analyze.

Just one comment. I don't think that the software technique that, in order to hide latency, runs several Horner evaluations in parallel is called software pipelining. It is a form of loop unrolling, I don't know what's it official academic name, but software pipelining it is not.
Now, some readers would understand what you mean, but you somehow managed to accumulate a bad karma on this forum, so not everybody will make an effort.


>
>Linear algebra doesn't get a latency benefit from FMA, but it's the perfect candidate
>to benefit from having more than one FMA unit (note that GF104 also does superscalar FMA issue).
>
>So unless I missed a major class of multiply-add heavy algorithms it would all
>benefit from adding FMA support and bring the CPUs architecture closer to that of the GPU.
>

For exmaple, FFT don't benefit from FMA all that much.

The point is not a whether 2xFMAs are faster or not than 1 FADD + 1 FMUL. It's all about the cost.
1) of 2 more register file ports. By itself, it is probably manageable in SNB-like design that appears to have FP physical register file which is separate from heavier loaded Int register file
2) OoO scheduler that tracks 3 input registers instead of 2. That's a major change in microarchitecture. Besides, it's plain costly. Indeed, Power4/5 and Power7 do it, but Power4/5 has FP issue ports that are completely separated from Integer ALUs (fixed-point, in IBM speak) issue ports, so effectively it has FP scheduler almost independent from general scheduler. I don't remember (or don't know) whether Power7 OoO scheduler is split or unified. Anyway, SNB do have unified scheduler, so you really don't want to make changes in favor of less important FP side that could be even moderately harmful for more important Integer side. Unless, of course, you have no other choice.

>>Old-accumulator-to-result is on the critical path and software do have to pipeline
>>for it even on OoO, but FMA does not improve the latency of this path. Currently,
>>we have FADD latency = 3 on Intel and 4 on AMD.
>>Coming AMD Bulldozer (with FMA) would certainly not have old-accumulator-to-result
>>latency shorter than 4 and, very likely, will have it longer.
>>
>> [snip]
>>
>>In practice, even increasing FP_ADD latency to 4 clocks is o.k. since programmer's
>>tend to unroll by 4 even on current Intel machines that would work fine with just
>>3 accumulators. So I agree that in practice it is not the problem for SNB-derived CPUs.
>>But Bulldozer appear to have longer pipeline and here any additional latency will
>>hurt, esp. on binaries that were not specifically optimized for Bulldozer uArch.
>
>Why would Bulldozer have a longer add latency? Passing the operand directly to
>the adder part of the FMA unit takes about two port latencies, not a whole cycle.
>And Bulldozer is manufactured on a 32nm process and likely has a modest clock frequency
>so this delay doesn't appear much of a problem to me.
>
>Cheers,
>
>Nicolas


Now I recollect that Matt Waldhauer (a.k.a. dresdenboy) published BD latencies in his blog and posted the link here few months ago. FP_ADD latency is no good :) I don't know all the reasons, but my guess is that FMA-based design is the reason #1.

http://citavia.blog.de/2010/01/21/some-instruction-latency-numbers-of-bulldozer-7850137/