Wilco (Wilco.Dijkstra.delete@this.ntlworld.com) on April 6, 2014 5:29 am wrote:
> computational_scientist (brian.bj.parker99.delete@this.gmail.com) on April 5, 2014 6:50 pm wrote:
> > Linus Torvalds (torvalds.delete@this.linux-foundation.org) on April 4, 2014 4:56 pm wrote:
> > > TREZA (no.delete@this.ema.il) on April 4, 2014 3:18 pm wrote:
> > > >
> > > > Denormals is what made IEEE FP superior to VAX. It is the correct way of doing FP math!
> > >
> > > Don't get me wrong - I actually personally like denormals, and think they are
> > > definitely required for serious FP math, but the thing is, 99% of FP use isn't
> > > really serious, and most people don't really understand FP anyway.
> > >
> > > The kind of people who really understand FP and do serious math using it (to the point of
> > > caring about the order of operations, never mind denormals), those kinds of people really
> > > do know what they are doing, and sometimes (although not always) really do want denormals.
> > >
> > > But there's the other side of FP math, which really just wants good enough values quickly. There are a lot
> > > of people who are ok with single-precision and no denormals. And yes, quite often 24 bits of precision is
> > > not enough, and they decide they actually need double precision in order to avoid odd visual artifacts.
> > >
> > > Yeah, I'm talking about things like games.
> > >
> > > And the thing is, the defaults tend to be the wrong way around. People who don't know what they
> > > are doing with floating point basically *never* need denormals. You will generally hit other issues
> > > long before you hit the "oops, I lost precision because I didn't have denormals". But exactly
> > > *because* they don't know about floating point, they also don't know to disable them.
> > >
> > > So I think that from a hardware designer standpoint, it actually
> > > would make more sense if the default was "denormals
> > > are zero" (or "flush to zero"), because the people who do want denormals also know about them.
> > > So while
> > > I like denormals, the onus on disabling denormals when not needed is kind of the wrong way around.
> >
> > This is back to front reasoning: physicists, mathematicians and most computational scientists don't
> > know anything about the details of floating point and use the defaults like everyone else.
> > Even simple assumptions like x==y iff x-y==0 don't always hold in FTZ mode; default denormal mode
> > is the simplest floating point model mathematically. By contrast, people running gaming benchmarks
> > who know about these things can easily enable FTZ to speed them up for marketing purposes.
>

> You can't rely on x==y iff x-y==0 even with denormals. IEEE floating point is extremely complex
> with lots of odd non-intuitive corner cases. Very few mathematical identities work on IEEE,
> with or without denormals.

This is not true- it is a foundational theorem of IEEE floating point that subtraction is exact (provided the operands are with a factor of two). This is needed by compensated schemes that calculate and correct for floating point error.

> Denormals are a major pain in algorithms, for example you need
> special cases to avoid the catastrophic loss of precision (and the infinities/NaNs caused
> by calculations with a denormal) as well as the huge slowdown on many CPUs.

The underflow flag is set for exactly this reason- you need to test for such cases in FTZ in any case. (Kahan's notes have some good examples.)

> 8x slowdown is still unacceptable in my book - 10% is acceptable. POWER and ARM are at that level.

Well "tolerable" may have been a better term than "acceptable". AMD chips used to have very low overhead too.