Linus Torvalds (torvalds.delete@this.linux-foundation.org) on July 13, 2020 11:27 am wrote:
> Chester (lamchester.delete@this.gmail.com) on July 13, 2020 12:37 am wrote:
> >
> > Javscript numbers are FP64 so FP matters even for web browsing
>
> Others have pointed out type specialization, but I'll point out that a lot
> of FP use cases are "you use FP, but it doesn't have to be very intensive".
>
> Stupid JS interpreters very much fall under that heading. You don't really need to have a highly
> optimized vector FP unit. To be honest, it almost certainly doesn't even need to be pipelined.
> The old (bad) kind of thing people had 30 years ago is plenty good enough for a lot of loads.
>
> I agree that "FP is important" in the sense that it should exist. Soft-FP is too painful for words. It's
> easy to find places where soft-FP implementations will be too slow (or too power-hungry for embedded).
>
> So you want a FPU. I agree 100% with that. It's just that there's a big big difference
> between a chip that has "FPU hardware" and one that has "full AVX512".
>
> In fact, for a lot of stuff you don't even need double precision. I'm pretty sure I've seen
> JS for microcontrollers - ie the ARM M4 kind that does have a FPU, but it's SP only, no DP.
>
> > Photo editing/raw conversion - lots of FP there, often vectorized too
>
> Do it on the GPU if you actually want FP and performance.
>
> That said, I agree more about integer vectorization than I do about FP vectorization. There
> are actually lots of loads that can very effectively use (moderate) integer - particularly
> byte-wide - vectorization for some very basic parsing. Things you don't do on a FPU.
>
> > Video encode will use AVX if you use software, which gives higher quality than hardware encode
>
> That says more about the quality of the hw than about AVX.
>
> And honestly, if you care about performance, you will do it with a hardware encoder.
> Don't fool yourself. They aren't perfect, but they tend to generally be in the "good
> enough" to "pretty damn good" category. If you want something really special, you
> are really special, and you shouldn't claim that it's a common problem any more.
>

Hardware encoding is needed/proper for high speeds but that doesn't mean you don't need performance when doing software encoding!

The problem is such: ASICs are inherently too inflexible to achieve best (meaning best attainable, not best in theory) quality compression. They can do fast and energy-efficient but the image quality is average or more often sub-par. The reason is that they can't exploit the most of the huge search and decision tree of encoding tools + psychovisual optimization options a modern video format gives. This is something lots of people aren't aware of and it leads to all sorts of misconceptions about "GPU" (more precisely "ASIC") and hardware encoding. Or generally about video compression.

Maybe some day some breakthrough and paradigm shift will bring formats where ideal quality compression will be successfully achieved through specialised ASICs, but I don't see any path to that in the present state of this field.

So you have GPU/specialised ASIC/SoC encoding for fast and disposable encoding jobs, like encoding game footage for twitches or movies for phone and that's fine. But if you are encoding a movie to be pressed on bluray or to be distributed thousands or millions of times to end consumers over VoD web services *and you care about quality* (meaning, you don't want blurred backgrounds, banding in night battles, blocks and other artifacts and such mess), you want advanced video encoder and those are sofware, runs-on-CPU programs.
(Note: not with floating point ops, integer values and ahence integer SIMD ops are used in video).

Similarly as encoding, there is scope for software video decoding and that is also reason to have integer SIMD: because it is more flexible and you can use software decoders to decode anything, not being a slave of what your GPU's ASIC decoder can handle.

So for example with software decoding we could play VP8/VP9 some 3 years before is appeared in hardware, AV1 last year as opposed to next year (not to mention, we can keep using the same hardware and not buy new one). You can play 10-bit H.264 since 2011 while nothing supports it in hardware to this day.

Also even when you have hardware playback, it often comes with bugs and quirks that make decoding in software easier and less hassle as long as you don't hit prohibitive fan noise/power usage on a notebook.