Linus Torvalds (torvalds.delete@this.linux-foundation.org) on July 12, 2020 11:35 am wrote:
> Because this forum is about architecture design and implementation, isn't it?

It is. And I think one of the reasons for your issues with AVX-512 is that you're mixing the two - design and implementation. The AVX-512 design is not what the Skylake-X implementation did with it.

> So I think it's very fair to put down that gauntlet: AVX512 vs SVE2.

TL;DR: If FP is your thing, AVX-512 & SVE are both good and both better than the stuff before. If it isn't, they're both mostly a waste of silicon, like the stuff before.


First, context: I'm a HPC guy, and old school at that. Double-precision FP is my lifeblood. I couldn't care less about virtualization, web workloads, interpreted languages, or pure pointer-chasing stuff like compilers (though I rely on them heavily, and worked in a compiler company for a long time). Basically, I'm only interested in enough sequential performance to support parallelism (calling an MPI function is essentially sequential...), and in SIMD FP real-life performance (i.e. on useful codes, not on Linpack). In other words, I'm the kind of guy for whom AVX-512 or SVE were designed... so I like them :-)

For implementations:

AVX-512: KNL wasn't quite it (high latencies, missing DQ/VL, abysmal sequential performance, ...) and Skylake-X XCC has a major power/thermal issue, well documented. Lower-end Skylake-X are less catastrophic, though somewhat because they are slower to begin with (they don't have the 2nd FMA on port 5).

SVE2: There isn't any. SVE (without the 2, so missing some DSP-oriented stuff, as SVE2 doesn't bring much to the table for the FP crowd) only has one currently available. That is the (currently unobtainium outside of Fugaku itself) Fujitsu A64FX, which seems good from early reports. Time will tell.

As far as design goes, they're both very good, although they work on different assumptions. You certainly can build a case for or against either of them based on details, but broadly, they both do the job well, and better than previous solutions (MMX, SSE, AVX, NEON, VIS, MSA, AltiVec, ...).

AVX-512 is good, full stop. Sure, it assumes you know how much data is in a register and the kind of code you work with, so you can use any of the available width (128, 256 and 512 are all available if you have VL support, which only KNL is lacking) optimally. But it has all you need to generate code from a compiler on a wide range of cases: masking, scatter/gather, reasonable orthogonality, good feature coverage (they added back rotation, complex arithmetic support is still somewhat lacking). Compared to AVX2, there's no comparison, it's much easier/nicer to deal with, and you get an extra 16 registers as well. AVX2 doesn't have scatter, can't mask properly, etc. Semantically, it's just SSE on wider registers (and three operands, granted, the one redeeming feature of AVX).

SVE is basically the same as AVX-512 (though rotation is only in SVE2 and complex support is better), just with no fixed width. The idea is that binary code will be 'portable' between implementations of different width, a nice goal. Unfortunately, that doesn't play as nice with algorithms (or optimizations) with a fixed amount of work. You can't fully unroll your vectorized/cache-blocked loop if you don't know how many vector fit in the cache to begin with. And there's the weight of legacy, e.g. math libraries that assume they will know the width of the vector at compile time... So while the idea is attractive, it simply causes a different set of trade-offs than fixed-width ISA like AVX-512.

Which is better? For my job, hopefully SVE :-) But it's the quality of implementation that is going to make the difference. Codes don't magically run on the SIMD ISA. They go through a compiler first, and then they need to run on the whole core alongside an operating system, with only some successfully vectorized bits relying on the SIMD part of the design. Ultimately, as long as all the required features are there, the ISA really doesn't matter that much (heresy, I know).

As for general availability... Sure, AVX-512 is missing from most processors. But SVE* is currently missing from *all* of them. Sure, using 512-bits wide registers on Skylake is seldom a good idea. But that doesn't mean that AVX-512 sucks - just that 512 bits on Skylake sucks. That's why ICC or GCC default to 256 bits vector on Skylake: you can have all the goodies of AVX-512 without the frequency penalty.

Also - the whole set of extensions to AVX-512 is a mess, yes. For _that_ part, SVE is currently better, but the SVE2 specifications were released before any SVE implementation, so it may not stay that way :-(

Narrower SIMD ISA could get away without the advanced features from AVX-512/SVE, but that was because the cost of not vectorizing was low - cost being evaluated by comparing achieved performance vs. theoretical peak. So only vectorizing the 'easiest' case was OK. AVX-512 and SVE can vectorize a lot more cases; but that means a lot more is expected of them performance-wise. AMD took some HPC market share because they had a lot more memory bandwidth, and it could run with all the AVX/AVX2 experience accumulated in codes during the Sandy Bridge/Ivy Bridge/Haswell/Broadwell era. Whereas Skylake-X needs at least recompiling for AVX-512 (256 bits or otherwise), and sometimes a whole new retuning of the codes. SVE will have/already has the same issue, and for it the tools are less mature (a lot of AMD Naples/Rome sold in the HPC market run codes compiled with ICC/IFORT...).

As for RISC-V V - the design is not frozen yet, and there's no high-performance implementations (at least not public), so it's not fair to compare. Hopefully, the final design will be feature-rich enough to be generally usable, yet will still allow for high-performance implementations. The academic implementations (ETH-Zurich ARA, BSC VPU, maybe others) will give an idea of the starting point for high-performance designs. When it comes to extensions, given that it seems to be a selling point for RISC-V to have a lot of them, it's likely not going to be any better than for AVX-512. If the baseline is good enough, it shouldn't matter too much though.

For the HPC market, this are interesting times; for a long time the choice was only "will you take NVidia GPU with those Intel CPU?", now IBM has taken back some (thanks to NVlink, to a large extent), AMD has taken back some (mostly the bandwidth crowd), and Arm is stepping up as an alternative (some ThunderX2 deployment, the A64FX taking the #1 spot on the Top500, and maybe some future stuff).