I forgot one big advantage of auto-vectorization: it is way more future-proof than assembly or intrinsics. As new ISAs become available, compilers will just be able to use them on your original source, so things should only get faster over time.

In that respect it's a bit like REP MOVS for copies: "since forever" it hasn't been the fastest approach when compared to hand-written contemporary approaches: but although it lags behind the best it keeps improving along with the best at roughly the same rate as new ISAs are introduced. So that REP MOVSB you wrote 25 years ago is pretty damn fast today compared the byte copy loop you might have written that would have been faster then. REP MOVS also allows you to effectively access 32-byte registers on modern neutered chips where AVX isn't available (Pentium/Celeron whatevers): you can't use the full-width mov instructions, but behind the scenes rep movs does, so you can very slightly cheat the segmentation.

Jouni Osmala (a.delete@this.b.com) on August 3, 2018 10:22 pm wrote:
> 2) AVX-512 should be the target ISA level which autovectorization makes sense. The potentially
> vectorizable subset is far greater, and you can avoid epilogue by it. The features of > ...
> Here's the problem of current situation, Intel tied AVX-512 to 10nm process for
> the mainstream and their 10nm process got postponed several times. And that leads
> it not being reasonable target for general audience for a long time.

I think Intel could have introduced AVX-512 a long time ago on client chips if they wanted, around when they introduced Skylake-SP.

I think Skylake client probably has most of the machinery needed for need for AVX-512, but it's just disabled/fused off[1]. For example I think it might very well have a 512-bit wide PRF. There were a lot of changes in Skylake client, including rearranging some port assignments and reducing the L2 associativity that makes sense mostly if they wanted to re-use the same core for Skylake-SP.

Also, take a look at these die shots:

Skylake client:



Skylake server (including uncore):



Can someone who knows about this stuff point out the PRF? Maybe the regular blocks under the letter "OoO" in the first picture?

You can see how the rectangular core part of the Skylake server die looks just like the Skylake client part, plus the "extra" AVX-512 EU and 768KiB of L2 cache hanging off the side mixed in with the uncore. So it looks very much like Skylake client was designed in mind already with this extension, which means that any part of the die inside the "core rectangle" in the server part would have already had all the transistors needed for AVX-512. In particular, it would have had 1 512-bit FMA unit (which isn't even anything else extra since that's just the two 256-bit FMAs it already has, stitched together). More importantly, it probably has all the in-core support for mask registers, the new AVX-512 instructions, etc, etc.

I think that's why you get some cheaper Skylake-SP server chips with 1 FMA unit: they are the ones whose uncore doesn't have that extra AVX block hanging off the top. So in this case the missing FMA unit isn't arbitrary segmentation: it's actually a result of that EU hanging off the side of the main code and able to be omitted in some uncore designs.

So I think if Intel wanted AVX-512 everywhere, they could do it, but they don't/didn't want it.

I think they should: AVX-512 support and in general better wide vector support is one of the last remaining technical advantages they have over Ryzen which is a pretty damn good product.

---

[1] In fact, disabling it is very easy, because even chips with functional AVX-512 keep it powered down until you use it, including the high lanes of all the ALUs, probably the high parts of the PRF, etc. So to disable AVX-512, you just have to disallow any execution of the instructions, and the existing powersaving mechanisms mean you don't pay much cost.