j (invalid.delete@this.example.com) on July 13, 2020 12:29 am wrote:
> Geoff Langdale (geoff.langdale.delete@this.gmail.com) on July 11, 2020 7:49 pm wrote:
> > I do agree that the fragmentation that Intel has introduced is absolutely tooth grinding.
>
> From a HPC vector geek point of view, the aggravating thing is that we've known from at least 1970'ies
> vector supercomputers what's needed to make a good vector ISA that is usable by both compilers and
> asm programmers. No innovation needed here; Intel could just have copy-pasted the 1975 Cray I vector
> ISA and adapted it to x86, and the result would have been pretty decent. In particular:
>

Except that *everything* is different today, so expecting that old trade offs hold is beyond naive.

In particular, the main innovation of CRAY-1 - high ratio between vector register width and actual execution unit width (64 on CRAY-1, 32?? on followup) would be very bad idea today.
I am not a big fan of today's prevalent ratio of 1 or 2, but my personal favorites are 4 and 8. Certainly not 32-64. Instruction decoding and associated overheads today a not costly enough to justify ratio of more than 8. May be, even 8 is already too much.

> - Alignment restrictions. Should be enough that a vector is naturally aligned to the vector element
> size, not the entire length of the vector. This avoids the need for a loop prolog which bloats the
> code and makes it hard for a compiler to decide whether vectorizing a loop is profitable or not.
>

Today 32-bit is more important than 64-bit. And 16-bit becomes more and more important every day.
And if you bothered to go all way down to 16 bits it would be phenomenally stupid no not make the last step and remove alignment restrictions altogether.
Everybody realized it at least 10 years ago. A smarter ones - 15 years ago.


> - Strided and scatter/gather memory ops to drive MLP
>

Alpha Tarantula paper describes few nice optimizations made possible by strided memory operations.
But they apply to EV8/EV9 cache structure (big L2 made of lots of banks, that can be accessed independently and simultaneously) which is very different from today's prevalent caches that are built with non-vector power efficiency as a goal #1.
I still like an idea of strided accesses, but for different reasons - simplicity (relatively to full scatter/gather) of management.
As to full gather - that's a measure of last resort. It can't be made efficient, but could be a somewhat better than sequence of scalar loads+inserts. I am less sure about vector scatter. I was feel the impact of its absence in few of my AVX2 optimization attempts, but it looks like the reason for it is most because AVX2 does not provide good scalar alternative to scatter - ability to store arbitrary scalar from the middle of vector register.

> - A vector length register to avoid the need for a loop epilog,
> with similar code size benefits as avoiding the prolog.
>

Nice, but for sane vector length rarely critical.

> - Masking/predication.
>
> The one big innovation in vector ISA design since then is basically the
> vector length agnosticism in ARM SVE or the RISC-V vector extension.
>

Nice, but costly, both in encoding and in need to read additional register on every op.

As to "length agnosticism" of SVE, it could work for code that don't try to be "speed of light", but IMHO, anybody who want 70% or better utilization of EUs will have to ignore it an write normal width-aware stuff.

> Also features like shuffles etc. are certainly useful, particularly for non-HPC stuff like string
> processing you mention. But I don't personally know those very well so won't comment on them.
>
> With the basic ISA in place and stable, it would still be possible to market segmentate based
> on the vector execution width, or if the ISA has the SVE/RVV style length agnosticism, by
> the width of the vector registers as well. Also higher end parts could certainly benefit
> from beefier memory units able to load/store from more cache lines in parallel etc.
>
> Ideally I'd like to have seen Intel replace x87 with a basic scalar FP ISA. Say, the scalar
> part of SSE2 plus FMA, and some generation(s) later added the full vector ISA, and then
> as transistor budgets improve increase execution and register widths. Oh well..


SW-visible separate of "latency engine" vs "throughput engine" is attractive idea. But in case of x86, and most likely of ARM too, it's water under bridge.