j (invalid.delete@this.example.com) on July 13, 2020 2:58 am wrote:
> Michael S (already5chosen.delete@this.yahoo.com) on July 13, 2020 1:12 am wrote:
> > j (invalid.delete@this.example.com) on July 13, 2020 12:29 am wrote:
> > > Geoff Langdale (geoff.langdale.delete@this.gmail.com) on July 11, 2020 7:49 pm wrote:
> > > > I do agree that the fragmentation that Intel has introduced is absolutely tooth grinding.
> > >
> > > From a HPC vector geek point of view, the aggravating thing is that we've known from at least 1970'ies
> > > vector supercomputers what's needed to make a good vector ISA that is usable by both compilers and
> > > asm programmers. No innovation needed here; Intel could just have copy-pasted the 1975 Cray I vector
> > > ISA and adapted it to x86, and the result would have been pretty decent. In particular:
> > >
> >
> > Except that *everything* is different today, so expecting that old trade offs hold is beyond naive.
>
> Sure, though I meant the basic features of the vector ISA, not the particular
> implementation details. I'm certainly aware modern cache-based memory subsystems
> are vastly different from what was found on early vector supercomputers.
>
> > In particular, the main innovation of CRAY-1 - high ratio between vector register width and
> > actual execution unit width (64 on CRAY-1, 32?? on followup) would be very bad idea today.
>
> Probably yes. Unless we're talking about some throughput oriented design with very small
> caches, and/or where some cache levels are bypassed entirely for vector load/store.
>
> To some extent this is similar to what GPU's do, with many threads/warps
> mapped to a smaller number of execution units.
>
> > > - Strided and scatter/gather memory ops to drive MLP
> > >
> >
> > Alpha Tarantula paper describes few nice optimizations made possible by strided memory operations.
> > But they apply to EV8/EV9 cache structure (big L2 made of lots of banks, that can
> > be accessed independently and simultaneously) which is very different from today's
> > prevalent caches that are built with non-vector power efficiency as a goal #1.
> > I still like an idea of strided accesses, but for different reasons
> > - simplicity (relatively to full scatter/gather) of management.
> > As to full gather - that's a measure of last resort. It can't be made efficient, but could
> > be a somewhat better than sequence of scalar loads+inserts. I am less sure about vector scatter.
> > I was feel the impact of its absence in few of my AVX2 optimization attempts, but it looks
> > like the reason for it is most because AVX2 does not provide good scalar alternative to scatter
> > - ability to store arbitrary scalar from the middle of vector register.
>
> There are certainly challenges in making these operations fast on cache-based memory subsystems.
> But you get the same problem also for stride-1 access if you have looser alignment requirements
> and your vector ends up straddling a cacheline. But it doesn't need to be perfect from the
> get-go, it just needs to be better than the alternative scalar implementation.
>
> > > - A vector length register to avoid the need for a loop epilog,
> > > with similar code size benefits as avoiding the prolog.
> > >
> >
> > Nice, but for sane vector length rarely critical.
>
> The thing is, with looser alignment instructions and the vector length register it's
> (ideally at least) always profitable to vectorize a loop, no matter how short.
>
> >
> > > - Masking/predication.
> > >
> > > The one big innovation in vector ISA design since then is basically the
> > > vector length agnosticism in ARM SVE or the RISC-V vector extension.
> > >
> >
> > Nice, but costly, both in encoding and in need to read additional register on every op.
>
> There's a cost, yes, but OTOH it enables vectorization of more algorithms.
>
> > As to "length agnosticism" of SVE, it could work for code
> > that don't try to be "speed of light", but IMHO, anybody
> > who want 70% or better utilization of EUs will have to ignore it an write normal width-aware stuff.
>
> Certainly for core kernel libraries like BLAS, FFT, some image ops etc. you certainly
> want tuned implementations for each target microarchitecture, just like today.
> But it would be very nice if the ISA would just work everywhere.
>

The argument against SVE and scalable vector instructions in general sounds like what I've heard before where people said coherence for GPU's was an unnecessary expense and would reduce performance. The problem though is that working around a lack of coherence requires extra work and is error prone. Yes one could do it for the main things but there's just so much else that needs doing. Having coherence has made using a GPU far more usable and accessible.

If you look at the Riken project which adopted ARM with SVE a major requirement was that their supercomputer be easy to use by researchers. They were not aimining at a single problem nor did they want researchers to have to pay large amounts of money to programmers. If we get stuck on highest performance we'd still be programming everything in assembler. What Riken aimed at is where the mass market is. One should first and foremost satisfy the general market and only after that deal with special cases which can make a big difference.