Jukka Larja (roskakori2006.delete@this.gmail.com) on October 31, 2020 8:14 am wrote:
[...]
> I'm not sure what std::simd would include, but at least on a quick glance I don't see GCC's extensions
> would matter at all for us. It's not like writing a wrapper of our own (for (S)SSE(2/3) and Neon)
> was all that difficult or took a lot of time.

Template metaprogramming is not rocket surgery. It's not trivial either, though. And you have to keep validating that new compiler versions don't break performance.

I was thinking of the contrast illustrated, for example, here between a plain formula on the one hand, and a mess of nested intrinsics on the other hand:

https://nullprogram.com/blog/2015/07/10/

The GCC extension allows writing a plain formula, too, when targeting a SIMD backend.

> The problem is that we don't have much code that
> could make use of such abstraction (actually, we mostly use four wide float vectors as direct
> replacement of three wide float vectors. We can't even make use of the last item).
>
I think the terminology "vector" for SIMD parallelism is misleading people at large into thinking in terms of vector math. There is some overlap, but as you noticed, the mathy vectors aren't generally too useful for extracting SIMD parallelism.

Usually the problem needs to be "rotated by 90 degrees", i.e. in this case you'd have a SIMD vector with elements x1, x2, x3, x4, another SIMD vector with elements y1, y2, y3, y4, and so on. Effectively working with four mathy 3-vectors inside three SIMD 4-vectors.

> I'm sure there are some places where rethinking data structures or algorithms could
> allow making use of SIMD, but if we were to go through the trouble, we'd consider
> using GPU first. After that consideration, there's practically nothing left.
>
Plus, GPU programming interfaces don't mislead you as much, as they don't really make their SIMD width visible to the programmer.


BTW, I found it useful to think about re-arranging data structures not in terms of vectors, but in terms of cache friendliness. Locality, yes, but also things like: invariant data should not be interleaved with changing data (to save write back bandwidth).

That kind of data centric programming enables optimizations which also help ordinary scalar code. And they are a large step towards SIMD, where the programming model forces locality by treating a number of subsequent data items as one indivisible unit.

Doing optimizations for caches "naturally" tends to prefer structure of arrays format.