Jukka Larja (roskakori2006.delete@this.gmail.com) on July 13, 2020 8:38 am wrote:
> Jörn Engel (joern.delete@this.purestorage.com) on July 12, 2020 11:35 am wrote:
>
> > Simple parsers.
>
> Don't take this the wrong way, but do you think a parser is still simple, if you use SIMD to implement it? If
> I wrote a parser like that at work, either of two things would happen: 1) I'd be the only one ever touching
> the code, or 2) someone else would need to touch the code, and would promptly reimplement it without SIMD.

Any code you vectorize ends up less pleasant to read or work with. Not sure if simple is the right term, but just like assembler, you end up with a small subset of programmers still willing to look at the code and work with it.

I have vectorized simple parsers. If they had been much more complicated it would have overflowed both my brain capacity and the available registers.

> > Data modifications like matrix transpose or similar. Dot-products.
>
> We do some vector and matrix algebra with SIMD, but I can't say much about
> it. It's mostly based on vector4, so only makes use of four wide SIMD.

That sounds like you have 32bit or 64bit types. I agree, going 4x faster isn't that exciting. If you can get by with smaller data types and go 16x or 32x faster, you are willing to suffer a bit more for that advantage.

> > I think of vector programming similar to object orientation
> > or functional programming. If you read an article
> > and play around with it for a day you probably won't get it or see any benefit. But if you commit some
> > time and allow your brain to rewire you start to see more and more applications where it helps.
>
> I'm sure if I spent significant time with SIMD, I'd see more places where it could make sense. However, I've
> already spent more time with it than most programmers ever will. If I can't name a single place in our million
> line codebase to make real use of it, how could I expect it will have significance in the future?

Run a profiler and see what stands out. If you have one thing responsible for 10% of runtime, it might be worth it. Or if you have one thing you don't do because it costs too much. And just like OO or functional programming, you don't have to do it just because the cool kids seem to. Only do it if it makes sense to you.

> > And the biggest impediment is that we don't have a nice way to express vector code.
> > Assembler is, well, assembler. Intel intrinsics are assembler disguised as C functions
> > with a syntax that seems universally detested. I ended up writing my own small library
> > simply to be able to read my own code two months later without going crazy.
>
> The problem for me is that pretty much only abstraction that I know could be easily used
> would be Vec3 operations, and that at the best gives just 3x improvement[1]. Even the amount
> of matrix calculations in a game engine are low. All the hard work happens on GPU.

The abstraction I wanted and eventually noticed I could get is pretty much C syntax.

a += b;
a >>= 4;

That is code I can read and work with. It doesn't make a huge difference whether "a" is a u8, u64 or vector of u16. It took me about a year of suffering until I realized that it's possible to do that, you basically have to define your own types and then provide wrapper functions every time you want to use Intel intrinsics to do the type casting back and forth.

typedef uint16_t u16_256 __attribute__((vector_size(32), may_alias));
typedef uint8_t u8_256u __attribute__((vector_size(32), may_alias, aligned(1)));
static inline u8_256 read256(const void *buf) { return *(const u8_256u *)buf; }
...

Now I have all intrinsics confined inside a header-library. If I come across some old code using intrinsics, I convert it before I even start reading. To me it dramatically reduces the brain-tax I have to pay for vectorization. As a result I started using it more.

I think C syntax is a wonderful thing and Ken Thomson was absolutely right when he decided that BCPL was mostly a fine language, except for the syntax. It is absolutely important to express simple things in simple terms without a ton of noise distracting you.

a = _mm256_add_epi16(a, b);
a = _mm256_srl_epi32(b, 4);

Be honest, did you spot the two bugs I slipped into this code? If you didn't you were probably distracted by the noise. Having a single type for integer vectors of any size is another mistake. Typically if you start with u16, you want to continue with u16. That's why you select an appropriate type instead of specifying it with every single instruction. There are exceptions and you want the exceptions to stand out. If I see a cast I know that I need to pay attention. I also know that the programmer meant to do a cast instead of just typing epi32 when it should have been epi16.