Jörn Engel (joern.delete@this.purestorage.com) on July 13, 2020 10:10 am wrote:
> Jukka Larja (roskakori2006.delete@this.gmail.com) on July 13, 2020 8:38 am wrote:
> > Jörn Engel (joern.delete@this.purestorage.com) on July 12, 2020 11:35 am wrote:
> >
> > > Simple parsers.
> >
> > Don't take this the wrong way, but do you think a parser
> > is still simple, if you use SIMD to implement it? If
> > I wrote a parser like that at work, either of two things would happen: 1) I'd be the only one ever touching
> > the code, or 2) someone else would need to touch the code, and would promptly reimplement it without SIMD.
>
> Any code you vectorize ends up less pleasant to read or work with. Not sure
> if simple is the right term, but just like assembler, you end up with a small
> subset of programmers still willing to look at the code and work with it.
>
> I have vectorized simple parsers. If they had been much more complicated it
> would have overflowed both my brain capacity and the available registers.
>
> > > Data modifications like matrix transpose or similar. Dot-products.
> >
> > We do some vector and matrix algebra with SIMD, but I can't say much about
> > it. It's mostly based on vector4, so only makes use of four wide SIMD.
>
> That sounds like you have 32bit or 64bit types. I agree, going 4x faster
> isn't that exciting. If you can get by with smaller data types and go 16x
> or 32x faster, you are willing to suffer a bit more for that advantage.
>
> > > I think of vector programming similar to object orientation
> > > or functional programming. If you read an article
> > > and play around with it for a day you probably won't get it or see any benefit. But if you commit some
> > > time and allow your brain to rewire you start to see more and more applications where it helps.
> >
> > I'm sure if I spent significant time with SIMD, I'd see
> > more places where it could make sense. However, I've
> > already spent more time with it than most programmers ever
> > will. If I can't name a single place in our million
> > line codebase to make real use of it, how could I expect it will have significance in the future?
>
> Run a profiler and see what stands out. If you have one thing responsible for 10% of runtime, it might be worth
> it. Or if you have one thing you don't do because it costs too much. And just like OO or functional programming,
> you don't have to do it just because the cool kids seem to. Only do it if it makes sense to you.
>
> > > And the biggest impediment is that we don't have a nice way to express vector code.
> > > Assembler is, well, assembler. Intel intrinsics are assembler disguised as C functions
> > > with a syntax that seems universally detested. I ended up writing my own small library
> > > simply to be able to read my own code two months later without going crazy.
> >
> > The problem for me is that pretty much only abstraction that I know could be easily used
> > would be Vec3 operations, and that at the best gives just 3x improvement[1]. Even the amount
> > of matrix calculations in a game engine are low. All the hard work happens on GPU.
>
> The abstraction I wanted and eventually noticed I could get is pretty much C syntax.
>
> a += b;
> a >>= 4;
>
> That is code I can read and work with. It doesn't make a huge difference whether "a" is a
> u8, u64 or vector of u16. It took me about a year of suffering until I realized that it's
> possible to do that, you basically have to define your own types and then provide wrapper functions
> every time you want to use Intel intrinsics to do the type casting back and forth.
>
> typedef uint16_t u16_256 __attribute__((vector_size(32), may_alias));
> typedef uint8_t u8_256u __attribute__((vector_size(32), may_alias, aligned(1)));
> static inline u8_256 read256(const void *buf) { return *(const u8_256u *)buf; }
> ...
>
> Now I have all intrinsics confined inside a header-library. If I come across some old code
> using intrinsics, I convert it before I even start reading. To me it dramatically reduces
> the brain-tax I have to pay for vectorization. As a result I started using it more.
>

If I stat to work on your code base the first thing I'd likely do would be removing all wrapper and using bare intrinsics. To me it dramatically reduces the brain-tax.
To each his own.

> I think C syntax is a wonderful thing and Ken Thomson was absolutely right when he decided
> that BCPL was mostly a fine language, except for the syntax. It is absolutely important
> to express simple things in simple terms without a ton of noise distracting you.
>
> a = _mm256_add_epi16(a, b);
> a = _mm256_srl_epi32(b, 4);
>
> Be honest, did you spot the two bugs I slipped into this code? If you didn't you were probably distracted
> by the noise. Having a single type for integer vectors of any size is another mistake. Typically
> if you start with u16, you want to continue with u16. That's why you select an appropriate type instead
> of specifying it with every single instruction. There are exceptions and you want the exceptions
> to stand out. If I see a cast I know that I need to pay attention. I also know that the programmer
> meant to do a cast instead of just typing epi32 when it should have been epi16.