Chester (lamchester.delete@this.gmail.com) on July 11, 2020 5:26 pm wrote:
> So the argument is more like, we want AVX512 and all the conveniences it brings
> over AVX2. But we don't care for physically 512-bit wide execution units.

Well it's one argument.

Generally people care more about total width than per-vector width. Obviously you'd mostly prefer a machine that did 10x 128-bit ops per cycle than one that did a single 512-bit op.

There are some exceptions: especially lane crossing shuffles. These don't scale linearly. It takes something like sixteen 128-bit any-to-any shuffles to implement a 512-bit any-to-any shuffle, so there you'd really want the wider vectors. FP code rarely cares about such crazy, general shuffles, but they are a staple for integer code since they basically stand-in for small table lookups.

> Would decoding AVX512 insts into uops that match actual execution unit width be the best of both worlds?

That's what would happen if you had small EUs, but mostly that doesn't even need to happen. There isn't much difference between 2x256-bit units and 1x512-bit units, for the dominant lane-wise operations. On Intel, the 128-bit, 256-bit and 512-bit units are largely overlapped: they are the same units.

The 128-bit units are really just one half of the 256-bit units: with very few exceptions instructions that only have a 128-bit unit (so 256-bit has 1/2 the throughput). So the other half is wasted. This one isn't that interesting.

The 512-bit units, however, are shared differently. Here, most of the big units are overlapped so that 2x256-bit units are the same thing as 1x512-bit unit. This is "easy" for lane-wise operations. So many zmm ops have half the throughput of their ymm equivalents. The main exception are chips with the second FMA unit, which is only usable by 512-bit ops, but in ICL you see that this relationship is almost exact: the 512-bit "vector width" costs you very little because you just gang together the two 256-bit units you already had.

So I think you only need to go down the path of splitting uops when your *total* width is less than the size of the instruction.

Now there are some complications in this approach: see how Intel has to fuse p0 and p1 to implement this ganging. So it is not totally efficient when you are mixing vector widths, but maybe they will solve that.