Adrian (a.delete@this.acm.org) on July 26, 2021 3:41 am wrote:
> Michael S (already5chosen.delete@this.yahoo.com) on July 26, 2021 2:53 am wrote:
>
> > >
> >
> > > a.k.a. tensor instructions.
> >
> > If by "tensor instructions" you mean "outer product" (I think, nowadays
> > that's the most common meaning) than I tend to disagree.
> > IMHO, many tasks can be accelerated quite nicely by "inner product" instructions. And intuitively,
> > without being a chip architect, I would think that "inner product" is inherently more energy
> > efficient than "outer product", due to lower fan out to register files.
> >
> > But, as shown above, neither is needed for specific task of [big] matrix multiply.
> > Now, small or medium matrix multiply is completely different kettle of
> > fish. BTW, much more important for me, from practical perspective.
> >



You are right that the inner product is more energy efficient, so when the speed is less important, matrix operations should be decomposed into inner products, where possible.

However, the inner product has a latency that increases with the vector size, so for maximum speed it is frequently easier to use operations with lower latency, e.g. outer products or AXPY, even if they require more storage for the intermediate values.