By: ML will move to PIM (anon.delete@this.mailinator.com), March 22, 2021 2:51 am
Room: Moderated Discussions
wumpus (wumpus.delete.delete@this.this.lost.in.a.hole) on March 21, 2021 12:24 pm wrote:
> On the other hand, also expect that your new market will drag you away from any chance to jump to general
> purpose land.
Definitely. ML is going to go to full processing-in-memory soon enough.
The workload is one of the only things where large-scale PIM actually makes sense. Because the precision needed is so low, and the amount of data moved is so huge, bleeding edge ML accelerators already spend >>90% of their energy on data movement. If you make use of the fact that the weights need more precision than the inputs, and that the same input goes to a lot of different neurons, by keeping the weights stationary and broadcasting the inputs to the whole chip, you can cut the amount of data you need to move to a quarter or less. That's easily enough to make it worth jumping into a process that's shittier for computation and that supports dram on top of it.
> On the other hand, also expect that your new market will drag you away from any chance to jump to general
> purpose land.
Definitely. ML is going to go to full processing-in-memory soon enough.
The workload is one of the only things where large-scale PIM actually makes sense. Because the precision needed is so low, and the amount of data moved is so huge, bleeding edge ML accelerators already spend >>90% of their energy on data movement. If you make use of the fact that the weights need more precision than the inputs, and that the same input goes to a lot of different neurons, by keeping the weights stationary and broadcasting the inputs to the whole chip, you can cut the amount of data you need to move to a quarter or less. That's easily enough to make it worth jumping into a process that's shittier for computation and that supports dram on top of it.
