Peter Lewis (peter.delete@this.notyahoo.com) on June 4, 2022 7:06 pm wrote:
> One thing that might help x86 to survive is if more chip area on future processors is used for special-purpose
> accelerators, such as neural engines, rather than CPU cores. The smaller the fraction of transistors used
> for CPU cores, the less differences between x86 and ARM will matter. That theory didn’t work for Intel
> when they tried to make cell phone chips but maybe it will work for future processors with special-purpose
> accelerators. Intel’s future Meteor Lake CPU is rumored to contain a neural engine.
>
> Intel or AMD could make 3 versions of a processor with 25%, 50% and 75% of chip area used
> for CPU cores and 75%, 50% and 25% used for neural engines. This could be implemented by
> having two tile types and varying the fraction of each tile type on a module. The neural
> engine and the software stack that supports it would need to be as good as Nvidia’s.

Are Microsoft (or Android) making interesting use of neural engines (and their "equivalents" like GPUs) when they exist today? Will Intel be reluctant to add them because it will be ten years before MS takes advantage of them?

In the case of Apple it's unclear where the neural engine proper is being used.

Neural nets (of some sort) are being used on the local device for a few things like keyboard prediction (so, so) and Siri+text to speech to text (IMHO this part, the voice recognition and text to speech or speech to text work well; the stuff after that works less well, but that's also not on-local-device so not relevant). But it's unclear either of those are demanding enough to require an NPU.

Apple has three interesting vision technologies that appear to use the NPU (as opposed to other vision technologies, like recognizing poses, or memoji, or putting rectangles around faces which are probably done by the VPU).
One (possibly, details unclear) is FaceID.
The second two (definite) are recognizing stuff in photos. The UI for this is kinda clunky, but it does work pretty well. When you can figure the UI (good luck...) the device will recognize a variety of things, from landmarks to famous works of art to animal or plant types. Along with recognizing stuff, there is also recognizing text in images (also clunky, different, UI) which also works pretty well. In a sense it's "just" OCR, but it's OCR that works very well, and handles a lot of handwriting, fancy scripts, weird cases like vertical writing, and so on. These both don't *require* the NPU but does use it when available, and runs faster and lower power as a result.

Other possibilities like translation maybe use the NPU (you can, I think, in the latest OSs force downloads and get local translation without requiring a connection) but the use of NPU hasn't been validated.

Point is: NPU takes up a not-negligible amount of area, has been iterated on repeatedly, and yet doesn't seem to do much. What it does is nice, and is a little faster and lower power on a phone, but works just as well on an Intel Mac. So what's the value in the NPU?
The current fashionable answer (just you wait for WWDC, just you wait...) is Neural Radiance Fields which are, to hear certain people say it, the next thing in 3D technology, past both polygons and ray tracing. Is there anything to this? I have no idea.

But a skeptic could say that,
- right now, there appears no *compelling* reason for Intel to add one to their SoCs
- Apple (so far...) also can't figure out what to do with it except minor energy savings for a fairly rare (recognize image or text) operation

Is the magic bullet AR? Is it language? Beats me. But whatever it is, what's the maximum speed at which MS will implement anything taking advantage of an NPU?
It's fine to wave your hands and say Open Source, but I'm unaware of any great innovative used by open source of any sort of NPU. For example comskip could, I suspect, do a vastly better job of recognizing commercial breaks by using some ML, but that's hasn't happened yet.