--- (---.delete@this.redheron.com) on June 5, 2022 6:34 pm wrote:
> Are Microsoft (or Android) making interesting use of neural engines (and their
> "equivalents" like GPUs) when they exist today? Will Intel be reluctant to add
> them because it will be ten years before MS takes advantage of them?
>
> In the case of Apple it's unclear where the neural engine proper is being used.
>
> Neural nets (of some sort) are being used on the local device for a few things like keyboard prediction
> (so, so) and Siri+text to speech to text (IMHO this part, the voice recognition and text to speech
> or speech to text work well; the stuff after that works less well, but that's also not on-local-device
> so not relevant). But it's unclear either of those are demanding enough to require an NPU.
>
> Apple has three interesting vision technologies that appear to use the NPU
> (as opposed to other vision technologies, like recognizing poses, or memoji,
> or putting rectangles around faces which are probably done by the VPU).
> One (possibly, details unclear) is FaceID.
> The second two (definite) are recognizing stuff in photos. The UI for this is kinda clunky, but
> it does work pretty well. When you can figure the UI (good luck...) the device will recognize a
> variety of things, from landmarks to famous works of art to animal or plant types. Along with recognizing
> stuff, there is also recognizing text in images (also clunky, different, UI) which also works pretty
> well. In a sense it's "just" OCR, but it's OCR that works very well, and handles a lot of handwriting,
> fancy scripts, weird cases like vertical writing, and so on. These both don't *require* the NPU
> but does use it when available, and runs faster and lower power as a result.
>
> Other possibilities like translation maybe use the NPU (you can, I think, in the latest OSs force downloads
> and get local translation without requiring a connection) but the use of NPU hasn't been validated.
>
> Point is: NPU takes up a not-negligible amount of area, has been iterated on repeatedly,
> and yet doesn't seem to do much. What it does is nice, and is a little faster and lower power
> on a phone, but works just as well on an Intel Mac. So what's the value in the NPU?
> The current fashionable answer (just you wait for WWDC, just you wait...) is Neural Radiance
> Fields which are, to hear certain people say it, the next thing in 3D technology, past
> both polygons and ray tracing. Is there anything to this? I have no idea.
>
> But a skeptic could say that,
> - right now, there appears no *compelling* reason for Intel to add one to their SoCs
> - Apple (so far...) also can't figure out what to do with it except minor
> energy savings for a fairly rare (recognize image or text) operation
>
> Is the magic bullet AR? Is it language? Beats me. But whatever it is, what's the
> maximum speed at which MS will implement anything taking advantage of an NPU?
> It's fine to wave your hands and say Open Source, but I'm unaware of any great innovative used
> by open source of any sort of NPU. For example comskip could, I suspect, do a vastly better
> job of recognizing commercial breaks by using some ML, but that's hasn't happened yet.

You are ignoring some metrics here, most neural network work that affects user interface have a latency limit, and a better unit not only meets that limit easier with less power, but meats with better work, there is a quality metric here, do you remeber the shitty spelling corrector of Android a few years ago? Did Android miss-corrected your text recently? So, that's not just performance and power.