A Look Inside Apple’s Custom GPU for the iPhone

Pages: 1 2

Strategic Advantages of Custom Designs

Apple’s vertical integration is unique in the consumer electronics world. For the iPhone and the iPad, the company controls nearly everything – from the basic circuit design of the processor to the OS and customer-facing services like Maps, iMessage, and the Camera. This control and integration enables Apple’s hardware and software teams to differentiate with system-level trade-offs and features that are difficult or impossible for the competition to copy.

Looking back to the original iPhone, the overall trend is crystal clear – at every step along the way, Apple has increased its control over the platform and ecosystem. Initially, Apple used standard ARM-based CPUs and outsouced most of the work to Samsung, but eventually designed its own proprietary ARMv8 compatible CPU cores that now outperform the competition. Similarly, the company bought Anobit and used the team and technology to create a custom storage controller for the flash memory in the iPhone and iPad. For Apple, designing a custom GPU is simply the next logical step to create sustainable strategic advantages.

The most obvious benefit is that Apple’s GPU is better (i.e., faster and more efficient) than the mobile competition, which includes GPUs licensed from ARM or Imagination, as well as proprietary designs from Qualcomm. Superior performance directly translates into a better user experience and battery life for gaming, as well as for imaging and machine learning applications.

Apple’s Metal Performance Shaders library includes dozens of highly optimized shaders that run on the GPU, giving a rich set of tools to developers [4]. These shaders include neural networks for classification, such as convolution, pooling, and rectified linear unit layers. The library also image processing kernels such as image resizing using the Lanczos resampling algorithm, which produces higher quality images than linear sampling for photographs. For example, Instagram uses Apple’s GPU to apply tonemapping and local contrast enhancement. It is even possible that Apple’s camera app uses the GPU for various imaging effects. For imaging and neural networks, half-precision is a natural fit and Apple’s shader architecture provides better performance and efficiency for those applications than the PowerVR shader core.

The second benefit is that Apple can create new features and fix bugs in the GPU without its competitor benefitting. On the other hand, with standard third-party IP, any time Apple fixes a bug or requests a new feature, sooner or later that will be available to Samsung, Mediatek, and any other of Imagination Technology’s customers.

The third benefit is time-to-market and scheduling. Apple’s A-series processors are on an aggressive annual release cycle that is driven by the iPhone. Since the iPhone is a premium product, Apple must deliver superb performance to customers and a significant generation-on-generation upgrade to stimulate demand. As a result, Apple is typically the lead customer for new process technology (e.g., TSMC’s 10nm node), which entails tremendous risk. With the custom GPU, Apple can choose to invest as much time and energy as necessary to hit the ship date, whereas Imagination simply has less money and fewer engineers.

Apple must carefully orchestrate design, validation, high-volume manufacturing and the whole software ecosystem to launch millions of phones or tablets with delightful features into the hands of customers. The months leading up to a launch are a frenzied cycle of detecting and fixing bugs, and updating the graphics software and hardware. Since the GPU is now mostly in-house, this cycle is under Apple’s direct control with fewer external dependencies, which helps the company ensure that products will launch on time.

As a hypothetical example, if Apple’s engineers detect a high-priority logical bug in the shader core, they can make a fix that same day and check the solution. Using third-party IP means that the IP provider must first consider the bug a priority and approve any changes or work-arounds, which could take some time since the IP provider will also need to coordinate with other customers. Samsung’s Exynos 5410 is a case-study in the perils of integrating third-party IP. It was designed with ARM’s Cortex A15 and A7 in a Big.Little configuration to save power, but a coherency bug forced Samsung to disable the power saving features. In-house designs reduce this type of risk considerably, since there are no competing interests and communication within a single company is much easier than with third parties.

The last benefit of internally designing the GPU is that Apple is reducing its reliance on its suppliers, which gives the company a favorable negotiating position and reduces business risk. Creating alternatives to key suppliers, whether internal or external, is a longstanding Apple principle. For example, Apple used to wholly depend on Qualcomm for their LTE modem in the iPhone. Once Intel finally developed a competetive LTE modem, Apple deployed it for most GSM carriers (e.g., AT&T and T-Mobile in the US). In the long run, this keeps Apple’s cost down and also encourages unique opportunities.

Apple’s custom GPU creates an implicit alternative to licensing IP from Imagination in the future. The company has already built a GPU design team and a driver team that together implemented a large portion of the A8, A9 and A10 GPUs. If Imagination Technologies was acquired, or fell behind technically, Apple could simply design its own fixed-function hardware to replace the remaining PowerVR IP.

Imagining Apple’s Next Steps

After years of recruiting graphics architects, Apple has designed its own custom GPU, which is already shipping in the A8, A9, and A10 processors that power the iPhone 6, 6S, and 7. The GPU in Apple’s processors still retains some fixed-function hardware from PowerVR, but based on publicly available evidence it is clear that the shader core in Apple’s GPU is architecturally very different from Imagination Technologies PowerVR line. This further implies that Apple wrote its own Metal and OpenGL ES compiler for its GPUs and almost certainly wrote the entire driver as well.

Given Apple’s history, it is not surprising that they chose to design a custom GPU. The tangible performance benefits are readily apparent. Other benefits such as tighter ecosystem control, better time-to-market, and reducing bugs are difficult to observe, but equally important for Apple.

Going forward, Apple has three options. The status quo is licensing some or all of the fixed-function hardware from Imagination Technologies to complement internally designed components such as the shader core. In this scenario, Apple might eventually upgrade to a newer version of PowerVR, but presumably while negotiating a better deal for licensing and royalties. A second option is simply buying Imagination Technologies, although that would come with considerable extra baggage (e.g., the MIPS processor line) and Apple already passed on this opportunity earlier in 2016. On the other hand, Apple could continue to customize more and more of their GPU – eventually designing out Imagination Technologies. Ultimately Apple will have to decide whether they can do a better job on their own, but so far the company seems to be excellent at developing world-class expertise in new areas.

References

[1] Alex Kan and Fiona, Advanced Metal Shader Optimization. WWDC 2016. https://developer.apple.com/videos/play/wwdc2016/606/ and http://devstreaming.apple.com/videos/wwdc/2016/606oluchfgwakjbymy8/606/606_advanced_metal_shader_optimization.pdf
[2] PowerVR Series6 Compiler Instruction Set Reference. March 17, 2016. https://community.imgtec.com/?do-download=50691
[3] PowerVR Performance Recommendations. March 17, 2016. https://community.imgtec.com/?do-download=50705
[4] Metal Performance Shaders Framework. https://developer.apple.com/reference/metalperformanceshaders


Pages: « Prev  1 2  

Discuss (26 comments)