Some Thoughts on Apple’s Metal API

Though it seems like Apple’s hardware divisions can hardly keep a secret these days due to the realities of mass production, the same is fortunately not true for their software divisions. Broad strokes aside, Apple managed to pack in a number of surprises in their OS X and iOS presentations at WWDC yesterday, and there’s nothing that ended up being quite as surprising to me as the announcement of the Metal API for iOS.

Later this week Apple will be holding their Metal developers sessions, at which time we’ll hopefully get some further details on the API and just how Apple intends to have developers use it. In the meantime with the preliminary Metal programming guide posted over on Apple’s developer website, I wanted to spend a few minutes musing over yesterday’s announcement, how Apple ended up developing their own API, and what this may mean for users and game developers.

Why Low-Overhead APIs?

First and foremost, let’s quickly recap just what exactly Apple has announced. Metal is Apple’s forthcoming low-overhead/low-level graphics and compute API for iOS. Metal is primarily geared towards gaming on iOS, and is intended to offer better graphics performance than the existing OpenGL ES API by curtailing driver overhead and giving developers more direct control over the GPU.

As our regular readers are no doubt well aware, Metal is the latest in a wave of low-level graphics APIs to be introduced over the last year in the GPU space, joining the ranks of AMD’s Mantle and Microsoft’s DirectX 12. In the case of Metal, as has been the case of all of these APIs, the idea is rooted in the fact that while high level APIs provide a number of important features from libraries to hardware abstraction, the overhead from this functionality is not worth the benefits, especially in the hands of highly seasoned programmers who have the experience and the means to go close-to-metal and bang on the hardware directly. The situation facing developers in these cases is that at a time when GPU performance growth is rapidly outpacing CPU performance growth, the API and driver overhead has gone from problematic to intolerable, leading to developers wanting to access the hardware directly.

How The Low-Level Mantle API Benefitted DICE's Frostbite Engine

Metal in turn is the API through which Apple will provide this access. By peeling back the driver and API stack to the bare minimum, developers get to tell the GPU exactly what they’re doing and how they want it done, bypassing large chunks of CPU-intensive code that would previously do this for the developer. Whenever we’re talking about these low-level APIs it’s important to note that they’re merely ways to improve efficiency and are not miracle workers, but when faced with the most applicable bottleneck, the draw call – what’s essentially a single function call for the GPU – the increase in throughput can be remarkable. We won’t spend too much more time on the why’s of Metal, as we’ve written much longer outlines on low-level APIs before that don’t need repeated here, but it’s important to establish a baseline for evaluating Metal.

Are SoCs Draw Call Limited?

Upon hearing Apple’s Metal announcement, perhaps the greatest surprise was that iOS developers were in a position where they needed and could benefit from a low-level API like Metal. In the PC space we’ve been seeing low-level APIs rolled out as a solution to the widening gap between CPU and GPU performance, however the SoC class processors in Apple’s iOS devices are a very different beast. As one would expect for a mobile product, neither the CPU nor the GPU is high performance by PC standards, so why should a low-level API be necessary.

The answer to that is that while SoCs are lower performance devices, the same phenomena that has driven low-level APIs on the PC has driven them on mobile devices, just on a smaller scale. GPU performance is outgrowing CPU performance on the SoC level just as it has been the PC level, and even worse, SoC class CPUs are so slow that even small amounts of driver overhead can have a big impact. While we take 4000 draw calls for granted on desktop hardware – overhead and all – it’s something of a sobering reminder that this isn’t possible on even a relatively powerful SoC like the A7 with OpenGL ES, and that it took Metal for Crytek to get that many draw calls in motion, never mind other CPU savings such as precompiled shaders. If Apple intends to further gaming on iOS (and all signs are that they do), then capable programmers are going to want low level GPU access to maximize their graphical quality, the same as they do on the desktop and on game consoles.

Apple Metal Thread Model (Note that no Apple SoC has more than 2 CPU cores yet)

Ecosystems & Portability

But on that note there’s quite a bit that goes into providing developers with these kinds of tools, which puts Apple in a very interesting position among hardware and OS vendors. Of the other low-level APIs we’ve seen so far – AMD’s Mantle and Microsoft’s DirectX 12 – the former is an API established by a hardware vendor who has to ride on top of other companies CPUs and OSes, and the latter is an OS vendor who has to ride on top of third party CPUs and GPUs. Apple on the other hand is in the enviable position of being as close as anyone can be to offering a fully vertical ecosystem. Apple designs their own CPUs, configures their own SoCs, and writes their own OS. The only portion of the chain that Apple doesn’t control is the GPU, and even then the company has exclusively used Imagination Technologies’ PowerVR GPUs for the last 7 years with no signs of this changing. So for all practical purposes Apple has a closed ecosystem that they control from top to bottom, and can design for accordingly.

A closed ecosystem in turn means that Apple can achieve a level of OS, hardware, and programming language integration that no one else can achieve. Metal doesn’t need to take into consideration any other GPU architectures (though Apple in all likelihood has left it generic enough to be portable if the situation arises) and the OS around it can be tailored to the API, rather than making the API fit within the confines of the OS. This doesn’t necessarily mean Apple is going to make significant use of this integration, but it will be interesting to see just what Apple does do with so much control.

A7 SoC Floorplan (Image Courtesy Chipworks)

Another interesting thing to see as Metal plays out is how Apple handles portability from OpenGL ES, that is if they try to handle it at all. On the whole, it’s accepted that a low-level API like Metal will have minimal portability from higher level languages such as OpenGL ES. The exception to this thus far has been that due to the fundamentally low level nature of shader programs, that shader programs have been more portable. In the case of AMD’s Mantle, for example, we have seen AMD specifically support DirectX’s shader language – HLSL – to make porting to Mantle easier. Shader programs are just one part of a bigger picture, but their growing complexity and low level nature means that there are still benefits to being able to port them among APIs even when the API commands themselves are not portable.

At least for the moment, Apple’s Metal programming guide makes no mention of porting from the existing OpenGL ES API. Looking at the Metal shader language and comparing it to the OpenGL ES shader language (GLSL ES), while it’s initially promising since both languages are based on C++, it’s also clear that for better or worse Apple hasn’t held back from eclipsing OpenGL ES here. Metal’s shader language is based on a newer version of C++, C++11, and consequently includes features not available in GLSL ES. Furthermore comparing the function libraries there are again a number of identical functions, and yet more functions that the two shader languages do not have in common. Portability out of Metal aside, it’s not at all clear whether GLSL ES shaders are meaningfully portable into Metal; if they aren’t then that means additional work for developers, a specific concern if Apple is trying to land console-like games for iOS devices. So it will be interesting to see how this plays out.

Of course Android portability is also going to raise a flag here, though at first glance it actually seems unlikely that this will be a concern. Without an equivalent API – and the OpenGL AZDO concept isn’t going to be fully applicable to OpenGL ES – the games that benefit the most from Metal are also the games least likely to be on Android, so while portability from Android looks far from easy, there also appears to be little need to handle it. Android portability would seem to be best handled by traditional porting methods using OpenGL ES, which retains its common API status and will be sufficient for the kinds of games that will run on both ecosystems.

Metal Computing

On a final note, while we’ve discussed graphics almost exclusively thus far, it’s interesting to note that Apple is pitching Metal as an API for GPU compute as well as graphics. Despite being one of the initial promoters of the OpenCL API, Apple has never implemented OpenCL or any other GPU compute API on iOS thus far, even after they adopted the compute-friendly PowerVR Rogue GPU for their A7 SoC. As a result GPU compute on iOS has been limited to what OpenGL ES can be coaxed into, which although not wholly incapable, it is an API designed for dealing with images as opposed to free form data.

The low-level nature of Metal on the other hand means that it’s a good (or at least better) fit for GPU computing, as the lack of abstraction for graphics makes it more capable of handling the workflows and data types of compute tasks. This is one area in particular where the Metal shader language being based on a subset of C++11 is a benefit to Apple, as it provides a solid foundation for writing compute kernels. None the less it remains to be seen just how adaptable Metal is – can it match the compute functionality of OpenCL 1.2 or even OpenGL 4.x compute shaders – but even if it’s only of limited use it means Apple is finally ready to approach GPU computing on iOS devices.