AI For The Rest Of Us
Apple announces Apple Intelligence at WWDC 2024
Apple’s ’One More Thing’ moment at WWDC on Monday was the announcement of Apple Intelligence, Apple’s branded take on artificial intelligence across their ecosystem and devices. Apple have been in the AI game for a long time, but have rarely talked much about the technology directly, preferring to let the features powered by AI do all the talking for them. WWDC this year saw that change in a big way with half of their 90-minute+ keynote presentation dedicated to introducing Apple Intelligence.
If you haven’t already watched Apple’s WWDC keynote, you can find it in full here. There’s also a good 5 minute overview of all the Apple Intelligence features announced here.
Personal Intelligence
One of the first things that Apple highlighted when introducing Apple Intelligence was that it is all about personal knowledge and context. The frontier Large Language Models (LLMs) that many of us have had access to for the past 18 months have an incredible amount of ‘world knowledge’ but know absolutely nothing about us, the users. This is incredibly limiting, especially if one of the ambitions for LLMs is to power new personal AI agents/assistants in the near future.
Apple are uniquely placed to solve this challenge as our phones are always with us and ‘know’ so much about us. The reality is that right now, our phones have a lot of data on us, but don’t ‘know’ anything. That’s where Apple’s new on-device semantic index comes in - it turns on device data into the personal knowledge and context that powers Apple Intelligence (more on this later).
The Power of Privacy
One of the big challenges facing AI companies in the pursuit of building LLM powered personal AI agents/assistants is convincing users to share their personal data. This is no trivial challenge, given all the headlines over the last 18 months highlighting how LLMs are being trained on all the data they can get their hands on. Few users will feel comfortable with their personal data being used in the training of LLMs and it’s hard to trust the companies training these models with our personal data when up until know they’ve shown such little regard for data ownership and copyright.
In contrast, Apple has been very serious about privacy for a very long time. Commentators have often claimed that it’s Apple’s cautious approach to user privacy that has held them back from being able to compete with Google et al. on AI. It now looks like the exact opposite will be true - Apple’s stance on privacy will be their greatest competitive advantage when it comes to shipping the next generation of AI technologies.
A core part of Apple Intelligence is on-device processing, where as many requests as possible are run locally on the device. If the request requires more processing power, then it will be handed off to cloud servers in what Apple is calling Private Cloud Compute. There are two really big innovations here:
Scalable processing - This is something that hasn’t been seen yet - for example, every request to ChatGPT gets the same amount of processing, no matter how simple of complicated it is. Apple’s approach is a rudimentary solution to this but if LLMs can determine how much processing they need for every request this will lead to both cheaper and more efficient processing, and allow LLMs to ‘think’ before answering and (in theory) be able to tackle more complex problems that they currently struggle with.
Privacy in the Cloud - Nobody has really tried to tackle this before, and so everything that people do that involves cloud services (which is most things online these days) at best creates a log or record of some description and at worse just stores all the data that’s been shared. By using the same privacy-safe hardware from iPhones, iPads and Macs in building their new servers, Apple is ensuring that their users can experience the same privacy protections in the cloud as they can on their devices. All of this can be independently verified and Craig Federighi even said during the keynote that Apple Intelligence will refuse to speak to servers if they haven’t been made available for independent privacy verification.
Both of these innovations from Apple are a bigger deal than many commentators are making out and I think we’ll see all the large AI companies following suit with some version of this over the coming months. Privacy and personalisation go hand-in-hand, so getting this right will be crucial as companies try to deliver on the idea of personal AI agents/assistants.
New Capabilities
In introducing Apple Intelligence, Apple positioned it as being able to deliver four big new capabilities - creating language, creating images, taking actions and understanding personal context.
Both language and image generation are fairly rudimentary and we’ve seen a lot of these sorts of capabilities before from generative AI models. However, Apple Intelligence has some novel and interesting applications of them that move the game on. The most interesting capabilities announced are about taking actions and understanding personal context, which will take us towards Siri becoming a truly personal AI agent/assistant.
Language
The language capabilities that Apple introduced were mostly packaged up in something they call ‘Writing Tools’ which has very Co-pilot like abilities to summarise, re-write and transcribe any text or audio. Nothing too exciting here.
However, the language capabilities that will now be on Apple devices enable some really interesting new features such as prioritising messages, emails and notifications by understanding their context and also a new focus mode to reduce interruptions that I really liked the look of.
Images
The image capabilities were a bit more interesting. Yes, Apple Intelligence can generate images much like MidJourney, DALL-E or many of the other image generation models, but Apple are approaching this in a very Apple way:
Image Playground
Images can be generated using a new app and interface they call ‘Image Playground’ which allows users to build a description (and I’m deliberately not using the word prompt here!) of the image they want in a very intuitive way. The user interface is fantastic, very simple and allows users to adjust their description and choose different versions really easily. Image Playground also has knowledge of your photos and so can generate caricatures of the people in your life - it’ll be interesting to see how Apple is safeguarding around this as I’m not sure people are asked permission for their images to be used and they didn’t talk about the guardrails they’ve built into Image Playground to stop offensive images being created.
Image Wand
Apple also introduced something called the ‘Image Wand’ where you can turn any sketch into a generated image, making adjustments and choosing versions in the same intuitive way that Image Playground works. This looks really cool and I’m sure many users will find a lot of value in this. I have the same questions around safeguards and guardrails though.
Genmoji
Finally, we had Genmoji where users can generate any emoji just by describing what they’re looking for. On the face of it, this sounds a bit gimmicky, but I actually really like the idea of being able to create any emoji that has a style that is consistent with all the other Apple emojis. It’s a great idea and something I think that will really take off.
Actions & Personal Context
I’ve grouped these last two capabilities together because they’re intrinsically linked and the actions on their own are quite honestly the things that you’d hope that Siri was already able to do in 2024. The examples of actions that Apple shared were things like finding files or photos, and playing a podcast that someone shared with the user. Quite frankly Siri should have been capable of doing these things years ago.
However, when you start being able to chain actions together and add personal context, this becomes way more interesting and powerful. This is really what Apple is aiming for with Apple Intelligence. The personal knowledge and context side of this is powered by the on-device semantic index and Apple were clear to state that the ability for Apple Intelligence to chain tasks together would be fairly basic to start off with. The reason for this is that to be able to accomplish more complex tasks and be able to chain multiple actions together, Apple Intelligence will need to be able to plan the actions required to accomplish a given task. This isn’t something that we’ve seen LLMs have much success with yet beyond some research previews and proof of concepts. I suspect that it’s this planning capability that will need to be handed off to larger, more sophisticated models running on Apple’s servers or even to GPT-5 when that’s released towards the end of the year. It’s worth watching the demo (below) that Apple shared on how this will work though, as it’s an interesting glimpse of what’s to come:
New Experiences
There were three other AI-powered experiences that Apple announced at WWDC that weren’t included in the Apple Intelligence section, but are really interesting new capabilities that I think will have a big impact when released later this year. They’re all features that are so beautifully simple, intuitive, and give users both an ‘a-ha’ moment but also a ‘why hasn’t this been done before?’ moment:
Math Notes
Once you’ve seen Math Notes, you’ll ask yourself why this hasn’t been done before and I think over time this will become a favourite feature for many Apple users. From a technical perspective, this must have been really hard to pull off, but from a user perspective it’s so simple and intuitive. Math Notes really is Apple at its best!
Smart Script
Smart Script on the iPad “makes handwritten notes fluid, flexible, and easier to read, all while maintaining the look and feel of a user’s personal handwriting.” This is another super simple feature for users that will have been technically very difficult to pull off. But again, it makes total sense and now I’ve seen it, and I’m sure once people have experienced it later this year, we’ll wonder how we ever lived without it.
Safari Highlights
Safari Highlights surfaces key information about a webpage. For example, users can review a summary to get the gist of an article; quickly see the location of a restaurant, hotel, or landmark; or listen to an artist’s track right from an article about the song or album. This is a really great way to access the key information on a webpage really quickly.
The Next Growth Cycle
There’s a huge amount to unpack from Apple’s announcements from WWDC, and I think we’ll see more news on the Apple Intelligence front as the summer unfolds, more betas are released, and we get closer to the release of iOS 18 in September. One thing is clear though - Apple Intelligence will drive the next growth cycle for Apple. Since the announcements on Monday, Apple’s stock price has gained 10%, which is worth over $300bn 🤑.
Due to the nature of Apple Intelligence’s on-device models, it needs some of the latest Apple silicon to run. This means that people with older devices will need to upgrade to the latest devices to get access to Apple Intelligence, which I think they’ll want to do.
The release of iOS 18 with Apple Intelligence will also drive the next growth cycle for OpenAI and ChatGPT, which has stalled at about 150m monthly users, only 50% more than the 100m users it reached in record-breaking time when it was launched at the end of 2022. The deal with Apple will instantly give ChatGPT hundred of millions of new users which can only help OpenAI in the long run.
With Apple Intelligence, Apple have given us all a glimpse of what a future could look like with private, personal AI experiences that will over time build towards becoming a more personal assistant like experience as Apple Intelligence gains the ability to plan, execute, and chain of multiple actions together. The coming months and years are looking exciting and it’s great to see where the rapid developments in generative AI are likely to take us!
“The future is already here, it’s just not evenly distributed.“
William Gibson