Apple Intelligence is the name of Apple’s multi-modal, cross-platform approach to the current AI computing trend. It is coming to almost every Apple platform and most new Apple devices. The new platform includes many features, including generative AI capabilities like writing and image creation, as well as an enhanced Siri assistant.

Apple Intelligence comes with an approach to privacy that is definitively unique to Apple. What do we mean? It means that information will be shared with as few people as possible, even with Apple itself. We don’t know the deep details of this issue, but the tech giant certainly inspires more confidence compared to other companies. We mention privacy first because Apple initially highlighted this issue.

Apple Intelligence will be available on the iPhone 15 and devices with the M1 chip or later, starting this fall. As you know, the iPhone 15 and iPhone 15 Plus use the A16 Bionic chip rather than the A17 Pro chips. Apple says this is necessary due to the increased processing power of the two devices. All future models are likely to continue offering the full range of features within the Apple Intelligence ecosystem.

Apple and OpenAI Partnership

Most of you are already familiar with ChatGPT and its parent company, OpenAI. ChatGPT-4o is coming to Apple’s new operating systems as part of agreements between the two companies. Moreover, it’s free and does not require an external account.

Apple says users can switch from internal LLMs to ChatGPT. While making a Siri query, the option to access ChatGPT will be offered. Then, you will be asked to allow ChatGPT to handle the heavy lifting from that point on. Basic access will be provided for free, but those subscribed to ChatGPT’s premium features can integrate their accounts to access these functions.

In short, ChatGPT will also be part of the Apple Intelligence platform. As a result of the OpenAI partnership, Siri will be able to leverage ChatGPT’s expertise to answer specific questions when needed. According to Apple, you will be asked if you want to use ChatGPT before using the service to get answers. The company says the ChatGPT integration includes privacy protections with masked IP addresses and no request storage by OpenAI.

Apple Intelligence Features

We will discuss the features in more detail, but let’s summarize briefly. Apple Intelligence includes many generative AI features. However, it’s not just like a generative AI model like ChatGPT; we can generally say it’s a more robust platform. It will offer generative AI tools for writing and editing, image creation, and organization. Additionally, like current generative AI tools, it will be able to provide summaries.

Some of the features Apple introduced we’ve already seen from other companies. For example, the ability to summarize a long email, rewrite the text in your email to clarify it, or change its tone. Such features will be available system-wide across all of Apple’s operating systems. Here are some of the biggest AI features of Apple Intelligence:

• Improved photo editing with AI, such as object removal in Photos
• More Siri control over applications and actions, including asking Siri to find emails or edit photos
• Creating custom emojis with AI based on text prompts
• Generating quick summaries of notes, text message threads, emails, and other texts
• Automatically suggesting replies for emails and messages
• An advanced Mail app that can categorize emails and create messages
• Automatically transcribing voice notes
• AI enhancements for Xcode to auto-complete code

The Biggest Innovation: Siri Update

Siri was first introduced as Apple’s personal assistant 13 years ago. However, most industry observers felt that Siri lagged behind its competitors in functionality for a long time. This situation will change with the new Intelligence-supported version, which can accept voice or written commands and build on your previous queries instead of handling each new request independently.

Siri’s voice now sounds more natural and understands the language better. For example, if you mix up your words or need to regroup your thoughts during a query, Siri will still be able to process the information smoothly. Apple’s assistant will also have “screen awareness,” meaning it can accept text inputs.

These new capabilities will also cover images in the Photos app. During the keynote, a presenter asked Siri, “Show me pictures of Stacey in a pink coat in New York.” The results were quickly presented to the user. After selecting one of the photos, the presenter told Siri, “Add this photo to my note along with Stacey’s bio.” Siri was able to establish the context and act accordingly to fulfill the request.

The new features are spreading to all of Apple’s first-party apps and even third-party apps using the App Intents API. Siri can help you find specific pieces of information, even if you don’t remember if it came from a text, email, or previous note. For example, if you need your driver’s license number to fill out a form, Siri can search your device, find an image of your license, and add the relevant number to the required text field.

In another demo, the presenter asked when their mother’s flight would land. The AI quickly fetched the flight itinerary, including the updated landing time, from the airline’s app. Then, the presenter asked about the lunch plans they made with their mother, which were also displayed. Finally, they asked Siri how long it would take to get from the airport to the lunch spot, and Siri plotted the route using Apple Maps.

Image Generation

Apple is also rolling out an Image Playground feature accessible through the Messages app. You can create your own caricature images using text or voice commands. Image Playground will also be available as a standalone app, and third-party apps will be able to leverage the API.

With Image Playground, users can create images using AI in Messages, Photos, Notes, Keynote, Freeform, Pages, and third-party apps. Images can be created in Animation, Illustration, and Sketch styles.

For example, if you are creating an image to add to a birthday message for your mother, Apple Intelligence will create an image using your mother’s photo along with other birthday images in one of several preset styles you can choose from.

Photo/Video Search and Editing

Like Google Pixel’s Magic Eraser feature, Apple is bringing a new AI-supported photo editing feature called Clean Up to its devices. This tool detects and removes distracting objects in the background and reconstructs the background.

Apple is adding a natural language photo and video search feature to the Photos app. This allows users to enter a command to search for a picture, video, or even a part of a video. Let’s say you went on vacation last year and took a picture by the sea. You can search for an object you found on the beach by name. Thousands of photos can easily be found based on this object or anything else.

In Photos, users can also give a text command to create “Memory” videos. These will be compilations of videos and photos that follow a description, accompanied by music suggestions from Apple Music.

Genmoji

One of the cool things Apple showed was a new feature in Messages called Genmoji. On iOS 18, iPadOS 18, and macOS Sequoia, you can create your own Genmojis. In one example, the input “smiling face relaxing with cucumbers on its eyes” was used, and the necessary Genmoji outputs in the familiar yellow face tone were provided. Users can then choose from different variations of the output to find the one they like best. In summary, users can create an emoji or drawing just by describing it, and even add pictures of their friends.

Text Editing

Apple Intelligence will offer suggestions to improve your writings wherever you work with texts. It can rewrite, edit, and summarize text for you.

Message and Email

It will be aware of your incoming Mail messages and notifications and try to prioritize what’s important to you. Like Microsoft and Google, Apple also wants to use AI to organize your email inbox. A new feature called Priority Messages will display the most urgent emails, and you’ll be able to see email summaries without opening a message.

Users will be able to see summaries of messages in their inbox without opening each email’s first few lines. For long conversation threads, Apple’s new “Mail Assistant” can read entire threads and offer a single summary. Furthermore, this assistant can suggest replies and prioritize which emails need replies.

Apple Xcode

Developers also have some tools to look forward to. Apple Intelligence is coming to Xcode, which will help developers write and debug code. Apple is adding some small suggestions to developers in its coding environment. Developers will also be able to select code and get suggestions to improve it, similar to GitHub Copilot. It will also allow finding certain lines of code in a large project.

Other New Features

The new release of Apple Intelligence offers many other features. Here are a few examples:

• Enhanced functionality with AI in Apple Music to create mixes
• Intelligent summaries of your health data with suggestions for improvements
• Expanded memory in the Notes app for recalling past meetings and suggestions for meeting follow-ups
• More intelligent and accurate workout suggestions in the Fitness app
• Search features for intelligent, natural language-based text files in Finder

iOS 18 and iPadOS 18

The company is not changing its naming structure for the next iOS release. The new iOS 18 will come with many features powered by Apple Intelligence. The biggest change is undoubtedly the new Siri assistant.

The enhanced Photos app now features powerful object removal tools, a search feature, and the ability to create animated memories from your videos and images. You will also be able to create emoji suggestions from Photos.

The updated Messages app will include improved suggestions, Genmojis, and image creation tools. There will also be an automatic reply feature that can be triggered using custom keywords, which is especially useful for things like “meeting mode.”

The new Mail app is much more advanced with auto-generated responses, enhanced summaries, and categorization. Additionally, Apple Music will offer mixes based on your daily activities and workout suggestions in the Fitness app.

macOS Sequoia

Apple is making some significant changes with the new macOS Sequoia, although it still retains the old naming convention for its operating system.

The new operating system is named after the Sequoia trees found in California’s forests. It comes with a new file system that offers better performance and a redesigned Finder. Additionally, it will include Siri, which can manage local files and applications. Siri’s screen awareness will work on Mac as well.

As part of its commitment to privacy, Apple says that all AI processing is done locally on the device using Apple’s M series chips. While most AI companies leverage powerful server farms, Apple says they will only use the cloud when necessary. Otherwise, almost all computing is done on the device. This approach aims to improve users’ privacy and make applications more responsive.