Apple is trying to get ‘LLM Siri’ back on track

Apple Intelligence has been a wreck since its first features rolled out last year, and a big new report from Bloomberg’s Mark Gurman details why — and how Apple is trying to piece things back together. And much of its effort hinges on rebuilding Siri from the ground up.

Gurman has reported in the past that Apple is working on what it’s internally calling ‘LLM Siri’ — a reworked, generative AI version of the company’s digital assistant. Apple’s previous approach of merging the assistant with the existing Siri hasn’t been working. Gurman describes in great detail a number of reasons why, but here’s a quick summary:

• Apple software chief Craig Federighi was “reluctant to make large investments in AI.” The company doesn’t like to invest in a goal without a clear endpoint, Gurman writes, but where AI is concerned, one unnamed Apple executive told him “…you really don’t know what the product is until you’ve done the investment.” That would have meant expensive GPUs, which the company didn’t rush to buy and later didn’t have enough of to keep up with competitors.
• Apple started late. Another executive told Gurman that Apple Intelligence “wasn’t even an idea” before ChatGPT launched in late 2022.
• Apple AI chief John Giannandrea thought people didn’t want AI chatbots. He told employees that customers commonly want to be able to disable tools like ChatGPT.
• Old Siri didn’t work with new Siri. Apple apparently saw bolting generative AI features onto the old Siri as the fastest way to catch Apple up in AI, but it wasn’t working. “It’s whack-a-mole. You fix one issue, and three more crop up,” an employee told Gurman.
• Giannandrea didn’t “fit in” with Apple’s inner circle. Giannandrea was a rare outside executive hire when he came on in 2018, and he didn’t have the same “forceful” personality as others in company leadership. He didn’t fight hard enough to get big funding amounts, the report says. Apple employees told Gurman that Giannandrea didn’t push his workers hard enough, and that he doesn’t see big AI companies like OpenAI or Google as an urgent threat to Apple.
• Marketing got out over its skis. The company’s AI marketing focused heavily on promised features like an improved Siri or Apple Intelligence being able to take context from apps across your system before they were ready — features that it has since been forced to delay.

Now the company is trying to rejigger its approach. Part of that is a total overhaul of Siri, rather than just trying to make generative AI work in concert with the old Siri. According to Gurman, Apple has its AI team in Zurich working on a new architecture that will “entirely build on an LLM-based engine.” Gurman reported in November last year that the company was working on this, and the idea is that it will make the assistant “more believably conversational and better at synthesizing information.”

Another part of the solution is leveraging iPhones and differential privacy to improve Apple’s synthesized data — comparing fake training data with language from iPhone users’ emails, but doing so on-device and sending only the synthesized data back to Apple for AI training. And one way the company is discussing improving Siri is letting the LLM version loose on the web to “grab and synthesize data from multiple sources.” Basically, Siri as an AI web search tool not unlike Perplexity, which is one of the companies Apple has approached about partnering for AI search in Safari.

Whatever the outcome, apparently Giannandrea won’t be a direct part of it, having been taken off of product development, Siri, and robotics projects in the spring. According to Gurman, Apple execs have talked about putting him “on a path to retirement,” but are concerned that some of the research and engineering folks he brought with him would leave with him, too. Whatever the case, Gurman says Giannandrea plans to stick around, “relieved Siri is now someone else’s problem.”