Inside Amazon's agonizing attempt to save Alexa with AI

• Amazon has faced repeated delays in launching a new AI-powered Alexa.
• Integration with partners like Uber and Ticketmaster has complicated troubleshooting processes.
• Latency and compatibility issues have also caused delays.

Amazon's Alexa seems like the perfect product for the generative AI era.

Getting this powerful technology to actually work well with the digital assistant is a monumental challenge that's been plagued by gnarly technical problems and repeated delays.

Customer-friction concerns, partnership hiccups, compatibility questions, latency problems, and accuracy issues have snarled progress, according to internal Amazon documents and multiple people involved in the project.

The Alexa team is under immense pressure to get something out. A decade ago it launched with Echo speakers and became a household name. But that early success fizzled and the business has so far failed to become profitable, leading to drastic cutbacks and layoffs in recent years.

Some company insiders consider this AI moment to be a seismic opportunity for Alexa, and potentially the last chance to reignite consumer interest in the voice assistant through the power of large language models.

A product of this scale is "unprecedented, and takes time," an Amazon spokesperson told Business Insider. "It's not as simple as overlaying an LLM onto the Alexa service."

"RED" warning

One of the main challenges facing the new Alexa relates to how the digital assistant will interact with other companies and services, and who is responsible for customers if their requests, orders, and payments don't go smoothly.

In late August, Amazon was working on integrating 8 third-party applications, including Uber and Ticketmaster, into the upcoming AI-powered Alexa to handle various user inquiries.

At that time, the goal was to launch the new Alexa around mid-October, according to one of the internal documents obtained by Business Insider. However, it was still unclear which companies would be responsible for customer support issues, like payment and delivery errors, this document stated.

The lack of clarity could cause Amazon to send "frequent" customer contacts to the partner companies. Then, those partners would sometimes redirect the users back to Amazon, the document explained.

"This level of support would cause significant customer friction, when some of the orders/purchases are time-sensitive (meal orders or rideshare trips) and purchase mistakes can be expensive (e.g. buy Taylor Swift tickets)," the document said, assigning it a "RED" warning.

Release dates pushed back

Snafus like this have caused Amazon to push back the release date, almost on a weekly basis, according to some of the people involved in the project, which has been codenamed "Banyan" or "Remarkable Alexa." BI's sources asked not to be identified because they're not authorized to talk to the press.

For example, without more clearly defined responsibilities with third-party partners, Amazon expected further delays in the launch. "Alignment on customer support plans between Product teams and the 3P partners may push this timeline further out if any delays occur," one of the documents warned.

The company had once planned for a June launch, but after repeated delays, it told employees late last month that the new Alexa would launch "no earlier" than November 20, one of the documents said.

A few of people BI spoke with recently are even talking about the Alexa upgrade rolling out in early 2025, which would miss the key holiday period. Bloomberg earlier reported on a 2025 launch plan.

As of late October, Amazon had not settled on an official brand for the updated voice assistant, and instructed employees to simply call it the "new Alexa" until further notice, one of the documents said.

Alexa's huge potential

To be sure, Alexa has significant long-term potential in the generative AI era — as long as Amazon can iron out problems relatively quickly.

Time is of the essence, partly because the existing Alexa business has lost momentum in recent years. According to a recent report from eMarketer, user growth for major voice assistants, including Alexa, has declined significantly in recent years.

The sudden rise of ChatGPT has showcased what is possible when powerful AI models are integrated smoothly with popular products that consumers and companies find useful.

Some Amazon leaders are bullish about the AI-powered Alexa and a new paid subscription service that could come with it. At least one internal estimate projected a 20% conversion rate for the paid subscription, one of the people said. That would mean that out of every 100 existing Alexa users, roughy 20 would pay for the upgraded offering. Amazon doesn't publicly disclose the number of active Alexa users but has said it has sold more than 500 million Alexa-enabled devices.

An internal description of the new Alexa shows Amazon's grand ambitions: "A personalized digital assistant that can handle a wide range of tasks, including drafting and managing personal communications, managing calendars, making reservations, placing orders, shopping, scouting for deals and events, recommending media, managing smart home devices, and answering questions on virtually any topic," one of the documents said.

Customers will be able to access the new Alexa "through natural language using voice, text, email, shared photos, and more across all their devices like Echo, Fire TV, mobile phones, and web browsers," it added.

Amazon CEO Andy Jassy shared a similar vision during last month's earnings call, saying the new Alexa will be good at not just answering questions, but also "taking actions."

In an email to BI, Amazon's spokesperson said the company's vision for Alexa is to build the world's "best personal assistant."

"Generative AI offers a huge opportunity to make Alexa even better for our customers, and we are working hard to enable even more proactive and capable assistance on the over half a billion Alexa-enabled devices already in homes around the world. We are excited about what we're building and look forward to delivering it for our customers," the spokesperson said.

Smaller AI models

Still, the project has grappled with several challenges, beyond customer friction and partnership problems.

Latency has been a particularly tough problem for the AI Alexa service. In some tests, the new Alexa took about 40 seconds to respond to a simple user inquiry, according to three people familiar with the test results. In contrast, a Google Search query takes milliseconds to respond.

To speed up, Amazon considered using a smaller AI model, like Anthropic's Claude Haiku, to power the new Alexa, one of the people said. But that dropped the quality and accuracy of the answers, leaving Amazon in limbo, this person said. In general, smaller language models generate quicker responses than larger models but can be less accurate.

Amazon had initially hoped to use a homegrown AI model, one of the people said. Last year, Alexa head scientist Rohit Prasad left the team to create a new Artificial General Intelligence group at Amazon. The stated goal of the new team was to create Amazon's "most expansive" and "most ambitious" large language models.

However, this AGI team has not produced notable results so far, which led Amazon to consider Anthropic's main Claude offering as the primary AI model for the new Alexa, this person said. Reuters previously reported that Amazon was going to mainly power Alexa with Claude.

Amazon's spokesperson said Alexa uses Amazon Web Services's Bedrock, an AI tool that gives access to multiple language models.

"When it comes to machine learning models, we start with those built by Amazon, but we have used, and will continue to use, a variety of different models — including Titan and future Amazon models, as well as those from partners — to build the best experience for customers," the spokesperson said.

The spokesperson also added a note of caution by highlighting the difficulties of successfully integrating large language models with consumer applications. These models are great for conversational dialogue and content creation, but they can also be "non-deterministic and can hallucinate," the spokesperson added.

Getting these models "to reliably act on requests (rather than simply respond) means it has to be able to call real-world APIs reliably and at scale to meet customer expectations, not just in select instances," the spokesperson explained.

New risks

In late August, Amazon discovered several new risk factors for the AI Alexa service.

Only 308 of more than 100,000 existing Alexa "skills," or voice-controlled applications, were compatible with the new Alexa, presenting a "high risk for customers to be frustrated," one of the documents explained.

Some older Echo devices would not be able to support the AI-powered Alexa, the document also warned. And there were no plans for expanding the new service to dozens of overseas markets where Alexa is currently available, leaving a large user base out of touch, it also noted. Fortune previously reported some of these risk factors.

Integration headaches

As of late August, Amazon had 8 "confirmed" partner companies to handle certain tasks for the new Alexa, as BI previously reported. The company hopes to onboard roughly 200 partners by the third year of the new Alexa's launch, one of the documents said.

Integrating with some of these companies has already created headaches. One document said that Amazon struggled to develop a consistent troubleshooting process across every partner service. Companies including Uber, Ticketmaster, and OpenTable have deprecated their existing Alexa skills, further disconnecting them from the voice assistant.

Amazon's spokesperson said that, as with any product development process, a lot of ideas are discussed and debated, but "they don't necessarily reflect what the experience will be when we roll it out for our customers."

Amazon has also anticipated customer complaints, at least in the early launch phase. One internal document from late August stated that the new Alexa was projected to receive 176,000 customer contacts in the first three months of its release. At one point, Amazon considered launching a new automated troubleshooting service for issues related to its devices and digital services, including Alexa, according to one of the internal documents. That was later shelved.

Do you work at Amazon? Got a tip?

Contact the reporter, Eugene Kim, via the encrypted-messaging apps Signal or Telegram (+1-650-942-3061) or email (ekim@businessinsider.com). Reach out using a nonwork device. Check out Business Insider's source guide for other tips on sharing information securely.