AutomotiveCloudWatch

Signals · Article · Automotiveシグナル · 記事 · 自動車

In-Cabin AI [Explained]

Language models reached car cabins at record speed. The harder questions are what they may control, who owns them, and what cabin sensors may see.

Published 11 min read

On 6 February 2025, Geely announced that its in-house AI model had completed deep integration with DeepSeek R1, the Chinese reasoning model that had shaken the AI industry a few weeks earlier. Within days, Zeekr, Dongfeng, Changan, Leapmotor, IM Motors, GAC, Great Wall and others announced their own DeepSeek integrations. The rush was not mainly about self-driving. It was mostly about the voice assistant in the cockpit.

The same race is running everywhere. Mercedes opened a ChatGPT beta to more than 900,000 US cars in 2023 and now uses a Google Gemini agent for navigation in its CLA. BMW built its newest assistant with Amazon Alexa+. Tesla added xAI’s Grok. Apple’s CarPlay Ultra takes over the instrument cluster, and GM is phasing out phone projection and bringing a Gemini-based assistant to its cars.

This article explains how in-cabin AI works, from voice assistants and language models to the sensors that detect who is in the car, why the leading systems split work between the car and the cloud, why chatting with a car is different from letting AI control it, who is fighting to own the cabin, and how child presence rules and privacy law are shaping the technology. The driver assistance explainer in this series covers driver monitoring for hands-off driving, and the vehicle data explainer covers what happens to driving data after it leaves the car.

Three systems in one cabin

In-cabin AI is really three systems that share the cabin’s sensors and computers. The first is conversational. A wake word such as “Hey Mercedes” is detected on the car, speech is converted to text, a language model or intent engine works out what the occupant wants, and the answer is spoken back or turned into an action such as changing the temperature.

The second is perception. Cameras and short-range radar in the headliner or dashboard classify who is in each seat, whether a child seat is fitted, and whether anyone has been left behind when the car is locked. These occupant monitoring systems are related to the driver monitoring cameras that enable hands-off driving, but they watch the whole cabin rather than only the driver’s eyes.

The third is personalization, the layer that ties both to an identity. A driver profile linked to a phone key or a face can set seats, mirrors, climate, music and navigation preferences, and increasingly shapes how the assistant talks and what it remembers.

Each layer has its own engineering tradeoffs, but they converge on the same cockpit computer and the same question of how much of the cabin’s data leaves the car.

From voice commands to language models

Voice control in cars is not new. Nuance supplied automotive speech recognition for about two decades before spinning out its automotive business as Cerence in October 2019. Cerence inherited contracts with about 60 automakers, and by its own count its technology now sits in 52% of the world’s car production and more than 525 million cars.

Those earlier systems worked on fixed commands. A driver had to learn the right phrases, and anything outside the script failed. Large language models changed that by understanding loosely worded requests and answering open questions.

Mercedes moved first among the large Western automakers. On 16 June 2023, it opened an optional beta of ChatGPT through Microsoft’s Azure OpenAI Service to more than 900,000 cars with its MBUX system in the United States. Drivers joined with a spoken request to the assistant, and the beta was planned to run for three months. In January 2025, Mercedes announced that Google Cloud’s Automotive AI Agent, built on Gemini and Google Maps data covering 250 million places, would bring conversational navigation to the assistant in the new CLA and hold context across follow-up questions.

Chinese automakers built their own. NIO deployed NOMI GPT, which it describes as a device-to-cloud multimodal model, to its Banyan system vehicles in April 2024. The DeepSeek wave of February 2025 showed how quickly a new model could spread across Chinese cockpits once automakers had the software plumbing in place.

Other automakers chose a partner. BMW announced at CES in January 2026 that its Neue Klasse iX3 would bring an assistant built with Amazon Alexa+, which BMW said made it the first automaker to extend its voice assistant with Alexa+, rolling out from the second half of 2026 in Germany and the United States to models running BMW Operating System 9 and X. Tesla began shipping Grok on 12 July 2025 to cars with AMD infotainment processors.

Edge and cloud

The leading systems split work between the car and the cloud. Cerence’s language model family is explicitly divided in two, a cloud model called CaLLM and an embedded small language model called CaLLM Edge, parts of which run on NVIDIA’s DRIVE AGX Orin. Cerence says the split reduces latency but has not published figures. NIO likewise describes NOMI GPT as a combination of on-device perception and cloud models.

The engineering logic is straightforward. A car spends time in tunnels, parking garages and rural areas with poor coverage, and a voice command that fails there feels broken. Conversation also depends on quick turn-taking, so every network round trip matters. And every query sent to a cloud model costs the automaker money over a vehicle life of a decade or more.

So the common pattern is to run the wake word, basic vehicle commands and offline functions on the car, and send open-ended questions to the cloud when a connection is available. That also requires more memory and AI processing in the cockpit chip, which is part of why cockpit computers keep growing.

Chat is not control

The most important design choice in in-cabin AI is what the language model is allowed to touch. Tesla’s Grok launched in July 2025 as a cloud-only assistant that required Premium Connectivity or Wi-Fi and could not issue commands to the car, leaving climate, navigation and media to Tesla’s existing voice commands. A year later, Tesla’s 2026.26 update let Grok adjust the climate, place calls, play music and open the glovebox, while some functions still run through the older command system.

Tesla took a year to cross that line, and it did so through a fixed list of functions. The caution is deliberate. A language model can produce confident wrong answers, and an automaker does not want a misunderstood sentence to open a window on the highway or change a driving setting. So most production systems keep a deterministic layer between the model and the vehicle. The model works out what the occupant wants, and a fixed set of approved functions decides what actually happens.

Google’s approach shows the same boundary from another side. When Gemini began rolling out to existing cars with Google built-in on 26 June 2026, starting with Volvo, it could adjust climate, heated seats and wipers, but how much it can control depends on how deeply each automaker connects its vehicle functions. Drivers could also choose to stay with Google Assistant.

The line between chat and control will move as models become more reliable, but it will move slowly. Safety-relevant functions in a car are engineered to be predictable, and a language model is not predictable in the same way.

Who owns the cabin

Behind the technology sits a commercial contest. The cabin is where drivers spend their attention, and whoever runs the assistant shapes which maps, music, shopping and services they use. Automakers must decide whether to hand that relationship to a technology company or build it themselves.

Apple has pushed furthest into the car. CarPlay Ultra began rolling out on 15 May 2025 with Aston Martin in the United States and Canada. It takes over the instrument cluster as well as the center screen, controls climate and radio, and uses Siri for vehicle functions. Apple has said Hyundai, Kia and Genesis are committed to it.

GM has made the opposite bet. It is phasing out Apple CarPlay and Android Auto, starting with its EVs, and said in October 2025 that Gemini-based conversational AI would come to its vehicles from 2026, followed later by its own AI built for individual vehicles, which with owner permission would draw on vehicle data and driver preferences. Mercedes uses a partner model but keeps the assistant under its own MBUX brand on its own operating system.

The supplier in the middle is being squeezed. Cerence’s revenue fell from $331.5 million in fiscal 2024 to $251.8 million in fiscal 2025, though the earlier year was inflated by revenue from a legacy Toyota contract. It now positions itself as the layer that orchestrates third-party language models inside the car, with its first cars using its new xUI platform expected on roads in 2026.

The software-defined vehicle explainer in this series covers the operating systems this contest runs on.

Child presence detection

The same cabin sensors are also being shaped by safety rules. Children left in hot cars die every year, and regulators and rating agencies have pushed automakers to detect them.

Under Euro NCAP’s 2026 rating scheme, child presence detection is worth up to 5 points within occupant monitoring, and only systems that directly sense a child, by movement, breathing or heartbeat, can score. The initial warning must come within 15 seconds of locking, escalation within 90 seconds of the first warning ending, and intervention, such as running the climate system or alerting a caregiver remotely, within 10 minutes of locking or 5 minutes of the first escalation, or immediately if the cabin becomes dangerously hot. Indirect systems that only remind the driver based on which rear door was opened stopped earning points in 2025.

Direct sensing usually means 60 GHz radar, which can detect the tiny movements of breathing through blankets and child seats where a camera might be blocked. Suppliers such as Vayyar market imaging radar for this job, and the same sensor can help classify occupants for airbags and seat belt reminders.

The United States has moved more slowly. The 2021 infrastructure law directed NHTSA to issue a rear-seat reminder rule by November 2023. As of mid-2026, NHTSA had still not proposed a rule, missing both that deadline and its own April 2025 target, according to the safety group Kids and Car Safety and Senator Richard Blumenthal. Most US rear-seat reminders remain voluntary, and many rely on the door logic that Euro NCAP no longer rewards.

Privacy in the cabin

A car with cameras, microphones and radar watching its occupants is a privacy question as much as an engineering one. In September 2023, the Mozilla Foundation reviewed the privacy policies of 25 car brands and found that all of them failed, calling cars the worst product category it had ever reviewed. Some policies allowed collection of deeply personal information.

China has written the strictest rules. Its automotive data security provisions, in force since 1 October 2021, favor processing data inside the car by default and not collecting personal data unless the driver opts in. Audio, video and images from the cabin are treated as sensitive personal information. In the United States, the Federal Trade Commission finalized an order against GM in January 2026 over the sale of location and driving data, a reminder that regulators now treat connected-car data as consumer data subject to binding long-term restrictions. The connected vehicle data explainer covers that case.

For in-cabin AI, the practical consequence is that what happens on the car matters. A system that processes voice and camera data locally and sends only what is needed to the cloud is easier to defend to regulators and customers than one that streams the cabin to a data center.

What in-cabin AI really means

In-cabin AI is three systems sharing one cabin. Conversational AI is becoming a hybrid of on-car and cloud models, occupant sensing is being driven by child safety ratings, and personalization ties both to a driver’s identity. The hardest decisions are not about the models themselves but about what they may control, who owns the relationship with the driver, and how much of the cabin leaves the car.

The useful questions for any program are which functions the assistant can actually control and through what safeguards, what runs on the car and what goes to the cloud, whether occupant sensing detects a child directly, and whose brand and data sit behind the voice.

Words you will hear

  1. Wake word. An always-listening, on-car detector for a phrase such as “Hey Mercedes” that decides when audio is sent for heavier processing.

  2. Small language model. A compact language model that runs on the car’s cockpit computer, handling commands and offline use.

  3. Hybrid edge and cloud. A design where each request is answered by the on-car model or a cloud model, trading speed, coverage and cost.

  4. Occupant monitoring system. Cabin cameras or radar that classify every occupant, distinct from driver monitoring.

  5. Child presence detection. A system that detects a child left in a parked car and warns, escalates and intervenes.

  6. Direct and indirect sensing. Direct systems sense the occupant with radar or cameras. Indirect systems infer a child from which doors were opened.

  7. Non-collection by default. China’s principle that personal data is not collected unless the driver opts in, with processing kept in the car where possible.

This article is part of the AutomotiveCloudWatch Explained series. Before it, read Software-Defined Vehicle [Explained] and Zonal Architecture [Explained].

If this helped, subscribe free at automotivecloudwatch.com to get each new explainer when it publishes.

All opinions are my own and do not reflect those of my employer.

Get the next Executive Briefing by email

Each new briefing, in your inbox.

Subscribe

Next to read次に読む

Go deeperさらに深く

A relevant next step

Relevant to the decision How to evaluate an SDV platform

Advisory & diligence

Diligence a target, a thesis, or a supply chain before you commit to it.

See what you get → Start an inquiry about this

Nothing is sent until you submit the form. You can change the offer or leave out the page reference there.