Why Tesla’s AI trainers don’t trust its self-driving tech – or its safety stats

First of two parts

In a Utah office, hundreds of Tesla workers scrutinize video collected by vehicles using the automaker’s Full Self-Driving (FSD) feature. Some clips show the cars hitting cats, dogs or deer, along with more-routine accidents. Sometimes, they don’t brake before impact. Often, they speed. Occasionally, the workers see near-misses of children playing in the street.

Known as “data labelers,” these staffers train Tesla’s AI-powered driver-assistance software. They annotate incidents of good and bad driving and escalate problems to engineers working to improve the system.

Tesla CEO Elon Musk says FSD will soon make all Teslas fully autonomous. But interviews with nine former labelers and a former Tesla self-driving engineer show that the technology continued to struggle in recent months to execute basic maneuvers — such as avoiding emergency vehicles or stopping for school buses loading or unloading students.

Despite such dangerous shortcomings, Musk and other executives have increasingly touted FSD’s safety as they pushed Tesla to stage public displays of the fully autonomous capability the CEO has promised investors every year for a decade. The displays include a robotaxi pilot in Austin, Texas — launched last June with some human safety monitors in the cars and others working remotely.

Inside Tesla, as these events approached, staffers worked long hours mapping routes and training the software on specific hazards to make the company’s self-driving technology appear more capable than it really is, four of the former Tesla employees told Reuters. The staffers said these labor-intensive safeguards are impossible to deploy on a broad scale.

Those efforts, which haven’t been previously reported, undermine Musk’s long-stated claim that Tesla’s self-driving technology will soon work anywhere globally and doesn’t require the same laborious local mapping of roads and hazards employed by rivals. Musk has said Tesla takes a simpler approach, relying solely on cameras and AI, that will allow it to scale up its robotaxi service at “hyperexponential” speed and offer current Tesla owners full autonomy through software updates.

Musk and other Tesla leaders have bolstered the impression of robo-competence by citing company safety statistics that they say prove FSD is already up to 10 times safer than human drivers.

A Reuters examination of Tesla’s statistical methodology and interviews with company insiders show Tesla isn’t close to safely delivering self-driving vehicles at scale — a central promise underpinning the automaker’s $1.6-trillion stock-market value.

The examination included a Reuters analysis of how Tesla compares its own crash data to federal crash data; a review of the comparatively rigorous methodology employed by robotaxi competitor Waymo; and interviews with 11 traffic-safety researchers who reviewed Tesla’s methodology for Reuters. The review found several invalid data comparisons underlying the statistics in Tesla’s FSD safety report, which 10 researchers said amounted to misleading marketing rather than a serious investigation into a critical safety issue.

Tesla, for instance, exaggerates the technology’s safety by comparing a rate of crashes in FSD-piloted Teslas that triggered airbag deployments to a federal crash rate for all vehicles that includes far less-severe accidents. The company also compares its cars to the average U.S. vehicle – which is much older than the average Tesla. That distorts the results because all automakers have recently launched new safety features that reduce crashes, the researchers said.

“Any new car is dramatically safer than a 12-year-old car,” said Phil Koopman, a Carnegie Mellon University engineering professor and autonomous-vehicle safety expert. “It’s like saying: ‘My jet airplane is faster than your World War II bomber.’ Yeah, so, what’s your point?”

Tesla didn’t respond to detailed questions from Reuters for this report.

Tesla’s CFO, Vaibhav Taneja, first made the 10-times-safer claim last July, after Tesla’s Austin robotaxi launch. Tesla Board Chair Robyn Denholm repeated it at a November meeting where shareholders approved a pay package granting Musk up to $1 trillion in Tesla stock. Musk, at the same meeting, displayed a chart with the slightly more modest claim of “85% less crashes,” based on recently revised Tesla methodology.

“We almost feel comfortable allowing people to text and drive, which is kind of the killer app,” Musk told shareholders. “In the next month or two – we’re going to look closely at the safety statistics – but we will allow you to text and drive, essentially.”

Six months later, Tesla hasn’t greenlit texting and driving with FSD. The fine print on its FSD website continues to warn: “Currently enabled features require active driver supervision and do not make the vehicle autonomous.” Tesla has often cited such disclaimers when sued over serious accidents.

FSD is widely regarded as capable of navigating many driving situations, sometimes for long periods. But full autonomy has proven elusive for Tesla and other companies, as it demands flawless execution by the technology – including in the most complex driving scenarios.

Seven of the former data labelers told Reuters they wouldn’t trust FSD to drive them. “We have all seen it fail,” one said. Another said he wouldn’t ride in a Tesla robotaxi “if you fucking paid me.” One veteran self-driving engineer, who reviewed Tesla crash data for years, called its safety claims “bullshit.”

“Definitely,” the engineer said, “don’t trust Elon on this.”

‘Trauma team’ reviews near misses

Tesla’s data labelers get a close-up view of FSD’s capabilities as they review footage from vehicles equipped with eight exterior cameras.

The former employees reported regularly seeing FSD fail at basic tasks, including pulling over for emergency vehicles and giving motorcyclists enough space. Sometimes, they saw FSD-piloted vehicles fail to brake on freeway off-ramps, including a case where a Tesla hit a concrete wall. (The footage, they said, didn’t show whether anyone was hurt.) Two employees said clips showed FSD failing to avoid construction zones. In one such incident, a Tesla drove into the zone, nearly striking workers, one of the people said.

Reuters did not review the videos; this account is drawn from the former staffers’ descriptions of footage they viewed.

Inside Tesla, managers carefully controlled access to the videos. Because employees only see clips they’re assigned, they may or may not see FSD’s worst failures.

One data-labeling team focused on near-misses of pedestrians, three employees said. Known informally as the “trauma team,” one source said, these employees worked in Palo Alto, California, with special permissions to view the footage. Engineers closely guarded the trauma-team clips, but some footage would occasionally “slip through” to other teams, the person said.

The person and another employee said they saw clips showing drivers manually taking over at the last second when FSD failed to recognize pedestrians in crosswalks. Two other former employees recalled seeing videos last year of FSD-piloted Teslas nearly hitting children.

Tesla has for years faced federal investigations and lawsuits involving crashes, including fatalities, that drivers or regulators blamed on failures of FSD or its older Autopilot advanced driver-assistance system.

The US National Highway Traffic Safety Administration (NHTSA) opened an investigation into Autopilot in 2021 after a series of collisions involving Teslas striking emergency vehicles. The investigation led to a 2023 recall in which Tesla installed software upgrades to better detect when drivers stop paying attention and alert them.

NHTSA has four active investigations into FSD and Autopilot, including one involving dozens of cases where vehicles using FSD failed to stop for red lights or turned into oncoming traffic. Another one probes whether Tesla’s 2023 upgrades to Autopilot were sufficient to address the safety problems. The agency is also investigating at least nine FSD-involved incidents, including a fatal crash, where the system failed because of reduced visibility in conditions such as fog or sun glare.

Tesla last year was hit with a $243 million verdict after an Autopiloted Tesla crashed in Florida, killing a 22-year-old woman and severely injuring her boyfriend. Tesla has appealed. The company has settled several similar cases involving serious crashes without disclosing terms.

When asked by Reuters, NHTSA didn’t address the findings in this story about FSD safety and Tesla’s methodology. The agency referred questions about Tesla’s safety claims to the U.S. Federal Trade Commission (FTC).

The FTC declined to comment on Tesla’s safety statistics. Some consumer advocacy groups and U.S. senators have called on the FTC to investigate Tesla’s marketing of Autopilot and FSD.

The FTC has brought no enforcement actions against Tesla. – Rappler.com