Your robotaxi might be a narc

In early September, two teenagers got into a Waymo, but then ended up in the back of a police car. The robotaxi company said it detected “a violation of our terms of service involving a firearm,” pulled the car over, and alerted emergency services, according to the Los Angeles Times. Police arrested the passengers after allegedly finding a loaded AR-style “ghost gun” in the vehicle.

The arrests exposed a lot of unanswered questions about privacy and autonomous vehicles, including what they are recording, what they do with the footage, and how much privacy riders should expect while riding inside them.

Waymo’s camera-covered cars have already been drawn into the debate over privacy and surveillance. On a number of occasions, law enforcement agencies have obtained search warrants for footage captured by its cars, according to Bloomberg. Last year, the LAPD obtained footage of a hit-and-run that was recorded by a nearby Waymo vehicle. During anti-ICE protests in Los Angeles, five Waymos were vandalized and set on fire amid questions about whether the vehicles could be capturing images of protesters that might be used as evidence to identify or arrest them. Waymo does not publicly disclose how many law enforcement requests it receives or how often it complies.

Those concerns echo the backlash against Flock’s network of automated license-plate readers, which have given police the ability to track cars across jurisdictions and have prompted growing fears about government surveillance and misuse. Unlike fixed Flock cameras, Waymos move through cities carrying cameras both outside and inside their vehicles.

The latest case of the arrested teens takes those concerns a step further. Police did not approach Waymo after a crime looking for footage its cameras happened to capture. Waymo detected something inside the car and initiated the police response itself.

Waymo spokesperson Julia Ilina told The Verge in an email that the company uses its internal cameras to help enforce its terms of service, including detecting prohibited items and behavior. She said it uses automated technology to help Waymo’s teams identify potential violations, such as the presence of firearms inside the vehicle. “If Waymo detects a firearm in the vehicle, we may reach out to local law enforcement,” she added.

Waymo did not explain how its automated system determines whether something may be a firearm, how confident it has to be before something gets flagged, what role humans play between an automated alert and a call to police, or what other violations of its terms of service might prompt a police response. Those are important omissions. This wasn’t a human taxi driver deciding to call 911. It was a driverless car, equipped with cameras and an AI brain, escalating a situation involving its passengers to police.

Bob Somers, a former engineer at autonomous vehicle company Zoox and technical adviser to the AV-safety nonprofit SAVE-US, said he would expect Waymo’s system to look for a range of potentially dangerous weapons and behaviors. The software may not need to decide with certainty that it has seen a firearm. It could instead flag something that resembles one for further review, he said.

“If there is a certain set of objects detected, then that would probably loop in a remote operator to take a look at what’s going on,” Somers said.

Phil Koopman, a Carnegie Mellon University professor emeritus who researches autonomous vehicle safety, said computer vision systems can be trained to recognize objects and potentially dangerous situations.

“If you can train a computer to look at the road and decide how to drive, you can train a computer to decide if somebody’s holding a gun,” Koopman said. But, he added, those systems are “never 100 percent accurate.”

That’s where the unanswered question about human review becomes important.

A human taxi driver who sees a passenger holding something has context. They’ve been in the car for the entire ride. They can hear what the passengers are saying and ask questions. They can decide whether the object is a real gun, a toy, or something else entirely. Back in July, Waymo reported two teens to the police for underage drinking and shooting a toy gun loaded with water beads from the vehicle. It was not clear whether the teens ultimately faced any charges.

A remote worker brought in to review a flagged moment might not have all of that context because they haven’t experienced the entire ride the way a human driver would. Waymo says its interior microphones are generally active only during calls with Rider Support or when riders enable them, meaning a reviewer may not be able to hear what preceded the flagged moment. As of earlier this year, Waymo has about 70 “remote assistance agents” on duty at a given time in the US and the Philippines, who “provide advice and support to the Waymo Driver but do not directly control, steer, or drive the vehicle.” Waymo has not said whether the people reviewing possible violations inside its cars are among those agents.

“It’s really important that Waymo gets it right when they alert the police to something dangerous,” Koopman said. Beyond undermining riders’ trust in robotaxis, a false alarm could put passengers at risk during a police encounter.

Then there’s the question of what Waymo tells riders to expect. Without a driver in the front seat, robotaxis can feel more private than they actually are. In 2023, the San Francisco Standard spoke with four people who said they had hooked up or had sex in Cruise robotaxis, despite the cameras inside them. The publication could not find anyone who said they had done so in a Waymo, but the episodes illustrate how readily passengers can mistake the absence of a human driver for the absence of a witness.

Waymo’s terms of service requires riders to comply with applicable laws and prohibits them from creating a dangerous condition, carrying weapons “or anything that resembles a weapon, e.g. toy gun,” or using drugs. Riders must be at least 18 to ride alone in San Francisco. The two riders arrested this month allegedly violated both the age and weapons restrictions.

The terms say Waymo can suspend or terminate a rider’s access if it believes the rider has violated the rules. Earlier this year, Waymo suspended the account used to book a ride after three boys were photographed hanging out the windows of the vehicle and taking selfies. But they do not plainly tell passengers that the company may call the police based on what it sees inside the car.

Waymo’s public camera policy says its interior cameras can be used to “check that in-car rules are being followed” and “promote safety and security.” It says Waymo Support may review recorded video after an issue is brought to the company’s attention and, in urgent circumstances, access live video during a trip.

The policy also says Waymo may share certain data with law enforcement as needed to comply with legal requirements, enforce agreements, or protect the safety of riders and others. But it doesn’t specify that automated monitoring inside the vehicle may trigger Waymo itself to contact police.

Hayley Tsukayama, director of state affairs at the Electronic Frontier Foundation, called Waymo’s public policies “very broad.” Riders shouldn’t have to infer from language about “safety and security” that their behavior could result in a police response, she said, adding that Waymo should clearly disclose that possibility before passengers get into the car.

“I’m not condoning illegal behavior in rideshares,” she said, but “if they’re going to share it with authorities, then that should be clearly stated in the policy.” As it stands, Tsukayama said, “it is outside of the expectations of consumers.”

Then again, robotaxis themselves are outside the norm. As they become more common, riders may have to develop a new set of expectations about what happens inside them. A human driver can decide how to respond to what happens in their car. In a robotaxi, cameras and software can flag what happens inside, remote workers may become involved, and the vehicle can pull over while police are summoned.

Waymo’s system may have helped police recover an illegal firearm in this case. But the incident also raises a more basic question about what kind of space a robotaxi is: a ride with cameras for safety, a continuously monitored space, or a system that can call the cops on its own passengers.

That question is already showing up in fiction. Robotaxis are making their debut as villains in Road Rage, an upcoming short film about a driverless car that turns against humans. One of the film’s more unsettling ideas is that a robotaxi could end up acting less like transportation and more like law enforcement. This month offered a glimpse of what that might look like.