Waymo has 7.1 million driverless miles — how does its driving compare to humans?

For years, Waymo has been claiming that its driverless vehicles have the potential to be safer than humans. Now the company says it has the data to back it up.

Waymo analyzed 7.13 million fully driverless miles in three cities — Phoenix, Los Angeles, and San Francisco — and compared the data to human driving benchmarks to determine whether its cars were involved in fewer injuring-causing and police-reported crashes. And it was the first time the company studied miles from fully driverless operations only, rather than a mix of autonomous and human-monitored driving.

The conclusion? Waymo’s driverless cars were 6.7 times less likely than human drivers to be involved a crash resulting in an injury, or an 85 percent reduction over the human benchmark, and 2.3 times less likely to be in a police-reported crash, or a 57 percent reduction. That translates to an estimated 17 fewer injuries and 20 fewer police-reported crashes compared to if a human driver would have driven the same distance in the cities where Waymo operates.

Waymo’s analysis comes at a fraught moment for autonomous vehicles. The company’s main competitor, Cruise, has paused operations nationwide after a crash in San Francisco resulted in a pedestrian being dragged 20 feet by one of the company’s driverless cars. Cruise allegedly withheld video footage of the incident from regulators and is now facing up to $1.5 million in fines from the state. Lawmakers and labor advocates are calling for a crackdown on autonomous vehicles, regardless of company ownership.

But Waymo said the timing of the release of its safety analysis — coming at a time when its main rival is facing its worse crisis in years — is just a coincidence. Still, Waymo’s director of safety research and best practices Trent Victor said the data can help the public understand that not every company developing autonomous vehicles is the same.

“What we’d like to do is give a clearer picture to allow the people to see the difference” between Waymo and other AV companies, Victor said in a briefing with reporters. “Another difference is we are scaling responsibly.”

The company is using public data to form its conclusions, inviting other, third-party researchers to replicate its results. Waymo and other AV operators are required to report every crash to the National Highway Traffic Safety Administration as part of the agency’s efforts to shine a light on self-driving cars in the US.

Waymo’s millions of miles were not totally incident-free. The company said that in total, over the entire 7 million-plus miles in all three cities, its vehicles were only involved in three crashes that resulted in injuries: two in Phoenix and one in San Francisco. All three injuries were minor, according to Kristofer Kusano, safety researcher at Waymo and a co-author of the study.

Still, that’s significantly less than the crash rate for human drivers. Another way of looking at it is to look at the crash rate per million miles of driving. The human benchmark is 2.78 incidents per million miles. Waymo’s benchmark for its driverless vehicles was only 0.41.

One of the biggest challenges for Waymo was controlling for various factors when comparing its vehicles to human drivers. In order to present a fair comparison, Waymo needed to address statistical biases in the data, such as human drivers failing to report minor crashes, or differences in driving conditions.

For example, Waymo’s vehicles operate in geofenced areas in the three cities where they drive, which excludes highways. Human drivers don’t avoid these types of roads. Human drivers also tend not to report certain low-level crashes, like minor fender benders. In contrast, Waymo is legally required to report every contact event with another vehicle, no matter how minor. As a result, Waymo needed to adjust its model to account for those factors.

Waymo’s analysis included “underreporting adjustments for police-reported crashes or were derived from naturalistic driving study databases,” the company said, citing “a literature review of 12 past studies and 1 book comparing AV and human crash rates.”

“The goal was to say, here’s what the human crash population would have looked like if it had driven under similar conditions as what the automated vehicle did,” said John Scanlon, safety researcher at Waymo and co-author of the study.

The analysis comes on the heels of a study that Waymo published in conjunction with Swiss Re that found that the company’s driverless vehicles reduced the frequency of bodily injury claims by 100 percent, compared to Swiss Re’s human baseline of 1.11 claims per million miles. 

The current state of the self-driving car industry is very unsettled. AVs are operating in a small handful of cities, but many people say they are skeptical of the technology and wouldn’t necessarily trust it over their own driving. The negative attention around the Cruise incident, as well as the recent recall of Tesla’s Autopilot, are just the latest in a series of bad headlines that have led many to conclude that self-driving cars are a fad, or even worse, more dangerous than human drivers.

Waymo has gone further than other AV companies in the use of data and statistical analysis to make the safety case for autonomous vehicles. Waymo insists that driverless cars are necessary as an antidote to the crisis of traffic fatalities, of which there are around 40,000 a year in the US. The company often points out that driverless cars never get drunk, tired, or distracted and are able to avoid the human errors that so often lead to crashes and deaths.

Last year, the company produced two scientific papers comparing autonomous vehicle performance to human driving. The first analyzed response times when a crash is imminent, while the other presented a novel methodology to evaluate how well autonomous driving systems avoid crashes. Waymo has also sought to measure the safety of its AVs by simulating dozens of real-world fatal crashes that took place in Arizona over nearly a decade. The Google spinoff discovered that replacing either vehicle in a two-car collision with its robot-guided vehicles would nearly eliminate all deaths.

It is an inarguable fact that there are far fewer AVs on the road than human-driven vehicles and, thus, less data from which to draw conclusions. Humans drive close to 100 million miles between fatal crashes. Some experts assert that we’ll need hundreds of millions of miles from autonomous vehicles before we can start to make more meaningful comparisons about safety.

But Waymo’s efforts seem like a step in the right direction. “These reports represent a good-faith effort by Waymo to evaluate how the safety of its [autonomous driving system] compares with the safety of human driving,” said David Zuby, chief research officer of the Insurance Institute for Highway Safety. “The results are encouraging and represent one step in our evolving understanding of ADS safety.”

Other researchers agree. “This provides the strongest evidence I’ve seen yet that ADS have lower crash rates than humans,” Carol Flannagan, a researcher at the University of Michigan Transportation Research Institute. “I think the more complicated question is what this means in the broader journey to determining when ADS are ‘safer than humans.’”