This Arizona college student has taken over 60 driverless Waymo rides

[Karlston]

This Arizona college student has taken over 60 driverless Waymo rides

Human drivers are pretty safe, making it hard to prove driverless tech is safer.

Enlarge / Joel Johnson with a driverless Waymo vehicle.

Joel Johnson

172 with 100 posters participating

Waymo has long kept details about its industry-leading self-driving technology under wraps. The company has done millions of miles of testing in Arizona and California—including thousands of driverless miles with no one behind the wheel. But until last month, almost everyone who experienced those driverless rides was bound by a strict non-disclosure agreement.

 

In October, Waymo finally pulled back the curtain on its driverless technology. Today customers near the Phoenix suburb of Chandler can hail a fully driverless taxi. They can record rides, publish videos, and talk to reporters about their experiences.

 

One young Arizonan in particular has leapt at the chance to document the real-world performance of Waymo's driverless taxis. Joel Johnson is an Arizona State University student who is taking a break from college during the pandemic. He lives near Waymo's service territory and has been using some of his free time to put Waymo's driverless taxis through their paces. He says he has taken more than 60 driverless rides in the two months since Waymo opened driverless service up to the public. He has posted more than a dozen videos.

 

The most striking thing about these videos is how boring they are. In nearly five hours of driving, I didn't see Waymo's vehicles make a single significant mistake. That contrasts with the "full self-driving" software Tesla released in beta back in October. I watched three hours of videos of customers testing out Tesla's technology. Drivers intervened more than a dozen times—including two cases where a crash seemed imminent.

 

Johnson's experience hasn't been like that at all. "It's been rock solid," he told Ars in a phone interview.

“They forget that there’s nobody driving”

Enlarge / Johnson without his mask atop another type of vehicle that steers itself.

Joel Johnson

Johnson has offered rides to a number of others. In addition to friends and family, Johnson says he has played host to industry insiders and YouTubers who made special trips to the Phoenix area to see Waymo cars in action.

 

"Everyone I've taken along with me in private rides, they trust it," Johnson said. "They forget that there's nobody driving except the computer because of how smooth the experience is."

 

"I love how the braking and acceleration is so—you don't even notice," one of Johnson's companions said during a ride. "It's really getting very smooth."

 

Johnson has been riding in Waymo vehicles since mid-2019, when he joined Waymo's closed Early Rider program. He says he has seen significant progress.

 

"They've really ironed out stuff like unprotected lefts," Johnson said in one video. "It's definitely improved over time."

 

"That was awesome," a passenger said to Johnson at the end of one ride. "It's getting smarter. That was a lot better than in March."

 

Johnson also said Waymo's vehicles are getting better at dealing with pedestrians.

 

In an October video, a Waymo car was driving through a Costco parking lot crowded with pedestrians. It waited patiently until they were out of the way, then moved forward confidently.

 

"This amount of pedestrians would have caused whiplash-inducing brake usage in March 2020," Johnson wrote in an on-screen note. "And it would have completely given up in July 2019. No longer!"

 

The vehicles are still a little too cautious around pedestrians. In one recent video, Johnson called a Waymo vehicle to a crowded retail parking lot, hit "start ride," and then had to wait almost 3 minutes before the vehicle moved a significant distance. There were apparently so many pedestrians and other vehicles around that the Waymo car didn't feel safe moving forward.

 

A human driver almost certainly would have moved sooner. But it's hard to blame Waymo for this—far better to be a little slow than to risk running someone over.

Proving safety isn’t easy

Enlarge / A Waymo car.

Joel Johnson

Of course, four hours of perfect driving—or 40 or 400 hours for that matter—wouldn't be enough to prove that Waymo's cars are safe. To properly evaluate the safety of Waymo's vehicles requires a lot of data. And Waymo has more than 20 million miles of real-world driving data. Almost all of that mileage is on public roads with a safety driver behind the wheel. A small fraction—65,000 miles (105,000km) up until September 2020—were fully driverless.

 

Until recently, Waymo kept this data under wraps, making it difficult for the public to evaluate the technology. In October, Waymo took a big step toward greater transparency by releasing data about the real-world performance of its vehicles. It covered 6.1 million miles (10 million km) the company logged in the Phoenix metro area in 2019 with a safety driver behind the wheel—plus 65,000 miles (105,000km) of driverless operation from the start of 2019 through September 2020.

 

In six million miles of driving, Waymo's vehicles were involved in 18 crashes. Of course, for most of those 6 million miles, the vehicles had safety drivers who were supposed to intervene in case of an imminent crash. To estimate how well the cars would have performed without a safety driver, Waymo performed simulations of every situation where a safety driver took control. These simulations predicted that another 29 crashes would have occurred if the safety drivers hadn't intervened.

 

While 47 crashes might seem like a lot, it's important to remember the denominator. Waymo's vehicles got into a crash—or likely would have gotten into a crash without human intervention—once every 130,000 miles (210,000km) or so. That's equivalent to more than 10 years of driving for a typical human being who drives 1,000 miles per month.

 

It's surprisingly difficult to figure out what the comparable rate would be for a typical human driver. Some of the 47 collisions Waymo reported were extremely minor. For example, a pedestrian walked into the side of a stationary Waymo vehicle at 2.7 miles per hour (4.3km/h). Two simulated crashes involved a bike and a skate boarder rolling into the sides of stationary Waymo vehicles at speeds of 2.2 and 5.9 miles per hour, respectively.

 

Such minor low-speed collisions would never be reported to the police or other authorities, so we don't know how many "crashes" like this a typical human driver experiences.

 

More important, most of those 47 incidents appeared to be the fault of another driver. For example, a third of real and simulated crashes were rear-ending incidents. All but one of these—14 actual collisions and one simulated crash—involved another vehicle rear-ending a Waymo car. The final rear-ending was a simulated crash where the Waymo car would have rear-ended another vehicle at a speed of 1mph.

 

Most sideswipe crashes (eight out of ten) involved the other vehicle changing lanes into the Waymo vehicle's lane. In one of the cases where the Waymo changed lanes, Waymo says the other car was traveling 30mph above the speed limit.

 

Over six millions miles of driving, there were only three collisions (and five simulated crashes) severe enough to trigger the deployment of an airbag. Waymo says that none of these resulted (or would have resulted) in severe or life-threatening injuries.

 

To sum up: over six million miles of driving, Waymo had a low rate of crashes, had no life-threatening crashes, and most of the crashes that did occur were the fault of the other driver. These results make it plausible that Waymo's vehicles are safer than the average human driver in the vast majority of situations.

Waymo’s extreme gradualism

Enlarge / A seat-back display updates customers on the state of the ride.

Joel Johnson

There's still one big open question, however: whether Waymo's driver is less likely to cause a fatal crash. The difficulty here is that the US has just one highway fatality for every 100 million miles traveled. So even six million miles of near-flawless driving doesn't come close to proving that Waymo's driver is less likely to kill someone than a human driver.

 

There's a difficult chicken-and-egg problem here because even a company with Waymo's deep pockets probably can't afford to test its technology for hundreds of millions of miles before launching a commercial product. Yet it's risky to launch a driverless car before you've proved that it's safer than a human being.

 

So Waymo is approaching the problem very gradually. Its cars are limited to roads with a maximum speed of 45 miles per hour. That makes sense since high-speed highway crashes are most likely to get someone killed. Waymo is also rolling out its fully driverless service at a glacial pace. Two years after it had hoped to launch a driverless commercial service, the company is still only doing about a few hundred rides a week.

 

I would guess there's an army of people behind the scenes monitoring and analyzing each ride to make sure it goes flawlessly. If those evaluations are positive, the company will presumably increase the number of cars on the road. Eventually, Waymo will become confident enough in its strategy to expand to a larger area—first across the Phoenix area and then in other metropolitan areas.

 

It's a very different approach than Tesla, which tends to push out new software updates with relatively little testing and relies on customers to monitor the system for mistakes. Since 2016, at least three Tesla customers in the US have died after they failed to correct bad decisions by the Autopilot software.

 

Waymo seems more determined to keep its near-flawless driving record. The question is whether this cautious strategy will allow it to scale up quickly enough to be a viable business.

 

Correction: This story originally said that Waymo has a waiting list but the company recently ended its waiting list. I also understated the number of Waymo rides—the company says it's now doing hundreds of rides per week.

 

 

This Arizona college student has taken over 60 driverless Waymo rides