9 - Boris Sofman, Waymo: How do you train autonomous vehicles?

What’s the deal with autonomous vehicles? For starters, saving 40,000 lives per year and answering some of the most interesting questions in artificial intelligence.

Boris Sofman, Head of Perception and Trucking Engineering at Waymo, came in to discuss the methods used to train autonomous vehicles, covering where datasets are gathered, how performance is evaluated, and the potential for these approaches to be leveraged across different industries.

Read on for the transcript!

Hey, it’s Monte. I’m doing a couple talks this January: one is the second Feature Store Meetup, where I’ll be speaking along with a technical lead from Uber. The other is TWiMLCon, a machine learning conference lead by my fellow podcaster Sam Charrington. You can find out more at mlminutes.com/events.

Monte 0:10

Hi, I’m Monte Zweben, CEO of Splice Machine. You’re listening to ML Minutes, where cutting-edge thought leaders discuss Machine Learning in one minute or less.

This episode, our guest is Boris Sofman, head of Perception and Trucking Engineering at Waymo, an autonomous driving technology company. Prior to Waymo, Boris was the co-founder and CEO of Anki, where he developed and shipped over three and a half million racing and entertainment robots and devices all over the world. Welcome Boris.

Boris 0:40

Thank you, Monte. Real pleasure to be on.

Monte 0:43

Well, Boris, you've done really interesting things, from your PhD thesis in robotics to what you're doing now at Waymo. Tell us about your journey. How did you get to where you are now?

Boris 0:54

I started really getting excited about artificial intelligence and robotics. Did a PhD at Carnegie Mellon, where I was focused on off-road autonomous vehicles. There was a lot of the early research at the time was happening at Carnegie Mellon was just super fun. I took a detour from the space for about nine or 10 years, and started Anki, which was a consumer robotics startup focused on a variety of entertainment products, which was a lot of fun, we shipped products all over the world and were able to push on unique capabilities at low cost. And now I am at at Waymo, leading the perception team and the autonomous trucking team.

Monte 1:34

Excellent. I can't imagine how much fun it is to go from just studying this to putting toys out there for kids to enjoy robotics all the way through to working with autonomous vehicles. Just for our audience to truly understand, why is it so important to have self-driving vehicles? Why is autonomous driving so important?

Boris 1:58

It's without a doubt one of the biggest technology potentials in the world, both in terms of safety and efficiency. So on the safety front, on the consumer side, you have just in the US 40,000 people a year that die from automotive accidents, and you know that that's just a safety argument. On the efficiency side, we've seen the stats on how much people-- how much time people spend in traffic. And on trucking, there's a shortage of drivers that's 65,000 today, it'll be 225,000 within a handful of years, and the average age is 50. Because nobody wants to go into this field while the demand is actually getting higher and higher. And so you have this like backbone of both personnel and goods transportation, that is now constrained by all these awkward things like commute time, or caps on driver distance, and so forth, that can completely rethink both our urban environments are laid out as well as how our goods get to us.

Monte 3:03

Interesting. So you're not only solving a convenience problem, this is a true supply chain optimization problem that's constrained by the capacity of drivers.

Boris 3:14

Yeah, absolutely.

Monte 3:15

Tell us about the problems that you're working on now to solve the driverless experience at Waymo.

Boris 3:22

Yeah, very different flavor of problems than before. Definitely different constraints and different scale of challenges. But it's, it's the most exciting robotics and AI problem I've ever seen in my life. So I get to focus on the autonomous trucking vertical, it's interesting, because it's a slightly more structured environment that you can lean into, than the consumer side, but at the same time, we're dealing with higher speeds and higher safety risks. And so there's different types of complexities. The-- what our goal is, is to get large scale autonomous trucks and vehicles in general, in the case of Waymo, into mass market use, and to do it safely and reliably. In the process, we get to tackle some of the most interesting AI problems that I could have ever imagined and definitely felt a lot of appreciation for them.

Monte 4:14

Excellent. So in your experience, in this more structured but higher speed environment on the trucking side, what are some of the main uses of machine learning for autonomous driving?

Boris 4:31

So a lot of the core technology is actually shared between the car side and the trucking side, which has been a really massive advantage because Waymo has 10 years of experience in working on this problem. So it's one of the areas that's most mature in terms of leveraging machine learning as a perception system, as a team I've gotten to work with very closely. We have incredibly large scale data coming in from a large variety of sensors. And at the end of the day, the problem of understanding robustly what is happening in the world around us is a pretty mass scale machine learning problem with a wide variety of categories from objects to semantics to even down to kind of base filtering. And so we're leveraging a huge amount of models simultaneously. And over time, we're actually starting to use far more machine learning approaches, even on the planning side as well, which was traditionally a much more search-driven problem.

Monte 5:27

Excellent. I read somewhere in the materials that are on the Waymo blog that often some of the model building may actually have human input, or at least be evaluated by human input with drivers that are going along in the testing vehicles there. Can you talk a little bit about how that human expert is in injected into the process?

Boris 5:56

Yeah, sometimes that's actually one of the best signals that you have where it's hard, particularly when you think about planning and behavior of the vehicle itself, it's very hard to come up with exactly the right answer the way you would on a classifier for, for vehicles or for other animals. In the case of something like planning, oftentimes, the right answer is a human expert driver, whether it's in a truck or a car. And the best metrics could end up being comparisons to that human driver, like how, how were you relative to the level of aggressiveness or conservatism? What choice did you make, versus what they made? Or even in the case of, of a prediction system to understand what might happen in the future for particular other agents in the environment? The best answer is exactly what they did do in the future. And so you could actually use that to have an automated training feedback loop to go back and actually train your systems in a way that's much more geared towards the real world.

Monte 6:59

Excellent. So I think what you're saying is that, on the planning side of the use of machine learning, where you're determining in a current state, what is the next best action to take to get into a future state that's closer to your objective or goal, that you may actually compare what the system did to what a human would do in that case, and measure the differences and learn from those differences. Is that a fair assessment?

Boris 7:29

Yeah, that's a fair assessment. And what's really interesting is that that expert could either be your vehicle and what the driver did, or in some classes of problems, what other vehicles did. And so we would, the advantage of one of the advantages that Waymo has is the sheer amount of data that we've collected, where we have over 20 million miles of autonomous driving with all of the data that comes with that. And so there's a giant amount of historical data that we can use to train these large scale systems. And sometimes it's to directly train a behavior or a response to a situation. And sometimes it's actually indirectly propagating those signals back and actually training things like cost functions, or heuristics that help us better influence the overall behavior of the vehicle.

Monte 8:15

Excellent. Okay. Let's turn to the stack now. What are some of the tools that you're using in your tech stack throughout each l