Toyota’s Gill Pratt on Self-Driving Cars and the Reality of Total Autonomy
After wrapping up the DARPA Robotics Challenge in 2015, Gill Pratt helped to launch the Toyota Research Institute (TRI), which is investing over a billion dollars in robotics and artificial intelligence over the next five years. As you might expect, a major concentrate of TRI is automotive autonomy: Toyota is just as interested as any automotive manufacturer at using autonomous systems to make cars safer, more efficient, and more pleasant to drive.
At Toyota’s CES press conference earlier this month, Pratt took the stage to address some of the challenges facing anyone working on automotive autonomy. There are many of these, and frequently, the amount of progress that the industry is making towards total autonomy is misunderstood, or even periodically misrepresented. With that in mind, he spent a solid twenty minutes providing the audience a much needed reality check.
For more, IEEE Spectrum contributing editor Evan Ackerman spoke with Pratt at Toyota’s booth at CES. The following has been edited and condensed for clarity.
No One Is Close to Achieving True Level five Autonomy
IEEE Spectrum: At the Toyota press conference, you said that Level five autonomy—when a car can drive totally autonomously in any traffic or weather condition—is “a wonderful purpose but none of us in the automobile or IT industries are close to achieving true Level five autonomy.” This certainly isn’t what it seems like when you walk around the automotive hall at CES, not to mention the host of self-driving demos we’ve seen recently.
Gill Pratt: The most significant thing to understand is that not all miles are the same. Most miles that we drive are very effortless, and we can drive them while daydreaming or thinking about something else or having a conversation. But some miles are truly, indeed hard, and so it’s those difficult miles that we should be looking at: How often do those demonstrate up, and can you ensure on a given route that the car will actually be able to treat the entire route without any problem at all? Level five autonomy says all miles will be treated by the car in an autonomous mode without any need for human intervention at all, ever.
So if we’re talking to a company that says, “We can do total autonomy in this pre-mapped area and we’ve mapped almost every area,” that’s not Level Five.
That’s Level Four. And I wouldn’t even stop there: I would ask, “Is that at all times of the day, is it in all weather, is it in all traffic?” And then what you’ll usually find is a little bit of hedging on that too. The trouble with this Level four thing, or the “full autonomy” phrase, is that it covers a very broad spectrum of possible competencies. It covers “my car can run fully autonomously in a dedicated lane that has no other traffic,” which isn’t very different from a train on a set of rails, to “I can drive in Rome in the middle of the worst traffic they ever have there, while it’s raining,” which is fairly hard.
Because the “full autonomy” phrase can mean such a broad range of things, you indeed have to ask the question, “What do you truly mean, what are the actual circumstances?” And usually you’ll find that it’s geofenced for area, it may be restricted by how much traffic it can treat, for the weather, the time of day, things like that. So that’s the elaboration of why we’re not even close.
BACK TO TOP↑
The Over-Trusting Autonomy Paradox
And all this creates an expectation problem for consumers because they’re hearing all of this stuff and they don’t know what it means.
Right. And that’s particularly true because, as consumers, we have a tendency to attempt to build a model in our brains to predict how good the autonomy is going to be. That model tends to be an anthropomorphic one: We concentrate on certain things, and we don’t see when the autonomy doesn’t treat certain other things, and we tend to either over-trust or under-trust how good the autonomy is.
[Toyota’s] view is that consumer education is absolutely significant to truly understand the limitations of the technology. It’s very significant that the industry as a entire strive to put themselves in the boots of the customer and say, “How do we make sure they indeed understand how good this is, what it can do, but most importantly, what it can’t do,” so people don’t over-trust their vehicles.
That over-trusting is another big problem, because once you give the car to a consumer and the consumer practices some of what it can do in a favorable environment.
It’s truly difficult. As human beings, we tend to make predictions, that’s sort of how our brains are wired. Things like, “The weather was good today, the weather was good the day before, so therefore the weather is going to be good forever.” As we improve the technology for autonomous cars, they’re going to require human intervention less and less. And this will exacerbate the over-trust problem, because the tendency will be that people will say, “It hasn’t needed me before, it doesn’t need me now, so it will never need me.”
In some ways, the worst case is a car that will need driver intervention once every 200,000 miles. And so an ordinary person who has a car every 100,000 miles would never see it. But every once in a while, maybe once for every two cars that I own, there would be that one time where it all of a sudden goes “beep beep beep, now it’s your turn!” And the person, typically having not seen this for years and years, would entirely over-trust the car and not be ready when that happened. So we worry about this paradox: The better we do, the worse the over-trust problem could become.
BACK TO TOP↑
If Self-Driving Cars Need to Be Better Than Human Drivers, How Good Is Good Enough?
Do you think utter Level five autonomy in that context is realistic, or even possible?
I think that it is possible, and our standard is some numerous of how good human beings are at the task of driving. I asked this question recently [at Toyota’s CES press conference], which I think is utterly significant, and which society has not answered yet: What standards should we use, and how good does the system need to be?
It will be many years before autonomous cars are flawless, and so some crashes are likely to happen. Are we as a society ready to accept ten percent better than human driving? One percent better than human driving? Ten times better than human driving? I don’t know, and to be fair, it’s not our role as technologists to tell society what the reaction is. It’s the government and everybody else’s role who will be affected by this to weigh in and say, “Well, maybe it’s good enough to save one life.” Or should we actually say, “We’re not going to use it until it’s ten times better.” I’m not sure, but I think until we have an response to that question, we have to be very careful to not introduce technology that completes up out of line with what society’s expectations are.
In the context of autonomous cars and safety, I’ve attempted to make the point at times—and others have as well—that if autonomous cars are even just one percent safer than humans, we should still be using them.
From a rational point of view, you’re exactly right, but it’s an emotional thing, right? People aren’t rational: If you look at plane crashes as an analogy, planes are without a doubt, by far, the safest way to travel, per mile. You should never worry that you’re going to have a crash in a plane. But plane crashes do happen, and when they happen, we have this sort of exaggerated response, where we become worried and afraid that our plane is going to crash. Rationally, it makes no sense at all: it’s much more likely that we’ll have a crash in a car than we will have in a plane, and yet it’s plane crashes that are in the news all the time, so we end up fearing the wrong things.
For an accident in a human-driven car, it’s a situation where we can put ourselves in the boots of the driver of that car and say, “That could have happened to me. I could have made that mistake.” If it’s a machine, I worry that the empathy won’t be there, and that people will expect the machine to be flawless.
“AI systems are not ideal . . . Every once in a while, as we progress this technology, there will be cases where the perception system will get it wrong, and that may lead to an accident. What are we going to say when that happens?”
And yet, as we know, AI systems, especially ones that are based on machine learning, are not flawless. The space of possible inputs to the sensors is so large, the car will see things it’s never been trained on before, and we’re expecting it to come out with a reasonable perception of the world based on what it sees. Every once in a while, as we progress this technology, there will be cases where the perception system will get it wrong, and that may lead to an accident. And what are we going to say when that happens? Who are we going to blame? We don’t know the reaction, but we know it’s a very significant question.
BACK TO TOP↑
Machine Learning and “The Car Can Explain” Project
James Kuffner [who helped develop the Google self-driving car and is now CTO at TRI] was talking about this on a CES robotics panel in the context of cloud robotics. The fact that autonomous cars are going to get into accidents is a certainty, but is it the case that every time one gets into an accident, that the manufacturer will be able to tell what caused it, and then update the software in all of their other cars so that we won’t have that kind of accident again?
I think it’s very likely, in fact, I would be astonished if it’s not the case, that we will have very precise logs of what happens in an autonomous car accident. Now, you asked a very interesting question: Will we be able to figure out why?
What’s interesting about this—because machine learning systems, and deep learning in particular, are very high spectacle but don’t actually explain how they figure out the answer—is that getting the car to explain what it does is actually hard. We have research that we’re doing at MIT, and other places, where we’re attempting to make progress on that issue: the one that I can point you to is Professor Gerald Sussman at MIT, and the project that he’s working on with our funding is called “The Car Can Explain.”
“Globally, cars travel around ten trillion miles. So if you’re testing millions of miles, which car manufacturers are doing on physical cars, that doesn’t get you anywhere near close coverage of possible things that could happen to all cars . . . You have to boost it in some way, and doing accelerated testing simulation is a key part to that.
So the logs will be there, but what’s responsible for the mistake? That’s a much tighter question. Will we do something to make sure that that mistake doesn’t happen again? Of course we will, but the dimensionality of the input space immense. And so telling, “Okay, I’ve plugged this leak in the dam, I’ve plugged that leak in the dam…” It’s a truly big dam, and there are lot of places that things could go wrong. This actually brings us to the thickest part of our work: It turns out that testing is the number one thing that we need to do, and this is that entire trillion mile challenge that I talked about last year. Globally, cars travel around—within an order of magnitude—10 trillion miles. So if you’re testing millions of miles, which car manufacturers, including us, are doing on physical cars, that doesn’t get you anywhere near close coverage of possible things that could happen to all cars in the world in the course of a year. You have to boost it in some way, and doing accelerated testing simulation is a key part to that. We’re not going to simulate the “Sunday drive,” when the weather’s nice and everything’s good and there’s hardly any cars on the road. We’re going to simulate when things are terrible.
Rod Brooks has this wonderful phrase: “Simulations are fated to succeed.” We’re very well aware of that trap, and so we do a lot of work with our physical testing to validate that the simulator will be good. But inevitably, we’re pushing the simulator into regimes of operation that we don’t do with cars very often, which is where there are crashes, where there are near misses, where all kinds of high-speed things happen. For example, an autonomous car has to be ready for the bad actor, the sort of byzantine case where there’s another car, perhaps a human-driven car, that’s driven by a person with road rage, and your car has to do the right thing, even tho’ the other car was bad. And so, what’s the reaction there? We’re not going to test that very often in the physical world, because half the time, they’re going to crash, so we’ll test it once in a while, but we need to boost it with simulation and that’s fairly hard to do.
And that eventually brings up yet another area, which is, can formal methods help? There’s this tremendous promise of formal methods in programming, to say that we can prove correct operation within this regime of the state space. Presently, it’s very hard to do, so we’re looking at hybrid approaches of simulation and formal methods, blended together to attempt to get at the dimensionality of the problem. But fundamentally a thing your readers should know is that this indeed comes down to testing. Deep learning is wonderful, but deep learning doesn’t ensure that over the entire space of possible inputs, the behavior will be correct. Ensuring that that’s true is very, very hard to do.
BACK TO TOP↑
Simulating the Crazy Things Human Drivers Do
Especially right now, with autonomous cars or any car with any aspect of autonomy, I would imagine that humans are the least predictable thing on the road. Going through on-road testing, a lot of weird human-related stuff has to come up, but how do you even attempt to simulate all of the crazy things humans might do?
We attempt to model human beings in the same way we model weather or we model traffic. It’s hard to model human beings, because we have ten to the ten neurons in our brains: Every person’s different, and there’s a entire broad range of behaviors, but we know that to some extent it’s possible. As a human driver, we have theory of mind, of how other human drivers act. We run a little simulation in our head, like when we’re at a four-way stop, telling if I were that person, I’d act like this, so I’m going to do this. Theory of mind is this amazing thing that means that making a simulation is possible, because we can build statistical models to predict what other human beings are going to do.
Sometimes, tho’, there’s a difference inbetween how a human being is likely to act, and what the safe (or legal) way of acting is. How do you train a car what to do in circumstances like that?
It’s an interesting question. What speed should you travel on the highway? If the speed limit is posted at fifty five miles an hour, should you go 55, or should you go the median speed of what the other cars are doing? What’s the safest thing to do? I don’t want to give an official response, but it’s a conundrum, and we have discussed this with the legal department at Toyota. They scrape their goes and say, “That’s indeed rough.” So what do you do? And who’s responsible, right? It may be the case that driving slower isn’t the safest. How do we resolve that issue? It’s very hard to figure out.
BACK TO TOP↑
“We Actually Need a Revolution in Computer Hardware Design”
Right after the press conference, you mentioned something about how cooling and powering the computers in electrified cars is actually a big problem. We’re very focused on the challenges involving decision making, but what other things do we still have to figure out?
What I love about this stuff is that I have this entire side of me that’s the hardware fellow, but I also did a lot of my own research in the past in neurophysiology. This business of computational efficiency is indeed significant. Our brains use around fifty watts of power, while the autonomy systems for most [autonomous and semi-autonomous cars] in the field, not all of them, but most of them, use thousands of watts of power, and that’s a tremendous difference. And it’s not even that our brain is using all fifty watts to do the driving task: we think about all kinds of other things at the same time, and so perhaps ten watts is going into the driving task.
“The idea that only the chauffeur mode of autonomy, where the car drives for you, that that’s the only way to make the car safer and to save lives, that’s just false . . . There are tremendous numbers of ways to support a human driver.”
We also don’t know that the amount of computational power that’s in current prototype autonomous cars is the suitable level. It may be that if we scaled the computation by a factor of Ten, that the car would perhaps not get ten times better, but would get significantly better than what it does now. Or what would happen if we had one hundred times more computational power? It may be that it actually doesn’t plateau, but proceeds to get better. I suspect the reaction is that it does keep on growing, albeit I don’t know what the curve is. And so in addition to all the software stuff and all the testing we need to do, we actually need a revolution in computer hardware design: We need to somehow get more towards the power efficiency that the brain has.
To response your question of other things [that remain to be solved], I think sensors still have a long way to go. Lidar is an amazing thing, but it’s still not that superb. The point density is still low, and for looking at cars in the distance, it doesn’t compare with human vision. Particularly, for example, with Level three autonomy, the amount of time you need in order to wake up the driver to say, “Hey, you’re going to need to deal with this thing that’s just happening in one thousand five hundred feet in front of you,” that’s not going to happen unless your sensor can see and understand what’s happening that far away.
The sensor also needs to be cheap. It needs to be stimulation resistant. It needs to last for ten years or maybe twenty years and not break. Automotive quality, which most people think of as being low, is actually very, very high. I’ve worked a lot, in my DARPA work, with mil-spec and things like that, but automotive quality is ridiculously hard to do. And so a cellphone camera is not going to work—you’re going to need special stuff that indeed can withstand the desert and Alaska and going back and forward. How do you deal with salt and rust, with the screen getting utter of dust from the road… Making sure that’s going to work, all of the time, that’s truly hard.
So to sum this thing up, I think there’s a general desire from the technical people in this field to have both the press and particularly the public better educated about what’s truly going on. It’s very effortless to get misunderstandings based on words like or phrases like “full autonomy.” What does utter actually mean? This actually matters a lot: The idea that only the chauffeur mode of autonomy, where the car drives for you, that that’s the only way to make the car safer and to save lives, that’s just false. And it’s significant to not say, “We want to save lives therefore we have to have driverless cars.” In particular, there are tremendous numbers of ways to support a human driver and to give them a kind of blunder prevention device which sits there, inactive most of the time, and every once in a while, will very first warn and then, if necessary, intervene and take control. The system doesn’t need to be competent at everything all of the time. It needs to only treat the worst cases.
BACK TO TOP↑
Human and Machine: Who Should Guard Whom?
And often the worst cases for human drivers are things that robots are particularly good at right now, like paying attention all the time to the car in front of you, or watching where the lines are, right?
Correct. People and machines have complementary abilities: machines are very good at staying vigilant. People are very good at treating difficult driving situations with theory of mind and all that kind of stuff, and machines are particularly bad at that. So how can we put the two together in a way that they complement each other, rather than sort of fight each other? Another way of thinking about this is who guards whom, right? Should the human being guard the autonomy? Should the autonomy guard the human being? It’s a higher threshold to make this system where the human being has to guard the machine and eventually doesn’t have to guard anything at all to get the AI to work, so that’s why at Toyota we’re taking a dual treatment to things.
Most companies at CES are here to sell. At Toyota, we don’t have any trouble selling cars, so I took the chance to use this to attempt to educate. To say, “Look, this is all excellent, but let’s make sure we’re all talking about the same thing.” If we could define the capabilities, take the SAE scale [of driving automation] to heart. But even within the SAE scale, there’s so much confusion.
Yes, so NHTSA had one to Four, and SAE was very clever, they said we’ll take Level four and we’ll split it into two things, four and Five, and the difference is that Level five is everywhere at any time, and four is only some places at some times. But otherwise they’re basically the same. The SAE levels are fine, but people keep making mistakes. And so Level two systems are often called Level Three, which is wrong. A key thing of Level three is that the person does not need to supervise the autonomy. So no guarding of the machine by the human being.
And so when the driver can’t take over, what’s the Plan B? I reminisce when Freightliner introduced its autonomous truck, its Plan B was that the truck would just stop in the middle of the highway because it doesn’t know what else to do.
Buried inwards of SAE stuff, it does talk about some Plan B things where the car will pull off the road if it has to stop. I love that, if only it could be true all the time. I grew up in Fresh Jersey. There are a lot of highways in Fresh Jersey where there’s no place to pull over, and so what are you going do? What if you’re in the tunnel and there’s no side lane, no pull-off lane? It has to treat every case, and that’s what so hard about this thing.
There have been proposals for things like a remote driver, where if the car gets into difficulty, it’ll roll over to a remote driver in a call center somewhere. Very first of all, those things assume the Internet is working, there are no hackers, there’s no natural disasters. But most of all, it’s the same issue we discussed earlier, that you assume a person at a call center can abruptly wake up and say, “Okay, trouble,” and treat the situation. People are very bad at that. We’re very bad at all of a sudden being astonished and needing to orient ourselves and coming up with a correct response. So we’re not telling that that’s unlikely, but we’re telling it may be very difficult, for the reasons that we’ve outlined.
Toyota s Gill Pratt on Self-Driving Cars and the Reality of Utter Autonomy – IEEE Spectrum
Toyota’s Gill Pratt on Self-Driving Cars and the Reality of Utter Autonomy
After wrapping up the DARPA Robotics Challenge in 2015, Gill Pratt helped to launch the Toyota Research Institute (TRI), which is investing over a billion dollars in robotics and artificial intelligence over the next five years. As you might expect, a major concentrate of TRI is automotive autonomy: Toyota is just as interested as any automotive manufacturer at using autonomous systems to make cars safer, more efficient, and more pleasant to drive.
At Toyota’s CES press conference earlier this month, Pratt took the stage to address some of the challenges facing anyone working on automotive autonomy. There are many of these, and frequently, the amount of progress that the industry is making towards total autonomy is misunderstood, or even periodically misrepresented. With that in mind, he spent a solid twenty minutes providing the audience a much needed reality check.
For more, IEEE Spectrum contributing editor Evan Ackerman spoke with Pratt at Toyota’s booth at CES. The following has been edited and condensed for clarity.
No One Is Close to Achieving True Level five Autonomy
IEEE Spectrum: At the Toyota press conference, you said that Level five autonomy—when a car can drive totally autonomously in any traffic or weather condition—is “a wonderful objective but none of us in the automobile or IT industries are close to achieving true Level five autonomy.” This certainly isn’t what it seems like when you walk around the automotive hall at CES, not to mention the host of self-driving demos we’ve seen recently.
Gill Pratt: The most significant thing to understand is that not all miles are the same. Most miles that we drive are very effortless, and we can drive them while daydreaming or thinking about something else or having a conversation. But some miles are truly, truly hard, and so it’s those difficult miles that we should be looking at: How often do those demonstrate up, and can you ensure on a given route that the car will actually be able to treat the entire route without any problem at all? Level five autonomy says all miles will be treated by the car in an autonomous mode without any need for human intervention at all, ever.
So if we’re talking to a company that says, “We can do utter autonomy in this pre-mapped area and we’ve mapped almost every area,” that’s not Level Five.
That’s Level Four. And I wouldn’t even stop there: I would ask, “Is that at all times of the day, is it in all weather, is it in all traffic?” And then what you’ll usually find is a little bit of hedging on that too. The trouble with this Level four thing, or the “full autonomy” phrase, is that it covers a very broad spectrum of possible competencies. It covers “my car can run fully autonomously in a dedicated lane that has no other traffic,” which isn’t very different from a train on a set of rails, to “I can drive in Rome in the middle of the worst traffic they ever have there, while it’s raining,” which is fairly hard.
Because the “full autonomy” phrase can mean such a broad range of things, you indeed have to ask the question, “What do you truly mean, what are the actual circumstances?” And usually you’ll find that it’s geofenced for area, it may be restricted by how much traffic it can treat, for the weather, the time of day, things like that. So that’s the elaboration of why we’re not even close.
BACK TO TOP↑
The Over-Trusting Autonomy Paradox
And all this creates an expectation problem for consumers because they’re hearing all of this stuff and they don’t know what it means.
Right. And that’s particularly true because, as consumers, we have a tendency to attempt to build a model in our brains to predict how good the autonomy is going to be. That model tends to be an anthropomorphic one: We concentrate on certain things, and we don’t see when the autonomy doesn’t treat certain other things, and we tend to either over-trust or under-trust how good the autonomy is.
[Toyota’s] view is that consumer education is absolutely significant to truly understand the limitations of the technology. It’s very significant that the industry as a entire strive to put themselves in the footwear of the customer and say, “How do we make sure they truly understand how good this is, what it can do, but most importantly, what it can’t do,” so people don’t over-trust their vehicles.
That over-trusting is another big problem, because once you give the car to a consumer and the consumer practices some of what it can do in a favorable environment.
It’s indeed difficult. As human beings, we tend to make predictions, that’s sort of how our brains are wired. Things like, “The weather was good today, the weather was good the day before, so therefore the weather is going to be good forever.” As we improve the technology for autonomous cars, they’re going to require human intervention less and less. And this will exacerbate the over-trust problem, because the tendency will be that people will say, “It hasn’t needed me before, it doesn’t need me now, so it will never need me.”
In some ways, the worst case is a car that will need driver intervention once every 200,000 miles. And so an ordinary person who has a car every 100,000 miles would never see it. But every once in a while, maybe once for every two cars that I own, there would be that one time where it all of a sudden goes “beep beep beep, now it’s your turn!” And the person, typically having not seen this for years and years, would fully over-trust the car and not be ready when that happened. So we worry about this paradox: The better we do, the worse the over-trust problem could become.
BACK TO TOP↑
If Self-Driving Cars Need to Be Better Than Human Drivers, How Good Is Good Enough?
Do you think utter Level five autonomy in that context is realistic, or even possible?
I think that it is possible, and our standard is some numerous of how good human beings are at the task of driving. I asked this question recently [at Toyota’s CES press conference], which I think is enormously significant, and which society has not answered yet: What standards should we use, and how good does the system need to be?
It will be many years before autonomous cars are ideal, and so some crashes are likely to happen. Are we as a society ready to accept ten percent better than human driving? One percent better than human driving? Ten times better than human driving? I don’t know, and to be fair, it’s not our role as technologists to tell society what the reaction is. It’s the government and everybody else’s role who will be affected by this to weigh in and say, “Well, maybe it’s good enough to save one life.” Or should we actually say, “We’re not going to use it until it’s ten times better.” I’m not sure, but I think until we have an reaction to that question, we have to be very careful to not introduce technology that completes up out of line with what society’s expectations are.
In the context of autonomous cars and safety, I’ve attempted to make the point at times—and others have as well—that if autonomous cars are even just one percent safer than humans, we should still be using them.
From a rational point of view, you’re exactly right, but it’s an emotional thing, right? People aren’t rational: If you look at plane crashes as an analogy, planes are without a doubt, by far, the safest way to travel, per mile. You should never worry that you’re going to have a crash in a plane. But plane crashes do happen, and when they happen, we have this sort of exaggerated response, where we become worried and afraid that our plane is going to crash. Rationally, it makes no sense at all: it’s much more likely that we’ll have a crash in a car than we will have in a plane, and yet it’s plane crashes that are in the news all the time, so we end up fearing the wrong things.
For an accident in a human-driven car, it’s a situation where we can put ourselves in the boots of the driver of that car and say, “That could have happened to me. I could have made that mistake.” If it’s a machine, I worry that the empathy won’t be there, and that people will expect the machine to be ideal.
“AI systems are not flawless . . . Every once in a while, as we progress this technology, there will be cases where the perception system will get it wrong, and that may lead to an accident. What are we going to say when that happens?”
And yet, as we know, AI systems, especially ones that are based on machine learning, are not ideal. The space of possible inputs to the sensors is so large, the car will see things it’s never been trained on before, and we’re expecting it to come out with a reasonable perception of the world based on what it sees. Every once in a while, as we progress this technology, there will be cases where the perception system will get it wrong, and that may lead to an accident. And what are we going to say when that happens? Who are we going to blame? We don’t know the reaction, but we know it’s a very significant question.
BACK TO TOP↑
Machine Learning and “The Car Can Explain” Project
James Kuffner [who helped develop the Google self-driving car and is now CTO at TRI] was talking about this on a CES robotics panel in the context of cloud robotics. The fact that autonomous cars are going to get into accidents is a certainty, but is it the case that every time one gets into an accident, that the manufacturer will be able to tell what caused it, and then update the software in all of their other cars so that we won’t have that kind of accident again?
I think it’s very likely, in fact, I would be astonished if it’s not the case, that we will have very precise logs of what happens in an autonomous car accident. Now, you asked a very interesting question: Will we be able to figure out why?
What’s interesting about this—because machine learning systems, and deep learning in particular, are very high spectacle but don’t actually explain how they figure out the answer—is that getting the car to explain what it does is actually hard. We have research that we’re doing at MIT, and other places, where we’re attempting to make progress on that issue: the one that I can point you to is Professor Gerald Sussman at MIT, and the project that he’s working on with our funding is called “The Car Can Explain.”
“Globally, cars travel around ten trillion miles. So if you’re testing millions of miles, which car manufacturers are doing on physical cars, that doesn’t get you anywhere near close coverage of possible things that could happen to all cars . . . You have to boost it in some way, and doing accelerated testing simulation is a key part to that.
So the logs will be there, but what’s responsible for the mistake? That’s a much stiffer question. Will we do something to make sure that that mistake doesn’t happen again? Of course we will, but the dimensionality of the input space immense. And so telling, “Okay, I’ve plugged this leak in the dam, I’ve plugged that leak in the dam…” It’s a truly big dam, and there are lot of places that things could go wrong. This actually brings us to the thickest part of our work: It turns out that testing is the number one thing that we need to do, and this is that entire trillion mile challenge that I talked about last year. Globally, cars travel around—within an order of magnitude—10 trillion miles. So if you’re testing millions of miles, which car manufacturers, including us, are doing on physical cars, that doesn’t get you anywhere near close coverage of possible things that could happen to all cars in the world in the course of a year. You have to boost it in some way, and doing accelerated testing simulation is a key part to that. We’re not going to simulate the “Sunday drive,” when the weather’s nice and everything’s fine and there’s hardly any cars on the road. We’re going to simulate when things are terrible.
Rod Brooks has this wonderful phrase: “Simulations are fated to succeed.” We’re very well aware of that trap, and so we do a lot of work with our physical testing to validate that the simulator will be good. But inevitably, we’re pushing the simulator into regimes of operation that we don’t do with cars very often, which is where there are crashes, where there are near misses, where all kinds of high-speed things happen. For example, an autonomous car has to be ready for the bad actor, the sort of byzantine case where there’s another car, perhaps a human-driven car, that’s driven by a person with road rage, and your car has to do the right thing, even however the other car was bad. And so, what’s the response there? We’re not going to test that very often in the physical world, because half the time, they’re going to crash, so we’ll test it once in a while, but we need to boost it with simulation and that’s fairly hard to do.
And that eventually brings up yet another area, which is, can formal methods help? There’s this tremendous promise of formal methods in programming, to say that we can prove correct operation within this regime of the state space. Presently, it’s very hard to do, so we’re looking at hybrid approaches of simulation and formal methods, blended together to attempt to get at the dimensionality of the problem. But fundamentally a thing your readers should know is that this indeed comes down to testing. Deep learning is wonderful, but deep learning doesn’t assure that over the entire space of possible inputs, the behavior will be correct. Ensuring that that’s true is very, very hard to do.
BACK TO TOP↑
Simulating the Crazy Things Human Drivers Do
Especially right now, with autonomous cars or any car with any aspect of autonomy, I would imagine that humans are the least predictable thing on the road. Going through on-road testing, a lot of weird human-related stuff has to come up, but how do you even attempt to simulate all of the crazy things humans might do?
We attempt to model human beings in the same way we model weather or we model traffic. It’s hard to model human beings, because we have ten to the ten neurons in our brains: Every person’s different, and there’s a entire broad range of behaviors, but we know that to some extent it’s possible. As a human driver, we have theory of mind, of how other human drivers act. We run a little simulation in our head, like when we’re at a four-way stop, telling if I were that person, I’d act like this, so I’m going to do this. Theory of mind is this amazing thing that means that making a simulation is possible, because we can build statistical models to predict what other human beings are going to do.
Sometimes, tho’, there’s a difference inbetween how a human being is likely to act, and what the safe (or legal) way of acting is. How do you instruct a car what to do in circumstances like that?
It’s an interesting question. What speed should you travel on the highway? If the speed limit is posted at fifty five miles an hour, should you go 55, or should you go the median speed of what the other cars are doing? What’s the safest thing to do? I don’t want to give an official reaction, but it’s a conundrum, and we have discussed this with the legal department at Toyota. They scrape their goes and say, “That’s indeed harsh.” So what do you do? And who’s responsible, right? It may be the case that driving slower isn’t the safest. How do we resolve that issue? It’s very hard to figure out.
BACK TO TOP↑
“We Actually Need a Revolution in Computer Hardware Design”
Right after the press conference, you mentioned something about how cooling and powering the computers in electrical cars is actually a big problem. We’re very focused on the challenges involving decision making, but what other things do we still have to figure out?
What I love about this stuff is that I have this entire side of me that’s the hardware man, but I also did a lot of my own research in the past in neurophysiology. This business of computational efficiency is indeed significant. Our brains use around fifty watts of power, while the autonomy systems for most [autonomous and semi-autonomous cars] in the field, not all of them, but most of them, use thousands of watts of power, and that’s a tremendous difference. And it’s not even that our brain is using all fifty watts to do the driving task: we think about all kinds of other things at the same time, and so perhaps ten watts is going into the driving task.
“The idea that only the chauffeur mode of autonomy, where the car drives for you, that that’s the only way to make the car safer and to save lives, that’s just false . . . There are tremendous numbers of ways to support a human driver.”
We also don’t know that the amount of computational power that’s in current prototype autonomous cars is the adequate level. It may be that if we scaled the computation by a factor of Ten, that the car would perhaps not get ten times better, but would get significantly better than what it does now. Or what would happen if we had one hundred times more computational power? It may be that it actually doesn’t plateau, but resumes to get better. I suspect the reaction is that it does keep on growing, albeit I don’t know what the curve is. And so in addition to all the software stuff and all the testing we need to do, we actually need a revolution in computer hardware design: We need to somehow get more towards the power efficiency that the brain has.
To response your question of other things [that remain to be solved], I think sensors still have a long way to go. Lidar is an amazing thing, but it’s still not that excellent. The point density is still low, and for looking at cars in the distance, it doesn’t compare with human vision. Particularly, for example, with Level three autonomy, the amount of time you need in order to wake up the driver to say, “Hey, you’re going to need to deal with this thing that’s just happening in one thousand five hundred feet in front of you,” that’s not going to happen unless your sensor can see and understand what’s happening that far away.
The sensor also needs to be cheap. It needs to be stimulation resistant. It needs to last for ten years or maybe twenty years and not break. Automotive quality, which most people think of as being low, is actually very, very high. I’ve worked a lot, in my DARPA work, with mil-spec and things like that, but automotive quality is ridiculously hard to do. And so a cellphone camera is not going to work—you’re going to need special stuff that indeed can withstand the desert and Alaska and going back and forward. How do you deal with salt and rust, with the screen getting utter of dust from the road… Making sure that’s going to work, all of the time, that’s truly hard.
So to sum this thing up, I think there’s a general desire from the technical people in this field to have both the press and particularly the public better educated about what’s indeed going on. It’s very effortless to get misunderstandings based on words like or phrases like “full autonomy.” What does total actually mean? This actually matters a lot: The idea that only the chauffeur mode of autonomy, where the car drives for you, that that’s the only way to make the car safer and to save lives, that’s just false. And it’s significant to not say, “We want to save lives therefore we have to have driverless cars.” In particular, there are tremendous numbers of ways to support a human driver and to give them a kind of blunder prevention device which sits there, inactive most of the time, and every once in a while, will very first warn and then, if necessary, intervene and take control. The system doesn’t need to be competent at everything all of the time. It needs to only treat the worst cases.
BACK TO TOP↑
Human and Machine: Who Should Guard Whom?
And often the worst cases for human drivers are things that robots are particularly good at right now, like paying attention all the time to the car in front of you, or watching where the lines are, right?
Correct. People and machines have complementary abilities: machines are very good at staying vigilant. People are very good at treating difficult driving situations with theory of mind and all that kind of stuff, and machines are particularly bad at that. So how can we put the two together in a way that they complement each other, rather than sort of fight each other? Another way of thinking about this is who guards whom, right? Should the human being guard the autonomy? Should the autonomy guard the human being? It’s a higher threshold to make this system where the human being has to guard the machine and eventually doesn’t have to guard anything at all to get the AI to work, so that’s why at Toyota we’re taking a dual treatment to things.
Most companies at CES are here to sell. At Toyota, we don’t have any trouble selling cars, so I took the chance to use this to attempt to educate. To say, “Look, this is all excellent, but let’s make sure we’re all talking about the same thing.” If we could define the capabilities, take the SAE scale [of driving automation] to heart. But even within the SAE scale, there’s so much confusion.
Yes, so NHTSA had one to Four, and SAE was very wise, they said we’ll take Level four and we’ll split it into two things, four and Five, and the difference is that Level five is everywhere at any time, and four is only some places at some times. But otherwise they’re basically the same. The SAE levels are fine, but people keep making mistakes. And so Level two systems are often called Level Trio, which is wrong. A key thing of Level three is that the person does not need to supervise the autonomy. So no guarding of the machine by the human being.
And so when the driver can’t take over, what’s the Plan B? I reminisce when Freightliner introduced its autonomous truck, its Plan B was that the truck would just stop in the middle of the highway because it doesn’t know what else to do.
Buried inwards of SAE stuff, it does talk about some Plan B things where the car will pull off the road if it has to stop. I love that, if only it could be true all the time. I grew up in Fresh Jersey. There are a lot of highways in Fresh Jersey where there’s no place to pull over, and so what are you going do? What if you’re in the tunnel and there’s no side lane, no pull-off lane? It has to treat every case, and that’s what so hard about this thing.
There have been proposals for things like a remote driver, where if the car gets into difficulty, it’ll spin over to a remote driver in a call center somewhere. Very first of all, those things assume the Internet is working, there are no hackers, there’s no natural disasters. But most of all, it’s the same issue we discussed earlier, that you assume a person at a call center can abruptly wake up and say, “Okay, trouble,” and treat the situation. People are very bad at that. We’re very bad at abruptly being astonished and needing to orient ourselves and coming up with a correct response. So we’re not telling that that’s unlikely, but we’re telling it may be very difficult, for the reasons that we’ve outlined.
No comments