Driving a human-machine collaboration

Here’s the scenario: A driver falls asleep at the wheel. But their car is equipped with a dashboard camera that detects the driver’s eye condition, activating a safety system that promptly guides the vehicle to a secure halt.

That’s not just an idea on the drawing board. The system, called Guardian, is being refined at the Toyota Research Institute (TRI), where MIT Professor John Leonard is helping steer the group’s work, while on leave from MIT. At the MIT Mobility Forum on Friday, Leonard and Avinash Balachandran, head of TRI's Human-Centric Driving Research Department, presented an overview of their work.

The presenters offered thoughts on multiple levels about automation and driving. Leonard and Balachandran discussed particular TRI systems while also suggesting that — after years of publicity about the possibility of fully automated vehicles — a more realistic prospect might be the deployment of technology that aids drivers, without replacing them.

Anticipating that kind of future might make sense, “instead of thinking about an all-or-nothing world, where it’s full automation or exclusively human control,” said Leonard, the Samuel C. Collins Professor of Mechanical and Ocean Engineering in the Department of Mechanical Engineering. “Is there a sort of future where there’s more of the blend of the best of human and machine?”

Leonard walked the audience through the steps the Guardian system takes, from recognizing the lack of driver awareness, to taking over control of the vehicle, then ultimately reaching a point where — given an alert driver — the system no longer operates the vehicle itself.

“Now guardian has stepped in; it’s driving the car for you,” Leonard said, showing the system at work on screen. “And now it will offer at some point to give it back to you.” He added: “Our vision is that we can use some of the advanced techniques from automation to help human drivers more.”

The presentation, titled “Human-Centered Driving Research at Toyota Research Institute,” took place online Friday before a public audience of over 200 people. The MIT Mobility Forum is a weekly series of talks covering all aspects of transportation. It is presented by the MIT Mobility Initiative, an Institute-wide effort to spur innovation and strategic thinking around the development of sustainable, accessible, and safe modes of transportation.

In his remarks, Leonard positioned himself as both an advocate of the auto industry’s efforts to develop higher-technology vehicles, and a realist about the way those technologies will likely reach everyday life.

“I’ve been a bit of a skeptic of full automation in terms of the timelines. [It] is going to take a lot longer to have this sort of ubiquitous robotaxi fleet, whereby, you know, a teenager today would never need a drivers license or never need to have a real human Uber driver, because all cars would drive themselves autonomously,” Leonard emphasized.

More likely, he suggested, “We can expect multiple pathways and a longer time horizon than many have forecast for the rollout of that.”

That said, Leonard and Balachandran both highlighted the sophistication of current technologies. Balachandran showed the audience a demo video of a TRI system that can help prevent crashes when a vehicle is drifting dangerously and about to leave the road due to inattention, bad conditions, or other factors.

That system is meant “to amplify the human, to have better overall system performance. That includes things like safety, but also other elements that we think [make driving] pleasurable,” Balachandran said, terming this kind of human-machine cooperation “HCID,” for human-centric, intelligent driving.

In the TRI video about dealing with drifting, a car navigates a winding test track with obstacles placed on the circuit as well. If a car is about to drift off the road, the system can quickly react and control the vehicle more effectively than many drivers would at a moment of crisis.

“This gives us access to a capability that’s much more outside what a regular driver could do,” Balachandran said.

The event was hosted by Jinhua Zhao, associate professor of transportation and city planning in MIT’s Department of Urban Studies and Planning, and director of the MIT Mobility Initiative.

Leading off the question-and-answer portion of the event, Zhao inquired about the relative challenges everyday safety issue on city streets, as opposed to refining the high-end car-handling abilities found in, say, Forumula One drivers.

“When a lot of people think of uncertainty, they think of the uncertainty of the perception of the environment and the prediction of agents,” Balachandran said. “And that’s true, [it] is a very difficult problem.”

However, Balachandran added, a vehicle’s ability to perform object recognition is “a more defined kind of problem. … The dynamic uncertainty [of driving maneuvers] is actually a lot higher.” In each domain, Balachandran observed, it is “relatively easier to get to a 90 percent capability. It’s very hard to get from 90 to 100 percent.”

For his part, Leonard emphasized that the need for better traffic safety is a pressing one, given developments such as the increase in traffic fatalities per mile driven since the beginning of the Covid-19 pandemic.

“The upsurge in traffic accidents and fatalities since Covid is just really heartbreaking,” Leonard said. “I’m really disappointed … that we haven’t found a way to use this advanced technology to make cars safer. And so I think that the human piece is critical.”

And, while the vision of fully automated vehicles whizzing around on streets without any problems may be appealing, it seems quite likely that hundreds of millions of drivers will still be on the roads in the foreseeable future. An alternate vision, then, is helping those people drive as effectively as possible.

“I believe deeply, personally, that we want to use this technology to help human drivers,” Leonard said.