Self-driving cars: Hype-filled decade ends on sobering note

There are other challenges. The gains in AI that propelled self-driving vehicles have come with drawbacks. The systems are black boxes, making it hard to know why the car drives as it does. And when a self-driving vehicles makes an ill-advised decision, it’s extremely difficult to debug the vehicles.
“We’re between a rock and hard place. Machine learning is right now not explainable or certifiable,” said Raj Rajkumar, a Carnegie Mellon University professor, who like Johnson-Roberson, competed in the government-sponsored races that helped spawn the self-driving industry.
Researchers are trying to build tools to help AI explain itself, as well as build simulation technology, so the cars can be certified. There are also unanswered ethical questions over whether a car should, say, protect its passengers’ lives over those of pedestrians.