What went wrong is that a test vehicle killed Elaine Herzberg ? —?not a fully autonomous vehicle. The vehicle involved wasn’t supposed to have perfect autonomy technology at all. Rather, it had unproven systems still under development, and a safety driver who was supposed to ensure that failures in the technology did no harm. Whether the autonomy failed to detect a mid-block pedestrian crossing at night isn’t the real issue. The real issue is why the safety driver didn’t avoid the mishap despite any potential technology flaw.

The expedient approach of blaming the safety driver (or the victim) won’t make test vehicles safer. We’ve known for decades that putting a single human in charge of supervising a self-driving car with no way to ensure attentiveness is asking for trouble. And we know that pedestrians don’t always obey traffic rules. So to really be sure these vehicles are safe on public roads, we need to dig deeper…

Philip Koopman is an associate professor at the Carnegie Mellon University. Prof. Koopman has been helping government, commercial, and academic self-driving developers improve safety for 20 years.
More>>