Category: Dangerous Activities

Potential Source of Harm: Autonomous Vehicles

Updated December 16, 2023

 

Nature of Harm

There is strong reason to believe that the benefits of autonomous vehicles will ultimately substantially outweigh the harms, including because of improvements in road safety. However, given the very large importance of automobiles to contemporary society and the dangers associated with driving, the potential for harm deserves significant attention (as it has received since automobiles were first invented).

 

In these early days of progress with autonomous vehicles, actual and perceived risks have been a significant barrier to deployment. These risks relate generally to the current inability of self-driving algorithms to properly recognize the many risks present in the driving environment, and in particular the complications of sharing of the road between autonomous and non-autonomous vehicles.

 

Example of incidents involving autonomous vehicles that have attracted public attention include:

• a self-driving Uber (with a human safety driver) killing a pedestrian in Arizona in 2018, leading to suspension of Uber's autonomous vehicles program

• multiple accidents by Teslas in Autopilot mode, including a first fatality in 2016 and a 2023 recall of 2 million vehicles following a US National Highway Traffic Safety Administration (NHTSA) investigation

• incidents involving Cruise robotaxis in San Francisco, including a serious accident that led the company to suspend operations.

Many accidents involve the complex interactions of AI with humans (who tend to use AI other than as intended).

 

Regulatory and Governance Solutions

The automotive sector is highly regulated, and the continuation of strict regulation for self-driving automobiles is likely to mean that harms will be kept to a manageable level (particularly in comparison to the large number of deaths and injuries that are already experienced from traffic accidents). In general, self-driving automobiles are not permitted to operate unless licensed by a local jurisdiction.

 

Detailed regulation of the automotive sector means that non-regulatory governance initiatives may have relatively lesser importance than for other AI harms, although there are numerous important automotive standards developed by the private sector.

 

Technical Solutions

Autonomous vehicles are inherently a technical solution, and potentially one of the most important ones to result from advances in AI. The Society of Automotive Engineers (SAE) in its Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles, has defined six levels of automation:

• Level 0: No Driving Automation

• Level 1: Driver Assistance

• Level 2: Partial Driving Automation

• Level 3: Conditional Driving Automation

• Level 4: High Driving Automation

• Level 5: Full Driving Automation.

Level 3 is the highest level of automation in widespread use, through Tesla Autopilot and similar offerings from other manufacturers. There have been some limited tests of Level 4 automation, such as the now-suspended Cruise robotaxi service in San Francisco.

 

There are a wide variety of technical approaches to autonomous driving. Notable ones include:

• Tesla Autopilot, which primarily uses cameras as sensors and relies heavily on machine learning

• Mobileye, which uses both cameras and light distance and ranging (LIDAR) and uses a more rules-based system rather than relying primarily on machine learning

• Wayve, which uses cameras and radar and relies most heavily on machine learning, taking an "end-to-end deep learning" approach that involves driving systems "learn[ing] from experience" (YouTube video interview with Wayve CEO Alex Kendall).

 

Government Entities

Essentially every country has a government regulator with authority over autonomous vehicles. Regulators may operate both at the national level (e.g. US NHTSA or UK Vehicle Certification Agency) and sub-national level (e.g. California Department of Motor Vehicles and Public Utilities Commission).

 

Private Entities

The companies developing automous vehicles and self-driving systems, like those discussed above, are the key private entities for addressing potential harms from autonomous vehicles.