External Control Layer

A test vehicle detects a pedestrian and is not allowed to brake hard

Uber’s automated driving system detected a pedestrian 5.6 seconds before impact, but its design relied on the operator, who was distracted, to brake in an emergency. The vehicle’s factory emergency braking was switched off whenever the system was driving.

Date
March 2018
Kind of system
Perception and control
Outcome
Harm

Gaps found

G1G2G3G4G5G6G7G8G9G10G11G12G13

G7 Review in name only G9 Warning with no authority G11 Safeguard off

What happened

On 18 March 2018 in Tempe, Arizona, the automated driving system of an Uber test vehicle first detected a pedestrian 5.6 seconds before impact and repeatedly reclassified her as a vehicle, a bicycle, and an unknown object. It did not predict that she would enter its path until 1.2 seconds before impact. It then held off braking for one second, a delay Uber built in to avoid reacting to false alarms, and by design did not brake hard. It sounded an alert and began slowing 0.2 seconds before the crash.

Volvo’s factory collision warning and automatic emergency braking were switched off whenever Uber’s system was driving. In Volvo’s simulation, they would have avoided the collision in 17 of 20 variations. While the SUV was moving on public roads, the operator spent 34% of her time looking down toward the bottom of the center console. The NTSB found that the probable cause was her failure to monitor the road because she was distracted by her phone, and that Uber had not managed the risk of automation complacency.

What allowed the action to take effect

  • G7 Review in name only

    A person was supposed to catch the error but lacked the information, time, training, or practical means to do it, so the review did not work as a check.

    Safety depended on one operator watching the road. The NTSB found Uber “did not adequately recognize the risk of automation complacency and develop effective countermeasures,” and that removing the second operator increased the task demands on the one who remained.

    Kind of evidence: finding. Source: National Transportation Safety Board, 19 November 2019, Findings 11, 12.

    Related guidance: P5 Human authority. Contract clause C-5.

  • G9 Warning with no authority

    Signs that the action was wrong existed before or during the harm, but none of them could stop or hold the action, and nothing required anyone to act on them.

    The system tracked the pedestrian for 5.6 seconds. The NTSB found: “The system design precluded activation of emergency braking for collision mitigation, relying instead on the operator’s intervention to avoid a collision or mitigate an impact.”

    Kind of evidence: finding. Source: National Transportation Safety Board, 19 November 2019, Executive summary, p. v; Finding 5.

    Related guidance: P3 Fail-closed and P8 Sequence awareness. Contract clauses C-3 and C-8.

  • G11 Safeguard off

    A check that could have caught the action existed but was switched off, bypassed, set to approve automatically, or not working when the action happened.

    “When the SUV was operated in autonomous mode (controlled by the ATG ADS), all the Volvo ADAS components were automatically deactivated.” The NTSB found this “removed a layer of safety redundancy.”

    Kind of evidence: finding. Source: National Transportation Safety Board, 19 November 2019, § 1.6.3; Finding 7.

    Related guidance: P3 Fail-closed. Contract clause C-3.

Considered and left out

G12 Unreviewed change
The NTSB criticized Uber’s safety risk assessment and its move to a single operator, but the parts of the report that were reviewed do not show that the change went into service without review.

Entries with gaps in common

Sources

Cite this entry

External Control Layer. (2026). Entry 5: A test vehicle detects a pedestrian and is not allowed to brake hard. In Control Gap Record (Version 0.3). G-14, Inc. https://externalcontrollayer.ai/record-5

@misc{ecl_cgr_5,
  author = {{External Control Layer}},
  title = {Entry 5: A test vehicle detects a pedestrian and is not allowed to brake hard},
  howpublished = {Control Gap Record, version 0.3},
  publisher = {G-14, Inc.},
  year = {2026},
  url = {https://externalcontrollayer.ai/record-5},
  note = {Released 2 November 2026}
}

Uses Control Gap Classification 0.2.

Correct this entry

Email ecl@g14.ai with “ECL record 5” in the subject line.

Write the email