External Control Layer

Autopilot runs on a road it was not designed for

Autopilot was engaged on a highway with cross traffic, outside the conditions it was designed for, and nothing in the system prevented its use there. Its design also allowed the driver, its only supervisor, to disengage from the driving task for long periods.

Date
May 2016
Kind of system
Perception and control
Outcome
Harm

Gaps found

G1G2G3G4G5G6G7G8G9G10G11G12G13

G4 Boundary not enforced G7 Review in name only

What happened

On 7 May 2016 a Tesla Model S with Autopilot engaged struck a tractor-semitrailer crossing its path on US-27A near Williston, Florida. The system was not designed to identify a truck crossing in front of it, so the car did not slow, and neither the collision warning nor emergency braking acted. Autopilot had been active for 37 minutes of the 41-minute trip, and in that time it sensed the driver’s hands on the wheel, through steering torque, for a total of 25 seconds.

The NTSB found the probable cause was the truck driver’s failure to yield, combined with the car driver’s inattention due to overreliance on the automation. It found that the system’s design permitted “his use of the automation in ways inconsistent with guidance and warnings from the manufacturer,” and recommended that makers limit such systems to the conditions they were designed for. After another fatal crash, in Mountain View, California, in 2018, the NTSB wrote in 2020 that Tesla “does not restrict where Autopilot can be used” and reclassified its recommendation to Tesla as open, with an unacceptable response. In February 2021 the NTSB reported that Tesla had told it such limits do not apply to driver-assistance systems “because the driver determines the acceptable operating environment.”

What allowed the action to take effect

  • G4 Boundary not enforced

    The action reached systems, data, or operating conditions outside those the system was built or approved for, and nothing enforced that boundary when it acted.

    The NTSB later described Williston as a crash “that involved a driver operating outside the manufacturer’s ODD,” and found that the design allowed “his use of the automation in ways inconsistent with guidance and warnings from the manufacturer.” The system treated the highway as a “preferred roadway” on which it would have allowed speeds up to 90 mph.

    Kind of evidence: finding. Sources: National Transportation Safety Board, 1 February 2021; National Transportation Safety Board, 12 September 2017. References: NTSB comments of 1 February 2021, p. 7; HAR-17/02, probable cause and § 1.3.4.

    Related guidance: P9 Declared scope and P1 Complete mediation. Contract clauses C-1 and C-2.

  • G7 Review in name only

    A person was supposed to catch the error but lacked the information, time, training, or practical means to do it, so the review did not work as a check.

    The driver was the system’s only supervisor, and its design “permitted his prolonged disengagement from the driving task.” Over 37 minutes of Autopilot use, it sensed his hands on the wheel for a total of 25 seconds.

    Kind of evidence: finding. Source: National Transportation Safety Board, 12 September 2017, Probable cause; § 1.3.5.

    Related guidance: P5 Human authority. Contract clause C-5.

Considered and left out

G11 Safeguard off
The collision warning and emergency braking were on but were not designed to detect a vehicle crossing the car’s path.

Entries with gaps in common

Sources

Cite this entry

External Control Layer. (2026). Entry 3: Autopilot runs on a road it was not designed for. In Control Gap Record (Version 0.3). G-14, Inc. https://externalcontrollayer.ai/record-3

@misc{ecl_cgr_3,
  author = {{External Control Layer}},
  title = {Entry 3: Autopilot runs on a road it was not designed for},
  howpublished = {Control Gap Record, version 0.3},
  publisher = {G-14, Inc.},
  year = {2026},
  url = {https://externalcontrollayer.ai/record-3},
  note = {Released 2 November 2026}
}

Uses Control Gap Classification 0.2.

Correct this entry

Email ecl@g14.ai with “ECL record 3” in the subject line.

Write the email