Classification/Practice

Find the gaps in an incident

Read what happened in an incident from the record and decide which of the 13 gaps the facts support, then compare your answer with the record’s, gap by gap, with the evidence for each. One incident takes about 10 minutes.

Your answers stay in this browser and are not sent anywhere.

Incident

The incident

The practice tool needs JavaScript. Each entry’s facts, gaps, and evidence are on its page in the record.

Your answer

Tick each gap the facts support. If the facts say nothing about a control, leave it unticked. The rules for each gap are under it.

A. Where the limit lived

G1 Instruction-only limitIf the AI ignored every instruction it had been given, what would stop this action?

Rules for G1

What it means: The rule that should have stopped the action existed only as an instruction, prompt, or trained behavior of the AI system, so it held only as long as the system followed it.
Count it when: Sources show that a user or the organization relied on telling the system what not to do, or on behavior its maker trained into it, and the system acted anyway.
Do not count it when: A check outside the system existed and failed. Count what made it fail instead, such as G11. If no rule existed at all, count the gap that left the action uncontrolled, such as G3.

G2 Direct accessWhich credentials, sessions, or connections does the AI hold that reach records, money, messages, or equipment without passing a check it does not control?

Rules for G2

What it means: The system held the credentials or connection to carry out the action itself, so nothing outside it stood between its decision and the effect.
Count it when: The system or its tools could write, delete, pay, order, or send on their own authority, using keys, sessions, or connections they held.
Do not count it when: The output was words sent to people; count G3 instead. Access that passed through a check able to refuse the action is not direct access.

G3 Unchecked statementWhat does this system say that a customer, a court, or the public could hold you to, and what checks it before it is sent?

Rules for G3

What it means: Words the system produced reached customers or the public in the organization’s name, on matters people rely on such as prices, policy, law, or health, and nothing checked them first.
Count it when: A chatbot, assistant, or automated account made a statement, offer, or piece of advice that the organization had not approved, and no check compared it with approved text or policy before it was sent.
Do not count it when: A person approved the statement without the means to catch the error; count G7 instead. If the statement passed an automated screen that missed it, it was checked, and this gap does not apply.

G4 Boundary not enforcedWhat stops this system from acting on data, systems, or conditions outside the ones it was approved for?

Rules for G4

What it means: The action reached systems, data, or operating conditions outside those the system was built or approved for, and nothing enforced that boundary when it acted.
Count it when: A documented boundary existed, such as a design domain, a project folder, a development environment, or a freeze window, and the sources show that this action crossed it. A finding that the boundary was unenforced in general is not enough.
Do not count it when: The boundary appears only as guidance to the people who deploy the system, such as a model card’s advice against high-risk uses, and not as a limit on the running system.

B. What the check could see

G5 Blind to the totalWhat is the most this system could do in an hour if each action passed its checks?

Rules for G5

What it means: Each action was allowed one at a time. Nothing tracked what the actions added up to in volume, rate, exposure, or repetition until the total did the harm.
Count it when: The harm came from many actions or repeated activations, and sources show no limit on the total, or a limit that was not connected to the action. Harm to many people from many separate actions also counts as a total when the sources show that nobody tracked it.
Do not count it when: A single action did the harm on its own, and the sources show no untracked volume or repetition.

G6 No independent evidenceWhen this system’s output is approved, what evidence does the approver have that did not come from the system?

Rules for G6

What it means: The action, or the approval of it, was based on the system’s own output or on a single unverified input. Evidence that would have shown the error was never gathered, was thrown away, or was available and not consulted.
Count it when: A reviewer checked the AI’s output by asking the AI, the source material was erased, a model’s match or estimate was treated as proof, or a second input that could have caught the error was not used.
Do not count it when: The only fact is a perception or prediction error, with no sign that independent evidence was available. That error is recorded as the trigger (what started the incident), and this gap does not apply. An account given after the action is G13.

C. Who could decide and stop

G7 Review in name onlyFor each approval step, what does the approver see, how long do they have, and how often do they say no?

Rules for G7

What it means: A person was supposed to catch the error but lacked the information, time, training, or practical means to do it, so the review did not work as a check.
Count it when: Sources describe approvals given without the evidence needed, supervisors who were not told how the system behaves, or supervision that relied on people paying attention when nothing in the design made sure they would.
Do not count it when: No person was involved at all; count G8 instead.

G8 No human decisionWhich of this system’s actions are supposed to need a person, and what makes sure they get one?

Rules for G8

What it means: A decision that needed a person went through without one. The need can come from law, from policy, from the product’s own design, or from the user’s explicit instruction.
Count it when: Determinations with legal or financial effect were fully automated, or an agent acted where its user or its maker required confirmation first.
Do not count it when: A person did decide, but without the means to decide well; count G7 instead. A mode or flag had switched the confirmation off; count G11 instead.

G9 Warning with no authorityWhen this system’s own monitoring detects trouble, what holds the next action automatically?

Rules for G9

What it means: Signs that the action was wrong existed before or during the harm, but none of them could stop or hold the action, and nothing required anyone to act on them.
Count it when: Alerts, error messages, detections, complaints, legal advice, or published findings reached the organization or the system before the harm or while it continued, and the action went on.
Do not count it when: The warning came only after the harm was complete, or it was general guidance, such as a model card or a recommendation to an industry, rather than a sign about this system’s actions.

G10 No independent stopIf this system stopped responding to you, how would you stop it, and how long would that take?

Rules for G10

What it means: Nothing could stop the system in time without relying on the system itself. In the agent incidents with this gap, the only way to stop the action went through the agent.
Count it when: People tried to stop the system through the system and failed, had to reach the machine in person, or had no procedure to halt it while the harm went on.
Do not count it when: Nobody needed to stop the system before the harm was done.

D. How the safeguards and the record were kept

G11 Safeguard offWhich of this system’s safeguards can be switched off, set to approve automatically, or stop working without anyone noticing?

Rules for G11

What it means: A check that could have caught the action existed but was switched off, bypassed, set to approve automatically, or not working when the action happened.
Count it when: A safety function was disabled to reduce false alarms or friction, an auto-run mode was on, a confirmation was skipped with a flag, or an alert was broken.
Do not count it when: The check never existed. Count the gap its absence left.

G12 Unreviewed changeWho reviews a change to this system’s prompts, tools, models, and dependencies before it goes live, and where is that review recorded?

Rules for G12

What it means: The change that introduced the harmful behavior went live without independent review. The change can be code, a prompt, a model, a package, a configuration, or a procedure.
Count it when: Sources show a deployment without a second check, a modification that went around code review, an update adopted without review, or a design change whose safety analysis was not updated.
Do not count it when: The change was reviewed and approved.

G13 Self-reported recordIf this system’s own logs and reports were wrong, what independent record would show what it did?

Rules for G13

What it means: After the action, the only account of what the system did, or of what could be undone, came from the system itself or from the party that ran it, and that account was wrong or incomplete.
Count it when: An agent misreported what it had done or whether it could be reversed, and the user relied on it; or an operator’s report was the record a regulator or court relied on, and it left out what happened.
Do not count it when: The account was checked against an independent record before anyone relied on it, or nothing shows it was wrong.

No gaps are ticked. Each incident in the record has at least one.

How your answer compares

Read the full entry Correct this entry

Reviewing entries for version 1.0

How the review works

How the comparison works

For each incident, the result first gives how many of the record’s gaps you found and how many you marked that the record does not. The match across all 13 gaps is given second, because it also counts the gaps that both left out, and most gaps do not apply to most incidents.
The record was marked from the full sources, which are listed with each incident. Some gaps depend on details found only in those sources, so a difference is not always a mistake.
From five incidents on, the tool also shows Cohen’s kappa, the usual measure of agreement between two people after allowing for the agreement expected by chance, and names its band on the scale of Landis and Koch (1977), from slight to almost perfect. With fewer incidents the figure varies too much to be meaningful.
If the record still seems wrong after reading the evidence, send a correction from the entry’s page. Corrections and the decisions on them are published on the record page.