External Control Layer

Check a system

Answer 13 questions about one AI system that can take actions. As you answer, the page shows the system’s gaps, the incidents in the record that had the same gaps, and the guidance that covers them, and at the end it produces a short report for whoever runs the system.

Your answers stay in this browser and are not sent anywhere.

Choose one system and one kind of action it can take, such as issuing refunds, merging code, or answering customers about fees. Controls differ by action, so a system that is well controlled for one action can have gaps for another. Answer “Yes” only if you have seen the control work, and “Not sure” when you would have to ask. A “Not sure” is treated as a gap until someone checks.

Fills in answers for a fictional customer support agent to show a finished report. Clear the answers to start your own.

A. Where the limit lived

G1 The limits on this action are enforced outside the AI. If the AI ignored its instructions, something else would still refuse the action.

If the AI ignored every instruction it had been given, what would stop this action?

G2 The AI holds no credential, session, or connection that reaches the target system directly. Its actions pass through a check it does not control.

Which credentials, sessions, or connections does the AI hold that reach records, money, messages, or equipment without passing a check it does not control?

G3 Whatever the AI tells customers or the public about prices, policy, law, health, or eligibility is checked against approved text before it is sent.

What does this system say that a customer, a court, or the public could hold you to, and what checks it before it is sent?

G4 The systems, data, and conditions the AI may act on are written down, and actions outside them are refused automatically.

What stops this system from acting on data, systems, or conditions outside the ones it was approved for?

B. What the check could see

G5 There are limits on what the AI’s actions can add up to, such as volume, rate, or total value, and crossing one stops or holds the next action.

What is the most this system could do in an hour if each action passed its checks?

G6 Approvals of the AI’s actions are based on evidence the AI did not produce, and that evidence is kept.

When this system’s output is approved, what evidence does the approver have that did not come from the system?

C. Who could decide and stop

G7 People who approve the AI’s actions see the evidence they need, have time to check it, and sometimes say no.

For each approval step, what does the approver see, how long do they have, and how often do they say no?

G8 Every action that needs a person’s decision, by law, policy, or design, is held until a person decides.

Which of this system’s actions are supposed to need a person, and what makes sure they get one?

G9 When monitoring detects a problem, the next action is held automatically. No warning depends on someone happening to read it.

When this system’s own monitoring detects trouble, what holds the next action automatically?

G10 Someone can stop the AI, or revoke its access, without its cooperation, and you know how long that takes.

If this system stopped responding to you, how would you stop it, and how long would that take?

D. How the safeguards and the record were kept

G11 The system’s safeguards cannot be switched off or set to approve automatically without someone independent knowing, and if a safeguard stops working, it blocks actions instead of letting them through.

Which of this system’s safeguards can be switched off, set to approve automatically, or stop working without anyone noticing?

G12 Changes to the AI’s prompts, tools, models, and dependencies are reviewed by someone else before they go live, and the review is recorded.

Who reviews a change to this system’s prompts, tools, models, and dependencies before it goes live, and where is that review recorded?

G13 There is a record of what the AI did that the AI cannot write or edit, and it is what you would rely on after an incident.

If this system’s own logs and reports were wrong, what independent record would show what it did?

No answers yetReport

Your report

Answer the questions above to see which gaps this system has.

The report downloads as a plain-text file (Markdown) with a summary, a table of the gaps with the question to ask about each, the controls to put in first, and the incidents in the record with the same gaps.

Using the report

  • Take each gap to whoever runs the system, with the question under it. If a vendor runs the system, the matching contract clause puts that question in writing.
  • Priorities depend on the system. In the record, direct access (G2) and instruction-only limits (G1) recur in agent incidents and unchecked statements (G3) in incidents involving a conversational assistant, and review in name only (G7) appears in most incidents in which people approved AI output.
  • The implementation guide describes one way to add an outside check to a single workflow, and the specifications page shows which published specifications address each gap.