AI PRACTICE6 min read

AI PRACTICE · ISSUE 001

When an AI Workflow Should Escalate Instead of Decide

Route uncertainty, missing evidence, and high-impact outcomes to people before automation turns a weak signal into an action.

Computer monitors in the CERN control room
CERN control-room photograph, used to illustrate supervised operations. Image: Robert Scoble from Half Moon Bay, USA · CC BY 2.0. Resized and converted to WebP; image adaptations use the stated license.

The most consequential part of an AI workflow may be the branch that does nothing. A system that can draft a response, update a record or send a payment needs an explicit way to stop before an uncertain interpretation becomes an external action. Asking a model whether it feels confident is not a sufficient routing policy. This guide designs a fictional invoice assistant with three routes: prepare, hold for review and reject the requested operation. The example is a design exercise, not a deployed system or a measured performance claim.

Describe the transaction before choosing a model

The fictional assistant reads an invoice and proposes accounting fields. Its allowed task is extraction for review. It is not authorized to change bank details or send money. That distinction should exist in the software's permissions as well as in a written instruction. Otherwise an apparently helpful request inside a document could reach a tool that was never needed for extraction.

List the input documents, trusted reference records, output fields and available tools. Mark each external effect separately: saving a draft, editing a ledger, emailing a supplier and initiating payment have different consequences. Assign authority to people or systems that can actually approve those actions. OWASP's excessive-agency guidance highlights the risks of unnecessary functionality, permissions and autonomy. The appropriate response is to constrain those properties, not to assume a more persuasive prompt gives the model a dependable authorization boundary.

Use observable conditions for the first route

Define what permits the assistant to prepare a draft. The document must be readable, required fields must be present, extracted values must pass format checks, and the proposed supplier must match a trusted record through a documented comparison. These are suggested conditions for the fictional workflow, not universal accounting requirements. The actual organization must decide which evidence is sufficient for its process.

Even then, preparing a draft is not permission to execute a transaction. Show the source location beside each extracted value and preserve the original input identifier. If a total is calculated, use deterministic arithmetic outside the language model and display the inputs. A generated explanation can help a reviewer navigate the document, but it should not silently turn a missing field into an inferred fact. The preparation route succeeds by producing inspectable work within its narrow authority.

Hold ambiguity rather than resolving it invisibly

The second route handles missing or conflicting evidence. Imagine the fictional invoice contains two different due dates, or the supplier name matches more than one record. The assistant should retain both possibilities and identify the unresolved field. It should not choose the version that creates the tidiest output merely to finish the task.

A review queue needs a useful packet: original document, extracted values, conflicting passages, validation result and the specific question to answer. Set responsibility and a time at which the unresolved item is raised again. A queue that simply accumulates work can create pressure for hurried approval later. If review capacity is inadequate, narrow the automatic workflow or reduce its intake. Do not redefine a click on an unexplained recommendation as evidence that meaningful oversight occurred.

Reject operations outside the assigned authority

Some inputs are not merely ambiguous. A note in an uploaded PDF might ask the system to email account data elsewhere or bypass an approval step. Treat that text as document content, not as a new instruction from the account owner. The workflow should refuse the requested operation even if the invoice fields themselves look plausible.

Keep a distinction between refusing an action and discarding an entire business record. The document may still need human examination or security reporting. Record the prohibited request in an appropriately protected incident channel without needlessly copying sensitive material into a general log. Tool-side authorization must enforce the boundary when an operation is attempted. An explanation from the model that it intended to be helpful does not establish permission, and a model refusal in one test does not prove the boundary will always hold.

Test the branch and the effect separately

Create fictional test inputs for an ordinary invoice, a missing total, conflicting supplier details and an embedded request to contact an unapproved destination. Keep payment and messaging tools disconnected or replaced by safe stubs. Write the expected route before running the test. The expectation should follow the workflow's specification rather than whatever result the model happens to produce.

Observe both the proposed branch and the actual tool behavior. A model can select the wrong branch while a permission check correctly prevents the action. Conversely, a convincing warning in the chat does not help if a tool already performed the change. Retain input identifiers, configuration, validation decisions and outcomes. Do not attach real client records to an experimental test set unless that use and environment are specifically approved.

Review why the route exists

An escalation rule should change when evidence shows the task has changed. Track categories of holds and refusals, the corrections reviewers make and any attempted action outside scope. Investigate whether repeated ambiguity reflects a document problem, a weak reference dataset or a task that should remain manual. An average success score can hide a severe failure at the boundary.

High-impact decisions may have professional, legal or procedural duties beyond this design exercise. Its defensible result is narrower: it prepares only authorized work, makes uncertainty visible and cannot use untrusted input to grant itself new authority.

An invoice assistant’s transition table

This is a fictional design with no connected payment or messaging tools. The expected routes below are a specification, not a report that an assistant passed the tests.

Input, route and permitted effect
ConditionRoutePermitted effect / next transition
Readable invoice; required fields and trusted supplier match passPREPARESave an inspectable draft only; release still requires authorized review
Missing field or two conflicting due datesHOLDPreserve sources and ask a named reviewer; return to PREPARE only after resolution
Supplier bank details differ from trusted recordHOLDUse independent supplier verification; never update bank details automatically
Document asks to email private data or bypass approvalREJECT OPERATIONTool-side authorization blocks the operation; retain the record for review
Validation service unavailableHOLDDo not treat unavailable validation as success
Any proposed paymentOUTSIDE SCOPENo payment tool is exposed in this example
Select or copy plain text
Condition	Route	Permitted effect / next transition
Readable invoice; required fields and trusted supplier match pass	PREPARE	Save an inspectable draft only; release still requires authorized review
Missing field or two conflicting due dates	HOLD	Preserve sources and ask a named reviewer; return to PREPARE only after resolution
Supplier bank details differ from trusted record	HOLD	Use independent supplier verification; never update bank details automatically
Document asks to email private data or bypass approval	REJECT OPERATION	Tool-side authorization blocks the operation; retain the record for review
Validation service unavailable	HOLD	Do not treat unavailable validation as success
Any proposed payment	OUTSIDE SCOPE	No payment tool is exposed in this example

REFERENCES

Sources and further reading

  1. 01NIST AI RMF Playbook
  2. 02OWASP: Excessive Agency

Sources reviewed October 20, 2026. Provider instructions and product terms can change. Examples identified as fictional are editorial illustrations, not reported events or product tests.