Agent assurance: audit the loop, not just the answer
An AI agent does not just answer — it understands, observes, locates, acts and verifies. Assurance has to follow every step of that loop.
A chatbot that gives a wrong answer is a quality problem. An agent that takes a wrong action is an incident.
That is the difference that makes agents harder to assure than the AI solutions that came before them. An agent plans, calls tools, reads documents, writes to systems and moves money. Judging it only on its final reply is like auditing a payment system by reading the confirmation emails.
The agent loop
Every agent, whatever framework it is built on, runs some version of the same loop:
- Understand — interpret the request, the goal and the constraints.
- Observe — gather context: the conversation, the user's permissions, the state of the systems it can see.
- Locate — find the right data, document or tool for the next step.
- Act — call the tool, write the record, send the message.
- Verify — check that the action achieved what was intended, and recover or hand over if it did not.
Each stage fails in its own way. An agent can understand the request perfectly and still locate the wrong customer record. It can act correctly and then fail to notice that the tool returned an error. It can follow an instruction hidden inside a document it was asked to summarise.
What an Agent Assurance Audit covers
An Agent Assurance Audit examines the controls around the agent and tests its behaviour at every stage of the loop.
Controls we audit
- Who owns the agent, and who may stop it.
- Which actions are irreversible, and whether a human confirms them.
- Whether each tool has only the permissions it needs.
- Whether every step, tool call and decision is logged well enough to replay.
- Budgets for steps, loops, time and cost.
- The incident, rollback and kill-switch procedures.
Behaviour we test
- Task completion on held-out tasks drawn from real work — not the demo path.
- Direct and indirect prompt injection, including instructions hidden in retrieved content.
- Tool-failure behaviour: timeouts, errors and malformed responses.
- Permission boundaries: can it be talked into acting beyond its authority?
- Step-level traces for every failure, so the root cause is located, not guessed.
What comes out
A findings register with severity, evidence and a recommended fix for each issue; a scorecard across safety, capability and return on investment; and a release-gate verdict — Go, Conditional or No-Go — against exit criteria agreed before testing began.
The same approach scales. For a single agent it is one audit. For an organisation running many agents and AI solutions, it becomes an AI Programme Assurance engagement that covers the whole portfolio and the policies that govern it.