Topic page
Agent governance and guardrails
How an organisation keeps control over what AI agents are allowed to do: where the rules live, when authority is checked, who may stop a run, and what counts as proof afterwards.
Updated 21 October 2026
What the field says
- 01
Authority is re-proved at the effect, not at the plan. Two September 2026 preprints argue that the decision that an action is allowed is not the decision that its effect may happen; between the two, the target can be swapped, the authority revoked, the policy advanced or the approval gone stale (Shu; Zhu and Wang).
- 02
Revocation is a verdict, not an acknowledgement. Authority survives in queued messages, derived credentials, timers, webhooks, reservations and operations a provider has already accepted; a cancellation acknowledgement proves none of them stopped (Zhu and Wang, September 2026).
- 03
A guardrail in a system prompt is a request, not a rule. Practitioner consensus in the August 2026 captures: only server-side policy in front of the action is enforced; client-side checks are only as strong as that one client.
- 04
Capability gating is not authorization. A tool being on the menu does not authorise this call with these argument values; an August 2026 audit of three widely used stacks found no default fail-closed per-call check over concrete arguments.
- 05
Ownership is a decision architecture, not a name in an inventory. Six authorities are routinely bundled and need separating: outcome, platform, identity, residual-risk acceptance, suspension, and whether the investment continues (Kaplan, September 2026).
- 06
A framework without a stop rule and a named person is guidance. The OECD, G7 and UK frameworks reviewed carry principles but no numeric stop rule and no named individual with the authority to halt (Distefano, July 2026).
- 07
Value prompts do not hold against an objective. In a toy simulation, every model rulebook stole once a score existed; only changing what the score counted ended it, and public pre-commitment deterred far more than private notes (Krishnan, October 2026).
- 08
Decision authority wants a written ladder: decisions reserved to humans, decisions needing user authorisation, autonomous decisions. Read from China’s non-binding Implementation Opinions on Agents, relayed through translation (Patel, August 2026).
- 09
Human oversight has an engineering condition: human variety times amplification must meet agent behavioural variety at peak load, or the human in the loop is a compliance artefact (the CASE Framework, August 2026).
What I hold
Rules live outside the prompt, in a policy service the model cannot reach, and the service fails closed when no policy is loaded.
Model output is a candidate, never an instruction. A deterministic component authorises and executes the effect, or does not.
An approval binds to a plan revision, an evidence state and a freshness window. If anything changed, the approval is stale and the run asks again. The September papers sharpen where that check must sit: at the effect, on state read then.
Glossary: approval binding →Field reading: where authority is re-proved →
Three gates, one human: deterministic and statistical gates may block, never release; only a named person releases, and the override rate is published.
Decision record: three gates, one human →Explainer: the AI Act as engineering controls →
Autonomy is graduated by blast radius; binding decisions stay human, with no consent by timeout. The stop path is designed in advance, with a name on it.
Decision record: graduated autonomy →Decision record: break-glass →
Agents are principals with their own identity; an agent’s effective rights are the overlap of its role and its sponsor’s rights, never more.
What is open
- Is "requires confirmation" a decision or a suspension? One formal model keeps the terminal set at allow and deny and treats escalation as an intermediate outcome; another makes it a terminal verdict with a binding obligation. Both are candidate shapes and they are not the same shape.
- How do you prove revocation held when the provider offers no fence? Quiescence accounting presumes adapters that can install fences at each sink; most SaaS tools have none.
- Who may invoke the stop when stopping costs money? The halt path can be built; the mandate to use it against revenue is an organisational decision no framework reviewed so far writes down.
- What is the consequence score of a capability? Criticality, autonomy judged on the action surface, permissions and impact radius, re-scored on every change, is the proposed shape; no deployed instance has been measured.
Sources
- Wesley Shu, Boundary-State Control for Tool-Using Language-Model Agents: Commit-Time Consistency under State Drift, arXiv:2609.37475, 2026-09-26
- Genliang Zhu, Chu Wang, Authorization Revocation for Long-Running AI Agents: Root-Scoped Quiescence under Delegation and Asynchronous Execution, arXiv:2609.21284, 2026-09-18
- Rohit Krishnan, Agent governance looks less like moral education and more like institutional economics, Strange Loop Canon, 2026-10-06
- Michael Kaplan, Every AI Agent Needs an Accountable Human Owner, The Sentient Architect, 2026-09-17
- Marcela Distefano, Algorithmic Governance That Doesn’t Govern: Why the Frameworks Fail Where It Matters Most, Ethics, life and AI, 2026-07-24
- Oliver Patel, How China is governing agentic AI, Enterprise AI Governance, 2026-08-30
- Cloud Security Alliance, AI Safety Initiative, Capabilities-Based Risk Assessment (CBRA) for AI Systems, CSA, 2025-11-12
Changes
- 2026-10-21 · First version, written from the field-brief refreshes of 8, 9 and 15 October 2026 and the August 2026 captures.