Prompting is not enforcement. A guardrail the agent could choose not to invoke is self-regulation, and the measured gap is large: prompt-level defences are collapsed by attacks written for the specific model, while the controls that actually moved a risk score in a published case study were ordinary database settings.

The mechanism

Any attack-success-rate figure quoted from a fixed benchmark suite measures how well a defence resists attacks nobody optimized against it. In a red-team exercise by the US and UK AI safety institutes on AgentDojo’s workspace environment, attack success against the strongest baseline attack was 11%. Against the strongest attack purpose-built for that model, it was 81%. Adaptive-attack evaluations are the thing to ask a vendor for, and a figure produced without them is marketing.

Layering follows directly. Prompt-level defence is the weakest rung and the most commonly sold. In one MCP study, cross-server attack success was 61.3% unmitigated, 47.2% with a system-prompt rule, and 8.7% with protocol-level isolation. The prompt rule bought fourteen points; the architecture bought fifty-three. (Unreviewed preprint, authors evaluating their own defence, no adversarial-bypass testing; the best head-to-head measurement available and still not strong evidence.) A separate simulation of collusion in multi-agent markets found a prompt-only “constitution” produced no statistically significant improvement over no governance at all, while runtime enforcement with consequences cut severe collusion from 50% of runs to 5.6%. Simulation, 90 runs, six model configurations: cite it as a mechanism, not a field result.

The reason is structural rather than empirical. An in-agent check is self-regulation: an agent that can write code can bypass its own guardrails, and a policy engine the agent may decline to call fails as a control. Enforcement has to be interposed on a path the agent cannot route around. A regulatory version of the argument carries an authority into the buying decision: under the Market Access Rule, FINRA has cited firms for relying on vendor-set thresholds without maintaining “direct and exclusive control over controls.” A control the platform vendor owns and can change belongs to the vendor, whatever the datasheet says. Path-dependence matters too: some violations exist only as sequences. Reading A is permitted, sending B is permitted, and reading A then sending B is the incident. A policy that cannot see execution history and shared state cannot express that, which is an expressiveness gap rather than an implementation gap.

What works, in order of demonstrated effect:

Tool-level privilege policies drive attack success to near zero. Progent reports 0% on three benchmark suites with hand-written policies while preserving utility, and 39.9% → 1.0% at 76.3% utility with LLM-generated policies. Both halves are true at once, and the split is the whole story: the 0% depends on the hand-authored policies that other work finds unscalable.

Learned allow-lists are the attempt to close that gap. AgentGuardian observes benign traces, builds a control-flow graph where an edge exists only if tool B directly followed tool A, clusters inputs so policies generalize instead of enumerating, and adds attribute constraints (max tokens, permitted daily window, max idle interval, max end-to-end duration) enforced below the agent’s business logic. It detected 18 of 20 policy-violation scenarios with a false-acceptance rate of 0.10 and a distinct false-rejection rate of 0.10 (those are two numbers that happen to coincide; don’t merge them), across twenty attack samples in two applications: research-stage evidence.

Deterministic resource limits are the underrated ones. In MATRA’s case study, the guardrails that moved a database-availability scenario from Very High to Moderate were Postgres settings rather than anything sold as AI security: statement_timeout, EXPLAIN cost validation and connection limits. The paper’s own conclusion is that the most effective controls are architectural (network-limited sandboxing, tool and command allow-listing, read-only database roles) because injection cannot be eliminated, so mitigation must target what injected instructions can reach.

OWASP’s prescriptions are worth lifting wholesale: a predetermined allowable-topic list with implicit deny, structured input formats instead of concatenation, memory hygiene (strip instruction-like tokens before storing, TTL on memories, read-only system-instruction buffers), and isolation by default with explicit egress allow-lists. Its ASI02 prevention guidance names the architecture: a policy enforcement point plus decision point interposed before execution, treating planner output as untrusted, validating intent and arguments, enforcing schemas and rate limits, issuing short-lived credentials, and revoking on drift.

Two failure modes to design around. ML-based guardrails are adversarially evadable, and tuning one to catch more attacks raises its false-positive rate on legitimate work; static block and allow lists go stale and limit utility. Constrained output formats avoid both because they are enforced algorithmically rather than judged.

What to do

Put enforcement at a chokepoint the agent must traverse, and treat everything in-agent as advisory. That chokepoint is the control plane; this page is what runs on it.

Start with deterministic limits, because they are free and they work: query timeouts, connection caps, read-only roles, per-tool allow-lists, rate limits. If a control can be expressed as a database or network setting, it depends on no model judgment and cannot be talked out of firing.

Write policies over paths rather than actions. Condition on what has already happened in the run and on shared state, default to deny-and-halt on violation, and fail closed.

Use learned baselines to propose policies and a human to approve them. Hand-written policies work and don’t scale; generated policies scale and lose about three points of utility. Generation plus review is the shape that survives both findings.

Constrain output format rather than filtering output content. A schema is enforced; a classifier is judged.

Demand adaptive-attack results from any vendor selling a guardrail, and re-run the firm’s own suite on every policy change. See evals.

How you’d know it’s working

The log contains path-level blocks that no static rule would have caught. That is the direct evidence the policy layer is doing something a permission list could not.

A deny has happened recently and someone can name it. The false-rejection rate is known: guardrails that never block anything and guardrails that block real work fail differently and both get switched off.

Rerunning the red-team suite after a policy change produces a different result. A number that never moves means the suite is testing something other than the policy.

What this doesn’t solve

Guardrails rank below architecture, and this page claims no more for them. Architectural controls cap impact; input-level detectors only reduce probability. Cut a leg of the lethal trifecta first, then put this on top of the cut rather than in place of it.

A chokepoint control governs traffic through the chokepoint. Agents on personal API keys, self-hosted runtimes or unsanctioned platforms are ungoverned by construction; see shadow agents.

Learned policies inherit a trust assumption their authors state out loud: the staging phase is assumed attacker-free. A poisoned observation window bakes the attacker’s behaviour into the allow-list as normal. There is also a structural tension between exhaustive coverage and over-generalization that no amount of tuning removes.

AI monitoring AI is a research programme, not a control. A monitor inherits the failure modes of what it monitors, broadens the blast radius of its own failure, and roughly doubles inference cost; cheaper monitors are weaker monitors. The term “guardian agents” is Gartner’s, and it names a market category rather than an architecture.

Real-time monitoring suits clear-rule violations (restricted activities, tool and permission limits, sensitive-data leakage, usage caps) and needs no log retention to work. Subtle or delayed harms need retained logs and retrospective analysis, which is a different control: logging and audit.

Cross-layer defence fails to transfer. One preprint argues, with a proof in supplementary material, that a control at one layer has zero detection power against an attack at another. The argument is sound and the epistemic label is “argued,” not “established”, though it predicts what practitioners already see, and it means a guardrail stack is only as broad as its layer coverage.

Two claims in circulation deserve discounting. The claim that an agent can monitor its own compliance during a reflection step is contradicted by the entire runtime-governance literature. The traffic-light action classification often attributed to Anthropic is a consultancy’s rendering; Anthropic’s actual mechanism is three permission states (always-allow, needs-approval, block), with the control living in the permission layer, outside the model’s decision.

See also

• Prompt injection and goal hijack — the failure this layer sits above, after the architectural cut.
• Multi-agent cascades — the compositional violations only path-aware policy can express.
• Cross-session delayed detonation — why policy must condition on persisted state.
• The agent control plane — the architecture that gives this policy an enforcement point.
• Guardrail products — what vendors actually ship, and what to ask them for.
• Human approval gates — the escalation path when policy says “ask a human,” and its own failure modes.
• Sandboxing — the deterministic containment that outperforms detection here.