Governing Citizen-Developed AI Agents

7 min read

Agents run on credentials scoped to a person and inherit every entitlement that person has, so the question is never whether the agent is authorised; it is whose authority it is spending and whether anyone can tell afterward.

The mechanism

The confused deputy is forty years old and agents have rebuilt it at scale. A deputy holds more privilege than the party asking it to act, and cannot distinguish a legitimate request from an injected one, so it spends its own authority on someone else’s instruction. The old name is worth using plainly rather than as novelty, because a security team already knows why the problem is hard.

What is new is how the deputy gets its privilege. Three default paths, all common:

Inheritance from the builder. Non-human identities are typically managed by mapping them onto a human owner’s roles (LDAP group membership, IAM role mapping), so the agent runs with its creator’s entitlements. For a citizen-built agent that means the agent can do everything its builder can do, which nobody decided and nobody reviewed.

A shared service account. Databricks describes the antipattern in its own documentation: agents run as a generic service account with broad privileges. Two consequences follow mechanically. Rotation becomes infeasible because the token lives in every user’s config. And every action lands in the audit log under one identity, so there is no user-level telemetry and incident response has nothing to work with.

Platform defaults. OWASP names low- and no-code agents with default privileges, unrestricted internet access among them, as un-scoped privilege inheritance. The maker delegates their full access context because that is the path of least resistance in the product.

Then the scope grows without anyone granting anything. Each tool call is individually authorised; the sequence accumulates reach that no single call implies. Access control that evaluates permissions per call cannot see it, so this survives a clean permissions audit. Microsoft’s taxonomy names permission chaining across agents, OWASP notes that agents chain tools in ways that bypass per-tool restrictions, and the sample failure is exactly the shape a fund should recognise: an HR agent returns raw personnel data alongside its derived recommendation, because the agent’s access exceeds that of every user it serves.

Multi-agent systems add a relay. Agents trust internal requests by default, so a compromised low-privilege agent can pass valid-looking instructions to a high-privilege one that executes without rechecking the original user’s intent. Microsoft’s 2026 taxonomy update promotes this to its own category, inter-agent trust escalation: a compromised agent asserting false identity or inflated permissions to an orchestrator that never independently verifies them. The category is grounded in a year of red-team engagements rather than forecast.

The MCP authorization layer is measurably broken, and this is the part to bring to a security review. In a May 2026 scan of 119 testable OAuth-enabled MCP servers, every single one had at least one authentication flaw, 325 in total. Dynamic client registration flaws affected 96.6%: in 114 servers the registration endpoint accepted any redirect_uri from an anonymous registrant, a direct account-takeover primitive: register a malicious callback, induce the victim to follow a crafted authorization URL, collect their authorization code, exchange it for a token, take the session. The researchers walked exactly that chain against a live error-monitoring server and obtained a CVE.

Worse, the one binding these clients depend on is the one that fails. MCP clients are typically local applications that cannot protect a client secret, so PKCE is the real link between the authorization request and the token exchange. PKCE downgrade, omitting the challenge or accepting plain, was confirmed in 68% of servers, and consent-page bypass in 60%.

Standard application security review misses this because the flaws are structural rather than single-endpoint misconfigurations. An MCP server is a resource server to the calling client and an OAuth client to upstream services simultaneously, so guarantees must hold across independently operated systems: coordination failures across layers, in the paper’s own framing.

Over-permissioning is the baseline even with no adversary. GitHub’s official MCP server exposed roughly ninety tools and tens of thousands of tokens of definitions as of late 2025, delete_file beside get_pull_requests; GitHub’s mitigation is a switch to turn toolsets off, conceding the default is broader than most users need. Tools can also be discovered dynamically, so a server can add capabilities at runtime that the agent picks up unapproved. And capability declarations are self-asserted: the spec requires both parties to use only negotiated capabilities, which binds the honest party and gives the receiving one no way to verify or enforce it.

What to do

Give each agent its own identity with minimum scopes and short-lived tokens, and stop reusing a human’s credentials. That is the whole first move, and it is in identity and access.

Check three specific things on every remote MCP server a firm permits: that dynamic client registration is closed or restricts redirect URIs, that PKCE with S256 is required rather than optional, and that the consent page cannot be skipped. Those three cover the dominant measured failure modes, and none of them is exotic OAuth work.

Authorise sequences, not just calls. Any control that evaluates one action at a time is structurally blind to accumulated reach. See runtime guardrails for policies that condition on history.

Assume denied builders route around the restriction. This is the best-evidenced pattern in the pre-agent literature: when IT would not provide an API, no-code applications emulated human operators to query and update production databases directly, and a citizen developer in the same research describes holding a platform role nearly as powerful as a developer’s. Blocking the sanctioned path produces an unsanctioned one with worse identity properties; see shadow agents.

Treat spending authority as a distinct tier. Agents can move money today through mainstream payment tooling: payment links, invoicing, single-use virtual cards, agent-native checkout. Those scopes belong behind human approval gates rather than behind a policy document.

How you’d know it’s working

Every action in the audit log resolves to a distinct non-human identity. Two agents sharing a token produce a log file, not an audit trail.

A scope-creep report exists and something has actually been clawed back. Unused permissions that nobody removes are the measurable form of this concern.

One agent’s credential can be rotated without a change window. If it cannot, the shared-token problem is already load-bearing.

What this doesn’t solve

Perfect scoping still permits harmful actions inside the granted scope. An agent authorised to email clients can email the wrong client. Scoping bounds reach, never judgment; that needs human approval gates and runtime guardrails.

Unverifiable identity schemes are worse than none. An unverifiable agent ID lets an attacker’s agent masquerade as someone else’s and launders their actions into an innocent party’s audit trail; the researchers proposing agent IDs say so themselves and add that IDs should not be the sole evidence in an investigation.

Network isolation fails to contain it. An agent connected to several MCP servers bridges the boundaries between them, so segmentation that assumes traffic stays in its zone breaks down.

The insider framing is a poor guide. The standard numbers (56% of insider incidents from negligence, 26% malicious, 18% credential theft) come from a 2022 study predating agentic systems entirely. Useful for arguing that non-malicious misuse dominates; useless for sizing this.

Two claims in circulation remain unevidenced: agents autonomously probing management APIs to grant themselves privileges, and behavioural impersonation of an agent by mimicking its patterns. Both are hypotheses in proposal papers. The compositional escalation above is well attested; these two are not, and the difference matters when deciding what to spend on.

See also

Graph View

Backlinks