Governing Citizen-Developed AI Agents

7 min read

Order is the strategy: visibility first, tiering second, gates and platform controls third; a control installed before the agents it governs are visible gets routed around on day one.

This page is the canonical Day 1 / Day 2 / Day 3 sequence, sized for firms of 20–500 people. Dimension hubs link here rather than restating it.

Three sources with entirely different vocabularies land on the same floor. A consultancy’s ITSM ordering puts monitoring and event management first, alongside asset inventory and access management, with registry tooling and policy-as-code after. WEF’s baseline says every agent, regardless of tier, runs under least-privilege access with clear task boundaries and audit logging. A security survey’s minimum production stack starts with action logging and tool sandboxing. Logging, inventory, identity. None of the three is evidence; they are three prescriptions that happen to converge, which is worth more than any one of them alone.

The checklist

Day 1 — see what exists

Inventory before policy. Discover what is already running, map what each thing connects to and what it exposes, then write the policy. Rules for a population that cannot be enumerated produce a document, not a control; see agent inventory.

Open a registration path, and an amnesty. The path has to be lower-friction than staying hidden, or the gate governs an empty set. See shadow agents.

Set the floor for every agent, tiered or not: least privilege, explicit task boundaries, action logging. Read-only access to knowledge bases first; write capability only behind an approval gate.

Reuse the machinery already in place. Access control, human approval and red-teaming are, in the AI-control literature’s own framing, the minimum viable product of AI control; build them to scale up rather than planning to replace them. (That paper’s threat model is misalignment rather than human misuse and its upper levels run ahead of practice; the framing transfers, the contents don’t.)

The evidence that this order is more than tidy: in one documented case a low-code platform was switched on for everyone before any governance existed, and more than 4,000 applications appeared. The global product owner’s own retrospective was that they should have started with structured top-down guidelines. Another firm in the same study called its approach trust by design: guardrails designed before rollout rather than after the maintenance problem. (That paper contradicts itself on which case produced the 4,000 applications, and asserts a one-month timeframe unsupported by its own evidence section, so what survives is the count and the retrospective.)

Day 2 — make governance proportionate

Tier assignment, so controls scale with blast radius instead of applying uniformly. See risk-tier assignment.

Promotion gates at the zone boundaries, with evidence requirements keyed to tier. See promotion gates.

The platform floor: identity for agents, egress control, secrets management. This is where the build-vs-buy decision starts costing money.

Reviewer capacity, named before the gate opens. See the reviewer pool.

Day 3 — automate and harden

Runtime guardrails, evals wired into the gate as acceptance evidence, lifecycle automation, replay testing against a production clone. Label this honestly: it runs ahead of practice for a firm this size, and the literature modelling its payoff does so in simulation. One maturity model finds that formal policies, a central catalogue, RBAC and human-in-the-loop constitute the minimum viable standard, that automated sprawl detection is the control with the largest effect above it, and that mature automated governance costs less than semi-manual governance. Every figure in it comes from 750 simulation runs parameterized from published reports rather than from production, so it reads as “in the model,” never as “firms at this level achieve.”

What shifts the cutlines

Headcount. A 40-person fund does Day 1 in a spreadsheet with an owner column; a 400-person firm needs registry tooling from the start, because the spreadsheet stops being queryable.

Retrofit cost, uneven across controls. Logging and approval gates can be added to a running deployment relatively cheaply. Injection hardening cannot, because it requires architectural review of every channel through which untrusted text reaches the agent. Put the expensive-to-retrofit items earlier than their risk score alone would suggest. (That taxonomy circulates misattributed to a model vendor. It is a consultancy’s, and we carry it as engineering judgment with the reasoning shown.)

What pays off unilaterally. The sharpest prioritization rule in the research, and the one built for resource-constrained firms: build the infrastructure that works whether or not anyone else adopts it (oversight layers, rollback, logging) before the infrastructure that only pays off once counterparties adopt it too, such as inter-agent protocols and cross-firm agent identity. Most of the second category is still a research agenda.

Cost direction. Each successive stage of an enablement program gets more expensive, not less, as data, compliance and integration requirements accumulate. Keep Day 1 cheap and exploratory, and budget for Day 2 rather than being surprised by it.

How you’d know it’s working

Coverage before sophistication. Percent of known agents registered and tiered beats any count of controls deployed. A Day 3 control at Day 1 coverage is theater.

Timeline sized to the firm, not copied from an enterprise program. Mid-market firms report roughly 90 days from pilot to full implementation where large enterprises report nine months or more. That comes from a self-described directionally-accurate preprint covering 52 organizations, whose authors promote their own protocol, so it supports sizing to scale rather than setting a target.

Readiness correlating with benefit, read honestly. In a survey of 603 technology decision-makers, 27% scored as readiness leaders, 50% followers and 24% laggards; 42% of leaders and 41% of followers reported improved productivity against 33% of laggards. Note what that actually says: the leader-versus-follower gap is nil. The gap is between having done the groundwork and not. It is cross-sectional self-report in a sponsored study, so it shows correlation, not that governance produced the benefit.

What this doesn’t solve

Sequencing tells firms when, not whether. It assumes the firm has decided to permit citizen agent development at all, and it is no substitute for the pilot evidence that justifies each next step.

It cannot settle how much time a firm has. The most-quoted urgency claim, that CIOs have a six- to twelve-month window before agentic complexity outruns them, is a consultancy’s position with a services motive, and its only quantitative support is a growth rate that resolves to no published vendor metric. What is first-party and dated, and worth reasoning from rather than borrowing someone else’s deadline: Microsoft reported roughly 40 million registered agents across tens of thousands of companies about two months after its agent-management product reached general availability in May 2026.

The sharpest objection to this page comes from the citizen-development literature’s own advocates: none of this has to be expensive or bureaucratic, and obstacles defeat the point of citizen development. That is right, and it argues for cheap Day 1 controls rather than for no controls. The firm that switched the platform on for everyone and stood up governance after four thousand applications is the counter-example. Most readers sit between the two: in one executive survey a quarter had a comprehensive roadmap, just over half were still refining a draft, and nobody reported having none at all.

See also

Graph View

Backlinks