Governing Citizen-Developed AI Agents

A reference wiki for CTOs, heads of technology, and technically fluent COOs at hedge funds and asset managers of 20–500 people, on governing AI agents built by people outside the technology org chart. It assumes readers already run regulated infrastructure and know what SSO, RBAC and a SIEM are, and assumes no prior acquaintance with a lethal trifecta or an MCP rug pull.

This wiki is fully AI-generated:

Start with a folder of source docs, use all the AIs, OpenAI, Claude, Gemini, Google search, Google Scholar to identify authoritative sources. Iterate and fill gaps. (source docs not shared due to IP/copyright concerns)
Have Claude help write a Claude.md - tell it the structure you are looking for, style etc.
Prompt, bootstrap the wiki using the docs per Claude.md … read each doc carefully, extract relevant parts and assign to the correct section of the wiki. Start with phsse 1, run through 10 phases (needs multiple sessions and uses a lot of tokens)
Iterate, if you notice gaps or style problems say, address this and fix the wiki.

Why this exists

Analysts, ops staff and PMs are already building agents. Natural language became a compiler, the measured productivity gains land hardest on the least expert, and none of it needs hardware, a purchase order, or sign-off from the technology org. Roughly two-thirds of surveyed firms permit employee-led AI building; about half concede they cannot see it. Among companies planning agentic deployment, around one in five report a mature governance model for autonomous agents.

The old citizen-development playbook — sanctioned platform, maker certification, tiered app review — assumed a reviewable artifact and a bounded blast radius. Agents break both. There is often no code any accountable human read, and the artifact acts: it sends the email, files the ticket, moves the position. Meanwhile the governance most firms have was designed for software that produces the same output twice.

This wiki is opinionated about what to do instead, shows its sources, and marks the places where the evidence is thin, vendor-funded, or drawn from firms a hundred times larger. One asymmetry to know going in: the evidence is graded carefully and the recommendations are not. Read every “What to do” section as practitioner judgment unless it cites something; the map explains why.

What it concludes

Order is the strategy. Visibility, then tiering, then gates and platform controls. A control installed before a firm can see the agents it governs gets routed around on day one. → sequencing
Agent security is architecture, not filtering. An agent that reads untrusted text, holds sensitive data and can reach the network will eventually do all three in the wrong order. Cut one leg in the design; guardrails go on top of the cut, never instead of it. → the lethal trifecta
Identity is what everything else rests on. An agent authenticating as its builder produces an audit trail naming the wrong actor, and no amount of logging fixes that afterwards. → non-human identity
A control the agent can decline is self-regulation. In-process guardrails run under the agent’s own authority. Out-of-process versus in-process is the buying criterion, and it appears on no comparison sheet. → runtime guardrails
Gate the transition, not the keystroke. Per-use approval fails to scale and manufactures shadow agents. Govern the move from personal to shared to production, and make the tiers physically real as separate environments. → promotion gates, three-zone architecture
Killing the process is not revocation. The token is still valid, the session is still open, and the sub-agents are still running under their own identities. → kill switches and revocation
Review capacity, not build capacity, is the binding constraint, and it is a staffing decision, not a discipline problem. → oversight decay
Buy the platform, build the agents on it, and treat the contract as the deliverable. Half the vendor map has no credible leader; where there is no leader, the answer is architecture rather than procurement. → build vs buy
Recordkeeping obligations did not pause for agents. The trace store holds prompts and tool outputs, which makes it a records system before it is a debugging tool. → recordkeeping and compliance gaps
Nothing has to go wrong for this to be a violation. For a registered adviser, inadequate written procedures is chargeable on its own, before any harm, and the penalty is rarely what ends a fund; the redemption is. → regulatory exposure

How to read it

Start here: the problem — what changed, how much of the evidence to believe, and three questions to ask this week.
The map — the full page tree and reading paths for an exec, a platform lead, and a builder.
Crosswalk — concern → dimension → control → gate → maturity, which is the table an examiner will ask for.
Changelog — what changed and when.