Governing Citizen-Developed AI Agents

8 min read

A gateway is the one place to put auth, authorization, logging and egress policy in front of every model and tool call. It is an enforcement point, not a detection point, and most buyers get that distinction wrong.

What this category solves (and what you did before it)

Before it: per-application API keys, no central log, spend nobody could attribute, and DLP that never saw a prompt. The gateway makes model and tool traffic pass through one controlled place. The premise is multi-model reality — 37% of a hundred surveyed enterprise CIOs ran five or more models in production in 2025, up from 29% the year before (large-enterprise skew, n=100). Gartner projects that by 2028, 70% of software engineering teams building multimodel applications will use an AI gateway, against 25% in 2025, and sizes the market growing from under $250M to about $1B over the same period. Note the denominator: software engineering teams building multimodel applications, not organizations generally. A widely-circulated version of that forecast quotes a “fewer than 5% in 2024” baseline that no Gartner text we could reach supports; don’t repeat it.

What actually differentiates products

Policy depth at the right granularity. OWASP’s specification of the job is exact: authenticate the agent, then authorize this agent, this tool, these parameters, not just this API key. Add per-agent and per-tool rate limits, and treat the gateway as the audit point for every tool interaction.

Whether it brokers tool traffic at all. Proxying chat completions is table stakes. Mediating MCP is the part that matters for agents, and it is a different product surface.

Rate-limit mechanics, read carefully. Limits typically exist at three tiers (whole service, a default for all users, and custom limits per user, service principal or group), with the more restrictive winning and group limits applying cumulatively. One detail worth checking per platform before relying on it: on Databricks, token-per-minute limits do not apply to MCP services, which support requests-per-minute only. So a token cap that appears to throttle a looping agent throttles its model calls and not its tool calls: precisely the runaway case.

One thing a gateway cannot do. Zero retention is a contract and configuration property of the upstream provider, not a gateway feature. A gateway can route only to zero-retention endpoints and prove in the logs that it did; it cannot make a provider stop retaining, so write the requirement that way.

Tier

Day 2. A chokepoint presumes someone knows what traffic exists, so inventory first and chokepoint second. See Day 3 sequencing.

Named examples

LiteLLM is the low-friction open-source entry: a unified interface across providers with virtual keys, per-project and per-user spend tracking, rate limiting, load balancing and logging. Still independent, with a commercial enterprise tier.

Bifrost, from Maxim (H3 Labs), is the other open-source entry and takes the tool-brokering question head-on. Apache-2.0, written in Go, self-hosted in both tiers, with virtual keys carrying hierarchical budgets across teams and customers, rate limiting, and native Prometheus and OpenTelemetry export. The enterprise tier is a strict superset of the open-source gateway: clustering and in-VPC deployment on AWS, GCP, Azure or on-premise; OIDC provisioning against Okta, Entra, Keycloak, Zitadel or Google Workspace; RBAC with row-level data scoping; HMAC-signed audit trails over configuration changes; automated log export to S3, GCS or BigQuery; and PII and secrets guardrails with Presidio and Bedrock integrations. The documentation index advertises “Immutable audit trails for SOC 2, GDPR, HIPAA, and ISO 27001 compliance”, while the audit-log page makes immutability conditional on the customer enabling S3 Object Lock and scopes the record to who changed what and when: a configuration audit that asserts nothing about prompts or about the vendor’s own certification.

Two things to settle before shortlisting it. The repository README says enterprise deployments add “adaptive load balancing, clustering, guardrails, MCP gateway” while the same README’s feature list presents MCP as core, so get in writing which tier brokers tool traffic, the half that governs agents. And the performance marketing deserves discounting: the product page claims 54x lower P99 latency, 9.5x throughput and 68% less memory against LiteLLM, with no methodology published and no independent run; the vendor’s own figures (11µs of added overhead at 5,000 requests per second on a t3.xlarge) come from a mock provider answering in 60ms. Microseconds of gateway overhead sit inside hundreds of milliseconds of model latency, so the fail-open/fail-closed semantics below and the audit export decide the purchase well ahead of the throughput ceiling.

Portkey was the other common recommendation and no longer exists as an independent vendor. Palo Alto Networks announced the acquisition on 30 April 2026 and closed on 29 May 2026; it is now the AI Gateway inside Prisma AIRS, integrated with that suite’s runtime security and agent-identity authentication, generally available from 16 July 2026. Portkey’s own documentation now says so. Any guidance recommending Portkey as a standalone choice predates this.

Unity AI Gateway is the platform-native option for firms already running Databricks: usage tracking, payload logging into governed tables, content and caller policies, rate limits across model and MCP services, traffic distribution with failover, and access control through the existing catalogue. (The product has appeared under at least three names across vendor material; use the current documentation’s name and don’t assume the rename history.)

Deployed examples exist at scale: Peloton, Natura, Experian, CRED and Stellantis all describe gateway deployments on the record. All are executive self-reports in a sponsored benchmarking report, and all are very large enterprises; one built its gateway in response to demand from a 250,000-person workforce. Read them as existence proof that the pattern ships, not as a benchmark for a forty-person fund.

MCP gateways: the subcategory that matters, and its honest trade

Brokering agent-to-tool traffic (server allowlisting, tool pinning, rug-pull detection, inline secrets scanning) is the governance chokepoint for agents. The strongest argument for it is what a measurement study found on the other side: across 1,118 OAuth-enabled remote MCP servers, every one advertised a dynamic client registration endpoint, and seven of the nine CVEs the authors obtained correspond to that failure class. The MCP specification makes dynamic registration a SHOULD while making exact redirect-URI matching, PKCE, resource indicators and audience validation MUSTs, requirements this population is failing. The gateway pitch is therefore precise: enforce at the gateway what the upstream authorization server was supposed to enforce and did not, the failure described in tool and supply-chain compromise.

The trade-offs are real and come from the people selling the architecture, which makes them credible: added latency in agent-tool communication, operational complexity, and a new single point of failure. Their own remedy list (redundancy, failover, health checks) is routine. The exception, and the question to ask first: what are the fail-open versus fail-closed semantics when the gateway is degraded? That single answer decides whether the gateway is a safety control or an outage.

The boundary: enforcement, not detection

OWASP states the limit of its own recommendation. A network-placed guardrail may miss multi-turn prompt injection because it lacks the session history and context needed to see it; in-application and in-agent placement gives the most detection context. The two placements are complementary: the gateway is the mandatory enforcement and audit chokepoint, and it cannot be the only detection point. Buy it for authorization and evidence, and put context-dependent detection where the context sits. See runtime guardrails and guardrail products.

The enterprise bar

SOC 2 Type II, ISO 27001, deployment mode, SSO and SCIM, RBAC, environment separation, exportable audit logs, zero retention (of the upstream endpoints it routes to), no-training terms, audit rights, viability and exit path. Two category-specific additions: the fail-open/fail-closed answer above, and whether the log format is exportable into an existing SIEM without a bespoke parser. A gateway whose audit trail only lives in its own console has not solved the recordkeeping problem.

M&A state and category maturity

LLM gateways are consolidating into security and networking incumbents; the Palo Alto acquisition of Portkey inside a month of announcement is the clean example, and it is one of several. MCP gateways remain pre-consolidation, with open-source projects, API-management incumbents extending sideways, and early-stage startups occupying the same space. Assume the vendor chosen is acquired or gone inside a typical contract term, and make the exit path (configuration export, log portability, the ability to fall back to direct provider calls) a term rather than an afterthought.

Where the category is immature

Rug-pull detection and tool pinning are marketed more confidently than they are evidenced; we found no independent evaluation of either. The MCP gateway also inherits the protocol’s own unsettled authorization story, so its policy surface will keep moving. That is an argument for a thin, replaceable gateway rather than a deep investment in one vendor’s policy language.

See also

Graph View

Backlinks