AI Governance & Security Vendor Wiki

5 min read

TrueFoundry

Primary category: ai-gateway. Also: mcp-gateway.

One-liner — A Kubernetes-native enterprise AI platform whose centerpiece is an LLM/AI gateway (plus MCP gateway and model-deployment/LLMOps tooling) that gives a company one governed control plane for all model and agent traffic.

What it does

TrueFoundry started as an LLMOps / model-deployment platform — deploy and serve any LLM, embedding, or custom model on your own Kubernetes (vLLM, TGI, Triton backends, fractional-GPU support) — and has pushed its AI Gateway to the front of the product. The gateway is a unified API in front of OpenAI, Anthropic, Gemini, Groq, Mistral and (vendor claim) 250+ models, adding the things an enterprise needs before it lets teams hit model APIs: centralized key management, latency-based routing and automatic fallback, per-user/team/endpoint rate limits and token/cost budgets, RBAC, SSO, immutable audit logging, and observability (token usage, latency, error rates, full request/response logs). Input/output guardrails (PII filtering, toxicity, prompt-injection checks, OpenAI Moderation and Bedrock Guardrails integration) are built in. The newer MCP Gateway extends the same control plane to agent tool calls — a central registry of approved MCP servers, OAuth/RBAC on every tool invocation, credential brokering (one token auto-exchanged for per-server tokens), and full tracing of agent→tool actions.

Where it sits in the stack

Lives in the model/prompt layer as the ai-gateway — the single exit door for model traffic — and as an mcp-gateway for agent tool access. Its primary security role is egress control: it is the chokepoint where outbound model/tool calls are routed, logged, budgeted and policy-checked, so it’s the natural place to enforce where data can and can’t go. The built-in guardrails add some secondary screening of untrusted input and sensitive data, but it is not a dedicated ai-runtime-security firewall. It sits between the apps/agents (yellow zone) and external model providers, and is most valuable when self-hosted so traffic and logs stay inside the trust boundary.

Deployment & architecture

API gateway, not an inline network proxy or browser agent. Distinctive trait among gateways: it is Kubernetes-native and designed to run inside your own environment — VPC, on-prem, air-gapped, hybrid/multi-cloud — with the vendor stressing data sovereignty and no external egress of prompt/response data. Also offered as SaaS. Performance claims (marketing): sub-3ms internal latency, 10B+ requests/month, ~30% cost optimization. Integrations: Kubernetes, Hugging Face, MLflow, AWS/Azure/GCP; OpenTelemetry-based tracing exporting to Grafana, Datadog, Prometheus; SSO via Okta/Azure AD; MCP servers (Slack, GitHub, internal tools). Compliance posture: SOC 2, HIPAA, GDPR (vendor stated).

Positioning & differentiators

TrueFoundry’s pitch is “one control plane for models, gateway, and agents,” with the self-hosted, Kubernetes-native, full-LLMOps story as the differentiator. Versus neighbors:

  • portkey — closest analog (gateway + guardrails + observability); Portkey is more gateway-focused, TrueFoundry bundles the gateway into a broader deploy/serve platform.
  • litellm — open-source/library and lightweight proxy; TrueFoundry is a heavier, commercial, enterprise-governed platform with UI, RBAC, deployment.
  • kong — API-gateway incumbent extending into AI/MCP gateways; Kong leads with general API-management heritage, TrueFoundry is AI/ML-native and ships model serving.
  • cloudflare — edge/SaaS AI Gateway tied to Cloudflare’s network; TrueFoundry is run-it-yourself inside your K8s rather than at a third party’s edge.
  • openrouter — a hosted multi-model marketplace/router (buy access to models); TrueFoundry is governance/infrastructure you run, not a model reseller.

Ownership, funding & M&A

Independent and VC-backed. Founded June 2021 by Nikunj Bajaj (CEO), Abhishek Choudhary and Anuraag Gutgutia. Dual HQ: San Francisco and Bengaluru. Raised a $2.3M seed (Sep 2022, Peak XV’s Surge) and a $19M Series A in Feb 2025 led by Intel Capital (Eniac Ventures, Peak XV’s Surge, new investor Jump Capital, plus angels), for ~$21.3M total. No acquisition — no seed M&A flag, and none found. Ownership confidence: high.

CTO / hedge-fund lens

An AI gateway is a Day-1 control: before you let desks and agents call models, you want one governed exit door for keys, budgets, rate limits, audit logs and a kill switch. TrueFoundry’s self-hosted/air-gapped, Kubernetes-native posture is attractive to a fund that wants prompts and outputs to never leave its own infrastructure (MNPI, data-residency concerns). The catch: it is a heavier, infra-centric platform that assumes you run Kubernetes and want model serving/LLMOps too. A 50-person fund that just wants a managed exit door may find portkey (or a thin litellm proxy) lighter; TrueFoundry fits a shop that already operates its own ML/K8s infra and wants gateway + serving + agent governance from one vendor. Not itself a model-risk/SR 11-7 governance tool, but its audit logs and policy enforcement feed that process. Note its smaller scale (single Series A, India+US startup) when weighing vendor risk.

Competitors / alternatives

portkey, litellm, kong, cloudflare, openrouter. Adjacent: ai-runtime-security (dedicated AI firewalls), llm-observability, authorization-engine (fine-grained agent/tool authz).

Open questions / to verify

  • Independent benchmark of the sub-3ms latency / 10B-requests claims (vendor-stated).
  • Maturity and standalone adoption of the MCP Gateway vs the core AI Gateway.
  • SaaS vs self-hosted split among customers; pricing model.
  • Whether any Series B or follow-on funding has closed since Feb 2025.

Sources

History

  • [2026-06-28] Stub created from seed registry.
  • [2026-06-28] Researched; established founded 2021, dual HQ (SF + Bengaluru), founders, $21.3M total funding ($19M Series A led by Intel Capital Feb 2025), independent VC-backed (no M&A, confidence high). Documented AI Gateway + MCP Gateway, Kubernetes-native self-hosted deployment, egress-control role, medium hedge-fund fit; compared to portkey/litellm/kong/cloudflare/openrouter. Cached 3 sources.

Graph View

Backlinks