Open-source LLM gateway self-hosted or cloud 83 providers
Ship AI faster with routing, visibility, and control.
Ferro Labs unifies provider routing, policy enforcement, cost controls, and observability behind one OpenAI-compatible endpoint for production AI teams.
Product demo
See the AI Gateway in action
A short walkthrough of routing, governance, caching, and observability — one OpenAI-compatible endpoint, end to end.
Works with every major provider
One OpenAI-compatible endpoint in front of 83 providers and 2,505 models. Point at any of them by name — no SDK swaps, no application rewrites.
OpenAI
Anthropic
Google Gemini
Meta Llama
Mistral
Hugging Face
Groq
Perplexity
Azure OpenAI
System Architecture
Enterprise AI Gateway Architecture
A governed routing layer between your applications and every AI provider, built to centralize access, failover, caching, policy, and request telemetry.
Your Apps
AI Gateway
Unified control plane
AI Providers
Request lifecycle
Every request, governed end to end
What happens in the gateway between your application and the model provider.
- receiveTLS · OpenAI-compatible0.1ms
- authenticateAPI key · JWT0.2ms
- governrate limits · budgets · PII0.4ms
- cachesemantic lookup · 0.921.1ms
- routeweighted · fallback0.3ms
- inferprovider call~840ms
- observetrace · metrics · log0.2ms
gateway overhead p99 ≈ 2.3ms · provider time dominates
Production AI gateway capabilities
Route requests, enforce policy, cache repeated prompts, and observe spend from one open-source control layer hardened for production teams.
Built for platform, product, and AI teams
Start with the open-source gateway, then standardize routing, governance, and observability as AI usage grows.
For startups
Ship AI features in minutes, not weeks. Switch providers without touching code and start free — no infra team required.
- 30 providers
- OpenAI-compatible
- Start free
For enterprise
Centralize LLM access with governed routing, audit logs, SSO/RBAC rollout paths, and deployment options for VPC or self-hosted environments.
- SSO / RBAC
- Audit logs
- Self-host · VPC
For developers
One OpenAI-compatible SDK for every model. Self-host the Apache-2.0 gateway or wire up MCP for agentic tool use.
- Apache-2.0
- MCP
- 8 strategies
Open infrastructure with enterprise control
Built for teams that need transparent core infrastructure, fast execution, and control over where AI traffic runs.
Open infrastructure
Open source, no closed core
Apache 2.0 end to end. Every routing strategy and core plugin ships in the OSS tier — nothing critical is paywalled.
Written in Go
A compiled gateway runtime designed for low overhead on the request path, without a Python proxy layer between your apps and providers.
MCP-native
A built-in Model Context Protocol server runs the agentic tool-use loop inside the gateway — for Cursor, Claude Desktop and custom agents.
Self-hostable by design
Run the gateway in your own VPC or on-prem. Keep provider keys and runtime control in your environment without a required Ferro-hosted control plane.
Frequently asked questions
Answers for teams evaluating routing, governance, self-hosting, and provider strategy.
Talk to the enterprise teamWhat is an enterprise AI gateway?
An enterprise AI gateway is a control layer between applications and AI providers. Ferro Labs centralizes provider routing, OpenAI-compatible access, policy controls, usage telemetry, fallbacks, and cost visibility across 83 providers and 2,505 catalog models.
Can Ferro Labs run in our VPC or self-hosted environment?
Yes. The Apache 2.0 gateway is self-hostable, and enterprise deployments can be planned for customer-controlled VPC or on-prem environments where provider keys and runtime controls stay under your operating model.
Does Ferro Labs replace OpenAI or Anthropic SDKs?
No. Teams can keep OpenAI-compatible SDKs and point the base URL to Ferro Labs. The gateway then routes requests across approved providers and models without requiring application rewrites.
How does Ferro Labs improve LLM reliability?
Ferro Labs supports fallback chains, retries, load balancing, rate limits, and observability so platform teams can shift traffic when a provider is slow, unavailable, or no longer the best fit for a workload.
What governance controls are available?
Teams can standardize virtual keys, rate limits, audit logging, guardrails, content filters, budget controls, and deployment-level controls. Enterprise security reviews are available before production rollout.
Who is Ferro Labs AI Gateway built for?
Ferro Labs is built for platform engineering, product, and AI infrastructure teams that need one governed API for production LLM applications across OpenAI, Anthropic, Google Gemini, AWS Bedrock, Azure OpenAI, and other providers.
Trust & compliance
Built for security review
The controls and documentation your security team will ask about, ready before they ask.
Single sign-on
- OIDC + SAML
- RBAC — 9 scopes / 4 roles
Audit & export
- structured audit logging
- SIEM export
Encryption
- TLS 1.3 transit · AES-256-GCM rest
- Ed25519-signed JWTs
Data handling
- no training on customer data
- retention 3 / 30 / 90 days
Fully managed, coming soon
The AI Gateway without the ops. Join the waitlist and we'll reach out as we open access.
No spam just a heads-up when your spot opens.