Skip to content
LLM cost control

LLM Cost Control and Budget Enforcement

Ferro Labs gives finance, platform, and engineering teams shared LLM cost control across providers, models, projects, and API keys.

Set workspace, provider, model, API key, and project budgets with alert-only or hard enforcement modes.

Deploy OSSAll Solutions

Buyer Problem

AI spend can move from experiment to budget incident quickly. Provider dashboards rarely match internal ownership models, and app teams need controls before requests are billed.

Target Outcome

Teams can set budgets at the workspace, provider, model, API key, and project level, choose alert-only or hard-enforce behavior, and attribute spend to the right owners.

Capabilities

Gateway controls for this solution

These are the gateway-level capabilities this solution depends on.

Workspace budgets for total AI spend limits across connected providers.
Provider and model budgets for constraining expensive vendors or model families.
Project budgets that align spend controls to product, environment, or team ownership.
API key budgets for per-app, per-user, or per-service usage control.
Alert-only budgets for early warning before enforcement affects traffic.
Hard-enforce budgets that return 429 budget_exceeded before additional spend is incurred.

Proof Points

Evaluation evidence to review

Workspace, provider, model, API key, and project budgets are implemented.

Budget modes include alert-only and hard enforcement.

Hard-enforced budgets return HTTP 429 with budget_exceeded.

Budget decisions can be reviewed through logs, analytics, and project usage views.

Status note

Live: workspace, provider, model, API key, and project budgets with alert-only and hard-enforce 429 budget_exceeded behavior.

Related Features

Feature deep dives

Rate Limiting & Access Control

Rate limiting works in three independent layers: per-IP HTTP middleware, a global token bucket applied before traffic reaches a provider, and per-API-key and per-user limits. Checks run in order — global, then per-key, then per-user — and the first exceeded limiter rejects the request with a distinct reason string.

Real-time Observability

Four observability layers ship in the gateway: Prometheus metrics at /metrics, opt-in OpenTelemetry tracing, structured JSON logs, and a deep /health endpoint. The OTel trace ID, the log trace_id, and the X-Request-ID header are all the same value, so you can jump from a log line straight into your tracing backend.

8 Routing Strategies

Choose from eight production-hardened routing strategies shipped out of the box. Switch between them with a single config key — no code changes required.

Semantic Caching

Semantic caching compares the meaning of a request, not its exact bytes, so near-duplicate prompts are served from cache. Queries are embedded and matched against an HNSW index in PostgreSQL with pgvector; a hit above the similarity threshold returns instantly without a billable provider call.

FAQ

Common evaluation questions

Can Ferro Labs enforce hard AI spend limits?

Yes. Budgets can be configured in hard-enforce mode so requests over budget receive HTTP 429 with budget_exceeded.

Can budgets be alert-only?

Yes. Alert-only budgets are implemented for teams that want warnings before they turn on enforcement.

Which budget scopes are supported?

Workspace, provider, model, API key, and project budgets are implemented.

Solution Review

Validate this solution against your deployment model

Start with the open-source gateway, review the docs, or scope a managed deployment with Ferro Labs.

Deploy OSSRead DocsBook Demo