routing
Circuit Breaker
Self-healing provider isolation
Per-provider circuit breakers open after a configurable number of consecutive failures, removing a failing provider from rotation before it cascades. The breaker stays open for a cool-down, then half-opens and allows one probe request, closing again once it succeeds.
What you get
Everything in the box
- Per-provider independent breakers
- Opens on consecutive failures (failure_threshold)
- Half-open probe after a timeout cool-down
- Closes after success_threshold successes
- Open targets excluded from every routing strategy
- Self-heals with no manual intervention
targets:
- virtual_key: openai
circuit_breaker:
failure_threshold: 5 # consecutive failures before opening
success_threshold: 2 # successes in half-open before closing
timeout: "30s" # how long the breaker stays open
- virtual_key: anthropic
When to use it
When to reach for this
Stop a flaky provider degrading latency
When a provider starts timing out, the breaker opens and the gateway routes around it instantly instead of waiting for each request to fail.
Ride out a provider incident
During an outage the breaker keeps the provider isolated and automatically probes for recovery, so traffic returns on its own once the provider is healthy.
Protect a weighted split
In load-balanced routing, a breaker stops a failing target from receiving its weighted share until it recovers, keeping the healthy providers carrying the load.
Alert on tripped providers
Watch gateway_circuit_breaker_state in Prometheus and alert when any provider's breaker opens, turning isolation into an early-warning signal.
FAQ
Frequently asked questions
When does a circuit breaker open?
After failure_threshold consecutive failures for that provider. The breaker then stays open for the configured timeout before testing recovery.
How does a breaker recover?
Once the timeout elapses it enters a half-open state and allows one probe request. After success_threshold successes it closes and full traffic resumes; if the probe fails it re-opens for another cool-down.
Do circuit breakers work with every routing strategy?
Yes. A target whose breaker is open is excluded from selection by all routing strategies — single, fallback, load balance, least-latency, cost-optimized, conditional, content-based, and A/B test.
Are breakers shared across providers?
No. Each target has its own independent breaker, so isolating one failing provider never affects the others.
Can I see when a breaker is open?
Yes. The Prometheus metric gateway_circuit_breaker_state reports each provider's state (0 closed, 1 open, 2 half-open), so you can dashboard and alert on it.
Related features
Feature deep dives
Smart Fallbacks & Retries
Fallback routing tries your providers in priority order: on an error or a retryable status code, the gateway moves to the next target with exponential backoff. Pair it with per-target circuit breakers and a single failing provider never takes your AI features down.
8 Routing Strategies
Choose from eight production-hardened routing strategies shipped out of the box. Switch between them with a single config key — no code changes required.
Real-time Observability
Four observability layers ship in the gateway: Prometheus metrics at /metrics, opt-in OpenTelemetry tracing, structured JSON logs, and a deep /health endpoint. The OTel trace ID, the log trace_id, and the X-Request-ID header are all the same value, so you can jump from a log line straight into your tracing backend.
Keep exploring
Solutions & docs
LLM Observability, Logs, and Tracing
Ferro Labs turns the gateway into the source of truth for LLM observability across request logs, traces, sessions, cost, latency, and provider errors.
Reliable LLM Routing, Fallbacks, and SLOs
Ferro Labs keeps production AI traffic moving with fallback routing, provider failover, retries, routing strategies, circuit breakers, SLO tracking, and incident workflows.
Read the docs
Full configuration reference and examples on docs.ferrolabs.ai