observability
Real-time Observability
See everything, miss nothing
Four observability layers ship in the gateway: Prometheus metrics at /metrics, opt-in OpenTelemetry tracing, structured JSON logs, and a deep /health endpoint. The OTel trace ID, the log trace_id, and the X-Request-ID header are all the same value, so you can jump from a log line straight into your tracing backend.
Request volume
What you get
Everything in the box
- Prometheus /metrics endpoint (gateway_* metrics)
- Per-provider latency histograms
- Token & cost attribution per provider/model
- OpenTelemetry tracing over OTLP (opt-in)
- Structured JSON logs with a unified trace ID
- Deep /health endpoint with per-provider status
// Async event hook — emit to any backend
gw.AddHook(func(
ctx context.Context,
subject string,
data map[string]any,
) {
switch subject {
case aigateway.SubjectRequestCompleted:
metrics.Record(
data["latency_ms"],
data["cost_usd"],
data["tokens_out"],
)
case aigateway.SubjectRequestFailed:
alerts.Send(data["error"])
}
})
// Prometheus scrape: GET /metrics
// Deep health check: GET /health
When to use it
When to reach for this
Build a spend dashboard
Scrape gateway_request_cost_usd_total and the token counters into Grafana to watch estimated spend by provider and model in real time.
Debug a slow request end to end
Copy the X-Request-ID from a client error into your tracing backend and follow the unified trace ID through plugins, routing, and the provider call.
Alert on provider trouble
Alert on gateway_provider_errors_total and gateway_circuit_breaker_state so you hear about a degrading provider before users do.
Trace without leaking prompts
Run tracing at privacy_level metadata so spans carry attributes and redacted errors but never raw prompt or response content.
FAQ
Frequently asked questions
What observability does the gateway expose out of the box?
Four layers: Prometheus metrics at /metrics, opt-in OpenTelemetry tracing over OTLP, structured JSON logs on stdout, and a deep /health endpoint with per-provider status.
Does enabling tracing slow down the gateway?
No. Tracing is opt-in. Until you configure an OTLP endpoint the gateway uses a zero-allocation no-op tracer, so there is no overhead on the hot path.
How do I correlate a log line with a trace?
The OpenTelemetry trace_id, the structured-log trace_id, and the X-Request-ID response header are all the same value, so any one of them looks up the same request in your tracing backend.
Can I keep prompts out of my tracing backend?
Yes. The privacy_level key controls trace content: none omits content and errors, metadata (the default) sends metadata with redacted errors, and full includes raw error text for trusted environments.
What does the health endpoint report?
GET /health returns per-provider healthy/latency status and responds 200 while at least one provider is up, 503 only when every provider is down.
Related features
Feature deep dives
Smart Fallbacks & Retries
Fallback routing tries your providers in priority order: on an error or a retryable status code, the gateway moves to the next target with exponential backoff. Pair it with per-target circuit breakers and a single failing provider never takes your AI features down.
Circuit Breaker
Per-provider circuit breakers open after a configurable number of consecutive failures, removing a failing provider from rotation before it cascades. The breaker stays open for a cool-down, then half-opens and allows one probe request, closing again once it succeeds.
Built-in Plugins
Plugins extend the request pipeline at three lifecycle stages — before_request, after_request, and on_error. Six ship built in, covering content filtering, token limits, response caching, request logging, rate limiting, and spend control. Each is declared in config.yaml and can be toggled independently.
Keep exploring
Solutions & docs
Enterprise AI Governance and RBAC
Ferro Labs gives platform teams a single control plane for enterprise AI governance across providers, projects, teams, and API keys.
LLM Observability, Logs, and Tracing
Ferro Labs turns the gateway into the source of truth for LLM observability across request logs, traces, sessions, cost, latency, and provider errors.
AI Guardrails for PII and Prompt Injection
Ferro Labs lets platform and security teams apply AI guardrails at the gateway before risky prompts, responses, or tool flows spread across applications.
Read the docs
Full configuration reference and examples on docs.ferrolabs.ai