Skip to content

guardrails

6 Built-in plugins

Built-in Plugins

Safety and control baked into every request

Plugins extend the request pipeline at three lifecycle stages — before_request, after_request, and on_error. Six ship built in, covering content filtering, token limits, response caching, request logging, rate limiting, and spend control. Each is declared in config.yaml and can be toggled independently.

Policy evaluation · single request

demo data
POST /v1/chat/completions · claude-sonnet-4-6trace 8f42c1d9
stage
ms
verdict
  • auth.api_key0.08mspass
  • tenant.rate_limit0.11mspass
  • budget.check0.19mspass
  • guardrails.word_filter0.21mspass
  • guardrails.pii_scan0.42mspass
  • cache.semantic_lookup1.1msmiss
  • route.select0.31msselected
  • provider.invoke838ms200
gateway added latency: 2.42ms · 0.3% of totalwall time 840.42ms

What you get

Everything in the box

  • word-filter — block configured words or phrases
  • max-token — cap tokens, messages, and input length
  • response-cache — exact-match in-memory caching
  • request-logger — structured logs, optional persistence
  • rate-limit — token-bucket request throttling
  • budget — cumulative USD spend caps per API key
Full configuration in the docs
config.yaml
plugins:
    - name: word-filter
      type: guardrail
      stage: before_request
      enabled: true
      config:
        blocked_words: ["confidential", "password", "secret"]
        case_sensitive: false

    - name: max-token
      type: guardrail
      stage: before_request
      enabled: true
      config:
        max_tokens: 4096
        max_messages: 50
        max_input_length: 20000

When to use it

When to reach for this

Keep secrets out of prompts

Use word-filter to block obvious credential patterns and banned terms before a request is forwarded, so they never reach a provider.

Cap runaway requests

Use max-token to bound output tokens, message count, and input length, protecting both latency and spend from oversized prompts.

Put a ceiling on per-key spend

Register the budget plugin at both stages with a shared store_id and a spend_limit_usd so a single API key can't blow through its allowance.

Build an audit trail

Enable request-logger with persistence to SQLite or Postgres and query per-request history through the admin API.

FAQ

Frequently asked questions

At what points in a request can plugins run?

Three lifecycle stages: before_request (before the call is forwarded), after_request (once the response returns), and on_error (when the provider returns an error).

Which plugins are built in?

Six: word-filter, max-token, response-cache, request-logger, rate-limit, and budget. Each is declared in config.yaml and can be enabled or disabled independently.

In what order do plugins execute?

In the order they appear in config.yaml. Within a stage, a plugin that sets reject stops the chain and returns an error, while skip bypasses only that plugin.

How does the budget plugin enforce spend limits?

It registers at both before_request and after_request sharing a store_id, tracking cumulative USD per API key. When a key reaches spend_limit_usd, further requests get HTTP 429. Spend is in-memory and resets on restart.

Can I add guardrails beyond the built-in six?

Yes. The same plugin mechanism powers further guardrails such as PII redaction, prompt-injection scoring, secret scanning, JSON-schema validation, and custom regex rules — declared in config.yaml at the appropriate stage.

Related features

Feature deep dives

Rate Limiting & Access Control

Rate limiting works in three independent layers: per-IP HTTP middleware, a global token bucket applied before traffic reaches a provider, and per-API-key and per-user limits. Checks run in order — global, then per-key, then per-user — and the first exceeded limiter rejects the request with a distinct reason string.

Real-time Observability

Four observability layers ship in the gateway: Prometheus metrics at /metrics, opt-in OpenTelemetry tracing, structured JSON logs, and a deep /health endpoint. The OTel trace ID, the log trace_id, and the X-Request-ID header are all the same value, so you can jump from a log line straight into your tracing backend.

MCP Server (Model Context Protocol)

Add MCP tool servers to your config and the gateway injects their tools into chat-completion requests, then runs the full agentic loop — tool call, execution, result injection — inside the gateway. Your client sends a standard request and receives the final text answer, streaming included, without implementing tool-calling logic.

Previous

Semantic Caching

Next

Real-time Observability

Ready to deploy?

Stand up your AI gateway in minutes — open source, Apache 2.0, no credit card.