caching
MCP Server (Model Context Protocol)
Native tool-use for agents and IDEs
Add MCP tool servers to your config and the gateway injects their tools into chat-completion requests, then runs the full agentic loop — tool call, execution, result injection — inside the gateway. Your client sends a standard request and receives the final text answer, streaming included, without implementing tool-calling logic.
What you get
Everything in the box
- Streamable HTTP MCP transport
- Full agentic loop runs inside the gateway
- allowed_tools whitelist per server
- Configurable max_call_depth per server
- Header auth with ${ENV_VAR} interpolation
- Streaming requests supported (stream: true)
mcp_servers:
- name: filesystem
url: "https://www.ferrolabs.ai/mcp"
allowed_tools: [read_file, list_directory, search_files]
max_call_depth: 4
timeout_seconds: 10
- name: database
url: "https://mcp-db.internal/mcp"
headers:
Authorization: "Bearer ${MCP_DB_TOKEN}"
allowed_tools: [query_readonly, list_tables]
max_call_depth: 5
When to use it
When to reach for this
Give every app governed tool-use
Wire MCP servers once at the gateway and every application gets agentic tool-calling without implementing the protocol, under the same access controls.
Read-only database access for agents
Expose only query_readonly and list_tables via allowed_tools so an agent can answer questions from a database without any ability to write.
Let an IDE assistant read the workspace
Connect a filesystem MCP server with read/list/search tools so a coding assistant can ground answers in real files through the gateway.
Stream agentic answers to clients
Use stream: true so users see the final answer arrive as SSE while the gateway resolves the tool loop behind the scenes.
FAQ
Frequently asked questions
What is an MCP gateway?
It is a gateway that connects to Model Context Protocol tool servers, injects their tools into model requests, and runs the agentic tool-calling loop itself — so clients get a final answer without implementing tool-calling logic.
Does my client have to handle tool calls?
No. The agentic loop runs inside the gateway. Your client sends a standard chat-completion request and receives the final text answer; the intermediate tool calls are transparent, including for streaming requests.
How do I limit which tools a model can call?
Set allowed_tools on the MCP server entry. Only the listed tool names are injected and callable, so write or delete tools can be left out entirely.
How are MCP server credentials kept secure?
Through the headers field with ${ENV_VAR} interpolation, so tokens come from the environment and are never hardcoded in config files.
How is the agentic loop bounded?
By max_call_depth (default 5), which caps the number of tool-call rounds per request, and timeout_seconds per call. Keeping max_call_depth conservative is part of the recommended security posture.
Related features
Feature deep dives
Built-in Plugins
Plugins extend the request pipeline at three lifecycle stages — before_request, after_request, and on_error. Six ship built in, covering content filtering, token limits, response caching, request logging, rate limiting, and spend control. Each is declared in config.yaml and can be toggled independently.
Real-time Observability
Four observability layers ship in the gateway: Prometheus metrics at /metrics, opt-in OpenTelemetry tracing, structured JSON logs, and a deep /health endpoint. The OTel trace ID, the log trace_id, and the X-Request-ID header are all the same value, so you can jump from a log line straight into your tracing backend.
8 Routing Strategies
Choose from eight production-hardened routing strategies shipped out of the box. Switch between them with a single config key — no code changes required.