AI Gateway
The AI Gateway is a component of Stacklok Enterprise.
The AI Gateway is a self-hosted control point between AI clients and large language model (LLM) providers. Platform and security teams use it to enforce identity, budget, routing, data protection, and audit policies for model traffic.
The AI Gateway governs model access. The Connector Gateway governs MCP tool access.
What you can do
- Set cost budgets for users and groups.
- Route model names across providers with weighting and failover.
- Screen requests and responses for prompt injection, payment card data, and personally identifiable information (PII).
- Record structured audit events and forward them to a security information and event management (SIEM) system.
- Associate requests with identities while keeping provider credentials in the gateway.
How configuration works
Apply AIGateway and AIPolicy custom resources to configure providers,
routing, resilience, and screening. The AI Gateway operator reconciles those
resources into running infrastructure.
Manage budgets in the console or use the management API for automation and bulk import. See Manage budgets and Budgets and pricing.
Before you start
Install the AI Gateway before applying the examples in this section. See Configure the AI Gateway.
Examples on these pages use -n <NAMESPACE>, where <NAMESPACE> is the
namespace your AI Gateway is installed into.
Contents
AI Gateway
Govern, secure, and control access to large language model providers from one self-hosted enterprise gateway.
Model providers
Connect OpenAI, Anthropic, AWS Bedrock, Azure OpenAI, Google Vertex AI, and OpenAI-compatible providers to the AI Gateway, and rotate their credentials.
Model routing
Map model names onto providers with a default route, weighted load balancing, priority failover, retries, and timeouts on the AI Gateway.
Budgets and pricing
Cap LLM spend per user or group in USD, publish a pricing catalog, and understand why an unpriced model is refused at the gateway.
Budgets
Create and edit group and user budgets in the console, set organization defaults, and control which group budget is charged first.
Prompt injection screening
Screen inbound prompts for injection attacks with AWS Bedrock Guardrails, including rollout from monitor to enforce, IAM setup, and cost.
PCI/PII controls
Scan prompts and responses for credit cards, national identifiers, and other sensitive data, and block, redact, or log what is found.
Audit logs
Enable structured audit events for model requests, detections, and key lifecycle, then ship them to Splunk, Elastic, Kafka, or Loki.