Overview

A drop-in gateway for AI coding agents and LLM applications. It speaks both the OpenAI and Anthropic APIs, understands what each request is doing, and sends it to the cheapest model that can handle it — with automatic failover, budgets, SSO and a full audit trail in front of every provider you use.

How it routes

An illustration of requests from different agents passing through the gateway.

Claude Code → AI Agent Gateway

Read src/config.ts

1. Understand the step
2. Pick the cheapest capable model
3. Deliver · fail over if needed

Routing log

Why traditional tools fall short

Agents overpay for easy work

Reading files, searching code and summarising run as dozens of calls per task — and each one is billed like the hardest step.

One premium model for everything

Coding agents send every step to the same frontier model, whatever the step is, because nothing in the path decides otherwise.

No one sees what is sent

Prompts and spend are spread across teams, personal keys and vendors, with no budget, no owner and no audit trail.

The ClarityNxt advantage

One URL change, every agent routed

Point Claude Code, Codex CLI, the OpenAI and Anthropic SDKs, LangChain or your own agents at the gateway by changing a base URL. Behind it, any OpenAI- or Anthropic-compatible endpoint works — OpenAI, Anthropic, Azure OpenAI, Gemini, OpenRouter, or models you host yourself with vLLM or Ollama.

See AI Agent Gateway in action

Screens from the product, shown with sample data. Click any screen to open it full size.

AI Agent Gateway dashboard: 30-day spend, savings versus the premium model, requests, tokens, error rate, latency and daily spend by routing tier — Dashboard — spend, savings against the premium model, and daily spend by routing tier.

Savings report comparing what requests would have cost on the most expensive model with what routing actually cost, per model — Savings, per model, against sending everything to your most expensive model.

Model catalog with providers, routing tiers, per-million-token prices, context windows and status — Model catalog — prices, tiers and capabilities, editable from the UI.

Audit log listing each request with user, team, model, tier, tokens, cost, latency and status — Audit log — who, which team, which model, tokens and cost for every request.

Key capabilities

Intent-aware routing

Each request is understood from what the person asked and what the agent is doing — reading, searching, configuring, designing or changing code — and served by the cheapest model with the tools, context window and image support that step needs.

Steady within a task

Routing holds for the length of a task, so a coding job stays on one strong model and the provider's prompt cache isn't thrown away mid-task. Sub-agents and lookups are routed on their own.

Automatic failover

Rate limits, quota errors, outages and context-length failures retry on the next capable model — never a weaker one — before the first byte reaches the agent, so it works for streaming too.

Native OpenAI and Anthropic APIs

When client and provider speak the same format, requests pass through untouched. When they don't, tool calls, images and streaming are translated in both directions.

Credits and budgets

USD credit limits for every user and team, monthly or one-off, plus daily and monthly token limits. Exhausted budgets get a clear error agents don't blindly retry.

SSO and self-serve keys

Sign in with Okta, Entra ID, Google Workspace, Auth0, Keycloak or Ping over OpenID Connect. IdP groups map to teams and admin roles, and developers create their own API keys.

Audit and chargeback

Every request records user, team, model, routing reason, tokens and cost. Export CSV for chargeback, store prompts as a hash instead of text if you prefer, and review every admin change.

Model catalog in the UI

Add models and providers, set prices and capabilities, and test the connection before saving — changes apply on the next request. Pasted provider keys are encrypted with AES-256-GCM and never returned.

Transparent and observable

Every response states the tier, model and reason it was routed. Spend, savings, latency and failovers are on the dashboard and exported as Prometheus metrics and OpenTelemetry traces.

Business impact

Pay frontier prices only for the steps that need a frontier model
See the savings: the dashboard compares actual spend with sending everything to the priciest model
One place to see and control AI spend across teams, keys and vendors
Fewer failed agent runs — provider outages and rate limits fail over automatically
No lock-in: add or switch providers, including self-hosted models, by configuration

How it works

Step 1
Change one URL
Point your agents and SDKs at the gateway instead of the provider. No code changes.
Step 2
Understand the step
The gateway reads what the person asked and what the agent is doing right now.
Step 3
Pick the cheapest capable model
Only models with the tools, context window and capabilities the step needs are considered.
Step 4
Deliver and fail over
The response streams back in the agent's own API format; a failing provider escalates to a stronger model.

Delivery

Runs in your environment as a single service or container, so prompts and provider keys stay with you. Its OpenTelemetry output plugs straight into ClarityNxt observability.