Best LLM Gateways and API Routers in 2026

OpenRouter now routes more than 200 trillion tokens a month across 400+ models from 70+ providers, serving over 10 million users, according to its own published numbers. That makes it arguably the most widely used LLM gateway (or API router, the two terms describe the same thing) in production today, and it's not a niche developer convenience anymore, it's critical infrastructure for a huge share of production LLM traffic. And 2026 has been a rough year to be an independent company in this space: Portkey was acquired by Palo Alto Networks in May, Helicone was folded into Mintlify in March and pushed into maintenance mode, Martian quietly stopped talking about routing altogether, and Not Diamond narrowed its entire pitch down to routing for coding agents. Four of the best-known LLM gateway and API router startups changed identity in the same twelve months.

If you're building anything on top of OpenAI, Anthropic, or Google's APIs and haven't put a gateway or API router in front of them yet, this is the year to understand why almost everyone eventually does. This guide covers what an LLM gateway actually does, the tools worth using in mid-2026, real pricing, working code, and the consolidation story behind the headlines.

What Is an LLM Gateway (or API Router)?

An LLM gateway (also called an AI gateway or API router) is a proxy layer that sits between your application and the model providers you call. Instead of your code talking directly to api.openai.com or api.anthropic.com, it talks to the gateway, which forwards the request, normalizes the response, and handles everything around the actual model call.

Concretely, a gateway typically gives you:

• A single, OpenAI-compatible API format for every provider, so switching from GPT-5 to Claude Sonnet 4.5 to a Llama model on Groq is a config change, not a rewrite.
• Automatic fallback and retries when a provider has an outage or returns a rate-limit error, so a single upstream incident doesn't take your product down with it.
• Centralized credentials. Your application talks to the gateway with one internal key; the gateway holds the real OpenAI, Anthropic, and Google keys and never exposes them to individual services.
• Caching, exact-match or semantic, so repeated or near-duplicate prompts do not hit the model API (and the bill) twice.
• Budgets, rate limits, and per-team cost attribution, because "how much did the support bot cost us last week" is a question someone will eventually ask you.

None of this is exotic anymore. It's the same reasoning that led teams to put a load balancer in front of their web servers instead of exposing them directly, just applied to model APIs.

Why Teams Actually Reach for One

The moment you have more than one model in production, or more than one team calling the same provider, a hardcoded API call starts to hurt. A few concrete scenarios that push teams toward a gateway:

Your app calls gpt-4o directly in twelve different services. OpenAI has a rough hour, latencies spike, and every one of those services starts timing out at the same time, because there's no shared fallback logic anywhere.

A support engineer needs to know why the AI feature's bill tripled last month. Without a gateway, that means grepping logs across services. With one, it's a dashboard filtered by API key or route.

A new intern's laptop somehow has a .env file with a production Anthropic key in it, because someone pasted it into three different repos to get a feature working. A gateway means every service uses a scoped virtual key that can be revoked without touching the real credential.

Quick Comparison

OpenRouter

OpenRouter is the closest thing the market has to a default choice for hitting many different models through one API. Per its own about page, it currently sits at 400+ models across 70+ providers, over 10 million users, and 200+ trillion tokens routed monthly.

The pricing model is straightforward: OpenRouter doesn't mark up per-token inference pricing, you pay the same rate the underlying provider charges. Revenue instead comes from a 5.5% fee (minimum $0.80) when you top up credits by card, 5% for crypto payments, and a 5% fee on bring-your-own-key requests once you exceed the first million free monthly requests on your own keys. Enterprise customers can negotiate lower rates with direct invoicing.

The API is a drop-in OpenAI-compatible endpoint. Model names use a provider/model slug, for example anthropic/claude-sonnet-4.5, which OpenRouter's live model page lists at $3 per million input tokens and $15 per million output tokens with a 1M-token context window.

Bash
Copy
curl https://openrouter.ai/api/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
  -d '{
    "model": "anthropic/claude-sonnet-4.5",
    "messages": [{"role": "user", "content": "Summarize the CAP theorem in two sentences."}]
  }'

Or with the official OpenAI Python SDK, changing only the base URL:

Python
Copy
from openai import OpenAI

client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key="<OPENROUTER_API_KEY>",
)

completion = client.chat.completions.create(
    model="anthropic/claude-sonnet-4.5",
    messages=[{"role": "user", "content": "Summarize the CAP theorem in two sentences."}],
)
print(completion.choices[0].message.content)

OpenRouter raised a $113 million Series B in May 2026, led by CapitalG (Alphabet's growth fund) with participation from Nvidia's NVentures, putting the company at roughly a $1.3 billion valuation and around $174 million raised in total since its 2025 Series A. It's the best default choice if you want the broadest model catalog with zero infrastructure to run yourself, and you're comfortable with a hosted, closed-source proxy sitting between you and every model call.

LiteLLM

LiteLLM takes the opposite approach: it's fully open source, maintained by BerriAI (a Y Combinator W23 company founded by Krrish Dholakia and Ishaan Jaffer), and designed to be run on your own infrastructure. The GitHub repository sits at roughly 55,000 stars as of mid-2026, and it unifies over 100 LLM APIs, including OpenAI, Anthropic, Azure OpenAI, Bedrock, Vertex AI, Groq, and self-hosted vLLM endpoints, behind a single OpenAI-compatible format.

You can use it two ways: as a Python SDK you import directly, or as a standalone proxy server that your whole team points requests at. The proxy is the more common production pattern, since it centralizes credentials and lets non-Python services (a Go backend, a Node service) get the same unified interface over plain HTTP.

Starting the proxy against a single model takes one command:

Bash
Copy
litellm --model huggingface/bigcode/starcoder
# Proxy now running on http://0.0.0.0:4000

For multiple providers behind one endpoint, define a config file:

yaml
Copy
model_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: azure/<your-deployment-name>
      api_base: <your-azure-api-endpoint>
      api_key: <your-azure-api-key>

litellm --config your_config.yaml

Then call it exactly like the OpenAI API, because that's exactly what it looks like from the outside:

Bash
Copy
curl --location 'http://0.0.0.0:4000/chat/completions' \
  --header 'Content-Type: application/json' \
  --data '{"model": "gpt-3.5-turbo", "messages": [{"role": "user", "content": "what llm are you"}]}'

Python
Copy
import openai

client = openai.OpenAI(api_key="anything", base_url="http://0.0.0.0:4000")
response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "this is a test request, write a short poem"}],
)

A real security note, not a hypothetical one. In mid-2026, security researchers disclosed CVE-2026-42271, a command-injection flaw (CVSS 8.7) in LiteLLM's MCP test endpoints (/mcp-rest/test/connection and /mcp-rest/test/tools/list), affecting versions 1.74.2 through 1.83.6. Any authenticated proxy API-key holder could execute arbitrary commands on the host, and when chained with a separate Starlette host-header validation bypass (CVE-2026-48710), it escalated to unauthenticated remote code execution. It was added to CISA's Known Exploited Vulnerabilities catalog with evidence of active exploitation in the wild, and fixed in LiteLLM 1.83.7 and later. If you self-host LiteLLM's proxy, this is a hard reminder to stay current on patch versions and to not expose the proxy's admin or test endpoints to the open internet.

LiteLLM's enterprise tier (SSO, RBAC, audit logs, SLA support) is sold with custom, negotiated pricing rather than a published price list, so treat any specific number you see quoted for it as unverified until you get a quote directly.

Portkey

Portkey built its reputation on pairing a fast open-source gateway with a genuinely useful observability and guardrails layer on top: caching, load balancing across providers, virtual keys, and content guardrails, with an open-source gateway repo that has around 12,500 GitHub stars and claims routing support for 1,600+ models with 50+ built-in guardrails.

The pricing ladder, per Portkey's own pricing page, runs: the self-hosted open-source gateway is free with no log limits; a hosted "Developer" tier is free for up to 10,000 logs a month with three-day retention (explicitly marked not for production use); "Production" is $49 a month for 100,000 logs (then $9 per additional 100,000), with 30-day retention, RBAC, and semantic caching; and "Enterprise" is custom-priced for 10 million-plus logs a month, with SSO, VPC or air-gapped deployment, and SOC 2, ISO 27001, GDPR, and HIPAA compliance support.

The important update for 2026: Palo Alto Networks announced its intent to acquire Portkey on April 30, 2026, and closed the deal on May 29, framed publicly as securing "the rise of AI agents," following PANW's earlier Protect AI and CyberArk acquisitions. This is a security company acquiring an AI-agent control layer, not a routing vendor being bought by another routing vendor. The open-source gateway is still live and usable today, but there's no public statement yet on what its long-term roadmap looks like inside a much larger security company. If you're betting on Portkey for a multi-year platform, that's worth watching rather than assuming nothing will change.

Cloudflare AI Gateway

Cloudflare AI Gateway is the easiest one to try if you are already running anything on Cloudflare. It is free on every Cloudflare plan and gives you analytics, request logging, caching, rate limiting, and automatic fallback/retry across providers including Workers AI, OpenAI, Anthropic, Google Gemini, and Replicate, without asking you to run or pay for a separate service.

The main paid add-on is Logpush, for streaming gateway logs to an external SIEM or data warehouse, priced at $0.05 per million records once you're past 10 million records a month (and it requires a Workers Paid plan). In 2026, Cloudflare also rolled out "Unified Billing," which lets you pay for third-party model usage (OpenAI, for instance) directly through your existing Cloudflare invoice, for a 5% convenience fee on top, instead of managing a separate billing relationship with every provider.

It's not trying to be the most feature-dense gateway on this list. It's trying to be the gateway you turn on in five minutes because you're already inside the Cloudflare ecosystem, and for a lot of teams building on Workers, that's exactly the right tradeoff.

Worth Knowing About: Kong AI Gateway and TrueFoundry

If you are an enterprise already running Kong for API management, Kong AI Gateway extends the same platform with semantic caching, prompt compression, PII sanitization and guardrails, and governance for MCP and agent-to-agent traffic. It makes the most sense if Kong is already your API gateway of record and you want AI traffic under the same governance model rather than a second, separate system.

TrueFoundry's AI Gateway targets a similar enterprise audience with a unified interface across 250+ models, RBAC, budget controls, and deployment options ranging from SaaS to fully on-prem. Its published pricing runs a free Developer tier (50,000 requests a month, 3 users), a $499/month Pro tier, $2,999/month Pro Plus, and custom Enterprise pricing. The company has raised roughly $21 million combined, across an earlier Peak XV-led seed and an Intel Capital-led Series A that Eniac Ventures also joined.

The 2026 Shakeout

It's worth naming this pattern directly, because it changes how you should think about which gateway is safe to build on long-term. In the space of about five months in 2026: Portkey was acquired by Palo Alto Networks. Helicone, which had processed over 14 trillion tokens for roughly 16,000 organizations, was acquired by the documentation platform Mintlify in March and, by Mintlify's own announcement, moved into maintenance mode with only security patches, bug fixes, and new model support continuing, no active feature development. Martian, which originally marketed itself as the inventor of "the first LLM router," now describes itself purely as an AI interpretability research company with named projects like ARES and K-Steering, and no longer positions a router as its product. Not Diamond, another early model-routing startup, narrowed its entire positioning from general-purpose routing down to routing specifically for coding agents.

None of this means the category is dying, quite the opposite: it means the category matured fast enough that bigger platforms wanted to own a piece of it, and some of the earliest, narrowest bets (routing for routing's sake) turned out not to be defensible as standalone businesses. For you as a developer, the practical takeaway is to prefer either a genuinely open-source option you can self-host and fork if needed (LiteLLM, Portkey's OSS gateway), or a hosted option backed by a large platform with an obvious reason to keep investing in it (OpenRouter, Cloudflare), over a smaller independent product whose roadmap could change hands again next quarter.

How to Set Up Your First LLM Gateway

If you want the fastest possible path to trying this yourself, here is a concrete, minimal setup using LiteLLM's proxy, since it requires no account or hosted service to get started.

1. Install LiteLLM. Run pip install litellm[proxy] in a virtual environment. This installs both the SDK and the proxy server CLI.
2. Set your provider credentials as environment variables, for example export OPENAI_API_KEY=sk-... and export ANTHROPIC_API_KEY=sk-ant-.... The gateway holds these; your application code never sees them.
3. Write a config.yaml listing every model you want available behind the gateway, mapping a friendly model_name to the real provider and deployment, as shown in the LiteLLM section above.
4. Start the proxy with litellm --config config.yaml. By default it listens on http://0.0.0.0:4000.
5. Point your application at the proxy by changing only the base_url in your existing OpenAI SDK client to http://localhost:4000, keeping the rest of your code unchanged.
6. Verify it works with a plain curl request to /chat/completions, as shown earlier, before wiring it into your real application.
7. Add a fallback model in config.yaml (LiteLLM supports a fallbacks list per model) so a provider outage degrades gracefully instead of returning errors to your users.

From there, layer in what you actually need: virtual keys per team, budget alerts, semantic caching, or swapping the proxy for a hosted option like OpenRouter or Cloudflare AI Gateway once you know exactly which features matter for your traffic.

Which One Should You Actually Use?

Start with OpenRouter for the widest model catalog with nothing to host. Pick LiteLLM if you want full control and are fine self-hosting a proxy, just stay current on patch versions given its 2026 CVE history. Go with Portkey if guardrails and per-team observability matter more than model selection. And if you're already on Cloudflare Workers, Cloudflare AI Gateway is close to a free lunch.

Either way, the 2026 lesson holds: calling model APIs directly from a dozen services was never going to survive the second provider outage. A gateway, or an API router if you prefer that name, is now standard infrastructure, not a nice-to-have.

Related DevToolLab Tools

• AI Token Counter - estimate token usage and cost for GPT-5, Claude, Gemini, and Llama before you route a request through any gateway.
• API Key Validator - check that your OpenAI, Anthropic, or Google AI keys are valid before wiring them into a gateway config.
• cURL to Code Converter - turn the curl examples above into ready-to-run fetch, Axios, or Python code for your own app.
• Rate Limit Header Analyzer - inspect X-RateLimit and Retry-After headers coming back through your gateway to debug throttling.

Related Guides

• Best LLM Observability Tools 2026 - tracing, evals, and cost tracking for whatever you route through a gateway
• Prompt Caching Guide - how caching works at the provider level, which pairs directly with a gateway's own cache layer
• Top Local LLM Tools and Models - running models yourself instead of routing to a hosted provider