LiteLLM
Open-source LLM proxy and unified API that lets you call 100+ LLM providers using the OpenAI format, with 50k+ GitHub stars.
LiteLLM is a lightweight Python library and hosted proxy server that translates LLM calls to a unified OpenAI-compatible API, supporting 100+ providers including Anthropic, Google, Cohere, and Azure. It handles load balancing, fallbacks, rate limiting, spend tracking, and logging across multiple LLM deployments from a single config file. Used by engineering teams at Airbnb, Instacart, and other enterprise organizations, LiteLLM processes millions of API calls daily. The project crossed 50,000 GitHub stars in 2025 and offers a managed proxy service alongside the open-source library for teams that want zero-ops deployment.
Key Features
- Call 100+ LLMs via unified OpenAI-compatible API with a single code interface
- Load balancing and automatic fallbacks across providers and models
- Spend tracking and budget limits per API key or team deployment
- Built-in support for streaming, function calling, and vision models across providers
- Prometheus metrics and logging integrations for Langfuse, Helicone, and Langsmith
- Proxy server mode for deploying a team-wide LLM gateway with access controls
Use Cases
- Engineering teams standardizing on a single API client across multiple LLM providers
- MLOps teams enforcing spend limits and routing policies across LLM deployments
- Developers testing prompt compatibility across GPT-4o, Claude, and Gemini simultaneously
- Enterprises needing an internal LLM gateway with audit logging and rate limiting
Pros
- 100+ providers in one library - switch models with a single line change
- Active maintainer with rapid releases and a responsive issue tracker
- Native load balancing, fallbacks, and retries built in with no extra configuration
Cons
- Proxy server mode adds infrastructure complexity for teams wanting zero-ops deployment
- Some provider-specific features require custom params outside the unified interface
- Self-hosted proxy needs a separate monitoring stack to match managed cloud alternatives
LiteLLM Alternatives
Explore similar tools and alternatives
Looking for alternatives to LiteLLM? Here are some similar tools you might like:
OpenRouter
Unified API that routes requests to 300+ AI models from OpenAI, Anthropic, Google, and Meta - with automatic fallback, cost controls, and model comparison.
Portkey
AI gateway that routes requests across 200+ LLMs with automatic fallbacks, load balancing, caching, and observability - YC W23, free tier available.
Together AI
Fast open-source LLM inference platform with 200+ models, fine-tuning, and the largest selection of serverless AI APIs outside OpenAI.
Ready to try LiteLLM?
Visit the official website to explore all features and get started with LiteLLM today.
Reviews
0 reviews for LiteLLM
Based on 0 reviews
Share your experience
Log in to write a review for LiteLLM
Ito
Only code review that runs your code. Provides runtime analysis with evidence (logs, video, screenshot) to show how code changes application actually work. Back-end, front-end, api, integration.
Cursor
AI-native code editor built on VS Code with built-in AI chat, autocomplete, and codebase understanding.
GitHub Copilot
AI pair programmer by GitHub/OpenAI that suggests code completions, functions, and entire files in your IDE.
Have an AI Tool?
List your AI tool for free, or go featured for top placement in your category - and reach thousands of potential users.
Submit Your Tool