LiteLLM review, pricing and verdict

Last updated: 2026-10-23

LiteLLM is an open-source AI gateway and Python SDK from BerriAI that routes requests to more than 100 LLM providers through one OpenAI-compatible API. Its GitHub repository lists over 61,000 stars. The core is MIT licensed and runs on your own infrastructure, adding virtual keys, budgets, fallbacks and an MCP gateway.

About LiteLLM

BerriAI, a Y Combinator Winter 2023 company, maintains LiteLLM as both a Python library and a self-hosted server. Its job is narrow and useful: your application speaks the OpenAI request format, and LiteLLM translates that call to whichever provider you point it at, from Anthropic and Azure to Bedrock and Vertex AI. The vendor lists more than 100 supported providers, and its GitHub repository has passed 61,000 stars.

There are two ways to run it. The Python SDK needs no server: you change a model string such as anthropic/claude-sonnet-5-5 and keep the same response shape. The gateway is a self-hosted FastAPI server, started with a one-line Docker quickstart that also brings up Postgres and an admin UI. Apps then use their own virtual keys, each with its own budgets, rate limits and model access, and the upstream credentials never reach client apps. The gateway serves the chat completions, responses and Anthropic messages routes, so existing OpenAI and Anthropic client libraries only need a new base URL.

The gateway also acts as a control point for tooling. Its MCP gateway puts many Model Context Protocol servers behind one endpoint with per-key access and logged tool calls, and the vendor says you can point coding agents such as Claude Code at it to track and cap their spend. Routing and fallbacks let a request move to a second provider during an outage, which matters if you depend on Anthropic or OpenAI models in production.

It fits platform and backend teams who want one governed entry point to many models. If you would rather not run any infrastructure, a hosted aggregator such as OpenRouter, CometAPI or AIML API is the simpler route, and teams that mainly run local models may pair a gateway with Ollama. Developers building chains and agents in code will often call LiteLLM underneath a framework such as LangChain. The core is MIT licensed and free; Enterprise adds single sign-on, audit logs and support under a commercial license.

Pricing

The open-source gateway and SDK are free to self-host ($0), with the MIT-licensed core including virtual keys, budgets, spend tracking and fallbacks. Enterprise is annual, sized to your gateway request capacity and support needs, and quoted through sales with no public price. Provider token costs are billed by each provider, not by LiteLLM.

ai/pricing on 2026-10-23.

Key Features

• OpenAI-format API for every provider: Calls to Anthropic, Azure, Bedrock, Vertex AI, Cohere and others return the same OpenAI response shape, so switching providers is a model-string change.
• Virtual keys with budgets: Each app, user or team gets its own key with a spend cap, rate limits and a model allow-list, while the real provider keys stay on your server.
• Routing and fallbacks: A router spreads load across deployments and retries on a second provider when the first one errors or goes down.
• MCP gateway: One endpoint serves every registered Model Context Protocol server over Streamable HTTP, SSE or stdio, with access granted per key, team or organization and each tool call logged.
• Spend tracking and admin UI: Usage is attributed by key, user, team and tag in a built-in admin UI, with Prometheus metrics and request logging backed by Postgres.
• Guardrails and signed images: Guardrails run at the gateway, and the container images published to GHCR are signed with cosign so you can verify them before deploying.

Pros

• Self-hosting keeps prompts, responses and provider keys inside your own network, which is the usual reason regulated teams pick it over a hosted router.
• The free MIT core already includes the parts most teams actually need, namely virtual keys, budgets, spend tracking and the admin UI, so governance does not sit behind a paywall.
• It runs as either a plain Python library or a full gateway, so a prototype can start without a server and grow into shared infrastructure without changing client code.
• Heavy adoption means provider quirks and new models are usually covered quickly, and the vendor names Netflix, Okta, Ramp and NASA among production users.

Cons

• The gateway needs Docker or an equivalent, plus a Postgres database, so it is real infrastructure that someone on your team has to patch, monitor and scale.
• Single sign-on, SCIM, audit logs, secret-manager integration and support SLAs are Enterprise features, and Enterprise pricing is not published.
• A single script that calls one provider gains little from a gateway, a point the vendor makes itself in its own guidance.

Visit LiteLLM Official Website