LiteLLM
LiteLLM is an open-source unified invocation library and proxy middleware for large language models. It provides a standardized API interface compatible with the OpenAI format and supports routing requests to more than 100 LLM providers, including OpenAI, Anthropic, Azure, and locally deployed models. Core features include multi-model load balancing, request format conversion, cost tracking, and access control, enabling developers to integrate and manage LLM services from different sources in a unified manner.
Core Facts
LiteLLM is an open-source unified LLM access layer project whose Proxy mode can uniformly convert API requests in different formats into OpenAI-compatible format.
LiteLLM, Portkey, and Kong AI Gateway are typical open-source or commercial implementations of AI gateways.
Open-source projects such as LiteLLM and OneAPI also provide similar unified interface layers.
Timeline (last 90 days)
Internet-facing PHP applications, LiteLLM instances exposing API keys, and GitLab instances hosting core code should be treated as high priority in vulnerability remediation prioritization.
LiteLLM supports passing user input through to upstream models; if the input is not filtered, this may be exploited for prompt injection or resource abuse.
Security recommendations for LiteLLM instances: enable authentication and rate limiting, store keys in a key management service rather than in plaintext environment variables, and rotate keys regularly to reduce the impact of leaks.
The Vercel AI SDK is not a team-level gateway; if you need a unified gateway with per-team key and budget allocation, LiteLLM is recommended.
The LiteLLM release pipeline was attacked, with version 1.107.0 implanted with malware that steals credentials and loads remote code; the malicious package and affected images have been taken down or replaced, and customers using the private enterprise distribution are unaffected.
Wiz disclosed that a malicious version, LiteLLM 1.107.0, was uploaded to PyPI, with the related GHCR and Docker Hub image tags also implanted, and it was subsequently quarantined and removed; the most recent secure version is 1.106.2.
LiteLLM is an AI tool with 3.4 million daily downloads, and the post claims it was implanted with a password stealer.
The security requirements for gateway components like LiteLLM should be raised to the same level as databases and key management services.
For machines and images confirmed to have installed the malicious version, it is recommended to rebuild directly rather than patch.
Version 1.107.0 of the LiteLLM PyPI package was confirmed to contain credential-stealing code and backdoor logic less than two hours after release, and was subsequently taken down.
40 more timeline events
All Facts (20)
LiteLLM is an open-source unified LLM access layer project whose Proxy mode can uniformly convert API requests in different formats into OpenAI-compatible format.
80%VerifiedLiteLLM, Portkey, and Kong AI Gateway are typical open-source or commercial implementations of AI gateways.
75%VerifiedOpen-source projects such as LiteLLM and OneAPI also provide similar unified interface layers.
75%VerifiedLiteLLM是一个开源的LLM API统一代理层，将数十家模型提供商的API统一封装为OpenAI兼容格式
75%VerifiedSimilar open-source LLM gateway solutions in the industry include LiteLLM, OneAPI, and PortKey.
75%VerifiedProjects such as OpenRouter and LiteLLM are also exploring multi-model orchestration solutions.
75%VerifiedRouteLLM and LiteLLM are open-source frameworks for implementing LLM intelligent routing functionality.
70%VerifiedOpenRouter is an AI model aggregation and routing platform that allows developers to access hundreds of models from multiple vendors through a unified OpenAI-compatible API interface.
70%VerifiedIn testing, the p99 latency per 160-millisecond tick was 147.4–147.5 milliseconds, with zero missed deadlines across three rounds totaling 84,000 session ticks measured.
65%VerifiedThe core of the OpenAI-compatible protocol is the /v1/chat/completions REST interface specification, which defines fields such as model, messages, and temperature, as well as the streaming response (Server-Sent Events) format.
65%VerifiedThe LiteLLM project is developed in Python.
65%VerifiedLiteLLM provides two usage modes: a Python SDK and a Proxy Server (AI Gateway).
65%VerifiedLiteLLM is an open-source project developed by the BerriAI team.
65%VerifiedLiteLLM is a unified multi-model proxy in the Python ecosystem, focused on API compatibility conversion.
65%VerifiedLLM routing is an inference optimization strategy that dynamically selects models based on task complexity, cost budget, and response speed requirements; open-source frameworks such as RouteLLM and LiteLLM are already exploring this direction.
65%UnverifiedLiteLLM的日志系统支持与Langfuse、Helicone、Datadog和Prometheus等可观测性平台集成
90%UnverifiedLiteLLM的护栏系统支持集成第三方审核服务如Lakera和Presidio
90%Unverified该方案的所有数据和推理都在本地完成，不会外传到第三方服务器，适合对数据隐私有严格要求的企业场景
90%Unverified通过设置环境变量ANTHROPIC_BASE_URL指向LiteLLM Proxy的本地端口，可以将Claude Agent SDK的请求重定向到本地模型，无需修改SDK源码
90%UnverifiedLiteLLM和OneAPI等开源项目将不兼容OpenAI格式的模型（如Claude的原生API、Gemini的Vertex AI接口）统一适配为OpenAI格式
85%