LiteLLM is an open-source library and proxy that unifies access to various Large Language Model (LLM) APIs, including OpenAI, Azure, Anthropic, Cohere, and Google. It simplifies LLM integration by enabling developers to call any LLM with a single completion() function, abstracting away provider-specific complexities. The platform offers essential features like caching, fallbacks, retries, load balancing, rate limiting, streaming, and cost tracking. In addition to its free, self-hostable open-source SDK and Proxy, LiteLLM provides a "Managed LiteLLM Proxy" service. This commercial offering allows users to deploy a hosted version of their proxy with advanced features, enterprise support, and usage-based billing, catering to teams that prefer a managed solution over self-hosting.
Quick Info