Question 1

What is LiteLLM and why use it?

Accepted Answer

LiteLLM is an open-source gateway and SDK that lets you call 100+ LLMs through a single, OpenAI-compatible API. Use it to switch providers easily, set fallbacks, and centralize rate limits and cost tracking.

Question 2

How does LiteLLM compare to OpenRouter?

Accepted Answer

LiteLLM is self-hostable and open-source, giving you private control with your own provider keys, while OpenRouter is a hosted aggregator. Choose LiteLLM for compliance and control, OpenRouter for quick access to many models.

Question 3

Can LiteLLM replace LangChain or LlamaIndex?

Accepted Answer

No. LiteLLM focuses on model abstraction and routing. Pair it with LangChain or LlamaIndex if you need agents, chains, or RAG pipelines while keeping provider flexibility.

Question 4

Does LiteLLM support fallbacks and retries?

Accepted Answer

Yes. You can define fallback chains so if one provider fails or rate-limits, LiteLLM automatically tries the next model, improving reliability for production workloads.

Question 5

Is LiteLLM free to use?

Accepted Answer

LiteLLM is open-source, so there’s no license cost, but you’ll pay for the underlying model usage. If you run the proxy, factor in operational costs for hosting and monitoring.

LiteLLM 是與每個 LLM 對話最簡單的方式嗎？實用評測

LiteLLM 評測：透過一個入口存取所有 LLM 的最簡單方式

什麼是 LiteLLM？

OpenRouter 將多個模型整合在一個 token 後方，提供簡單路由、公眾速率限制與市場風格的體驗。相對而言，LiteLLM 是開源，且多部署在自有基礎設施上。

- LangChain：擴展的協調框架（链、代理、工具、記憶）。你可在 LangChain 中使用 LiteLLM 做模型抽象。

## 安全與合規考量

- 審計能力：集中代理紀錄查詢、回應元資料與花費。