from litellm import completion
resp = completion(
model="gpt-4o", # or "azure/gpt-4o", "anthropic/claude-3-5-sonnet", "gemini/gemini-1.5-pro"
messages=.
- Run the quickstart code above.
- Goal: Make your first OpenAI-compatible request via LiteLLM.
- Practical builder
- Read the DataCamp tutorial and extend examples with streaming and retries.
- Add two providers and test fallbacks.
- Team/production owner
- Study the official Getting Started guide.
- Stand up the proxy, add observability and cost tracking.
- Enforce rate limits and PII redaction policies.
—
## Deep Dive: Patterns You’ll Use Weekly
### OpenAI Compatibility as an Interface Contract
- Treat OpenAI’s API shape as your app contract. All requests go to your LiteLLM proxy’s `/v1/*` endpoints.
- Swap models (e.g., `gpt-4o` → `claude-3-5`) by config, not code.
### Model Routing by Use Case
- Latency-sensitive path: route to fast, cheaper models.
- Reasoning path: route to higher-quality models for retrieval-augmented generation (RAG) or tool use.
- Privacy path: route to local/Ollama for PII segments.
### Cost Guardrails
- Tag requests with `user_id`/`team`.
- Set budgets per team/model.
- Log token usage to a central store and alert on anomalies.
### Resilience
- Enable retries with jitter.
- Configure timeouts per provider and circuit breakers on repeated failures.
- Define provider priorities and explicit fallbacks.
### Observability
- Capture request/response metadata, latency histograms, and model/version.
- Redact secrets/PII in logs.
- Correlate traces across services to find slow calls quickly.
—
## Example LiteLLM Proxy Config (Production-Ready Starter)
```yaml
# config.yaml
model_list:
- model_name: gpt-4o
litellm_params:
model: openai/gpt-4o
api_key: ${OPENAI_API_KEY}
- model_name: claude-3-5-sonnet
litellm_params:
model: anthropic/claude-3-5-sonnet
api_key: ${ANTHROPIC_API_KEY}
- model_name: gemini-1.5-pro
litellm_params:
model: google/gemini-1.5-pro
api_key: ${GOOGLE_API_KEY}
defaults:
timeout: 30s
max_tokens: 1024
routing:
- name: low-latency
models: .
- A practical, example-driven article.
- The official LiteLLM docs for getting started and proxy best practices.
—
## Action Plan: Your Next 7 Days
Day 1–2: Do the crash course and quickstart; make your first proxied request.
Day 3–4: Add a second provider and streaming; set timeouts, retries.
Day 5: Stand up the proxy with config; route by use case (latency vs reasoning).
Day 6: Add logging, cost tracking, and redaction.
Day 7: Load-test; simulate provider failures; verify fallbacks.
—
## Key Takeaways
- LiteLLM is the fastest path to multi-provider LLM apps without vendor lock-in.
- Start with an OpenAI-compatible interface, then level up to the proxy for governance.
- Invest early in routing, resilience, and observability—you’ll need them in week two, not month six.
- The tutorials above cover 80% of what you’ll use daily; the rest is your product’s secret sauce.
### FAQ
Q1:What is the best LiteLLM tutorial for beginners?
Start with the LiteLLM Crash Course on YouTube for a quick visual walkthrough, then read the official Getting Started guide for the proxy. The DataCamp tutorial provides practical examples you can copy.
Q2:How do I use LiteLLM as an OpenAI-compatible proxy?
Run the LiteLLM proxy and point your SDK’s base URL to the proxy’s `/v1` endpoints. Keep provider details in the LiteLLM config so your application code stays portable.
Q3:Can LiteLLM route between OpenAI, Anthropic, and Gemini automatically?
Yes. Define models and routing strategies in the LiteLLM config to switch between providers by latency, cost, or quality. You can also set fallbacks for reliability.
Q4:How do I enable streaming and tool/function calling with LiteLLM?
Use the OpenAI-compatible API via LiteLLM and enable `stream=True` (or SSE in your SDK). For tool calling, follow the OpenAI function-calling format—LiteLLM forwards it to the target provider.
Q5:What’s the fastest way to control costs with LiteLLM?
Centralize requests through the proxy, enable usage logging, and enforce per-key rate limits and budgets. Route different workloads to cost-optimized models and pin versions to avoid surprises.