Litellm Versions Save

Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)

v1.37.9-stable

4 days ago

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.37.9...v1.37.9-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.37.9-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

v1.37.9

4 days ago

What's Changed

feat(langfuse.py): Allow for individual call message/response redaction by @alexanderepstein in https://github.com/BerriAI/litellm/pull/3603
[Feat] - /global/spend/report by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3619
Fixes #3544 based on the data-type of message by @paneru-rajan in https://github.com/BerriAI/litellm/pull/3554
[UI] Filter Tag Spend by Date + Show Bar Chart by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3624
Default routing fallbacks by @krrishdholakia in https://github.com/BerriAI/litellm/pull/3625

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.37.7...v1.37.9

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.37.9

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	40	45.14681571397024	1.5067595942578198	1.5067595942578198	451	451	37.28894399998239	203.69157899997958
/health/liveliness	Failed ❌	38	43.774724098143416	15.65894061704302	15.65894061704302	4687	4687	36.20009499996968	219.30193999997982
/health/readiness	Failed ❌	38	42.98829494917115	15.314824789529593	15.314824789529593	4584	4584	36.154727999985425	234.44879100000549
Aggregated	Failed ❌	38	43.46756735054526	32.48052500083043	32.48052500083043	9722	9722	36.154727999985425	234.44879100000549

v1.37.7-stable

4 days ago

What's Changed

feat(langfuse.py): Allow for individual call message/response redaction by @alexanderepstein in https://github.com/BerriAI/litellm/pull/3603
[Feat] - /global/spend/report by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3619
Fixes #3544 based on the data-type of message by @paneru-rajan in https://github.com/BerriAI/litellm/pull/3554
[UI] Filter Tag Spend by Date + Show Bar Chart by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3624

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.37.7...v1.37.7-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.37.7-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

v1.37.7

5 days ago

What's Changed

[Feat] send weekly spend reports by Team/Tag by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3609
build(deps): bump next from 14.1.0 to 14.1.1 in /ui/litellm-dashboard by @dependabot in https://github.com/BerriAI/litellm/pull/3550

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.37.6...v1.37.7

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.37.7

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

v1.37.6

5 days ago

What's Changed

[Feat] Use csv values for proxy batch completions (OpenAI Python compatible) by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3587
Add gpt-4o metadata by @ConnorDoyle in https://github.com/BerriAI/litellm/pull/3613
Update FastAPI to update starlette to fix warnings by @msabramo in https://github.com/BerriAI/litellm/pull/3601

New Contributors

@ConnorDoyle made their first contribution in https://github.com/BerriAI/litellm/pull/3613

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.37.5-stable...v1.37.6

v1.37.5-stable

6 days ago

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.37.5...v1.37.5-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.37.5-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

v1.37.5

6 days ago

What's Changed

add additional models from openrouter by @Merlinvt in https://github.com/BerriAI/litellm/pull/3545
Initial OIDC support (Google/GitHub/CircleCI -> Amazon Bedrock & Azure OpenAI) by @Manouchehri in https://github.com/BerriAI/litellm/pull/3507
Fix tool calls tracking with Lunary by @vincelwt in https://github.com/BerriAI/litellm/pull/3424
✨ feat: Add Azure Content-Safety Proxy hooks by @Lunik in https://github.com/BerriAI/litellm/pull/3407
fix(exceptions.py): import openai Exceptions by @nobu007 in https://github.com/BerriAI/litellm/pull/3399
Clarifai-LiteLLM : Added clarifai as LLM Provider. by @mogith-pn in https://github.com/BerriAI/litellm/pull/3369
(fix) Fixed linting and other bugs with watsonx provider by @simonsanvil in https://github.com/BerriAI/litellm/pull/3561
feat(router.py): allow setting model_region in litellm_params by @krrishdholakia in https://github.com/BerriAI/litellm/pull/3582
[UI] Show Token ID/Hash on Admin UI by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3583
[Litellm Proxy + litellm.Router] - Pass the same message/prompt to N models by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3585
[Feat] - log metadata on traces + allow users to log metadata when existing_trace_id exists by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3581
Set fake env vars for client_no_auth fixture by @msabramo in https://github.com/BerriAI/litellm/pull/3588
[Feat] Proxy + Router - Retry on RateLimitErrors when fallbacks, other deployments exists by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3590
Make test_load_router_config pass by @msabramo in https://github.com/BerriAI/litellm/pull/3589
feat(bedrock_httpx.py): Make Bedrock-Cohere calls Async + Command-R support by @krrishdholakia in https://github.com/BerriAI/litellm/pull/3586

New Contributors

@Merlinvt made their first contribution in https://github.com/BerriAI/litellm/pull/3545
@mogith-pn made their first contribution in https://github.com/BerriAI/litellm/pull/3369

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.37.3-stable...v1.37.5

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.37.5

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

v1.37.3-stable

1 week ago

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.37.3...v1.37.3-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.37.3-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

v1.37.3

1 week ago

BETA support for Triton Inference Embeddings on 👉 Start here: https://docs.litellm.ai/docs/providers/triton-inference-server

⚡️ [Feat] Use Team based callbacks for failure_callbacks https://docs.litellm.ai/docs/proxy/team_based_routing#logging--caching

🛠️ [Test] Added Testing to ensure Proxy - uses the same OpenAI Client after 1 min

🛠️ [Fix] Upsert deployment bug on LiteLLM Proxy

🔥 Improved LiteLLM-stable load tests - added testing for Azure OpenAI, and using 50+ deployments on a proxy server

🚀 [Feat] support stream_options on litellm.text_completion

codeimage-snippet_11 (3)

What's Changed

[Fix] Upsert deployment bug by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3569
[Test] Proxy - uses the same OpenAI Client after 1 min by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3570
[Feat] Use Team based callbacks with litellm.failure_callbacks by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3573
enforce unique key and team aliases in the ui by @powerhouseofthecell in https://github.com/BerriAI/litellm/pull/3572
Huggingface classifier support by @krrishdholakia in https://github.com/BerriAI/litellm/pull/3571
[Feat] Add Triton Embeddings to LiteLLM by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3577
fix(langfuse.py): fix logging user_id in trace param on new trace creation by @krrishdholakia in https://github.com/BerriAI/litellm/pull/3576

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.37.2...v1.37.3

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.37.3

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

v1.37.2

1 week ago

What's Changed

feat(proxy_server.py): return litellm version in response headers by @krrishdholakia in https://github.com/BerriAI/litellm/pull/3535
[Fix] litellm.completion_cost(model="bedrock/anthropic.claude-instant-v1"..) by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3534
[UI] show End-User Usage on Usage Tab by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3530
Add support for async streaming to watsonx provider by @simonsanvil in https://github.com/BerriAI/litellm/pull/3479
feat(proxy_server.py): add CRUD endpoints for 'end_user' management by @krrishdholakia in https://github.com/BerriAI/litellm/pull/3536
Revert "Add support for async streaming to watsonx provider " by @krrishdholakia in https://github.com/BerriAI/litellm/pull/3546
[Feat] support stream_options param for OpenAI by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3537
Fix/error on get user role by @powerhouseofthecell in https://github.com/BerriAI/litellm/pull/3551
Globally filtering pydantic conflict warnings by @CyanideByte in https://github.com/BerriAI/litellm/pull/3555
[Feat] support stream_options on litellm.text_completion by @ishaan-jaff in https://github.com/BerriAI/litellm/pull/3547
feat(predibase.py): add support for predibase provider by @krrishdholakia in https://github.com/BerriAI/litellm/pull/3552
Expand access for other jwt algorithms by @duckboy81 in https://github.com/BerriAI/litellm/pull/3378

New Contributors

@powerhouseofthecell made their first contribution in https://github.com/BerriAI/litellm/pull/3551
@duckboy81 made their first contribution in https://github.com/BerriAI/litellm/pull/3378

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.37.0.dev2_completion_cost...v1.37.2

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.37.2

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	24	28.59593037362605	1.5197959088318929	1.5197959088318929	455	455	22.671621000029063	184.80915000003506
/health/liveliness	Failed ❌	23	27.673046850246536	15.568722485858137	15.568722485858137	4661	4661	21.451024999976198	1771.8764150000084
/health/readiness	Failed ❌	23	28.361425038412307	15.652227755574176	15.652227755574176	4686	4686	21.433796999986043	1998.6570389999656
Aggregated	Failed ❌	23	28.044976272087183	32.74074615026421	32.74074615026421	9802	9802	21.433796999986043	1998.6570389999656