Predibase Lorax Versions Save

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

5 months ago

LoRAX is the open-source framework for serving hundreds of fine-tuned LLMs in production for the price of one.

6 months ago

LoRAX is the open-source framework for serving hundreds of fine-tuned LLMs in production for the price of one.

6 months ago

What's Changed

add weight property to exllamav2 quanlinear by @flozi00 in https://github.com/predibase/lorax/pull/80
fix: Assign dtype of lora to base model dtype by @tgaddair in https://github.com/predibase/lorax/pull/82

Full Changelog: https://github.com/predibase/lorax/compare/v0.2.0...v0.2.1

6 months ago

Implement sparse SGMV by @tgaddair in https://github.com/predibase/lorax/pull/64
Implement tensor parallel SGMV by @tgaddair in https://github.com/predibase/lorax/pull/79
Add adapter support for all linear layers in Llama and Mistral by @tgaddair in https://github.com/predibase/lorax/pull/75
4 bit support by @flozi00 in https://github.com/predibase/lorax/pull/66
Exllamav2 by @flozi00 in https://github.com/predibase/lorax/pull/60

Updated to custom SGMV kernel to fix issue with certain ranks by @tgaddair in https://github.com/predibase/lorax/pull/70
fix: Allow using unsupported base models without adapter loading by @tgaddair in https://github.com/predibase/lorax/pull/76

Add DISABLE_SGMV env var to explicitly fallback to loop by @tgaddair in https://github.com/predibase/lorax/pull/69
Upgrade the README discord badge and use an invite link that doesn't expire. by @justinxzhao in https://github.com/predibase/lorax/pull/73

@justinxzhao made their first contribution in https://github.com/predibase/lorax/pull/73

Full Changelog: https://github.com/predibase/lorax/compare/v0.1.2...v0.2.0

6 months ago

Fixed adapter segments when batches contain multiple distinct adapters by @tgaddair in https://github.com/predibase/lorax/pull/62

Full Changelog: https://github.com/predibase/lorax/compare/v0.1.1...v0.1.2

6 months ago

Add Helm charts to deploy models by @abidwael in https://github.com/predibase/lorax/pull/27
change defaults for helm chart by @noyoshi in https://github.com/predibase/lorax/pull/38
add helm release wf by @noyoshi in https://github.com/predibase/lorax/pull/39
Added support for YARN scaling by @tgaddair in https://github.com/predibase/lorax/pull/45
Fixed tensor parallelism splits by @tgaddair in https://github.com/predibase/lorax/pull/47
enh: enable CodeLlama by @geoffreyangus in https://github.com/predibase/lorax/pull/48
Fallback when Punica is not installed by @tgaddair in https://github.com/predibase/lorax/pull/49
add transformers gptq weights by @flozi00 in https://github.com/predibase/lorax/pull/52
Add support for paged attention v2 and update flash attention v2 by @tgaddair in https://github.com/predibase/lorax/pull/54
Fixed adapter loading for GPTQ base models by @tgaddair in https://github.com/predibase/lorax/pull/58
Update gha to be able to automatically push images with release tags by @magdyksaleh in https://github.com/predibase/lorax/pull/59

@abidwael made their first contribution in https://github.com/predibase/lorax/pull/27
@flozi00 made their first contribution in https://github.com/predibase/lorax/pull/52

Full Changelog: https://github.com/predibase/lorax/compare/v0.1.0...v0.1.1

6 months ago

LoRAX is the open-source framework for serving hundreds of fine-tuned LLMs in production for the price of one.