:robot: The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.
Excited to announce the release of LocalAI v2.10.0! This version introduces significant changes, including breaking changes, numerous bug fixes, exciting new features, dependency updates, and more. Here's a summary of what's new:
trust_remote_code
setting in the YAML config file of the model are now consumed for enhanced security measures also for the AutoGPTQ and transformers backend, thanks to @dave-gray101's contribution (#1799). If your model relied on the old behavior and you are sure of what you are doing, set trust_remote_code: true
in the YAML config file.finish_reason
fields for better compatibility with the OpenAI API, fixed by @mudler (#1745).default.metallib
has been resolved, which should now allow running the llama-cpp backend on Apple arm64, fixed by @dave-gray101 (#1837).transformer
backend also on Intel GPUs, implemented by @mudler (#1746).stream: true
! This feature was introduced by @golgeek (#1749).ggerganov/llama.cpp
, donomii/go-rwkv.cpp
, mudler/go-stable-diffusion
, and others, ensuring that LocalAI is built on the latest and most secure libraries.trust_remote_code
by @dave-gray101 in https://github.com/mudler/LocalAI/pull/1799
Thank you to all contributors and users for your continued support and feedback, making LocalAI better with each release!
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.9.0...v2.10.0
This release brings many enhancements, fixes, and a special thanks to the community for the amazing work and contributions!
We now have sycl images for Intel GPUs, ROCm images for AMD GPUs,and much more:
hipblas
. For example, master-hipblas-ffmpeg-core. Thanks to @fenfir for this nice contribution!sycl
. You can find images with two flavors, sycl-f16 and sycl-f32 respectively. For example, master-sycl-f16. Work is in progress to support also diffusers and transformers on Intel GPUs.First off, a massive thank you to each and every one of you who've chipped in to squash bugs and suggest cool new features for LocalAI. Your help, kind words, and brilliant ideas are truly appreciated - more than words can say!
And to those of you who've been heros, giving up your own time to help out fellow users on Discord and in our repo, you're absolutely amazing. We couldn't have asked for a better community.
Just so you know, LocalAI doesn't have the luxury of big corporate sponsors behind it. It's all us, folks. So, if you've found value in what we're building together and want to keep the momentum going, consider showing your support. A little shoutout on your favorite social platforms using @LocalAI_OSS and @mudler_it or joining our sponsorship program can make a big difference.
Also, if you haven't yet joined our Discord, come on over! Here's the link: https://discord.gg/uJAeKSAGDy
Every bit of support, every mention, and every star adds up and helps us keep this ship sailing. Let's keep making LocalAI awesome together!
Thanks a ton, and here's to more exciting times ahead with LocalAI! 🚀
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.8.2...v2.9.0
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.8.1...v2.8.2
This is a patch release, mostly containing minor patches and bugfixes from 2.8.0.
Most importantly it contains a bugfix for https://github.com/mudler/LocalAI/issues/1333 which made the llama.cpp backend to get stuck in some cases where the model starts to hallucinate and fixes to the python-based backends.
First off, a massive thank you to each and every one of you who've chipped in to squash bugs and suggest cool new features for LocalAI. Your help, kind words, and brilliant ideas are truly appreciated - more than words can say!
And to those of you who've been heros, giving up your own time to help out fellow users on Discord and in our repo, you're absolutely amazing. We couldn't have asked for a better community.
Just so you know, LocalAI doesn't have the luxury of big corporate sponsors behind it. It's all us, folks. So, if you've found value in what we're building together and want to keep the momentum going, consider showing your support. A little shoutout on your favorite social platforms using @LocalAI_OSS and @mudler_it or joining our sponsorship program can make a big difference.
Also, if you haven't yet joined our Discord, come on over! Here's the link: https://discord.gg/uJAeKSAGDy
Every bit of support, every mention, and every star adds up and helps us keep this ship sailing. Let's keep making LocalAI awesome together!
Thanks a ton, and here's to more exciting times ahead with LocalAI! 🚀
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.8.0...v2.8.1
This release adds support for Intel GPUs, and it deprecates old ggml-based backends which are by now superseded by llama.cpp (that now supports more architectures out-of-the-box). See also https://github.com/mudler/LocalAI/issues/1651.
Images are now based on Ubuntu 22.04 LTS instead of Debian bullseye.
There are now images tagged with "sycl". There are sycl-f16 and sycl-f32 images indicating f16 or f32 support.
For example, to start phi-2 with an Intel GPU it is enough to use the container image like this:
docker run -e DEBUG=true -ti -v $PWD/models:/build/models -p 8080:8080 -v /dev/dri:/dev/dri --rm quay.io/go-skynet/local-ai:master-sycl-f32-ffmpeg-core phi-2
First off, a massive thank you to each and every one of you who've chipped in to squash bugs and suggest cool new features for LocalAI. Your help, kind words, and brilliant ideas are truly appreciated - more than words can say!
And to those of you who've been heros, giving up your own time to help out fellow users on Discord and in our repo, you're absolutely amazing. We couldn't have asked for a better community.
Just so you know, LocalAI doesn't have the luxury of big corporate sponsors behind it. It's all us, folks. So, if you've found value in what we're building together and want to keep the momentum going, consider showing your support. A little shoutout on your favorite social platforms using @LocalAI_OSS and @mudler_it or joining our sponsorship program can make a big difference.
Also, if you haven't yet joined our Discord, come on over! Here's the link: https://discord.gg/uJAeKSAGDy
Every bit of support, every mention, and every star adds up and helps us keep this ship sailing. Let's keep making LocalAI awesome, together.
Thanks a ton, and here's to more exciting times ahead with LocalAI! 🚀
github/
-> github.com/
by @Wansmer in https://github.com/mudler/LocalAI/pull/1672
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.7.0...v2.8.0
This release adds support to the transformer backend for LLM as well!
For now instance you can run codellama-7b with transformers with:
docker run -ti -p 8080:8080 --gpus all localai/localai:v2.7.0-cublas-cuda12 codellama-7b
In the quickstart there are more examples available https://localai.io/basics/getting_started/#running-models.
Note: As llama.cpp is ongoing with changes that could possible cause breakage, this release does not includes changes from https://github.com/ggerganov/llama.cpp/discussions/5138 (the future versions will).
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.6.1...v2.6.2
This is a patch release containing bug-fixes around parallel request support with llama.cpp models.
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.6.0...v2.6.1
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.5.1...v2.6.0
Patch release to create /build/models
in the container images.
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.5.0...v2.5.1
This release adds more embedded models, and shrink image sizes.
You can run now phi-2
( see here for the full list ) locally by starting localai with:
docker run -ti -p 8080:8080 localai/localai:v2.5.0-ffmpeg-core phi-2
LocalAI accepts now as argument a list of short-hands models and/or URLs pointing to valid yaml file. A popular way to host those files are Github gists.
For instance, you can run llava
, by starting local-ai
with:
docker run -ti -p 8080:8080 localai/localai:v2.5.0-ffmpeg-core https://raw.githubusercontent.com/mudler/LocalAI/master/embedded/models/llava.yaml
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.4.1...v2.5.0