:robot: The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.
Hey awesome people! I'm happy to announce the release of LocalAI version 2.15.0! This update introduces several significant improvements and features, enhancing usability, functionality, and user experience across the board. Dive into the key highlights below, and don't forget to check out the full changelog for more detailed updates.
The Chat WebUI now seamlessly integrates with the Vision API, making it easier for users to test image processing models directly through the browser interface - this is a very simple and hackable interface in less then 400L of code with Alpine.JS and HTMX!
System prompts can be set in the WebUI chat, which guide the user through interactions more intuitively, making our chat interface smarter and more responsive.
New to LocalAI or haven't installed any models yet? No worries! The updated welcome page now guides users through the model installation process, ensuring you're set up and ready to go without any hassle. This is a great first step for newcomers - thanks for your precious feedback!
Don't get lost with our new background operations indicator on the WebUI, which shows when tasks are running in the background.
As our model gallery balloons, you can now effortlessly sift through models by tag and category, making finding what you need a breeze.
LocalAI is expanding into offering single binary releases, simplifying the deployment process and making it easier to get LocalAI up and running on any system.
For the moment we have condensed the builds which disables AVX and SSE instructions set. We are also planning to include cuda builds as well.
This release introduces several exciting new models to our gallery, such as 'Soliloquy', 'tess', 'moondream2', 'llama3-instruct-coder' and 'aurora', enhancing the diversity and capability of our AI offerings. Our selection of one-click-install models is growing! We pick carefully model from the most trending ones on huggingface, feel free to submit your requests in a github issue, hop to our Discord or contribute by hosting your gallery, or.. even by adding models directly to LocalAI!
Want to share your model configurations and customizations? See the docs: https://localai.io/docs/getting-started/customize-model/
A gigantic THANK YOU to everyone whoβs contributedβyour feedback, bug squashing, and feature suggestions are what make LocalAI shine. To all our heroes out there supporting other users and sharing their expertise, youβre the real MVPs!
Remember, LocalAI thrives on community supportβnot big corporate bucks. If you love what we're building, show some love! A shoutout on social (@LocalAI_OSS and @mudler_it on twitter/X), joining our sponsors, or simply starring us on GitHub makes all the difference.
Also, if you haven't yet joined our Discord, come on over! Here's the link: https://discord.gg/uJAeKSAGDy
Thanks a ton, and.. enjoy this release!
trust_remote_code
to users // favicon support by @dave-gray101 in https://github.com/mudler/LocalAI/pull/2253
app_test.go
by @dave-gray101 in https://github.com/mudler/LocalAI/pull/2248
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.14.0...v2.15.0
We're excited to announce that our AIO image has been upgraded with the latest LLM model, llama3, enhancing our capabilities with more accurate and dynamic responses. Behind the scenes uses https://huggingface.co/NousResearch/Hermes-2-Pro-Llama-3-8B-GGUF which is ready for function call, yay!
Chat | TTS | Image gen |
---|---|---|
Our interfaces for Chat, Text-to-Speech (TTS), and Image Generation have finally landed. Enjoy streamlined and simple interactions thanks to the efforts of our team, led by @mudler, who have worked tirelessly to enhance your experience. The WebUI interface serves as a quick way to debug and assess models loaded in LocalAI - there is much to improve, but we have now a small, hackable interface!
The model gallery has received a substantial upgrade with numerous new models, including Einstein v6.1, SOVL, and several specialized Llama3 iterations. These additions are designed to cater to a broader range of tasks , making LocalAI more versatile than ever. Kudos to @mudler for spearheading these exciting updates - now you can select with a couple of click the model you like!
This update brings a series of crucial bug fixes and security enhancements to ensure our platform remains secure and efficient. Special thanks to @dave-gray101, @cryptk, and @fakezeta for their diligent work in rooting out and resolving these issues :hugs:
We're introducing OpenVINO acceleration, and many OpenVINO models in the gallery. You can now enjoy fast-as-hell speed on Intel CPU and GPUs. Applause to @fakezeta for the contributions!
We've updated our documentation and dependencies to keep you equipped with the latest tools and knowledge. These updates ensure that LocalAI remains a robust and dependable platform.
A special shout-out to our new contributors, @QuinnPiers and @LeonSijiaLu, who have enriched our community with their first contributions. Welcome aboard, and thank you for your dedication and fresh insights!
Each update in this release not only enhances our platform's capabilities but also ensures a safer and more user-friendly experience. We are excited to see how our users leverage these new features in their projects, freel free to hit a line on Twitter or in any other social, we'd be happy to hear how you use LocalAI!
First off, a massive thank you (again!) to each and every one of you who've chipped in to squash bugs and suggest cool new features for LocalAI. Your help, kind words, and brilliant ideas are truly appreciated - more than words can say!
And to those of you who've been heros, giving up your own time to help out fellow users on Discord and in our repo, you're absolutely amazing. We couldn't have asked for a better community.
Just so you know, LocalAI doesn't have the luxury of big corporate sponsors behind it. It's all us, folks. So, if you've found value in what we're building together and want to keep the momentum going, consider showing your support. A little shoutout on your favorite social platforms using @LocalAI_OSS and @mudler_it or joining our sponsors can make a big difference.
Also, if you haven't yet joined our Discord, come on over! Here's the link: https://discord.gg/uJAeKSAGDy
Every bit of support, every mention, and every star adds up and helps us keep this ship sailing. Let's keep making LocalAI awesome together!
Thanks a ton, and.. exciting times ahead with LocalAI!
config_file_watcher.go
- root all file reads for safety by @dave-gray101 in https://github.com/mudler/LocalAI/pull/2144
ul
by @mudler in https://github.com/mudler/LocalAI/pull/2226
troubleshooting mac
by @LeonSijiaLu in https://github.com/mudler/LocalAI/pull/2174
Build on mac
documentations by @LeonSijiaLu in https://github.com/mudler/LocalAI/pull/2171
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.13.0...v2.14.0
Hello folks, Ettore here - I'm happy to announce the v2.13.0 LocalAI release is out, with many features!
Below there is a small breakdown of the hottest features introduced in this release - however - there are many other improvements (especially from the community) as well, so don't miss out the changelog!
Check out the full changelog below for having an overview of all the changes that went in this release (this one is quite packed up).
This is the first release with model gallery in the webUI, you can see now a "Model" button in the WebUI which lands now in a selection of models:
You can choose now models between stablediffusion, llama3, tts, embeddings and more! The gallery is growing steadly and being kept up-to-date.
The models are simple YAML files which are hosted in this repository: https://github.com/mudler/LocalAI/tree/master/gallery - you can host your own repository with your model index, or if you want you can contribute to LocalAI.
If you want to contribute adding models, you can by opening up a PR in the gallery
directory: https://github.com/mudler/LocalAI/tree/master/gallery.
I'm excited to introduce a new backend for rerankers
. LocalAI now implements the Jina API (https://jina.ai/reranker/#apiform) as a compatibility layer, and you can use existing Jina clients and point to those to the LocalAI address. Behind the hoods, uses https://github.com/AnswerDotAI/rerankers.
You can test this by using container images with python (this does NOT work with core
images) and a model config file like this, or by installing cross-encoder
from the gallery in the UI:
name: jina-reranker-v1-base-en
backend: rerankers
parameters:
model: cross-encoder
and test it with:
curl http://localhost:8080/v1/rerank \
-H "Content-Type: application/json" \
-d '{
"model": "jina-reranker-v1-base-en",
"query": "Organic skincare products for sensitive skin",
"documents": [
"Eco-friendly kitchenware for modern homes",
"Biodegradable cleaning supplies for eco-conscious consumers",
"Organic cotton baby clothes for sensitive skin",
"Natural organic skincare range for sensitive skin",
"Tech gadgets for smart homes: 2024 edition",
"Sustainable gardening tools and compost solutions",
"Sensitive skin-friendly facial cleansers and toners",
"Organic food wraps and storage solutions",
"All-natural pet food for dogs with allergies",
"Yoga mats made from recycled materials"
],
"top_n": 3
}'
There is a new backend available for tts now, parler-tts
. It is possible to install and configure the model directly from the gallery. https://github.com/huggingface/parler-tts
Thanks to our outstanding community, we have enhanced the performance and stability of LocalAI across various modules. From backend optimizations to front-end adjustments, every tweak helps make LocalAI smoother and more robust.
First off, a massive thank you (again!) to each and every one of you who've chipped in to squash bugs and suggest cool new features for LocalAI. Your help, kind words, and brilliant ideas are truly appreciated - more than words can say!
And to those of you who've been heros, giving up your own time to help out fellow users on Discord and in our repo, you're absolutely amazing. We couldn't have asked for a better community.
Just so you know, LocalAI doesn't have the luxury of big corporate sponsors behind it. It's all us, folks. So, if you've found value in what we're building together and want to keep the momentum going, consider showing your support. A little shoutout on your favorite social platforms using @LocalAI_OSS and @mudler_it or joining our sponsors can make a big difference.
Also, if you haven't yet joined our Discord, come on over! Here's the link: https://discord.gg/uJAeKSAGDy
Every bit of support, every mention, and every star adds up and helps us keep this ship sailing. Let's keep making LocalAI awesome together!
Thanks a ton, and here's to more exciting times ahead with LocalAI!
max_tokens
for old backend by @mudler in https://github.com/mudler/LocalAI/pull/2094
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.12.4...V2.13.0
Patch release to include https://github.com/mudler/LocalAI/pull/1985
I'm happy to announce the v2.12.3 LocalAI release is out!
Ever wondered what to do after LocalAI is up and running? Integration with a simple web interface has been started, and you can see now a landing page when hitting the LocalAI front page:
You can also now enjoy Swagger to try out the API calls directly:
Now the default model for CPU images is https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF - pre-configured for functions and tools API support! If you are an Intel-GPU owner, the Intel profile for AIO images is now available too!
Now there is support for OpenVINO and transformers got token streaming support now thanks to @fakezeta!
To try OpenVINO, you can use the example available in the documentation: https://localai.io/features/text-generation/#examples
Thanks for our outstanding community, we have enhanced several areas:
First off, a massive thank you (again!) to each and every one of you who've chipped in to squash bugs and suggest cool new features for LocalAI. Your help, kind words, and brilliant ideas are truly appreciated - more than words can say!
And to those of you who've been heros, giving up your own time to help out fellow users on Discord and in our repo, you're absolutely amazing. We couldn't have asked for a better community.
Just so you know, LocalAI doesn't have the luxury of big corporate sponsors behind it. It's all us, folks. So, if you've found value in what we're building together and want to keep the momentum going, consider showing your support. A little shoutout on your favorite social platforms using @LocalAI_OSS and @mudler_it or joining our sponsors can make a big difference.
Also, if you haven't yet joined our Discord, come on over! Here's the link: https://discord.gg/uJAeKSAGDy
Every bit of support, every mention, and every star adds up and helps us keep this ship sailing. Let's keep making LocalAI awesome together!
Thanks a ton, and here's to more exciting times ahead with LocalAI!
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.11.0...v2.12.3
I'm happy to announce the v2.12.1 LocalAI release is out!
Ever wondered what to do after LocalAI is up and running? Integration with a simple web interface has been started, and you can see now a landing page when hitting the LocalAI front page:
You can also now enjoy Swagger to try out the API calls directly:
Now the default model for CPU images is https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF - pre-configured for functions and tools API support! If you are an Intel-GPU owner, the Intel profile for AIO images is now available too!
Now there is support for OpenVINO and transformers got token streaming support now thanks to @fakezeta!
To try OpenVINO, you can use the example available in the documentation: https://localai.io/features/text-generation/#examples
Thanks for our outstanding community, we have enhanced several areas:
First off, a massive thank you (again!) to each and every one of you who've chipped in to squash bugs and suggest cool new features for LocalAI. Your help, kind words, and brilliant ideas are truly appreciated - more than words can say!
And to those of you who've been heros, giving up your own time to help out fellow users on Discord and in our repo, you're absolutely amazing. We couldn't have asked for a better community.
Just so you know, LocalAI doesn't have the luxury of big corporate sponsors behind it. It's all us, folks. So, if you've found value in what we're building together and want to keep the momentum going, consider showing your support. A little shoutout on your favorite social platforms using @LocalAI_OSS and @mudler_it or joining our sponsors can make a big difference.
Also, if you haven't yet joined our Discord, come on over! Here's the link: https://discord.gg/uJAeKSAGDy
Every bit of support, every mention, and every star adds up and helps us keep this ship sailing. Let's keep making LocalAI awesome together!
Thanks a ton, and here's to more exciting times ahead with LocalAI!
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.11.0...v2.12.1
I'm happy to announce the v2.12.0 LocalAI release is out!
Ever wondered what to do after LocalAI is up and running? Integration with a simple web interface has been started, and you can see now a landing page when hitting the LocalAI front page:
You can also now enjoy Swagger to try out the API calls directly:
Now the default model for CPU images is https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF - pre-configured for functions and tools API support! If you are an Intel-GPU owner, the Intel profile for AIO images is now available too!
Now there is support for OpenVINO and transformers got token streaming support now thanks to @fakezeta!
To try OpenVINO, you can use the example available in the documentation: https://localai.io/features/text-generation/#examples
Thanks for our outstanding community, we have enhanced several areas:
First off, a massive thank you (again!) to each and every one of you who've chipped in to squash bugs and suggest cool new features for LocalAI. Your help, kind words, and brilliant ideas are truly appreciated - more than words can say!
And to those of you who've been heros, giving up your own time to help out fellow users on Discord and in our repo, you're absolutely amazing. We couldn't have asked for a better community.
Just so you know, LocalAI doesn't have the luxury of big corporate sponsors behind it. It's all us, folks. So, if you've found value in what we're building together and want to keep the momentum going, consider showing your support. A little shoutout on your favorite social platforms using @LocalAI_OSS and @mudler_it or joining our sponsors can make a big difference.
Also, if you haven't yet joined our Discord, come on over! Here's the link: https://discord.gg/uJAeKSAGDy
Every bit of support, every mention, and every star adds up and helps us keep this ship sailing. Let's keep making LocalAI awesome together!
Thanks a ton, and here's to more exciting times ahead with LocalAI!
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.11.0...v2.12.0
Hey everyone! π I'm super excited to share what we've been working on at LocalAI - the launch of v2.11.0. This isn't just any update; it's a massive leap forward, making LocalAI easier to use, faster, and more accessible for everyone.
Imagine having a magic box that, once opened, gives you everything you need to get your AI project off the ground with generative AI. A full clone of OpenAI in a box. That's exactly what our AIO images are! Designed for both CPU and GPU environments, these images come pre-packed with a full suite of models and backends, ready to go right out of the box.
Whether you're using Nvidia, AMD, or Intel, we've got an optimized image for you. If you are using CPU-only you can enjoy even smaller and lighter images.
To start LocalAI, pre-configured with function calling, llm, tts, speech to text, and image generation, just run:
docker run -p 8080:8080 --name local-ai -ti localai/localai:latest-aio-cpu
## Do you have a Nvidia GPUs? Use this instead
## CUDA 11
# docker run -p 8080:8080 --gpus all --name local-ai -ti localai/localai:latest-aio-gpu-cuda-11
## CUDA 12
# docker run -p 8080:8080 --gpus all --name local-ai -ti localai/localai:latest-aio-gpu-cuda-12
Getting started with AIO images is as simple as pulling from Docker Hub or Quay and running it. We take care of the rest, downloading all necessary models for you. For all the details, including how to customize your setup with environment variables, our updated docs have got you covered here, while you can get more details of the AIO images here.
Thanks to the great contribution from @richiejp now LocalAI has a new backend type, "vector stores" that allows to use LocalAI as in-memory Vector DB (https://github.com/mudler/LocalAI/issues/1792). You can learn more about it here!
This release contains major bugfixes to the watchdog component, and a fix to a regression introduced in v2.10.x which was not respecting --f16
, --threads
and --context-size
to be applied as model's defaults.
Model defaults has changed to automatically offload maximum GPU layers if a GPU is available, and it sets saner defaults to the models to enhance the LLM's output.
You can now run llava-1.6-vicuna
, llava-1.6-mistral
and hermes-2-pro-mistral
, see Run other models for a list of all the pre-configured models available in the release.
First off, a massive thank you (again!) to each and every one of you who've chipped in to squash bugs and suggest cool new features for LocalAI. Your help, kind words, and brilliant ideas are truly appreciated - more than words can say!
And to those of you who've been heros, giving up your own time to help out fellow users on Discord and in our repo, you're absolutely amazing. We couldn't have asked for a better community.
Just so you know, LocalAI doesn't have the luxury of big corporate sponsors behind it. It's all us, folks. So, if you've found value in what we're building together and want to keep the momentum going, consider showing your support. A little shoutout on your favorite social platforms using @LocalAI_OSS and @mudler_it or joining our sponsors can make a big difference.
Also, if you haven't yet joined our Discord, come on over! Here's the link: https://discord.gg/uJAeKSAGDy
Every bit of support, every mention, and every star adds up and helps us keep this ship sailing. Let's keep making LocalAI awesome together!
Thanks a ton, and here's to more exciting times ahead with LocalAI!
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.10.1...v2.11.0
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.10.0...v2.10.1
Excited to announce the release of LocalAI v2.10.0! This version introduces significant changes, including breaking changes, numerous bug fixes, exciting new features, dependency updates, and more. Here's a summary of what's new:
trust_remote_code
setting in the YAML config file of the model are now consumed for enhanced security measures also for the AutoGPTQ and transformers backend, thanks to @dave-gray101's contribution (#1799). If your model relied on the old behavior and you are sure of what you are doing, set trust_remote_code: true
in the YAML config file.finish_reason
fields for better compatibility with the OpenAI API, fixed by @mudler (#1745).default.metallib
has been resolved, which should now allow running the llama-cpp backend on Apple arm64, fixed by @dave-gray101 (#1837).transformer
backend also on Intel GPUs, implemented by @mudler (#1746).stream: true
! This feature was introduced by @golgeek (#1749).ggerganov/llama.cpp
, donomii/go-rwkv.cpp
, mudler/go-stable-diffusion
, and others, ensuring that LocalAI is built on the latest and most secure libraries.trust_remote_code
by @dave-gray101 in https://github.com/mudler/LocalAI/pull/1799
Thank you to all contributors and users for your continued support and feedback, making LocalAI better with each release!
Full Changelog: https://github.com/mudler/LocalAI/compare/v2.9.0...v2.10.0