Curated list of awesome resources for the Stable Diffusion AI Model.
This is a list of software and resources for the Stable Diffusion AI model.
Due to the fast-moving nature of the topic, entries in the list may be removed at an expedited rate until the ecosystem matures.
See Contributing.
The easiest way to get started for most people is to pick one of the available GUIs based on the desired platform and follow it's installation instructions.
Alternatively, most of the more developed forks (such as InvokeAI) come with their own user interfaces.
To get up and running, one usually needs:
All forks listed here add additional features and optimisations and are generally faster than the original release, as they keep the model in memory rather than reloading it after every prompt. Most forks seem to remove the Safety Classifier which may present a risk if used to provide public-facing services, such as Discord bots. The two most popular forks at this point are Automatic1111 and InvokeAI
AbdBarho/stable-diffusion-webui-docker - Easy Docker setup for SD with multiple user-friendly UI options including AUTOMATIC1111/stable-diffusion-webui, sd-webui/stable-diffusion-webui and invoke-ai/InvokeAI.
AUTOMATIC1111/stable-diffusion-webui - Likely the fastest moving, most feature rich branch at the moment. Gradio based UI with extensive features such as in and outpainting, previews, xy plots, upscaling, clip-interrogation, textual inversion, negative prompting, a variety of upscaling features, training, checkpoint merging and switching capabilities and more. Comes with a handy install script that takes care of most dependencies and addons.
basujindal/stable-diffusion - "Optimized Stable Diffusion"βa fork with dramatically reduced VRAM requirements through model splitting, enabling Stable Diffusion on lower-end graphics cards; includes a GradIO web interface and support for weighted prompts.
bes-dev/stable_diffusion.openvino - A fork for running the model using a CPU compatible with OpenVINO.
DreamArtist - With just one training image DreamArtist learns the content and style in it, generating diverse high-quality images with high controllability. Embeddings of DreamArtist can be easily combined with additional descriptions, as well as two learned embeddings. (standalone version)
fast-stable-diffusion - Speed focused fork with Dreambooth integration.
Gauss - Native MacOS Stable Diffusion App.
imaginAIry - Pythonic generation of stable diffusion images. Unique in that it supports complex text-based masking. Has an interactive CLI, upscaling, face enhancement, tiling, and other standard features. No GUI.
invoke-ai/InvokeAI - (formerly known as lstein/stable-diffusion) - Very active fork adding a conversational CLI, basic web interface and support for GFPGAN, ESRGAN, Codeformer, weighted prompts, prompt blending, negative prompting, img2img, tiling, textual-inversion as well as inference on Apple M1.
KerasCV StableDiffusion - High performance implementation of stable diffusion on KerasCV.
NickLucche/stable-diffusion-nvidia-docker - Multi (Nvidia) GPU capable docker setup of SD
replicate/copg-stable-diffusion - Cog machine learning container of SD v1.4.
stable-diffusion-jupyterlab-docker - A Docker setup ready to go with Jupyter notebooks for Stable Diffusion.
runwayml/stable-diffusion - Stable Diffusion Branch by RunwayML with specifically trained inpainting model for high quality inpainting.
These branches have not been updated in more than a months and are kept here for historic purposes
Checkpoints (.ckpt files) must be separately downloaded and are required to run Stable Diffusion. The latest stability ai release is 2.1. The last model containing NSFW concepts was 1.5 released by RunwayML.
For 2.x releases, there is a 768x768px resolution capable model trained off the base model (512x512 pixels). You need one of these models to use stable diffusion and generally want to chose the latest one that fits your needs.
Tools and models for use in conjuction with Stable Diffusion
Dreambooth-Stable-Diffusion - Implementation of Google's DreamBooth for stable diffusion, allowing fine-tuning of the model for specific concepts.
LORA - LORA (Low-rank Adaptation) for Faster Text-to-Image Diffusion Fine-tuning (UNET+CLIP), a faster, more efficient Fine tuning method requiring less RAM than Dreambooth. Huggingface demo here.
Stable Tuner - Stable Tuner, an easy to install Dreambooth trainer with a very comfortable user interface.
Stable Diffusion Trainer - Stable Diffusion trainer with scalable dataset size and hardware usage. Requires 10G of VRAM.
textual-inversion - Addition of personalized content to Stable Diffusion without retraining the model (Paper, Paper2).
Stable Dreamfusion - Text to 3D dreamfusion implementation based on stable diffusion.
Most of these GUIS, unless mentioned otherwise in their documentation, include stable-diffusion.
ποΈπ΅ Auto SD Workflow - A UI for lstein/stable-diffusion's dream.py with optimized UX for large-scale/production workflow around image synthesis. Video Walkthrough.
Carefree Creator (local version) - User friendly GUI with a creator/artist centric workflow.
cmdr2/stable-diffusion-ui - Another, simple to use UI for windows and Linux.
DiffusionBee - Self contained binary app for MacOS.
ποΈ DiffusionUI - web UI made with Vue.js inspired by Dall-e using diffusers, perfect for inpainting. Video demo
ποΈ KIRI.ART (formerly SD-MUI) - mobile-first PWA with multiple models (incl. waifu diffusion). Run free locally or use free & paid credits on the live site. Built with React + MaterialUI. (Source Code) MIT License
TypeScript
π΅ NMKD GUI - Windows UI, fully featured. Closed source. Pick your own price.
sd-webui/stable-diffusion-webui - Very active fork with optional, highly featureful Gradio UI and support for txt2img, img2img inpainting, GFPGAN, ESRGAN, weighted prompts, optimized low memory version, optional textual-inversion and more.
Stable Diffusion GRisk GUI - Windows GUI binary for SD. Closed source so use at your own risk.
Stable Diffusion Infinity - A proof of concept for outpainting with an infinite canvas interface. (requires powerful GPU).
Unstable Fusion - A Stable Diffusion desktop frontend with inpainting, img2img and more
stable-diffusion-webui-docker - A docker based frontend integrating the most popular forks.
ποΈ Synesthetic - Prompt search engine and generator focused on remixing
pip install realesrgan
pip install lama-cleaner
Prompts are the instructions given to diffusion models to manipulate their output.
Stable diffusion prompt book - OpenAI's stable diffusion prompt book, a very comprehensive resource on prompt engineering.
ai-art.com/modifiers - A visual reference guide for keywords.
aipromptguide.com - Visual Database of styles, modifier, artists and persons
ποΈ aiTransformer Prompt Builder - Help build text prompts intuitively using an Image To Text tool to generate detailed text descriptions from image, and 800+ special Textual Inversion terms and often-used generic terms with sample images.
krea.ai - Prompt search engine that also recommends similar prompts to the one that you click on.
Lexica.art - A searchable, visual database of images and the prompts settings used to create them.
pharmapsychotic/clip-interrogator - Jupyter notebook uses CLIP models to suggest a prompt for images similar to a given image (Demo).
ποΈPhraser - A visual prompt builder drawing on a database of examples. (Requires account creation)
ποΈPrompthero = Another visual prompt builder and reference library.
PromptoMania - A visual prompt construction tool.
rom1504/clip-retrieval - Searches for prompt keywords in the datasets used in training Stable Diffusion and other models (Online GUI). Some GUIS like Automatic1111 include this functionality.
Stable Diffusion Prompt Generator - Gives suggestions for improving a given text prompt.
PromptSearch - Yet another Stable Diffusion search engine but with public API
Same Energy - A visual search engine that returns images that have the same 'energy'.
PublicPrompts - Collection of PublicPrompts
Tutorials and high quality educational resources