Image inpainting tool powered by SOTA AI Model. Remove any unwanted object, defect, people from your pictures or erase and replace(powered by stable diffusion) any thing on your pictures.
Alt
/Option
and use the mouse wheel to adjust the mouseNew --interactive-seg-model
from Segment Anything in High Quality
Automatically expand the input box when prompt input is selected.
https://github.com/Sanster/IOPaint/assets/3998421/28700fbe-9e5a-4112-a7ac-e0c8a19a5232
Automatically open the browser after the service is started: iopaint start --inbrowser
.
RemoveBG add more model: iopaint start --enable-remove-bg --remove-bg-model briaai/RMBG-1.4
If a plugin is enabled, the model of the plugin can be switched on the frontend page.
You can check the main feature updates of IOPaint in the Beta Release. This release mainly includes:
--mode=Sanster/AnyText
https://github.com/Sanster/IOPaint/assets/3998421/a84811a8-520c-44f7-91b6-1e5b365b1331
It's been a while since the last release of lama-cleaner(now renamed to IOPaint.), partly due to the fact that during this time I have released my own first macOS application OptiClean and started playing Baldur's Gate 3, which has taken up a lot of my free time. Apart from time constraints, another reason is because the code for the project has become increasingly complex, making it difficult to add new features. I was hesitant to make changes to the code, but in the end, I made the decision to completely refactor the front-end and back-end code of the project, hoping that this change will help the project moving forward.
The refactoring includes: switch to Vite, new css framework tailwindcss, new state management library zustand, new UI library shadcn/ui, using more modern python libraries such as fastapi and typer. These refactors were painful and involved significant changes, but I think they were worth it. They made the project structure clearer, easier to add new features, and more accessible for others to contribute code.
Although I am not an AIGC artist or creator, I enjoy developing tools and I wanted to make this project more practical for inpainting and outpainting needs. Below are the new features/models that have been added with this beta release:
After pip install iopaint
, you can use the iopaint
command in the command line. iopaint start
is the command for starting the model service and web interface, while iopaint run
is used for batch processing. You can use iopaint start --help
or iopaint run --help
to view the supported arguments.
Thanks to fastapi, now all backend interfaces have clear API documentation. After starting the service with iopaint start
, you can access http://localhost:8080/docs
to view the API documentation.
Previously, lama-cleaner only supported the built-in anything4
and realisticVision1.4
SD inpaint models. iopaint start --model
now supports automatically downloading models from HuggingFace, such as Lykon/dreamshaper-8-inpainting
. You can search available sd inpaint models on HuggingFace. Not only inpaint models, but you can also specify regular sd models, such as Lykon/dreamshaper-8
. The downloaded models can be seen and switched on the front-end page.
Single file ckpt
/safetensors
models are also supported. IOPaint will search model under stable_diffusion
folder in the --model-dir
(default value ~/.cache) directory.
You can use Extender
to expand images, with optional directions of x, y, and xy. You can use the built-in expansion ratio or adjust the expansion area yourself.
https://github.com/Sanster/lama-cleaner/assets/3998421/ae4e6560-03d3-4911-822a-c482de503f2a
RemoveBG
and Anime Segmentation
are two segmentation models. In previous versions, they could only be used to remove the background of an image. Now, the results from these two models can be used to generate masks for inpainting.
It is possible to extend or shrink interactive segmentation mask or removebg/anime segmentation mask.
I have added more samplers based on this Issue A1111 <> Diffusers Scheduler mapping #4167.
https://huggingface.co/docs/diffusers/main/en/using-diffusers/inference_with_lcm_lora
Latent Consistency Models (LCM) enable quality image generation in typically 2-4 steps making it possible to use diffusion models in almost real-time settings. The model will be downloaded on first use.
https://huggingface.co/docs/diffusers/main/en/using-diffusers/freeu
FreeU is a technique for improving image quality. Different models may require different FreeU-specific hyperparameters, you can find how to adjust these parameters in the original project: https://github.com/ChenyangSi/FreeU
--model Sanster/PowerPaint-V1-stable-diffusion-inpainting
PowerPaint is a stable diffusion model optimized for inpainting/outpainting/remove_object tasks. We can control the model's results by specifying the task.
--model kandinsky-community/kandinsky-2-2-decoder-inpaint
--model diffusers/stable-diffusion-xl-1.0-inpainting-0.1
--interactive-seg-model mobile_sam
Lightweight SAM model, faster and requires fewer resources. Currently, there are many other variations of the SAM model. Feel free to submit a PR!left
or right
to switch ImageFor 1-click window installer users, first of all, I would like to thank you for your support. The current version of IOPaint is a beta version, and I will update the installation package after the official release of IOPaint.
During the development process, lots of coffee was consumed. If you find my project useful, please consider buying me a cup of coffee. Thank you! ❤️. https://ko-fi.com/Z8Z1CZJGY
Hi everyone, I am planning to create a macOS inpainting app, it would be completely native from UI to model, no JS/Python/PyTorch involved. The app will be small and has better local file support e.g. directly modify a local image file. Here is a demo video, the model runs MUCH faster on the M2 GPU than using pytorch(mps device).
https://github.com/Sanster/lama-cleaner/assets/3998421/2796826b-3a0a-4d41-a5bc-d9e5d956efcf
If you are interested, please fill out this form with your email address. I will send an email notification when the app is released and provide a 50% discount code for you. Thank you.
https://lama-cleaner-docs.vercel.app/models/controlnet
--sd-controlnet
: enable controlnet in stable diffusion--sd-controlnet-method
: set controlnet method used, method can be change in web ui
Anime Segmentation: --enable-anime-seg
fix exif issue: https://github.com/Sanster/lama-cleaner/issues/299
remove scikit-image to make install eaiser for python3.11
Use new font: Inter
Show Stable Diffusion inpainting progress:
https://github.com/Sanster/lama-cleaner/assets/3998421/5c8c109a-5bca-4666-8fc6-c6ef07056536
Show prev mask
https://github.com/Sanster/lama-cleaner/assets/3998421/cd96f0c1-4f90-4091-9003-e5c1944f551a
Use Segment Anything model to do interactive segmentation. See demo here: https://twitter.com/sfjccz/status/1643992289294057472?s=20
--enable-interactive-seg --interactive-seg-model=vit_l --interactive-seg-device=cuda
This version contains a lot of features, so I set the version number to 1.0. I hope these updates will help you in your work.
https://user-images.githubusercontent.com/3998421/229261236-c4d59f5c-d293-42ac-9d04-dd73f494f07f.mov
In the post-processing of image cleaning, in addition to erasing, algorithms such as facial repair or super-resolution are often used. Now you can directly use them in Lama Cleaner. See the Plugins Doc for how to use it.
--sd-controlnet
to enable it.--model sd.15 --sd-local-model-path /path/to/your/local/inpainting_model.ckpt
to enable it. You can learn how to create a inpainting in AUTO1111's webui here
New Stable Diffusion inpainting model, you can choose different models for different scenarios
--model=anything4
: converted from https://huggingface.co/andite/anything-v4.0
--model=realisticVision1.4
: converted from https://huggingface.co/SG161222/Realistic_Vision_V1.4
Original | sd1.5 | anything4 |
---|---|---|
--output-dir
is used: https://github.com/Sanster/lama-cleaner/issues/220