GIMP plugin that enable the style transfer functionality thanks to artificial intelligence algorithms
This is a prototype plugin (or a proof of concept, if you wish) intended to demonstrate the feasibility to expand GIMP abilities with state of the art machine learning algorithms.
This plugin adds the ability to perform Neural Style Transfer, which reconstruct a content image (the selected GIMP layer) with the style of a style image (selected with the plugin).
You may have used apps that do this kind of processing, like Prisma (or something similar).
You can apply style transfer in two ways:
At the moment, the following approaches are implemented in the plugin:
Based on Fast Style Transfer: transfer the style of well know paintings to your images.
I have not yet received authorization to release its code. Follow Install instruction below to get this implementation working.
Once installed, you will find it in the menu Filters/Style Transfer/Style Transfer...
.
Pros | Cons |
---|---|
Fast | Only 6 styles available |
Nice results | Eventually, need to train new models |
Style's model file size not too big | Training requires expensive GPUs and a lot of time |
By adding new styles, we need more storage space for them |
Based on Adaptive Style Transfer implementation: apply the style of a given artist to your image.
Once installed, you will find it in the menu Filters/Style Transfer/Artist Style Transfer...
.
Pros | Cons |
---|---|
Fast | Only 13 artists available |
Eventually, need to train new models | |
Training requires expensive GPUs and a lot of time | |
Big model file size (requires a lot of storage space) |
Based on Adaptive Style Transfer developed by the Tensorlayer team: transfer the style of any given image to your content images.
Once installed, you will find it in the menu Filters/Style Transfer/Arbitrary Style Transfer...
.
Pros | Cons |
---|---|
Fast | Little bit slower than the others |
No need to train new models for a new style | |
Overall small file size because it only stores the 2 pretrained models |
Please, take a look at the repositories linked above to get a visual idea of what this plugin allows us to do.
I developed this plugin on Ubuntu 18.04, GIMP 2.10.8, Python 2.7. AFAIK tensorflow requires Python 3.5+ on Windows to work, therefore I strongly believe this plugin will not work on Windows (feel free to try and prove me wrong).
Install Tensorflow on the Python environment used by GIMP (usually version 2.7) with pip install tensorflow
.
I tested this plugin with tensorflow
CPU only. Feel free to test it also with tensorflow-gpu
by installing the module pip install tensorflow-gpu
(you need a CUDA-enabled GPU card, with CUDA toolkit installed).
Note: if you are using an old CPU that does not support AVX instruction set, you will have to build the latest version from source (it may take a while on older CPU).
Install the following Python packages: PIL, numpy, scipy, tensorlayer
You also need to install python-tk
with the command: sudo apt-get install python-tk
Open the terminal, move to the GIMP plug-ins folder (usually ~/.config/GIMP/2.10/plug-ins
).
git clone https://github.com/Davide-sd/GIMP-style-transfer.git
cd GIMP-style-transfer
Change permission to allow execution: chmod +x GIMP-style-transfer.py
To complete the setup, you can either:
python install/download.py
, or:Since I have not yet received any reply to my inquire regarding the possibility of releasing that code, you have to follow these steps to get it working.
fast-style-transfer/src/transform.py
into the plugin folder GIMP-style-transfer/implementation_1/src
..ckpt
files into the folder GIMP-style-transfer/implementation_1/models
.At this point you should be able to use the plugin located in the menu Filters/Style Transfer/Style Transfer...
.
GIMP-style-transfer/implementation_2/models
.At this point you should be able to use the plugin located in the menu Filters/Style Transfer/Artist Style Transfer...
.
GIMP-style-transfer/implementation_3/models
At this point you should be able to use the plugin located in the menu Filters/Style Transfer/Arbitrary Style Transfer...
.
With the following plugins you can apply style transfer either to a single layer or a group layer (note that GIMP may crash when using a group layer). In the latter case, the selected style will be transferred to all the sub-layers.
Filters/Style Transfer/Artist Style Transfer...
or Filters/Style Transfer/Style Transfer...
Ok
and wait for the process to complete.Filters/Style Transfer/Arbitrary Style Transfer...
Ok
and wait for the process to complete.You can apply style transfer to a folder containing your content pictures.
File/Batch Style Transfer/
and select the desired implementation to run.OK
and wait for the transfer to complete.Style Transfer requires a huge amount of RAM. I would advise against testing this plugin on systems with less than 4GB of RAM.
To give you an example, with a 1200px X 800px
image I noticed a boost in RAM usage of 2.5GB (when executing the algorithms on CPU only).
The memory usage increases rapidly with the input image size! Start testing with smaller images, then gradually increase the image size.
It would be nice if someone (with a decent amount of RAM) could create a plot relating image size (pixels) vs memory used (see section TODO point 1).
Compute an estimate of the memory necessary to perform the computation on the input image. Do not perform the computation if this value exceeds the total amount of free memory (and inform the user).
Look at the possibility to extract only the meaningfull weights from the Artist Style Transfer implementation models, in order to significantly reduce their file sizes.
Look for strategies to get this plugin working on Windows.
This plugin is meant for GIMP developers. The objectives of this plugin are:
This plugin is NOT meant for the generic GIMP user. Don't even try to go through the hassle of setting up your system for this plugin. You probably would end up disappointed with the results...
Look at the respective implementations pages: you will find the instructions to train new models. This procedure is only recomended if you use a decent CUDA-enabled GPU card. If you are going to train new models, feel free to share them on this repository.