Language-based Colorization of Scene Sketches. (SIGGRAPH Asia 2019)
Paper | Supplementary Material | Project Page
This repository hosts the datasets and the code for the SketchyScene Colorization system (SIGGRAPH Asia 2019). Please refer to our paper for more information: Language-based Colorization of Scene Sketches.
Our system supports two-mode interactive colorization for a given input scene sketch and text-based colorization instructions, using three models, namely, the instance matching model, foreground colorization model, and background colorization model.
Please follow the instructions in the following three sections (Instance Matching, Foreground Instance Colorization, and Background Colorization) to download the dataset and pre-trained models and place them in the correct directories.
For the details of MATCHING dataset and the code, please refer to the Instance_Matching directory.
For the details of FOREGROUND dataset and the code, please refer to the Foreground_Instance_Colorization directory.
For the details of BACKGROUND dataset and the code, please refer to the Background_Colorization directory.
Our system allows users to colorize the sketches through language instructions. If the result is not satisfactory, users can also withdraw the last instruction.
:fire: We have provided some test examples in examples
directory.
To colorize a sketch, run the command like:
python3 sketchyscene_colorization_main.py --image_id 9996 \
--instruction 'the bus is orange with gray windows'
image_id
to the sketch you want.instruction
.You will see the results in outputs
directory.
To withdraw the last instruction, run the command like:
python3 sketchyscene_colorization_main.py --command 'withdraw' --image_id 9996
See what happens in outputs
directory :)
Please cite the corresponding paper if you found the datasets or code useful:
@article{zouSA2019sketchcolorization,
title = {Language-based Colorization of Scene Sketches},
author = {Zou, Changqing and Mo, Haoran and Gao, Chengying and Du, Ruofei and Fu, Hongbo},
journal = {ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH Asia 2019)},
year = {2019},
volume = 38,
number = 6,
pages = {233:1--233:16}
}
@inproceedings{zou2018sketchyscene,
title={Sketchyscene: Richly-annotated scene sketches},
author={Zou, Changqing and Yu, Qian and Du, Ruofei and Mo, Haoran and Song, Yi-Zhe and Xiang, Tao and Gao, Chengying and Chen, Baoquan and Zhang, Hao},
booktitle={Proceedings of the european conference on computer vision (ECCV)},
pages={421--436},
year={2018}
}