Image Colorization: A Survey and Dataset

This repository is for colorization survey introduced in the following paper

Saeed Anwar, Muhammad Tahir, Chongyi Li, Ajmal Mian, Fahad Shahbaz Khan, Abdul Wahab Muzaffar, "Image Colorization: A Survey and Dataset", arXiv, 2020

Contents

1. Introduction
2. Overview
3. Dataset
4. Results
5. Citation
6. Acknowledgements

Introduction

Image colorization is an essential image processing and computer vision branch to colorize images and videos. Recently, deep learning techniques progressed notably for image colorization. This article presents a comprehensive survey of recent state-of-the-art colorization using deep learning algorithms, describing their fundamental block architectures in terms of skip connections, input etc. as well as optimizers, loss functions, training protocols, and training data etc. Generally, we can roughly categorize the existing colorization techniques into seven classes. Besides, we also provide some additional essential issues, such as benchmark datasets and evaluation metrics. We also introduce a new dataset specific to colorization and perform an experimental evaluation of the publicly available methods. In the last section, we discuss the limitations, possible solutions, and future research directions of the rapidly evolving topic of deep image colorization that the community should further address.

Overview

An overview of the existing single-image colorization techniques are present below

Taxonomy of colorization networks where classification of the colorization networks are based on structure, input, domain, and type of network. The details of each network used for single-image colorization using deep networks are reported in detail in our paper.

Dataset

We aim to remove this unrealistic setting for image colorization by collecting images that are true to their colors. For example, a carrot will have an orange color in most images. Bananas will be either greenish or yellowish. We have collected 723 images from the internet distributed in 20 categories. Each image has an object and a white background. We name our dataset as Natural-Color Dataset (NCD). The following figures shows representative test images for each category from our proposed Natural-Color dataset (NCD).

Our NCD dataset GrayScale & Color-Groundtruth

State-of-the-art method image results on Our NCD Dataset

AutomaticColorizer

ColorCapsNet

ColofulColorization

InstanceAwareColorization

LetBeColor

RealTimeUserGuided

Results

Quantitative Results

Comparisons of the state-of-the-art methods for the colorization in terms of PSNR, SSIM, PCQI, and IQM on our Natural-Color Dataset. The higher value of the metrics indicates better performance.

Visual Results

Visual comparison of colorization algorithms on different fruit images from the Natural-Color Dataset. State-of-the-art colorization algorithms are unable to colorize the images effectively.

Qualitative comparison on a few sample images of vegetables from Natural-Color Dataset. Most of the algorithms fail to reproduce the original colors.

Citation

If you find the code helpful in your resarch or work, please cite the following papers.

@article{anwar2020ColorSurvey,
  title={Image Colorization: A Survey and Dataset},
  author={Anwar, Saeed and Tahir, Muhammad and Li, Chongyi and Mian, Ajmal and Khan, Fahad Shahbaz and Muzaffar, Abdul Wahab},
  journal={arXiv preprint arXiv:2008.10774},
  year={2020}
}

Acknowledgements