Semantic white balance: Semantic Color Constancy using Convolutional Neural Network (SCCCNN)

Note that color constancy using diagonal correction should be applied to the linear RAW image. Here, we did not use that because of the absence of semantic segmentation models for RAW images (just a proof of concept). Read the report for more information.

To generate semantic masks, you can download Refinenet from the following link:

https://github.com/guosheng/refinenet

Remember: you have to use the model trained on MIT ADE20K dataset (ResNet-152) for scene understanding

Download the semantic white balance network from the following link: https://drive.google.com/file/d/0B6CktEG1p54WVVpyMlhOMjBjZk0/view?usp=sharing&resourcekey=0-F3-dFS8yWTq8G0chYu2Lvg

Because of the 4-D images, you are going to get an error states the following: Error using imageInputLayer>iParseInputArguments (line 59) The value of 'InputSize' is invalid. Expected input image size to be a 2 or 3 element vector. For a 3-element vector, the third element must be 3 or 1.

To fix it, do the following: -Open Matlab (run as administrator) -Write: edit imageInputLayer.m

-replace the following code:

function tf = iIsValidRGBImageSize(sz)
tf = numel(sz) == 3 && sz(end) == 3;
end

with the modified function:

function tf = iIsValidRGBImageSize(sz)
tf = numel(sz) == 3 && (sz(end) == 3 || sz(end) == 4);
end

-save

You can read the report from here: https://arxiv.org/abs/1802.00153

If you use this code, please cite it as:

Mahmoud Afifi. "Semantic White Balance: Semantic Color Constancy Using Convolutional Neural Network." arXiv preprint arXiv:1802.00153 (2018).

Related Research Projects

• sRGB Image White Balancing:
  • When Color Constancy Goes Wrong: The first work for white-balancing camera-rendered sRGB images (CVPR 2019).
  • White-Balance Augmenter: Emulating white-balance effects for color augmentation; it improves the accuracy of image classification and image semantic segmentation methods (ICCV 2019).
  • Color Temperature Tuning: A camera pipeline that allows accurate post-capture white-balance editing (CIC best paper award, 2019).
  • Interactive White Balancing: Interactive sRGB image white balancing using polynomial correction mapping (CIC 2020).
  • Deep White-Balance Editing: A multi-task deep learning model for post-capture white-balance editing (CVPR 2020).
• Raw Image White Balancing:
  • APAP Bias Correction: A locally adaptive bias correction technique for illuminant estimation (JOSA A 2019).
  • SIIE: A sensor-independent deep learning framework for illumination estimation (BMVC 2019).
  • C5: A self-calibration method for cross-camera illuminant estimation (arXiv 2020).