[ECCV 2022] LEDNet: Joint Low-light Enhancement and Deblurring in the Dark
Paper | Project Page | Video | Replicate Demo
Shangchen Zhou, Chongyi Li, Chen Change Loy
S-Lab, Nanyang Technological University
(The datasets are hosted on both Google Drive and BaiduPan)
Dataset | Link | Number | Description |
---|---|---|---|
LOL-Blur | Google Drive / BaiduPan (key: dz6u) | 12,000 | A total of 170 videos for training and 30 videos for testing, each of which has 60 frames, amounting to 12,000 paired data. (Note that the first and last 30 frames of each video are NOT consecutive, and their darknesses are simulated differently as well.) |
Real-LOL-Blur | Google Drive / BaiduPan (key: fh32) | 1354 | 482 real-world night blurry images (from RealBlur-J Dataset) + 872 real-world night blurry images acquired by Sony RX10 IV camera. |
|
[1] This method use distorted image as reference. Please refer to the paper for details.
[2] Currently, only naive random forest regression is implemented and does not support backward.
requirements.txt
# git clone this repository
git clone https://github.com/sczhou/LEDNet
cd LEDNet
# create new anaconda env
conda create -n lednet python=3.8 -y
conda activate lednet
# install python dependencies
pip3 install -r requirements.txt
python basicsr/setup.py develop
Before training, you need to:
dataroot_gt
and dataroot_lq
in the corresponding option file.Training LEDNet:
# without GAN
python basicsr/train.py -opt options/train_LEDNet.yml
# with GAN
python basicsr/train.py -opt options/train_LEDNetGAN.yml
This project is built on BasicSR, the detailed tutorial on training commands and config settings can be found here.
Download the LEDNet pretrained model from [Release V0.1.0] to the weights
folder. You can manually download the pretrained models OR download by runing the following command.
python scripts/download_pretrained_models.py LEDNet
Inference LEDNet:
# test LEDNet (paper model)
python inference_lednet.py --model lednet --test_path ./inputs
# test retrained LEDNet (higher PSNR and SSIM)
python inference_lednet.py --model lednet_retrain --test_path ./inputs
# test LEDNetGAN
python inference_lednet.py --model lednetgan --test_path ./inputs
The results will be saved in the results
folder.
# set evaluation metrics of 'psnr', 'ssim', and 'lpips (vgg)'
python scripts/calculate_iqa_pair.py --result_path 'RESULT_ROOT' --gt_path 'GT_ROOT' --metrics psnr ssim lpips
(The released model was retrained using the BasicSR framework, which makes it easier to use or further develop upon this work. NOTE that the PSNR and SSIM scores of retrained model are higher than the paper model.)
Download the CE-ZeroDCE pretrained model from [Release V0.1.0] to the weights
folder. You can manually download the pretrained models OR download by runing the following command.
python scripts/download_pretrained_models.py CE-ZeroDCE
Run low-light generation:
python scripts/generate_low_light_imgs.py --test_path 'IMG_ROOT' --result_path 'RESULT_ROOT' --model_path './weights/ce_zerodce.pth'
To run containerized local inference with LEDNet using Cog, run the following commands in the project root:
cog run python basicsr/setup.py develop
cog predict -i image=@'path/to/input_image.jpg'
You can view this demo running as an API here on Replicate.
This project is licensed under S-Lab License 1.0. Redistribution and use for non-commercial purposes should follow this license.
This project is based on BasicSR. We calculate evaluation metrics using IQA-PyTorch toolbox. Thanks for their awesome works.
If our work is useful for your research, please consider citing:
@InProceedings{zhou2022lednet,
author = {Zhou, Shangchen and Li, Chongyi and Loy, Chen Change},
title = {LEDNet: Joint Low-light Enhancement and Deblurring in the Dark},
booktitle = {ECCV},
year = {2022}
}
If you have any questions, please feel free to reach me out at [email protected]
.