nighttime_dehaze (ACMMM'2023)

[ACMMM2023] "Enhancing Visibility in Nighttime Haze Images Using Guided APSF and Gradient Adaptive Convolution"

Introduction

This is an implementation of the ACM'MM2023 APSF paper.

Enhancing Visibility in Nighttime Haze Images Using Guided APSF and Gradient Adaptive Convolution
ACM International Conference on Multimedia (ACMMM2023)
Yeying Jin*, Beibei Lin*, Wending Yan, Yuan Yuan, Wei Ye, and Robby T. Tan

1. Real-world Nighttime Haze and Clean Reference Data

1. RealNightHaze We provide 440 night hazy images.
2. Internet_night_clean1 We provide 411 night clean images.
3. Internet_night_clean2 We provide 50 clean night images, serve as reference images for glow removal.

2. Synthetic Nighttime Haze and Clean Ground Truth

4. GTA5 nighttime fog
   

• ECCV2020 Nighttime Defogging Using High-Low Frequency Decomposition and Grayscale-Color Networks [Paper]
  Wending Yan, Robby T. Tan and Dengxin Dai

3. Nighttime Dehazing Test Code

3.1 For the RealNightHaze | Results,

1. Set the dataset path to RealNightHaze|Internet_night_fog,
2. Download the checkpoint dehaze.pt, put in results/dehaze/model,
3. Run the Python code,

CUDA_VISIBLE_DEVICES=1 python main_test.py --dataset dehaze --datasetpath /diskc/yeying/night_dehaze/dataset/Internet_night_fog/

4. Results are in results/dehaze/output
   Results for RealNightHaze|Internet_night_fog.

3.2 For the Synthetic Dataset | Results,

1. Set Line18 --have_gt to True, set the dataset path to GTA5 or NHR or NHM or NHC,
2. Download the checkpoint GTA5.pt, put in results/GTA5/model, Similarly, for NHR.pt, NHM.pt, NHC.pt,
3. Run the Python code,

CUDA_VISIBLE_DEVICES=1 python main_test.py --dataset NHM --datasetpath /diskc/yeying/night_dehaze/dataset/middlebury/testA/ 
CUDA_VISIBLE_DEVICES=1 python main_test.py --dataset NHC --datasetpath /diskc/yeying/night_dehaze/dataset/Cityscape/testA/ 
CUDA_VISIBLE_DEVICES=1 python main_test.py --dataset NHR --datasetpath /diskc/yeying/night_dehaze/dataset/NHR/testA/ 
CUDA_VISIBLE_DEVICES=1 python main_test.py --dataset GTA5 --datasetpath /diskc/yeying/night_dehaze/GTA5/testA/

4. Results on the Synthetic Benchmark.

4. APSF-Guided Nighttime Glow Rendering

Run the Matlab code to obtain the clean and glow pairs:

APSF_GLOW_RENDER_CODE/synthetic_glow_pairs.m

Change the data path nighttime_dehaze/paired_data/clean_data/,
the paired clean and glow results are saved in nighttime_dehaze/paired_data/clean/ and nighttime_dehaze/paired_data/glow/,
the visualization of light source maps are in nighttime_dehaze/paired_data/glow_render_visual/light_source/.

Run the Matlab code to visualize the Fig.3 in the main paper:

APSF_GLOW_RENDER_CODE/synthetic_glow_fig3_visualization.m

APSF and Alpha Matting are the implementations of the papers:

• CVPR03 Shedding Light on the Weather [Paper]
• CVPR06 A Closed-Form Solution to Natural Image Matting [Paper]

5. Results on the Synthetic Benchmark:

We provide the visualization results in 0_ACMMM23_RESULTS/NHR/index.html, inside the directory 0_ACMMM23_RESULTS/NHR/img_0/ are hazy inputs, 0_ACMMM23_RESULTS/NHR/img_1/ are ground truths, 0_ACMMM23_RESULTS/NHR/img_2/ are our results.

For results corresponding to GTA5, NHM or NHC, please refer to the respective directories.

Evaluation:

Set the dataset_name GTA5 or NHR or NHM or NHC, and run the Python code:

python calculate_psnr_ssim_NH_GTA5.py

License

The code and models in this repository are licensed under the MIT License for academic and other non-commercial uses.
For commercial use of the code and models, separate commercial licensing is available. Please contact:

• Yeying Jin ([email protected])
• Robby T. Tan ([email protected])
• Jonathan Tan ([email protected])

Citation

If this work or the Internet data is useful for your research, please cite our paper.

@inproceedings{jin2023enhancing,
  title={Enhancing visibility in nighttime haze images using guided apsf and gradient adaptive convolution},
  author={Jin, Yeying and Lin, Beibei and Yan, Wending and Yuan, Yuan and Ye, Wei and Tan, Robby T},
  booktitle={Proceedings of the 31st ACM International Conference on Multimedia},
  pages={2446--2457},
  year={2023}
}

@inproceedings{jin2022unsupervised,
  title={Unsupervised night image enhancement: When layer decomposition meets light-effects suppression},
  author={Jin, Yeying and Yang, Wenhan and Tan, Robby T},
  booktitle={European Conference on Computer Vision},
  pages={404--421},
  year={2022},
  organization={Springer}
}

If GTA5 nighttime fog data is useful for your research, please cite the paper.

@inproceedings{yan2020nighttime,
	title={Nighttime defogging using high-low frequency decomposition and grayscale-color networks},
	author={Yan, Wending and Tan, Robby T and Dai, Dengxin},
	booktitle={European Conference on Computer Vision},
	pages={473--488},
	year={2020},
	organization={Springer}
}