Deep Hybrid Camera Deblurring for Smartphone Cameras


SIGGRAPH 2024 Conference Track

Deep Hybrid Camera Deblurring for Smartphone Cameras

Jaesung Rim,     Junyong Lee,     Heemin Yang,     Sunghyun Cho

...

Abstract


Mobile cameras, despite their significant advancements, still have difficulty in low-light imaging due to compact sensors and lenses, leading to longer exposures and motion blur. Traditional blind deconvolution methods and learning-based deblurring methods can be potential solutions to remove blur. However, achieving practical performance still remains a challenge. To address this, we propose a learning-based deblurring framework for smartphones, utilizing wide and ultra-wide cameras as a hybrid camera system. We simultaneously capture a long-exposure wide image and short-exposure burst ultra-wide images, and utilize the burst images to deblur the wide image. To fully exploit burst ultra-wide images, we present HCDeblur, a practical deblurring framework that includes novel deblurring networks, HC-DNet and HC-FNet. HC-DNet utilizes motion information extracted from burst images to deblur a wide image, and HC-FNet leverages burst images as reference images to further enhance a deblurred output. For training and evaluating the proposed method, we introduce the HCBlur dataset, which consists of synthetic and real-world datasets. Our experiments demonstrate that HCDeblur achieves state-of-the-art deblurring quality.

The HCBlur Dataset


Hybrid Camera Blur dataset
...

Google Drive Link

Google drive link (Soon)

Postech Link

Postech link (Soon)

Method


We introduce a deep neural network equipped with two sub-networks: a Hybrid Camera Deblurring Network (HC-DNet) and a Fusion Network (HC-FNet). HC-DNet uses burst images to construct pixel-wise blur kernels and exploits the blur kernels to obtain a deblurred image of the wide image. While HC-DNet produces a deblurred result of superior quality compared to previous single-image deblurring methods thanks to the blur kernels, its results may still contain artifacts and remaining blur due to information loss caused by blur and inaccurate blur kernels. To mitigate this, HC-FNet, which is inspired by burst imaging techniques, refines the output of HC-DNet using the entire sequence of the burst images as reference images.
...

Results on HCBlur-Syn


Deblurring results on the HCBlur-Syn dataset.

Methods PSNR / SSIM
MIMO-UNet+ 22.42 / 0.5966
Uformer-B 23.31 / 0.6332
HINet 23.60 / 0.6360
NAFNet-32 23.61 / 0.6364
NAFNet-64 24.18 / 0.6607
LSFNet 22.32 / 0.6058
D2HNet 22.64 / 0.6125
NAFNet-Ref 24.37 / 0.6585
UFPNet 24.44 / 0.6701
MotionETR 25.24 / 0.6951
HCDeblur (only HC-DNet) 26.13 / 0.7251
HCDeblur 26.76 / 0.7373
...

Results on HCBlur-Real


Deblurring results on the HCBlur-Real dataset.

Methods NIQE / BRISQUE / TOPIQ
MIMO-UNet+ 5.49 / 27.87 / 0.44
Uformer-B 5.17 / 29.63 / 0.55
HINet 5.42 / 29.74 / 0.47
NAFNet-32 5.05 / 29.50 / 0.53
NAFNet-64 4.83 / 28.09 / 0.55
LSFNet 4.87 / 30.80 / 0.53
D2HNet 4.83 / 31.56 / 0.56
NAFNet-Ref 4.56 / 29.81 / 0.60
UFPNet 4.79 / 28.14 / 0.56
MotionETR 4.52 / 26.00 / 0.63
HCDeblur (only HC-DNet) 4.11 / 25.51 / 0.67
HCDeblur 3.95 / 24.65 / 0.69
...

Bibtex


@inproceedings{HCDeblur_rim,
 title={Deep Hybrid Camera Deblurring for Smartphone Cameras},
 author={Rim, Jaesung and Lee, Junyong and Yang, Heemin and Cho, Sunghyun},
 booktitle={Proceedings of ACM SIGGRAPH},
 year={2023}
}

Let's Get In Touch!


Please feel free to contact us with any feedback, questions, or comments