Skip to main content
Article
Burst image restoration and enhancement
2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition
  • Akshay Dudhane, Mohamed bin Zayed University of Artificial Intelligence
  • Syed Waqas Zamir, Inception Institute of Artificial Intelligence
  • Salman Khan, Mohamed bin Zayed University of Artificial Intelligence & Australian National University
  • Fahad Shahbaz Khan, Mohamed bin Zayed University of Artificial Intelligence & Linköping University, Sweden
  • Ming-Hsuan Yang, University of California & Yonsei University & Google Research
Document Type
Conference Proceeding
Abstract

Modern handheld devices can acquire burst image sequence in a quick succession. However, the individual acquired frames suffer from multiple degradations and are misaligned due to camera shake and object motions. The goal of Burst Image Restoration is to effectively combine complimentary cues across multiple burst frames to generate high-quality outputs. Towards this goal, we develop a novel approach by solely focusing on the effective information exchange between burst frames, such that the degradations get filtered out while the actual scene details are preserved and enhanced. Our central idea is to create a set of pseudo-burst features that combine complimentary information from all the input burst frames to seamlessly exchange information. However, the pseudo-burst cannot be successfully created unless the individual burst frames are properly aligned to discount inter-frame movements. Therefore, our approach initially extracts pre-processed features from each burst frame and matches them using an edge-boosting burst alignment module. The pseudo-burst features are then created and enriched using multi-scale contextual information. Our final step is to adaptively aggregate information from the pseudo-burst features to progressively increase resolution in multiple stages while merging the pseudo-burst features. In comparison to existing works that usually follow a late fusion scheme with single-stage upsampling, our approach performs favorably, delivering state-of-the-art performance on burst super-resolution, burst low-light image enhancement and burst denoising tasks. The source code and pre-trained models are available at https://github.com/akshaydudhane16/BIPNet. © 2022 IEEE.

DOI
10.1109/CVPR52688.2022.00567
Publication Date
9-27-2022
Keywords
  • Low-level vision,
  • Scene analysis and understanding
Comments

Open Access version, provided by Computer Vision Foundation.

Citation Information
A. Dudhane, S. W. Zamir, S. Khan, F. S. Khan and M. -H. Yang, "Burst Image Restoration and Enhancement," 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 5749-5758, doi: 10.1109/CVPR52688.2022.00567.