Skip to main content
Article
Multi-stage progressive image restoration
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
  • Syed Waqas Zamir, Inception Institute of Artificial Intelligence
  • Aditya Arora, Inception Institute of Artificial Intelligence
  • Salman Khan, Mohamed bin Zayed University of Artificial Intelligence
  • Munawar Hayat, Monash University
  • Fahad Shahbaz Khan, Mohamed bin Zayed University of Artificial Intelligence
  • Ming Hsuan Yang, UC Merced
  • Ling Shao, Inception Institute of Artificial Intelligence
Document Type
Conference Proceeding
Abstract

Image restoration tasks demand a complex balance between spatial details and high-level contextualized information while recovering images. In this paper, we propose a novel synergistic design that can optimally balance these competing goals. Our main proposal is a multi-stage architecture, that progressively learns restoration functions for the degraded inputs, thereby breaking down the overall recovery process into more manageable steps. Specifically, our model first learns the contextualized features using encoder-decoder architectures and later combines them with a high-resolution branch that retains local information. At each stage, we introduce a novel per-pixel adaptive design that leverages in-situ supervised attention to reweight the local features. A key ingredient in such a multi-stage architecture is the information exchange between different stages. To this end, we propose a two-faceted approach where the information is not only exchanged sequentially from early to late stages, but lateral connections between feature processing blocks also exist to avoid any loss of information. The resulting tightly interlinked multi-stage architecture, named as MPRNet, delivers strong performance gains on ten datasets across a range of tasks including image deraining, deblurring, and denoising. The source code and pre-trained models are available at https://github.com/swz30/MPRNet.

DOI
10.1109/CVPR46437.2021.01458
Publication Date
1-1-2021
Keywords
  • image denoising,
  • MPRNet,
  • high-level contextualized information,
  • complex balance,
  • multistage progressive image restoration,
  • image deraining,
  • feature processing blocks,
  • information exchange,
  • local information,
  • high-resolution branch,
  • encoder-decoder architectures,
  • contextualized features,
  • restoration functions,
  • multistage architecture,
  • image deblurring
Comments

IR Deposit conditions: non-described

Open Access version available on CVF:

Citation Information
S. W. Zamir et al., "Multi-stage progressive image restoration," in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2021, pp. 14816-14826, doi: 10.1109/CVPR46437.2021.01458.