Skip to main content
Article
Learning Enriched Features for Fast Image Restoration and Enhancement
arXiv
  • Syed Waqas Zamir, Inception Institute of Artificial Intelligence, United Arab Emirates
  • Aditya Arora, Inception Institute of Artificial Intelligence, United Arab Emirates
  • Salman Khan, Mohamed bin Zayed University of Artificial Intelligence
  • Munawar Hayat, University of California at Merced, and Google, United States & Terminus Group, China
  • Fahad Shahbaz Khan, Mohamed bin Zayed University of Artificial Intelligence
  • Ming-Hsuan Yang, Monash Univeristy, Melbourne, Australia
  • Ling Shao, University of California at Merced, and Google, United States & Terminus Group, China
Document Type
Article
Abstract

Given a degraded input image, image restoration aims to recover the missing high-quality image content. Numerous applications demand effective image restoration, e.g., computational photography, surveillance, autonomous vehicles, and remote sensing. Significant advances in image restoration have been made in recent years, dominated by convolutional neural networks (CNNs). The widely-used CNN-based methods typically operate either on full-resolution or on progressively low-resolution representations. In the former case, spatial details are preserved but the contextual information cannot be precisely encoded. In the latter case, generated outputs are semantically reliable but spatially less accurate. This paper presents a new architecture with a holistic goal of maintaining spatially-precise high-resolution representations through the entire network, and receiving complementary contextual information from the low-resolution representations. The core of our approach is a multi-scale residual block containing the following key elements: (a) parallel multi-resolution convolution streams for extracting multi-scale features, (b) information exchange across the multi-resolution streams, (c) non-local attention mechanism for capturing contextual information, and (d) attention based multi-scale feature aggregation. Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details. Extensive experiments on six real image benchmark datasets demonstrate that our method, named as MIRNet-v2, achieves state-of-the-art results for a variety of image processing tasks, including defocus deblurring, image denoising, super-resolution, and image enhancement. The source code and pre-trained models are available at https://github.com/swz30/MIRNetv2. © 2022, CC BY-NC-SA.

DOI
10.48550/arXiv.2205.01649
Publication Date
4-19-2022
Keywords
  • Contrast Enhancement,
  • Dual-pixel Defocus Deblurring,
  • Image Denoising,
  • Low-light Image Enhancement,
  • Multi-scale Feature Representation,
  • Super-resolution,
  • Color photography,
  • Convolution,
  • Convolutional neural networks,
  • Image denoising,
  • Image reconstruction,
  • Optical resolving power,
  • Remote sensing,
  • Restoration
Comments

Preprint: arXiv

Archived with thanks to arXiv

Preprint License: CC by NC SA 4.0

Uploaded 25 August 2022

Citation Information
S.W. Zamir et al, "Learning Enriched Features for Fast Image Restoration and Enhancement", 2022, arXiv:2205.01649