Skip to main content
Article
Learning digital camera pipeline for extreme low-light imaging
Neurocomputing
  • Syed Waqas Zamir, Inception Institute of Artificial Intelligence
  • Aditya Arora, Inception Institute of Artificial Intelligence
  • Salman Khan, Mohamad bin Zayed University of Artificial Intelligence
  • Fahad Shahbaz Khan, Mohamad bin Zayed University of Artificial Intelligence
  • Ling Shao, Inception Institute of Artificial Intelligence
Document Type
Article
Abstract

In low-light conditions, a conventional camera imaging pipeline produces sub-optimal images that are usually dark and noisy due to a low photon count and low signal-to-noise ratio (SNR). We present a data-driven approach that learns the desired properties of well-exposed images and reflects them in images that are captured in extremely low ambient light environments, thereby significantly improving the visual quality of these low-light images. The recent works on this problem only consider a pixel-level loss metric that ignores perceptual quality and thus generate outputs susceptible to visual artifacts. To address this problem, we propose a new loss function that exploits the characteristics of both pixel-wise and perceptual metrics, enabling our deep neural network to learn the camera processing pipeline to transform the short-exposure, low-light RAW sensor data to well-exposed sRGB images. The results show that our method outperforms the state-of-the-art according to psychophysical tests as well as pixel-wise standard metrics and recent learning-based perceptual image quality measures. In essence, the proposed model can potentially replace the conventional digital camera pipeline for the specific case of extreme low-light imaging.

DOI
10.1016/j.neucom.2021.04.076
Publication Date
9-10-2021
Keywords
  • Digital camera pipeline,
  • Learning-based ISP modeling,
  • Low-light image enhancement,
  • RAW to sRGB mapping
Comments

IR deposit conditions:

  • OA version (pathway b)
  • Accepted version
  • 24 month embargo
  • Must link to publisher version with DOI
Citation Information
S. W. Zamir, A. Arora, S. Khan, F. S. Khan, and L. Shao, “Learning digital camera pipeline for extreme low-light imaging,” Neurocomputing, vol. 452, pp. 37–47, Sep. 2021, doi: 10.1016/J.NEUCOM.2021.04.076.