Skip to main content
Article
Towards partial supervision for generic object counting in natural scenes
IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Hisham Cholakkal, Mohamed Bin Zayed University of Artificial Intelligence
  • Guolei Sun, Mohamed Bin Zayed University of Artificial Intelligence
  • Salman Khan, Mohamed Bin Zayed University of Artificial Intelligence
  • Fahad Shahbaz Khan, Mohamed Bin Zayed University of Artificial Intelligence
  • Ling Shao, ETH Zürich & Inception Institute of Artificial Intelligence
  • Luc Van Gool, Mohamed Bin Zayed University of Artificial Intelligence
Document Type
Article
Abstract

Generic object counting in natural scenes is a challenging computer vision problem. Existing approaches either rely on instance-level supervision or absolute count information to train a generic object counter. We introduce a partially supervised setting that significantly reduces the supervision level required for generic object counting. We propose two novel frameworks, named lower-count (LC) and reduced lower-count (RLC), to enable object counting under this setting. Our frameworks are built on a novel dual-branch architecture that has an image classification and a density branch. Our LC framework reduces the annotation cost due to multiple instances in an image by using only lower-count supervision for all object categories. Our RLC framework further reduces the annotation cost arising from large numbers of object categories in a dataset by only using lower-count supervision for a subset of categories and class-labels for the remaining ones. The RLC framework extends our dual-branch LC framework with a novel weight modulation layer and a category-independent density map prediction. Experiments are performed on COCO, Visual Genome and PASCAL 2007 datasets. Our frameworks perform on par with state-of-the-art approaches using higher levels of supervision. Additionally, we demonstrate the applicability of our LC supervised density map for image-level supervised instance segmentation.

DOI
10.1109/TPAMI.2020.3021025
Publication Date
3-1-2022
Keywords
  • Generic object counting,
  • object localization,
  • reduced supervision,
  • weakly supervised instance segmentation
Comments

IR deposit conditions:

  • OA version (pathway a)
  • No embargo
  • When accepted for publication, set statement to accompany deposit (see policy)
  • Must link to publisher version with DOI
Citation Information
H. Cholakkal, G. Sun, S. Khan, F. S. Khan, L. Shao and L. Van Gool, "Towards partial supervision for generic object counting in natural scenes," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 3, pp. 1604-1622, 1 March 2022, doi: 10.1109/TPAMI.2020.3021025.