Skip to main content
Article
Dynamic Prototype Convolution Network for Few-Shot Semantic Segmentation
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
  • Jie Liu, University of Amsterdam, Netherlands
  • Yanqi Bao, Northeastern University, China
  • Guo-Sen Xie, Nanjing University of Science and Technology, China & Mohamed bin Zayed University of Artificial Intelligence
  • Huan Xiong, Mohamed bin Zayed University of Artificial Intelligence
  • Jan-Jakob Sonke, The Netherlands Cancer Institute, Netherlands
  • Efstratios Gavves, University of Amsterdam, Netherlands
Document Type
Conference Proceeding
Abstract

The key challenge for few-shot semantic segmentation (FSS) is how to tailor a desirable interaction among sup-port and query features and/or their prototypes, under the episodic training scenario. Most existing FSS methods im-plement such support/query interactions by solely leveraging plain operations - e.g., cosine similarity and feature concatenation - for segmenting the query objects. How-ever, these interaction approaches usually cannot well capture the intrinsic object details in the query images that are widely encountered in FSS, e.g., if the query object to be segmented has holes and slots, inaccurate segmentation al-most always happens. To this end, we propose a dynamic prototype convolution network (DPCN) to fully capture the aforementioned intrinsic details for accurate FSS. Specifi-cally, in DPCN, a dynamic convolution module (DCM) is firstly proposed to generate dynamic kernels from support foreground, then information interaction is achieved by con-volution operations over query features using these kernels. Moreover, we equip DPCN with a support activation mod-ule (SAM) and a feature filtering module (FFM) to generate pseudo mask and filter out background information for the query images, respectively. SAM and FFM together can mine enriched context information from the query features. Our DPCN is also flexible and efficient under the k-shot FSS setting. Extensive experiments on PASCAL-5i and COCO 20i show that DPCN yields superior performances under both 1-shot and 5-shot settings. © 2022 IEEE.

DOI
10.1109/CVPR52688.2022.01126
Publication Date
6-19-2022
Keywords
  • Information filtering,
  • Semantic Segmentation,
  • Semantic Web,
  • Semantics,
  • And filters,
  • Cosine similarity,
  • Episodic trainings,
  • Feature filtering,
  • Information interaction,
  • Query images,
  • Query object,
  • Segmentation methods,
  • Semantic segmentation,
  • Training scenario
Comments

Open Access version provided by Computer Vision Foundation.

Archived and thanks to CVPR 2022 Open Access.

Uploaded: 15 Feb 2023

Citation Information
J. Liu, Y. Bao, G.S. Xie, H. Xiong, J.J. Sonke, and E. Gavves, "Dynamic Prototype Convolution Network for Few-Shot Semantic Segmentation", in 2022 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR 2022), New Orleans, pp. 11543-11552, June 2022, doi: 10.1109/CVPR52688.2022.01126