Part-aware prototype Network for Few-shot Semantic Segmentation

Model Overview

Abstract

Few-shot semantic segmentation aims to learn to segment new object classes with only a few annotated examples, which has a wide range of real-world applications. Most existing methods either focus on the restrictive setting of one-way few-shot segmentation or suffer from incomplete coverage of object regions. In this paper, we propose a novel few-shot semantic segmentation framework based on the prototype representation. Our key idea is to decompose the holistic class representation into a set of part-aware prototypes, capable of capturing diverse and fine-grained object features. In addition, we propose to leverage unlabeled data to enrich our part-aware prototypes, resulting in better modeling of intra-class variations of semantic objects. We develop a novel graph neural network model to generate and enhance the proposed part-aware prototypes based on labeled and unlabeled images. Extensive experimental evaluations on two benchmarks show that our method outperforms the prior art with a sizable margin.

Publication
In European Conference of Computer Vision 2020
Yongfei Liu
Yongfei Liu
PhD Students

My research interests include Cross-modal Reasoning, Scene Understanding, Commonsense Reasoning, few/low-shot learning

Xiangyi Zhang
Xiangyi Zhang
Master Student

My research interests include few-shot segmentation, biological image segmentation and interactive segmentation.

Songyang Zhang
Songyang Zhang
PhD Student

My research interests include few/low-shot learning, graph neural networks and video understanding.

Xuming He
Xuming He
Associate Professor

My research interests include few/low-shot learning, graph neural networks and video understanding.

Related