Dynamic Context Correspondence Network for Semantic Alignment

Abstract

Establishing semantic correspondence is a core problem in computer vision and remains challenging due to large intra-class variations and lack of annotated data. In this paper, we aim to incorporate global semantic context in a flexible manner to overcome the limitations of prior work that relies on local semantic representations. To this end, we first propose a context-aware semantic representation that incorporates spatial layout for robust matching against local ambiguities. We then develop a novel dynamic fusion strategy based on attention mechanism to weave the advantages of both local and context features by integrating semantic cues from multiple scales. We instantiate our strategy by designing an end-to-end learnable deep network, named as Dynamic Context Correspondence Network (DCCNet). To train the network, we adopt a multi-auxiliary task loss to improve the efficiency of our weakly-supervised learning procedure. Our approach achieves superior or competitive performance over previous methods on several challenging datasets, including PF-Pascal, PF-Willow, and TSS, demonstrating its effectiveness and generality.

Publication
In International Conference on Computer Vision, 2019
Shuaiyi Huang
Shuaiyi Huang
Master Student

My research interests broadly include Deep Learning and Computer Vision, with a focus on scene understanding and low-level vision using strong or weak supervision.

Qiuyue Wang
Qiuyue Wang
Master Student

My research interests include video understanding, video segmentation and semantic alignment.

Songyang Zhang
Songyang Zhang
PhD Student

My research interests include few/low-shot learning, graph neural networks and video understanding.

Shipeng Yan
Shipeng Yan
PhD Student

My research interests include few/low-shot learning, incremental learning and representation learning.

Xuming He
Xuming He
Associate Professor

My research interests include few/low-shot learning, graph neural networks and video understanding.

Related