Beyond Grids: Learning Graph Representations for Visual Recognition

Yin Li and Abhinav Gupta
Conference Paper, Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS '18), December, 2018


Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract

We propose learning graph representations from 2D feature maps for visual recognition. Our method draws inspiration from region based recognition, and learns to transform a 2D image into a graph structure. The vertices of the graph define clusters of pixels (“regions”), and the edges measure the similarity between these clusters in a feature space. Our method further learns to propagate information across all vertices on the graph, and is able to project the learned graph representation back into 2D grids. Our graph representation facilitates reasoning beyond regular grids and can capture long range dependencies among regions. We demonstrate that our model can be trained from end-to-end, and is easily integrated into existing networks. Finally, we evaluate our method on three challenging recognition tasks: semantic segmentation, object detection and object instance segmentation. For all tasks, our method outperforms state-of-the-art methods.


@conference{Li-2018-113275,
author = {Yin Li and Abhinav Gupta},
title = {Beyond Grids: Learning Graph Representations for Visual Recognition},
booktitle = {Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS '18)},
year = {2018},
month = {December},
} 2019-05-24T14:25:14-04:00