Beyond Categories: The Visual Memex Model for Reasoning About Object Relationships

Tomasz Malisiewicz and Alexei A. Efros
Neural Information Processing Systems (NIPS), December, 2009.


Download
  • Adobe portable document format (pdf) (1MB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
The use of context is critical for scene understanding in computer vision, where the recognition of an object is driven by both local appearance and the object's relationship to other elements of the scene (context). Most current approaches rely on modeling the relationships between object categories as a source of context. In this paper we seek to move beyond categories to provide a richer appearance-based model of context. We present an exemplar-based model of objects and their relationships, the Visual Memex, that encodes both local appearance and 2D spatial context between object instances. We evaluate our model on Torralba's proposed Context Challenge against a baseline category-based system. Our experiments suggest that moving beyond categories for context modeling appears to be quite beneficial, and may be the critical missing ingredient in scene understanding systems.

Keywords
computer vision, object recognition, scene understanding, object context

Notes
Sponsor: NSF GRFP, NSF grant CAREER IIS-0546547, Guggenheim Fellowship, Sloan Fellowship

Text Reference
Tomasz Malisiewicz and Alexei A. Efros, "Beyond Categories: The Visual Memex Model for Reasoning About Object Relationships," Neural Information Processing Systems (NIPS), December, 2009.

BibTeX Reference
@inproceedings{Malisiewicz_2009_6503,
   author = "Tomasz Malisiewicz and Alexei A. Efros",
   title = "Beyond Categories: The Visual Memex Model for Reasoning About Object Relationships",
   booktitle = "Neural Information Processing Systems (NIPS)",
   month = "December",
   year = "2009",
}