MSR Speaking Qualifier
Carnegie Mellon University
2:30 pm - 4:00 pm
Title: Detailed Image Captioning with Hierarchical Attention
Automatic image description is the task of generating a natural sentence which reflects the visual content of an image. A lot of deep learning architectures have been explored in the past few years. While researchers have made great improvement on generating syntactically correct sentences by learning from large image-sentence paired datasets, generating semantically rich content has remained a major challenge. We developed a hierarchical modular attention architecture to incorporate compositional components into a sequential model. We demonstrate superior performance in subcategories, i.e., expressing counts and color with examples and show results of human study on Amazon Mechanical Turk.
Jean Oh (advisor, RI)
Louis-Philippe Morency (LTI)
Gunnar Atli Sigurdsson (PhD, RI)