RI | Publications | Describing Complex Charts in Natural Language: A Caption Generation System

Text only version of this site

Describing Complex Charts in Natural Language: A Caption Generation System
V. Mittal, J. Moore, G. Carenini, and S.F. Roth
Computational Linguistics, Special issue on Natural Language Generation, Vol. 24, No. 3, 1998, pp. 431 - 467.

Jump to: Download | Abstract | Notes | Text Reference | BibTeX Reference

Download [Help]

Adobe portable document format (pdf) [382 KB]

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract

Graphical presentations can be used to communicate information in relational data sets succinctly and effectively. However, novel graphical presentations that represent many attributes and relationships are often difficult to understand completely until explained. Automatically generated graphical presentations must therefore either be limited to generating simple, conventionalized graphical presentations, or risk incomprehensibility. A possible solution to this problem would be to extend automatic graphical presentation systems to generate explanatory captions in natural language, to enable users to understand the information expressed in the graphic. This paper presents a system to do so. It uses a text planner to determine the content and structure of the captions based on: (1) a representation of the structure of the graphical presentation and its mapping to the data it depicts, (2) a framework for identifying the perceptual complexity of graphical elements, and (3) the structure of the data expressed in the graphic. The output of the planner is further processed regarding issues such as ordering, aggregation, centering, generating referring expressions and lexical choice. We discuss the architecture of our system and its strengths and limitations. Our implementation is currently limited to 2-D charts and maps, but, except for lexical information, it is completely domain independent. We illustrate our discussion with figures and generated captions about housing sales in Pittsburgh.

Notes

Associated center: CIMDS
Associated lab/group: Visualization and Intelligent Interfaces Group

Text Reference

V. Mittal, J. Moore, G. Carenini, and S.F. Roth, "Describing Complex Charts in Natural Language: A Caption Generation System," Computational Linguistics, Special issue on Natural Language Generation, Vol. 24, No. 3, 1998, pp. 431 - 467.

BibTeX Reference

@article{Mittal_1998_2854,
   author = "Vibbhu Mittal and Johanna Moore and Giuseppe Carenini and Steven F Roth",
   title = "Describing Complex Charts in Natural Language: A Caption Generation System",
   journal = "Computational Linguistics, Special issue on Natural Language Generation",
   year = "1998",
   volume = "24",
   number = "3",
   pages = "431 - 467"
}