Graphics enhanced version of this site

Real-time Computerized Annotation of Pictures
J. Li and J.Z. Wang
IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 30, No. 6, June, 2008.

Jump to: Download | Abstract | Notes | Text Reference | BibTeX Reference

Adobe portable document format (pdf) [3658 KB]

Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Developing effective methods for automated annotation of digital pictures continues to challenge computer scientists. The capability of annotating pictures by computers can lead to breakthroughs in a wide range of applications, including Web image search, online picture-sharing communities, and scientific experiments. In this work, the authors developed new optimization and estimation techniques to address two fundamental problems in machine learning. These new techniques serve as the basis for the Automatic Linguistic Indexing of Pictures - Real Time (ALIPR) system of fully automatic and high speed annotation for online pictures. In particular, the D2-clustering method, in the same spirit as k-means for vectors, is developed to group objects represented by bags of weighted vectors. Moreover, a generalized mixture modeling technique (kernel smoothing as a special case) for non-vector data is developed using the novel concept of Hypothetical Local Mapping (HLM). ALIPR has been tested by thousands of pictures from an Internet photo-sharing site, unrelated to the source of those pictures used in the training process. Its performance has also been studied at an online demonstration site where arbitrary users provide pictures of their choices and indicate the correctness of each annotation word. The experimental results show that a single computer processor can suggest annotation terms in real-time and with good accuracy.

Sponsor: National Science Foundation (at Penn State)
Grant ID: 0219272

Number of pages: 18

J. Li and J.Z. Wang, "Real-time Computerized Annotation of Pictures," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 30, No. 6, June, 2008.

@article{Li_2008_6021,
   author = "Jia Li and James Z Wang",
   title = "Real-time Computerized Annotation of Pictures",
   journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",
   month = "June",
   year = "2008",
   volume = "30",
   number = "6"
}