Learning with Scope, with Application to Information Extraction and Classification

David Blei, J. Andrew (Drew) Bagnell, and Andrew Mccallum
April, 2002.

  • Adobe portable document format (pdf) (132KB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

In probabilistic approaches to classiflcation and information extraction, one typically builds a statistical model of words under the assumption that future data will exhibit the same regularities as the training data. In many data sets, however, there are scope- limited features whose predictive power is only applicable to a certain subset of the data. For example, in information extrac- tion from web pages, word formatting may be indicative of extraction category in difier- ent ways on difierent web pages. The dif- flculty with using such features is capturing and exploiting the new regularities encoun- tered in previously unseen data. In this pa- per, we propose a hierarchical probabilistic model that uses both local/scope-limited fea- tures, such as word formatting, and global features, such as word content. The local regularities are modeled as an unobserved random parameter which is drawn once for each local data set. This random parame- ter is estimated during the inference process and then used to perform classiflcation with both the local and global features| a proce- dure which is akin to automatically retuning the classifler to the local regularities on each newly encountered web page. Exact inference is intractable and we present approximations via point estimates and variational methods. Empirical results on large collections of web data demonstrate that this method signifl- cantly improves performance from traditional models of global features alone.


Text Reference
David Blei, J. Andrew (Drew) Bagnell, and Andrew Mccallum, "Learning with Scope, with Application to Information Extraction and Classification," April, 2002.

BibTeX Reference
   author = "David Blei and J. Andrew (Drew) Bagnell and Andrew Mccallum",
   title = "Learning with Scope, with Application to Information Extraction and Classification",
   booktitle = "",
   month = "April",
   year = "2002",
   number= "CMU-RI-TR-",