A Latent Space Approach to Dynamic Embedding of Co-occurrence Data

Purnamrita Sarkar, Sajid Siddiqi, and Geoffrey Gordon
Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics (AI-STATS), 2007.


Download
  • Adobe portable document format (pdf) (269KB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
We consider dynamic co-occurrence data, such as author-word links in papers published in successive years of the same conference. For static co-occurrence data, researchers often seek an embedding of the entities (authors and words) into a low-dimensional Euclidean space. We generalize a recent static co-occurrence model, the CODE model of Globerson et al. (2004), to the dynamic setting: we seek coordinates for each entity at each time step. The coordinates can change with time to explain new observations, but since large changes are improbable, we can exploit data at previous and subsequent steps to find a better explanation for current observations. To make inference tractable, we show how to approximate our observation model with a Gaussian distribution, allowing the use of a Kalman filter for tractable inference. The result is the first algorithm for dynamic embedding of co-occurrence data which provides distributional information for its coordinate estimates. We demonstrate our model both on synthetic data and on author-word data from the NIPS corpus, showing that it produces intuitively reasonable embeddings. We also provide evidence for the usefulness of our model by its performance on an author-prediction task.

Notes

Text Reference
Purnamrita Sarkar, Sajid Siddiqi, and Geoffrey Gordon, "A Latent Space Approach to Dynamic Embedding of Co-occurrence Data," Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics (AI-STATS), 2007.

BibTeX Reference
@inproceedings{Siddiqi_2007_5688,
   author = "Purnamrita Sarkar and Sajid Siddiqi and Geoffrey Gordon",
   title = "A Latent Space Approach to Dynamic Embedding of Co-occurrence Data",
   booktitle = "Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics (AI-STATS)",
   year = "2007",
}