K-groups: Tractable Group Detection on Large Link Data Sets

Jeremy Martin Kubica, Andrew Moore, and Jeff Schneider
tech. report CMU-RI-TR-03-32, Robotics Institute, Carnegie Mellon University, October, 2003


Download
  • Adobe portable document format (pdf) (129KB)
Copyright notice: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Abstract
Discovering underlying structure from co-occurrence data is an important task in many fields, including: insurance, intelligence, criminal investigation, epidemiology, human resources, and marketing. For example a store may wish to identify underlying sets of items purchased together or a human resources department may wish to identify groups of employees that collaborate with each other.

Previously Kubica et. al. presented the group detection algorithm (GDA) - an algorithm for finding underlying groupings of entities from co-occurrence data. This algorithm is based on a probabilistic generative model and produces coherent groups that are consistent with prior knowledge. Unfortunately, the optimization used in GDA is slow, making it potentially infeasible for many real world data sets.

To this end, we present k-groups - an algorithm that uses an approach similar to that of k-means (hard clustering and localized updates) to significantly accelerate the discovery of the underlying groups while retaining GDA's probabilistic model. In addition, we show that k-groups is guaranteed to converge to a local minimum. We also compare the performance of GDA and k-groups on several real world and artificial data sets, showing that k-groups' sacrifice in solution quality is significantly offset by its increase in speed. This trade-off makes group detection tractable on significantly larger data sets.


Keywords
Link Analysis, Clustering

Notes

Text Reference
Jeremy Martin Kubica, Andrew Moore, and Jeff Schneider, "K-groups: Tractable Group Detection on Large Link Data Sets," tech. report CMU-RI-TR-03-32, Robotics Institute, Carnegie Mellon University, October, 2003

BibTeX Reference
@techreport{Kubica_2003_4489,
   author = "Jeremy Martin Kubica and Andrew Moore and Jeff Schneider",
   title = "K-groups: Tractable Group Detection on Large Link Data Sets",
   booktitle = "",
   institution = "Robotics Institute",
   month = "October",
   year = "2003",
   number= "CMU-RI-TR-03-32",
   address= "Pittsburgh, PA",
}