Finding (Un)Usual Events in Video

Abstract

We propose an algorithm for detecting and categorizing (un)usual human activity in a video which might be a few days long. The proposed approach is unsupervised, and uses the co-occurrence among large number of simple visual image features to define the activity categories, and to identify what are unusual events automatically. A video is divided into short segments(clips), and motion/color histogram is extract for the foreground object for each image frame. The image features are Vector Quantized into a smaller set of prototype features. A weighted graph is constructed by taking clips and prototype features as nodes, and the co-occurrence relationship between them as the graph edges. We compute an optimal graph embedding that maps the clips and prototype features in a common low dimensional space. This unified embedding ensures that all pair of co-occurring clip and feature are as close as possible. We define event categories by identify cluster of clips in this embedding space, and those isolated clusters are detected as unusual events. We can also classify a new video clips based on the embedding of its co-ocurring features. We demonstrated this algorithm on several long surveillance video recorded at a nursing home.