Abstract: For persistent surveillance of wide-area, busy scenes such as urban regions, our goal is to detect and track all moving objects, and to analyze their movements to determine whether threatening or suspicious behaviors are occurring. There are many challenges to be overcome – maintaining object identity through occlusions, tracking multiple objects in close proximity, determining whether any objects are involved in one of many possible threats, detecting abnormal behavior, handling tracking errors, and so on. To address these challenges, we have developed approaches to multi-object tracking and the recognition of complex activities and behaviors. The tracking problem is formulated as a grouping problem, such that high- confidence track fragments on individual objects are linked together using a global cost matrix. This shifts most of the tracking likelihood computation to a higher level, where more information is available and complexity is lower. For complex activity recognition, we have developed methods using Dynamic Bayesian Networks (DBNs) defined on relational semantic primitives to achieve robustness against nuisance factors such as viewpoint changes. The DBN framework is integrated with track linking to jointly address tracking and event recognition, by finding track linking solutions that maximize the probability of event occurrence. Results are shown on difficult scenes with dozens or hundreds of moving vehicles and people.
Joint work with Michael Chan, Amitha Perera, Glen Brooksby, Rahul Bhotika, Zhaohui Sun, Chukka Srinivas, John Schmiederer, Bob Kaucic.