Despite the hierarchical methods performing best, the state of the art hierarchical algorithms have two significant limitations: they require the full video to be loaded into memory, limiting the videos they can process, and the hierarchical output they produce often overwhelms the user with too much data; it is not always clear which level of the hierarchy should be used. The second part of the talk will discuss how we systematically address these two limitations. We have proposed an approximation framework for streaming hierarchical video segmentation motivated by data stream algorithms: each video frame is processed only once and does not change the segmentation of previous frames. We also propose a new criterion for flattening the hierarchy based on the notion of uniform motion entropy; select segments throughout the hierarchy so as to balance the amount of motion entropy within the selected segments. Time permitting, I will present an example of how video understanding can benefit from using the segmentation: for the video label propagation problem, our supervoxel-based propagation method is significantly more capable than the best of the state of the art pixel-based methods.
- This event has passed.
Fall 2012 GRASP Seminar – Jason Corso, University of Buffalo, “Advances in Segmentation for Video Understanding”
November 30, 2012 @ 11:00 am - 12:00 pm
Abstract: The use of video segmentation as an early processing step in video understanding lags behind the use of image segmentation for image understanding, despite many available video segmentation methods. The reasons for this are likely due to a general lack of critical analysis to help us understand which methods work well in which scenarios, and the simple fact that videos are an order of magnitude bigger than images. In this talk, I will cover recent advances in my group that address both of these reasons. First, I will discuss LIBSVX, a suite of five supervoxel methods coupled with a set of spatiotemporal metrics to evaluate the segmentation methods. Our evaluation summarily arrives at the conclusion that hierarchical segmentation methods, which reevaluate similarity at multiple scales within the hierarchy, perform best overall.