Causal Video Segmentation Using Superseeds and Graph Matching

Abstract

The goal of video segmentation is to group pixels into meaningful spatiotemporal regions that exhibit coherence in appearance and motion. Causal video segmentation methods use only past video frames to achieve the final segmentation. The problem of causal video segmentation becomes extremely challenging due to size of the input, camera motion, occlusions, non-rigid object motion, and uneven illumination. In this paper, we propose a novel framework for semantic segmentation of causal video using superseeds and graph matching. We first employ SLIC for the extraction of superpixels in a causal video frame. A set of superseeds is chosen from the superpixels in each frame using color and texture based spatial affinity measure. Temporal coherence is ensured through propagation of labels of the superseeds across each pair of adjacent frames. A graph matching procedure based on comparison of the eigenvalues of graph Laplacians is employed for label propagation. Watershed algorithm is applied finally to label the remaining pixels to achieve final segmentation. Experimental results clearly indicate the advantage of the proposed approach over some recently reported works.