Bilateral Filtering-Based Optical Flow Estimation with Occlusion Detection
Using the variational approaches to estimate optical flow between two frames, the flow discontinuities between different motion fields are usually not distinguished even when an anisotropic diffusion operator is applied. In this paper, we propose a multi-cue driven adaptive bilateral filter to regularize the flow computation, which is able to achieve the smoothly varied optical flow field with highly desirable motion discontinuities. First, we separate the traditional one-step variational updating model into a two-step filtering-based updating model. Then, employing our occlusion detector, we reformulate the energy functional of optical flow estimation by explicitly introducing an occlusion term to balance the energy loss due to the occlusion or mismatches. Furthermore, based on the two-step updating framework, a novel multi-cue driven bilateral filter is proposed to substitute the original anisotropic diffusion process, and it is able to adaptively control the diffusion process according to the occlusion detection, image intensity dissimilarity, and motion dissimilarity. After applying our approach on various video sources (movie and TV) in the presence of occlusion, motion blurring, non-rigid deformation, and weak textureness, we generate a spatial-coherent flow field between each pair of input frames and detect more accurate flow discontinuities along the motion boundaries.
KeywordsGaussian Kernel Anisotropic Diffusion Input Frame Occlude Region Smoothness Term
Unable to display preview. Download preview PDF.
- 8.Deriche, R., Kornprobst, P., Aubert, G.: Optical-flow estimation while preserving its discontinuities: a variational approach. In: Asian Conference on Computer Vision, pp. 290–295 (1995)Google Scholar
- 9.Farneback, G.: Very high accuracy velocity estimation using orientation tensors, parametric motion, and simultaneous segmentation of the motion field. In: International Conference on Computer Vision, pp. 171–177 (2001)Google Scholar
- 11.Ju, S., Black, M., Jepson, A.: Skin and bones: Multi-layer, locally affine, optical flow, and regularization with transparency. In: Computer Vision and Pattern Recognition, pp. 307–314 (1996)Google Scholar
- 12.Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)Google Scholar
- 16.Strecha, C., Fransens, R., Van Gool, L.: A probabilistic approach to large displacement optical flow and occlusion detection. In: Workshop on Statistical Methods in Video Processing, pp. 71–82 (2004)Google Scholar
- 17.Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: International Conference on Computer Vision, pp. 839–846 (1998)Google Scholar
- 18.Tschumperle, D., Deriche, R.: Vector-valued image regularization with pde’s: A common framework for different applications. In: Computer Vision and Pattern Recognition, pp. 651–656 (2003)Google Scholar
- 19.Weber, J., Marlik, J.: Bobust computation of optical flow in a multi-scale differential framework. International Journal of Computer Vision 2, 5–19 (1994)Google Scholar
- 20.Xiao, J., Shah, M.: Accurate motion layer segmentation and matting. In: Computer Vision and Pattern Recognition, pp. 698–703 (2005)Google Scholar