Bilateral Filtering-Based Optical Flow Estimation with Occlusion Detection

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3951)


Using the variational approaches to estimate optical flow between two frames, the flow discontinuities between different motion fields are usually not distinguished even when an anisotropic diffusion operator is applied. In this paper, we propose a multi-cue driven adaptive bilateral filter to regularize the flow computation, which is able to achieve the smoothly varied optical flow field with highly desirable motion discontinuities. First, we separate the traditional one-step variational updating model into a two-step filtering-based updating model. Then, employing our occlusion detector, we reformulate the energy functional of optical flow estimation by explicitly introducing an occlusion term to balance the energy loss due to the occlusion or mismatches. Furthermore, based on the two-step updating framework, a novel multi-cue driven bilateral filter is proposed to substitute the original anisotropic diffusion process, and it is able to adaptively control the diffusion process according to the occlusion detection, image intensity dissimilarity, and motion dissimilarity. After applying our approach on various video sources (movie and TV) in the presence of occlusion, motion blurring, non-rigid deformation, and weak textureness, we generate a spatial-coherent flow field between each pair of input frames and detect more accurate flow discontinuities along the motion boundaries.


Gaussian Kernel Anisotropic Diffusion Input Frame Occlude Region Smoothness Term 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Alvarez, L., Deriche, R., Papadopoulo, T., Sánchez, J.: Symmetrical dense optical flow estimation with occlusions detection. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 721–735. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  2. 2.
    Alvarez, L., Weickert, J., Sanchez, J.: Reliable estimation of dense optical flow fields with large displacements. International Journal of Computer Vision 39, 41–56 (2000)CrossRefzbMATHGoogle Scholar
  3. 3.
    Barron, J., Fleet, D., Beauchemin, S.: Performance of optical flow techniques. International Journal of Computer Vision 12, 43–77 (1994)CrossRefGoogle Scholar
  4. 4.
    Black, M., Anandan, P.: The robust estimation of multiple motions: parametric and piecewise smooth flow fields. Computer Vision and Image Understanding 63, 75–104 (1996)CrossRefGoogle Scholar
  5. 5.
    Black, M., Sapiro, G., Marimont, D., Heeger, D.: Bobust anisotropic diffusion. IEEE Trans. on Image Processing 7, 421–432 (1998)CrossRefGoogle Scholar
  6. 6.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  7. 7.
    Burt, P., Adelson, E.: The laplacian pyramid as a compact image code. IEEE Trans. on Communications 31, 532–540 (1983)CrossRefGoogle Scholar
  8. 8.
    Deriche, R., Kornprobst, P., Aubert, G.: Optical-flow estimation while preserving its discontinuities: a variational approach. In: Asian Conference on Computer Vision, pp. 290–295 (1995)Google Scholar
  9. 9.
    Farneback, G.: Very high accuracy velocity estimation using orientation tensors, parametric motion, and simultaneous segmentation of the motion field. In: International Conference on Computer Vision, pp. 171–177 (2001)Google Scholar
  10. 10.
    Horn, B., Schunck, B.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)CrossRefGoogle Scholar
  11. 11.
    Ju, S., Black, M., Jepson, A.: Skin and bones: Multi-layer, locally affine, optical flow, and regularization with transparency. In: Computer Vision and Pattern Recognition, pp. 307–314 (1996)Google Scholar
  12. 12.
    Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)Google Scholar
  13. 13.
    Memin, E., Perez, P.: Hierarchical estimation and segmentation of dense motion fields. International Journal of Computer Vision 46, 129–155 (2002)CrossRefzbMATHGoogle Scholar
  14. 14.
    Nagel, H., Enkelmann, W.: An inverstigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. on Pattern Analysis and Machine Intelligence 8, 565–593 (1986)CrossRefGoogle Scholar
  15. 15.
    Pernoa, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. on Pattern Analysis and Machine Intelligence 12, 629–639 (1990)CrossRefGoogle Scholar
  16. 16.
    Strecha, C., Fransens, R., Van Gool, L.: A probabilistic approach to large displacement optical flow and occlusion detection. In: Workshop on Statistical Methods in Video Processing, pp. 71–82 (2004)Google Scholar
  17. 17.
    Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: International Conference on Computer Vision, pp. 839–846 (1998)Google Scholar
  18. 18.
    Tschumperle, D., Deriche, R.: Vector-valued image regularization with pde’s: A common framework for different applications. In: Computer Vision and Pattern Recognition, pp. 651–656 (2003)Google Scholar
  19. 19.
    Weber, J., Marlik, J.: Bobust computation of optical flow in a multi-scale differential framework. International Journal of Computer Vision 2, 5–19 (1994)Google Scholar
  20. 20.
    Xiao, J., Shah, M.: Accurate motion layer segmentation and matting. In: Computer Vision and Pattern Recognition, pp. 698–703 (2005)Google Scholar
  21. 21.
    Xiao, J., Shah, M.: Motion layer extraction in the presence of occlusion using graph cuts. IEEE Trans. on Pattern Analysis and Machine Intelligence 27, 1644–1659 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  1. 1.Sarnoff CorporationUSA

Personalised recommendations