Moving Object Segmentation Using Optical Flow and Depth Information

  • Jens Klappstein
  • Tobi Vaudrey
  • Clemens Rabe
  • Andreas Wedel
  • Reinhard Klette
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5414)


This paper discusses the detection of moving objects (being a crucial part of driver assistance systems) using monocular or stereoscopic computer vision. In both cases, object detection is based on motion analysis of individually tracked image points (optical flow), providing a motion metric which corresponds to the likelihood that the tracked point is moving. Based on this metric, points are segmented into objects by employing a globally optimal graph-cut algorithm. Both approaches are comparatively evaluated using real-world vehicle image sequences.


Motion detection optical flow stereo segmentation 


  1. 1.
    Argyros, A.A., Lourakis, M.I., Trahanias, P.E., Orphanoudakis, S.C.: Qualitative detection of 3d motion discontinuities. In: Proc. IEEE/RSJ Int. Conf. Intelligent Robots Systems, vol. 3, pp. 1630–1637 (1996)Google Scholar
  2. 2.
    Armangué, X., Araújo, H., Salvi, J.: Differential epipolar constraint in mobile robot egomotion estimation. In: Proc. IEEE Int. Conf. Pattern Recognition, pp. 599–602 (2002)Google Scholar
  3. 3.
    Avidan, S., Shashua, A.: Trajectory triangulation: 3d reconstruction of moving points from a monocular image sequence. IEEE Trans. Pattern Analysis Machine Intelligence 22, 348–357 (2000)CrossRefGoogle Scholar
  4. 4.
    Badino, H.: A robust approach for ego-motion estimation using a mobile stereo platform. In: Proc. Int. Workshop Complex Motion (2004)Google Scholar
  5. 5.
    Baehring, D., Simon, S., Niehsen, W., Stiller, C.: Detection of close cut-in and overtaking vehicles for driver assistance based on planar parallax. In: Proc. IEEE Intelligent Vehicles Symposium (2005)Google Scholar
  6. 6.
    Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/Max-flow algorithms for energy minimization in vision. In: Figueiredo, M., Zerubia, J., Jain, A.K. (eds.) EMMCVPR 2001. LNCS, vol. 2134, pp. 359–374. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  7. 7.
    Chalimbaud, P., Berry, F., Marmoiton, F., Alizon, S.: Design of a hybrid visuo-inertial smart sensor. In: Proc. Workshop Integration Vision Inertial Sensors (in conjunction with IEEE Int. Conf. Robotics Automation) (2005)Google Scholar
  8. 8.
    Clauss, M., Bayerl, P., Neumann, H.: Segmentation of independently moving objects using a maximum-likelihood principle. In: Proc. Autonome Mobile Systeme (2005)Google Scholar
  9. 9.
    Franke, U., Rabe, C., Badino, H., Gehrig, S.: 6D-vision: Fusion of stereo and motion for robust environment perception. In: Kropatsch, W.G., Sablatnig, R., Hanbury, A. (eds.) DAGM 2005. LNCS, vol. 3663, pp. 216–223. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  10. 10.
    Giachetti, A., Campani, M., Torre, V.: The use of optical flow for road navigation. IEEE Trans. Robotics and Automation 14, 34–48 (1998)CrossRefGoogle Scholar
  11. 11.
    Hartley, R., Vidal, R.: The multibody trifocal tensor: Motion segmentation from 3 perspective views. In: Proc. IEEE Int. Conf. Computer Vision Pattern Recognition (2004)Google Scholar
  12. 12.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2003)Google Scholar
  13. 13.
    Heinrich, S.: Fast obstacle detection using flow/depth constraint. In: Proc. IEEE Intelligent Vehicles Symposium, vol. 2, pp. 658–665 (2002)Google Scholar
  14. 14.
    Ke, Q., Kanade, T.: Transforming camera geometry to a virtual downward-looking camera: Robust ego-motion estimation and ground-layer detection. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pp. I–390– I–397 (2003)Google Scholar
  15. 15.
    Klappstein, J., Stein, F., Franke, U.: Monocular motion detection using spatial constraints in a unified manner. In: IEEE Intelligent Vehicles Symposium, IV (2006)Google Scholar
  16. 16.
    Klappstein, J., Stein, F., Franke, U.: Applying Kalman filtering to road homography estimation. In: Proc. Workshop Planning Perception Navigation Intelligent Vehicles (in conjunction with IEEE Int. Conf. Robotics Automation) (2007)Google Scholar
  17. 17.
    Klappstein, J., Stein, F., Franke, U.: Detectability of moving objects using correspondences over two and three frames. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 112–121. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  18. 18.
    Martin, M.C., Moravec, H.: Robot evidence grids. Technical Report CMU-RI-TR-96-06, Robotics Institute, Carnegie Mellon University (1996)Google Scholar
  19. 19.
    Rabe, C., Franke, U., Gehrig, S.: Fast detection of moving objects in complex scenarios. In: Proc. IEEE Intelligent Vehicles Symposium, pp. 398–403 (2007)Google Scholar
  20. 20.
    Tomasi, C., Kanade, T.: Detection and tracking of point features. Carnegie Mellon University, Technical Report CMU-CS-91-132 (1991)Google Scholar
  21. 21.
    Vaudrey, T., Gruber, D., Wedel, A., Klappstein, J.: Space-time multi-resolution banded graph-cut for fast segmentation. In: Rigoll, G. (ed.) DAGM 2008. LNCS, vol. 5096, pp. 203–213. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  22. 22.
    Waxman, A.M., Duncan, J.H.: Binocular image flows: steps toward stereo-motion fusion. IEEE Trans. Pattern Analysis Machine Intelligence 8, 715–729 (1986)CrossRefGoogle Scholar
  23. 23.
    Wedel, A., Schoenemann, T., Brox, T., Cremers, D.: Warpcut - fast obstacle segmentation in monocular video. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 264–273. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  24. 24.
    Woelk, F., Koch, R.: Fast monocular bayesian detection of independently moving objects by a moving observer. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds.) DAGM 2004. LNCS, vol. 3175, pp. 27–35. Springer, Heidelberg (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Jens Klappstein
    • 1
  • Tobi Vaudrey
    • 2
  • Clemens Rabe
    • 1
  • Andreas Wedel
    • 1
  • Reinhard Klette
    • 2
  1. 1.Environment Perception Group, Daimler AGSindelfingenGermany
  2. 2.enpeda.. ProjectThe University of AucklandNew Zealand

Personalised recommendations