Featureless Motion Vector-Based Simultaneous Localization, Planar Surface Extraction, and Moving Obstacle Tracking

Part of the Springer Tracts in Advanced Robotics book series (STAR, volume 107)


Motion vectors (MVs) characterize the movement of pixel blocks in video streams and are readily available. MVs not only allow us to avoid expensive feature transform and correspondence computations but also provide the motion information for both the environment and moving obstacles. This enables us to develop a new framework that is capable of simultaneous localization, scene mapping, and moving obstacle tracking. This method first extracts planes from MVs and their corresponding pixel macro blocks (MBs) using properties of plane-induced homographies. We then classify MBs as stationary or moving using geometric constraints on MVs. Planes are labeled as part of the stationary scene or moving obstacles using MB voting. Therefore, we can establish planes as observations for extended Kalman filters (EKFs) for both the stationary scene and moving objects. We have implemented the proposed method. The results show that the proposed method can establish plane-based rectilinear scene structure and detect moving objects while achieving similar localization accuracy of 1-Point EKF. More specifically, the system detects moving obstacles at a true positive rate of 96.6 % with a relative absolution trajectory error of no more than 2.53 %.



Thanks for Y. Lu, J. Lee, M. Hielsberg, X. Wang, Y. Liu, S. Jacob, P. Peelen, Z. Gui, and M. Jiang for their inputs and contributions to the NetBot Laboratory, Texas A&M University.


  1. 1.
    Babu, R., Ramakrishnan, K.: Compressed domain motion segmentation for video object extraction. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. IV-3788–IV-3791 (2002)Google Scholar
  2. 2.
    Blanco-Claraco, J., Nas, F.M.D., Gozalez-Jimenez, J.: The màlaga urban dataset: high-rate stereo and lidars in a realistic urban scenario. Int. J. Robot. Res. (IJRR) (2013) doi: 10.1177/0278364913507326
  3. 3.
    Braillon, C., Pradalier, C., Crowley, J., Laugier, C.: Real-time moving obstacle detection using optical flow methods. In: IEEE Intelligent Vehicles Symposium, pp. 466–471. Tokyo, Japan (2006)Google Scholar
  4. 4.
    Civera, J., Grasa, O., Davison, A., Montiel, J.: 1-point RANSAC for extended Kalman filtering: application to real-time structure from motion and visual odometry. J. Field Robot. 27(5), 609–631 (2010)CrossRefGoogle Scholar
  5. 5.
    Denman, S., Fookes, C., Sridharan, S.: Improved simultaneous computation of motion detection and optical flow for object tracking. In: Digital Image Computing: Techniques and Applications, pp. 175–182 (2009)Google Scholar
  6. 6.
    Eade, E., Drummond, T.: Edge landmarks in monocular slam. In: British Machine Vision Conference (BMVC), pp. 7–16, Sept 2006Google Scholar
  7. 7.
    Favalli, L., Mecocci, A., Moschetti, F.: Object tracking for retrieval applications in MPEG-2. IEEE Trans. Circuits Syst. Video Technol. 10(3), 427–432 (2000)CrossRefGoogle Scholar
  8. 8.
    Flint, A., Mei, C., Reid, I., Murray, D.: Growing semantically meaningful models for visual slam. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 467–474. San Francisco, CA, June 2010Google Scholar
  9. 9.
    Gee, A., Chekhlov, D., Mayol, W., Calway, A.: Discovering planes and collapsing the state space in visual slam. In: BMVC, pp. 1–10 (2007)Google Scholar
  10. 10.
    Gee, A., Chekhlov, D., Calway, A., Mayol-Cuevas, W.: Discovering higher level structure in visual slam. IEEE Trans. Robot. 24(5), 980–990 (2008)CrossRefGoogle Scholar
  11. 11.
    Gil, A., Mozos, O., Ballesta, M., Reinoso, O.: A comparative evaluation of interest point detectors and local descriptors for visual slam. Mach. Vis. Appl. 21(6), 905–920 (2010)CrossRefGoogle Scholar
  12. 12.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)Google Scholar
  13. 13.
    Jeong, W., Lee, K.: Visual slam with line and corner features. In: IEEE/RSJ International Conference on Intelligent Robots and Systems. Beijing, China, Oct 2006Google Scholar
  14. 14.
    Li, W., Song, D.: Toward featureless visual navigation: Simultaneous localization and planar surface extraction using motion vectors in video streams. In: IEEE International Conference on Robotics and Automation. Hong Kong, China, May 2014Google Scholar
  15. 15.
    Li, H., Song, D., Lu, Y., Liu, J.: A two-view based multilayer feature graph for robot navigation. In: IEEE International Conference on Robotics and Automation (ICRA). St. Paul, Minnesota, May 2012Google Scholar
  16. 16.
    Lu, Y., Song, D., Xu, Y., Perera, A., Oh, S.: Automatic building exterior mapping using multilayer feature graphs. In: IEEE International Conference on Automation Scienece and Engineering. Madison, Wisconsin, Aug 2013Google Scholar
  17. 17.
    Lu, Y., Song, D., Yi, J.: High level landmark-based visual navigation using unsupervised geometric constraints in local bundle adjustment. In: IEEE International Conference on Robotics and Automation. Hong Kong, China, May 2014Google Scholar
  18. 18.
    Ohnishi, N., Imiya, A.: Dominant plane detection from optical flow for robot navigation. Pattern Recognit. Lett. 27, 1009–1021 (2006)CrossRefGoogle Scholar
  19. 19.
    Park, S., Lee, J.: Object tracking in MPEG compressed video using mean-shift algorithm. In: Proceedings of the 2003 Joint Conference of the Fourth International Conference on Information, Communications and Signal Processing, 2003 and Fourth Pacific Rim Conference on Multimedia, vol. 2, pp. 748–752 (2003)Google Scholar
  20. 20.
    Pietzsch, T.: Planar features for visual slam. In: KI 2008: Advances in Artificial Intelligence, pp. 119–126. Kaiserslautern, Germany, Sept 2008Google Scholar
  21. 21.
    Toldo, R., Fusiello, A.: Robust multiple structures estimation with J-linkage. In: European Conference on Computer Vision, pp. 537–547 (2008)Google Scholar
  22. 22.
    Wang, Y., Lin, M., Ju, R.: Visual slam and moving-object detection for a small-size humanoid robot. Int. J. Adv. Robot. Syst. 7(2), 133–138 (2010)Google Scholar
  23. 23.
    Wangsiripitak, S., Murray, D.: Avoiding moving outliers in visual slam by tracking moving objects. In: IEEE International Conference on Robotics and Automation. Kobe, Japan, May 2009Google Scholar
  24. 24.
    Yokoyama, T., Iwasaki, T., Watanabe, T.: Motion vector based moving object detection and tracking in the MPEG compressed domain. In: Seventh International Workshop on Content-based Multimedia Indexing, pp. 201–206 (2009)Google Scholar
  25. 25.
    Zhang, J., Song, D.: Error aware monocular visual odometry using vertical line pairs for small robots in urban areas. In: Special Track on Physically Grounded AI (PGAI), the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI-10). Atlanta, Georgia, USA, July 2010Google Scholar
  26. 26.
    Zou, D., Tan, P.: Coslam: collaborative visual slam in dynamic environments. IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 354–366 (2013)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringTexas A&M UniversityCollege StationUSA

Personalised recommendations