Multi-task Learning of Visual Odometry Estimators

  • Vitor Campanholo Guizilini
  • Fabio Tozeto Ramos
Part of the Springer Tracts in Advanced Robotics book series (STAR, volume 79)


This paper presents a novel framework for learning visual odometry estimators from a single uncalibrated camera through multi-task non-parametric Bayesian inference. A new methodology, Coupled Gaussian Processes, is developed to jointly estimate vehicle velocity while concomitantly inferring a full covariance matrix of all tasks. Matched image feature descriptors obtained from sequential frames act as inputs and the vehicle’s linear and angular velocities as outputs, allowing its position to be incrementally determined. This approach has three main benefits: firstly, it readily provides uncertainty measurements, thus allowing posterior data fusion with other sensors; secondly, it eliminates the need for camera calibration, as the system essentially learns the transformation between the optical flow and vehicle velocity spaces; thirdly, it provides motion estimation directly, not subject to scaling as in standard structure from motion techniques with monocular cameras. Experiments conducted using imagery collected in urban and off-road environments under challenging conditions show the benefits of the approach for trajectories of up to 2 km. Finally, the framework is integrated into a Exactly Sparse Extended Information Filter for deployment in a SLAM scenario.


Covariance Function Gaussian Process Iterative Close Point Visual Odometry Kernel Machine 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agrawal, M., Konolige, K.: Rough terrain visual odometry. In: Proc. Int. Conference on Advanced Robotics (ICAR) (August 2007)Google Scholar
  2. 2.
    Bonilla, E.V., Chai, K.M., Williams, C.K.I.: Multi-task gaussian process prediction. In: Platt, J.C., Koller, D., Singer, Y., Roweis, S. (eds.) NIPS, pp. 153–160. MIT Press (2008)Google Scholar
  3. 3.
    Boyle, P., Frean, M.: Multiple output gaussian process regression. Tech. rep., University of Wellington (2005)Google Scholar
  4. 4.
    Campbell, J., Sukthankar, R., Nourbakhsh, I.: Techniques for evaluating optical flow for visual odometry in extreme terrain. In: Proc. Int. Conference on Intelligent Robots and Systems, IROS (2004)Google Scholar
  5. 5.
    Cheng, Y., Maimone, M., Matthies, L.: Visual odometry on the mars exploration rovers. In: Int. Conference on Systems, Man and Cybernetics (October 2005)Google Scholar
  6. 6.
    Civera, J., Bueno, D.R., Davison, A.J., Montiel, J.M.M.: Camera self-calibration for sequential bayesian structure from motion. In: Proc. Int. Conference on Robotics and Automation (2009)Google Scholar
  7. 7.
    Corke, P., Detweiler, C., Dunbabin, M., Hamilton, M., Rus, D., Vasilescu, I.: Experiments with underwater robot localization and tracking. In: Proc. Int. Conference on Robotics and Automation, ICRA (April 2007)Google Scholar
  8. 8.
    Corke, P., Strelow, D., Singh, S.: Omnidirectional visual odometry for a planetary rover. In: Int. Conference on Intelligent Robots and Systems, IROS (2004)Google Scholar
  9. 9.
    Cressie, N.: Statistics for Spatial Data. Wiley (1993)Google Scholar
  10. 10.
    Faugeras, O.D., Luong, Q.-T., Maybank, S.J.: Camera Self-Calibration: Theory and Experiments. In: Sandini, G. (ed.) ECCV 1992. LNCS, vol. 588, pp. 321–334. Springer, Heidelberg (1992)Google Scholar
  11. 11.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press (2004)Google Scholar
  12. 12.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer Series in Statistics. Springer New York Inc., New York (2001)zbMATHCrossRefGoogle Scholar
  13. 13.
    Higdon, D.: Space and Space-Time Modeling Using Process Convolutions. In: Quantitative Methods for Current Environmental Issues, pp. 37–54. Springer (2002)Google Scholar
  14. 14.
    Howard, A.: Real-time stereo visual odometry for autonomous ground vehicles. In: Int. Conference on Intelligent Robots and Systems, IROS (September 2008)Google Scholar
  15. 15.
    Kelly, J., Saripalli, S., Sukhatme, G.: Combined visual and inertial navigation for an unmanned aerial vehicle. In: 6th Int. Conference on Field and Service Robotics (2007)Google Scholar
  16. 16.
    Kelly, J., Sukhatme, G.: An experimental study of aerial stereovisual odometry. In: Proc. 6th IFAC Symposium on Intelligent Autonomous Vehicles (2007)Google Scholar
  17. 17.
    Lemaire, T., Berger, C., Jung, I.-K., Lacroix, S.: Vision-based slam: Stereo and monocular approaches. International Journal of Computer Vision (2007)Google Scholar
  18. 18.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision (2004)Google Scholar
  19. 19.
    Lu, F., Milios, E.: Robot pose estimation in unknown environments by matching 2D range scans. In: IEEE Computer Vision and Pattern Recognition Conference, CVPR (1994)Google Scholar
  20. 20.
    MacKsay, D.J.C.: Information Theory, Inference and Learning Algorithms. Cambridge University (2002)Google Scholar
  21. 21.
    Micchelli, C.A., Pontil, M.: Kernels for multi-task learning. In: Advances in Neural Information Processing Systems 20. MIT Press (2004)Google Scholar
  22. 22.
    Moravec, H., Gennery, D.B.: Cart project progress report. Tech. rep., Stanford University (July 1976)Google Scholar
  23. 23.
    Moravec, H.P.: Obstacle Avoidance and Navigation in the Real World by a Seeing Robot Rover. PhD thesis, Stanford University (1980)Google Scholar
  24. 24.
    Neal, R.M.: Bayesian Learning for Neural Networks. Springer-verlag New York Inc. (1996)Google Scholar
  25. 25.
    Nistr, D., Naroditsky, O., Bergen, J.: Visual odometry for ground vehicle applications. Journal of Field Robotics (January 2006)Google Scholar
  26. 26.
    Rasmussen, C.E., Williams, K.I.: Gaussian Processes for Machine Learning. The MIT Press (2006)Google Scholar
  27. 27.
    Roberts, R., Nguyen, H., Krishnamurthi, N., Balch, T.: Memory-based learning for visual odometry. In: Proc. Int. Conference on Robotics and Automation, ICRA (2008)Google Scholar
  28. 28.
    Roberts, R., Potthast, C., Dellaert, F.: Learning general optical flow subspaces for egomotion estimation and detection of motion anomalies. In: Proc. Conference on Computer Vision and Pattern Recognition (June 2009)Google Scholar
  29. 29.
    Scaramuzza, D., Fraundorfer, F., Pollefeys, M., Siegwart, R.: Absolute scale in structure from motion from a single vehicle mounted camera by exploiting nonholonomic constraints. In: Proc. Int. Conference on Computer Vision, ICCV (October 2009)Google Scholar
  30. 30.
    Scaramuzza, D., Siegwart, R.: Appearance guided monocular omnidirectional visual odometry for outdoor ground vehicles. IEEE Transactions on Robotics, Special Issue on Visual SLAM (October 2008)Google Scholar
  31. 31.
    Se, S., Lowe, D., Little, J.: Vision-based mobile robot localization and mapping using scale-invariant features. In: Proceedings of the IEEE Int. Conference on Robotics and Automation (ICRA), pp. 2051–2058 (2001)Google Scholar
  32. 32.
    Sunderhauf, N., Konolige, K., Lacroix, S., Protzel, P.: Visual Odometry using Sparse Bundle Adjustment on an Autonomous Outdoor Vehicle. Tagungsband Autonome Mobile Systeme. Springer (2005)Google Scholar
  33. 33.
    Tardif, J.-P., Pavlidis, Y., Daniilidis, K.: Monocular visual odometry in urban environments using an omnidirectional camera. In: Int. Conference on Intelligent Robots and Systems (IROS), pp. 2531–2538 (September 2008)Google Scholar
  34. 34.
    Tomasi, C., Zhang, J.: Is structure-from-motion worth pursuing? In: Proc. 7th International Symposium on Robotics Research (ISRR), pp. 391–400 (October 1995)Google Scholar
  35. 35.
    Walter, M.R., Eustice, R.M., Leonard, J.J.: Exactly sparse extended information filters for feature-based slam. International Journal of Robotics Research 26(4), 335–359 (2007)CrossRefGoogle Scholar
  36. 36.
    Williams, C.K.I.: Computation with infinite neural networks. Neural Computation 10(5), 1203–1216 (1998)CrossRefGoogle Scholar
  37. 37.
    Zhu, Z.W., Oskiper, T., Naroditsky, O., Samarasekera, S., Sawhney, H.S., Kumar, R.: An improved stereo-based visual odometry system. In: Proc. Workshop of Performance Metrics for Intelligent Systems (2006)Google Scholar

Copyright information

© Springer-Verlag GmbH Berlin Heidelberg 2014

Authors and Affiliations

  • Vitor Campanholo Guizilini
    • 1
  • Fabio Tozeto Ramos
    • 1
  1. 1.Australian Centre for Field RoboticsUniversity of SydneySydneyAustralia

Personalised recommendations