Advertisement

Topometric Localization with Deep Learning

  • Gabriel L. OliveiraEmail author
  • Noha Radwan
  • Wolfram Burgard
  • Thomas Brox
Conference paper
Part of the Springer Proceedings in Advanced Robotics book series (SPAR, volume 10)

Abstract

Compared to LiDAR-based localization methods, which provide high accuracy but rely on expensive sensors, visual localization approaches only require a camera and thus are more cost-effective however their accuracy and reliability is typically inferior to LiDAR-based methods. In this work, we propose a vision-based localization approach that learns from LiDAR-based localization methods by using their output as training data, thus combining a cheap, passive sensor with an accuracy that is on-par with LiDAR-based localization. The approach consists of two deep networks trained on visual odometry and topological localization, respectively, and a successive optimization to combine the predictions of these two networks. Furthermore, we introduce a new challenging pedestrian-based dataset for localization with a high degree of noise. Results obtained by evaluating the proposed approach on this novel dataset demonstrate localization errors up to 10 times smaller than those obtained with traditional vision-based localization methods.

Notes

Acknowledgements

This work has been partially supported by the European Commission under the grant numbers H2020-645403-ROBDREAM, ERC-StG-PE7-279401-VideoLearn, the Freiburg Graduate School of Robotics.

References

  1. 1.
    Abadi, M., Agarwal, A., Barham, P., et al.: Tensorflow: large-scale machine learning on heterogeneous distributed systems (2016). arXiv:1603.04467
  2. 2.
    Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., Sivic, J.: NetVLAD: CNN architecture for weakly supervised place recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  3. 3.
    Arroyo, R., Alcantarilla, P.F., Bergasa, L.M., Romera, E.: Fusion and binarization of CNN features for robust topological localization across seasons. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (2016)Google Scholar
  4. 4.
    Badino, H., Huber, D., Kanade, T.: Visual topometric localization. In: IEEE Intelligent Vehicles Symposium (IV) (2011)Google Scholar
  5. 5.
    Chen, Z., Lam, O., Jacobson, A., Milford, M.: Convolutional neural network-based place recognition (2014). arXiv:1411.1509
  6. 6.
    Cleveland, W.S.: Robust locally weighted regression and smoothing scatterplots. J. Am. Stat. Assoc. 74(368), 829–836 (1979)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Clevert, D., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (elus) (2015). arXiv:1511.07289
  8. 8.
    Garcia-Fidalgo, E., Ortiz, A.: Vision-based topological mapping and localization methods: a survey. Robot. Auton. Syst. 64, 1–20 (2015)CrossRefGoogle Scholar
  9. 9.
    He, K., Zhang, X., Ren, R., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  10. 10.
    Huang, G., Liu, Z., Weinberger, K.Q., van der Maaten, L.: Densely connected convolutional networks (2016). arXiv:1608.06993
  11. 11.
    Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift (2015). arXiv:1502.03167
  12. 12.
    Jégou, H., Douze, M., Schmid, C., Pérez, P.: Aggregating local descriptors into a compact image representation. In: IEEE Conference on Computer Vision and Pattern Recognition (2010)Google Scholar
  13. 13.
    Kaiming, H., Xiangyu, Z., Shaoqing, R., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. In: European Conference on Computer Vision (2014)Google Scholar
  14. 14.
    Kendall, A., Grimes, M., Cipolla, R.: Posenet: a convolutional network for real-time 6-dof camera relocalization. In: Proceedings of the IEEE International Conference on Computer Vision (2015)Google Scholar
  15. 15.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization (2014). arXiv:1412.6980
  16. 16.
    Konda, K., Memisevic, R.: Learning visual odometry with a convolutional network. In: Proceedings of the 10th International Conference on Computer Vision Theory and Applications (2015)Google Scholar
  17. 17.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems (2012)Google Scholar
  18. 18.
    Kümmerle, R., Ruhnke, M., Steder, B., Stachniss, C., Burgard, W.: Autonomous robot navigation in highly populated pedestrian zones. J. Field Robot. 32(4), 565–589 (2015)CrossRefGoogle Scholar
  19. 19.
    Lowry, S., Sünderhauf, N., Newman, P., Leonard, J., Cox, D., Corke, P., Milford, M.J.: Visual place recognition: a survey. IEEE Trans. Robot. 32(1), 1–19 (2016)CrossRefGoogle Scholar
  20. 20.
    Mazuran, M., Boniardi, F., Burgard, W., Tipaldi, G.D.: Relative topometric localization in globally inconsistent maps. In: Proceedings of the International Symposium on Robotics Research (2015)Google Scholar
  21. 21.
    Melekhov, I., Kannala, J., Rahtu, E.: Relative camera pose estimation using convolutional neural networks (2017). arXiv:1702.01381
  22. 22.
    Mohanty, V., Agrawal, S., et al.: DeepVO: a deep learning approach for monocular visual odometry (2016). arXiv:1611.06069
  23. 23.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the International Conference on Machine Learning (2010)Google Scholar
  24. 24.
    Nicolai, A., Skeele, R., Eriksen, C., Hollinger, G.A.: Deep learning for laser based odometry estimation. In: RSS Workshop Limits and Potentials of Deep Learning in Robotics (2016)Google Scholar
  25. 25.
    Sprunk, C., Tipaldi, G.D., Cherubini, A., Burgard, W.: Lidar-based teach-and-repeat of mobile robot trajectories. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (2013)Google Scholar
  26. 26.
    Sutskever, I., Martens, J., Dahl, G.E., Hinton, G.E.: On the importance of initialization and momentum in deep learning. In: Proceedings of the International Conference on Machine Learning (2013)Google Scholar
  27. 27.
    Wu, J., Ma, L., Hu, X.: Delving deeper into convolutional neural networks for camera relocalization. In: Proceedings of the IEEE International Conference on Robotics and Automation (2017)Google Scholar
  28. 28.
    Zhou, B., Khosla, A., Lapedriza, A., Torralba, A., Oliva, A.: Places: an image database for deep scene understanding (2016). arXiv:1610.02055

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Gabriel L. Oliveira
    • 1
    Email author
  • Noha Radwan
    • 1
  • Wolfram Burgard
    • 1
  • Thomas Brox
    • 1
  1. 1.Department of Computer ScienceUniversity of FreiburgFreiburg im BreisgauGermany

Personalised recommendations