Advertisement

International Journal of Computer Vision

, Volume 124, Issue 1, pp 49–64 | Cite as

Trajectory-Based Place-Recognition for Efficient Large Scale Localization

  • Simon LynenEmail author
  • Michael Bosse
  • Roland Siegwart
Article

Abstract

Place recognition is a core competency for any visual simultaneous localization and mapping system. Identifying previously visited places enables the creation of globally accurate maps, robust relocalization, and multi-user mapping. To match one place to another, most state-of-the-art approaches must decide a priori what constitutes a place, often in terms of how many consecutive views should overlap, or how many consecutive images should be considered together. Unfortunately, such threshold dependencies limit their generality to different types of scenes. In this paper, we present a placeless place recognition algorithm using a novel match-density estimation technique that avoids heuristically discretizing the space. Instead, our approach considers place recognition as a problem of continuous matching between image streams, automatically discovering regions of high match density that represent overlapping trajectory segments. The algorithm uses well-studied statistical tests to identify the relevant matching regions which are subsequently passed to an absolute pose algorithm to recover the geometric alignment. We demonstrate the efficiency and accuracy of our methodology on three outdoor sequences, including a comprehensive evaluation against ground-truth from publicly available datasets that shows our approach outperforms several state-of-the-art algorithms for place recognition. Furthermore we compare our overall algorithm to the currently best performing system for global localization and show how we outperform the approach on challenging indoor and outdoor datasets.

Keywords

Localization Loop-closure Place recognition 

Supplementary material

Supplementary material 1 (mp4 4676 KB)

References

  1. Agarwal, S., Mierle, K., et al. Ceres solver. https://code.google.com/p/ceres-solver/
  2. Alahi, A., Ortiz, R., & Vandergheynst, P. (2012). Freak: Fast retina keypoint. In: Proceedings of the European conference on computer vision (ECCV).Google Scholar
  3. Arandjelovic, R., & Zisserman, A. (2013). All about VLAD. In: 2013 IEEE conference on computer vision and pattern recognition (CVPR). IEEE.Google Scholar
  4. Arth, C., Wagner, D., Klopschitz, M., Irschara, A., & Schmalstieg, D. (2009). Wide area localization on mobile phones. In: Proceedings of the international symposium on mixed and augmented reality (ISMAR).Google Scholar
  5. Babenko, A., & Lempitsky, V. (2012). The inverted multi-index. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).Google Scholar
  6. Bay, H., Tuytelaars, T., & Gool, L. V. (2006). Surf: Speeded up robust features. In: Proceedings of the European conference on computer vision (ECCV).Google Scholar
  7. Bay, H., Tuytelaars, T., & Van Gool, L. (2006). Surf: Speeded up robust features. In: IEEE European conference on computer vision (ECCV).Google Scholar
  8. Blanco, J. L., Moreno, F.A., & González-Jiménez, J. (2014). The málaga urban dataset: High-rate stereo and lidars in a realistic urban scenario. International Journal of Robotics Research, 33, 207–214. http://www.mrpt.org/MalagaUrbanDataset
  9. Bosse, M., & Zlot, R. (2009). Keypoint design and evaluation for place recognition in 2D lidar maps. Robotics and Autonomous Systems, 57, 1211–1224.CrossRefGoogle Scholar
  10. Cummins, M., & Newman, P. (2011). Appearance-only SLAM at large scale with FAB-MAP 2.0. The International Journal of Robotics Research, 30, 1100–1123.CrossRefGoogle Scholar
  11. Cummins, M., & Newman, P. (2008). Fab-map: Probabilistic localization and mapping in the space of appearance. The International Journal of Robotics Research, 27, 647–665.Google Scholar
  12. Darling, D. A. (1957). The Kolmogorov-Smirnov, Cramer-von Mises tests. The Annals of Mathematical Statistics.Google Scholar
  13. Dong, Z., Zhang, G., Jia, J., & Bao, H. (2009). Keyframe-based real-time camera tracking. In: Proceedings of the international conference on computer vision (ICCV).Google Scholar
  14. Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 4, 381–395.MathSciNetCrossRefGoogle Scholar
  15. Galvez-Lopez, D., & Tardos, J. D. (2012). Bags of binary words for fast place recognition in image sequences. IEEE Transactions on Robotics and Automation, 28, 1188–1197.CrossRefGoogle Scholar
  16. Ge, T., He, K., Ke, Q., & Sun, J. (2014). Optimized Product Quantization. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 33, 117–128.Google Scholar
  17. Glover, A. J., Maddern, W. P., Milford, M. J., & Wyeth, G. F. (2010). FAB-MAP+ RatSLAM: Appearance-based SLAM for multiple times of day. In: IEEE international conference on robotics and automation (ICRA).Google Scholar
  18. Grunert, J. (1841). Das pothenotische Problem in erweiterter Gestalt nebst Bemerkungen über seine Anwendungen in der Geodäisie. Grunerts Archiv fiir Mathematik und Physik, 1, 238–248.Google Scholar
  19. Hesch, J. A., Kottas, D. G., Bowman, S. L., Roumeliotis, S. I. (2014). Camera-imu-based localization: Observability analysis and consistency improvement. The International Journal of Robotics Research. Google Scholar
  20. Hesch, J. A., & Roumeliotis, S. I. (2011). A Direct Least-squares (DLS) solution for PnP. In: Proceedings of the international conference on computer vision (ICCV).Google Scholar
  21. Irschara, A., Zach, C., Frahm, J. M., & Bischof, H. (2009). From structure-from-motion point clouds to fast location recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).Google Scholar
  22. Jegou, H., Douze, M., Schmid, C. Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI).Google Scholar
  23. Jegou, H., Douze, M., Schmid, C. (2008). Hamming embedding and weak geometric consistency for large scale image search. In: Proceedings of the european conference on computer vision (ECCV).Google Scholar
  24. Johns, E., & Yang, G. Z. (2014). Generative methods for long-term place recognition in dynamic scenes. International Journal of Computer Vision, 106, 297–314.MathSciNetCrossRefzbMATHGoogle Scholar
  25. Kendall, A., Grimes, M., & Cipolla, R. (2015). Posenet: A convolutional network for real-time 6-dof camera relocalization. In: Proceedings of the IEEE international conference on computer vision.Google Scholar
  26. Kneip, L., & Lynen, S. (2013). Direct optimization of frame-to-frame rotation. In: IEEE international conference on computer vision (ICCV).Google Scholar
  27. Kuiper, N. (1960). Tests concerning random points on a circle. Proceedings of the Koninklijke Nederlandse Akademie van Wetenschappe.Google Scholar
  28. Leutenegger, S., Chli, M., & Siegwart, R. Y. (2011). BRISK: Binary robust invariant scalable keypoints. In: Proceedings of the international conference on computer vision (ICCV).Google Scholar
  29. Leutenegger, S., Lynen, S., Bosse, M., Siegwart, R., & Furgale, P. (2014). Keyframe-Based Visual-Inertial SLAM Using Nonlinear Optimization. International Journal of Robotics Research (IJRR), 34, 314–334.CrossRefGoogle Scholar
  30. Li, Y., Snavely, N., Huttenlocher, D., & Fua, P. (2010). Worldwide pose estimation using 3D point clouds. In: Proceedings of the european conference on computer vision (ECCV).Google Scholar
  31. Li, Y., Snavely, N., & Huttenlocher, D. P. (2010). Location recognition using prioritized feature matching. In: Proceedings of the European conference on computer vision (ECCV).Google Scholar
  32. Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision (IJCV), 60, 91–110.CrossRefGoogle Scholar
  33. Lynen, S., Bosse, M., Furgale, P., & Siegwart, R. (2014). Placeless place-recognition. In: 3DV.Google Scholar
  34. Maddern, W., Milford, M., & Wyeth, G. (2012). CAT-SLAM: Probabilistic Localisation and Mapping using a Continuous Appearance-based Trajectory. International Journal of Robotics Research (IJRR), 31, 429–451.CrossRefGoogle Scholar
  35. Mei, C., Sibley, G., & Newman, P. (2010). Closing loops without places. In: Proceedings of the IEEE/RSJ conference on intelligent robots and systems (IROS).Google Scholar
  36. Middelberg, S., Sattler, T., Untzelmann, O., & Kobbelt, L. (2014). Scalable 6-DOF localization on mobile devices. In: Proceedings of the European conference on computer vision (ECCV).Google Scholar
  37. Milford, M.J., & Wyeth, G.F. (2012). Seqslam: Visual route-based navigation for sunny summer days and stormy winter nights. In: IEEE international conference on robotics and automation (ICRA).Google Scholar
  38. Mourikis, A., Trawny, N., Roumeliotis, S., Johnson, A., Ansar, A., & Matthies, L. (2009). Vision-aided inertial navigation for spacecraft entry, descent, and landing. IEEE Transactions on Robotics (T-RO) 25, 264–280.Google Scholar
  39. Murphy, L., & Sibley, G. (2014). Incremental unsupervised topological place discovery. In Proceedings of the IEEE international conference on robotics and automation (ICRA).Google Scholar
  40. Naseer, T., Spinello, L., Burgard, W., & Stachniss, C. (2014). Robust visual robot localization across seasons using network flows. In: AAAI.Google Scholar
  41. Neyman, J., & Pearson, E. S. (1992). On the problem of the most efficient tests of statistical hypotheses. New York: Springer.CrossRefzbMATHGoogle Scholar
  42. Nistér, D. (2003). Preemptive RANSAC for live structure and motion estimation. In: Proceedings of the international conference on computer vision (ICCV).Google Scholar
  43. Nistér, D. (2004). An efficient solution to the five-point relative pose problem. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 26, 756–770.CrossRefGoogle Scholar
  44. Nister, D., & Stewenius, H. (2006). Scalable recognition with a vocabulary tree. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).Google Scholar
  45. Pepperell, E., Corke, P.I., & Milford, M.J. (2014). All-environment visual place recognition with smart. In: 2014 IEEE international conference on robotics and automation (ICRA). IEEE.Google Scholar
  46. Philbin, J., Chum, O., Isard, M., Sivic, J., & Zisserman, A. (2007). Object retrieval with large vocabularies and fast spatial matching. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).Google Scholar
  47. Sattler, T., Leibe, B., & Kobbelt, L. (2011). Fast image-based localization using Direct 2D-to-3D Matching. In: Proceedings of the international conference on computer vision (ICCV).Google Scholar
  48. Sattler, T., Leibe, B., & Kobbelt, L. (2012). Improving image-based localization by active correspondence search. In: Proceedings of the European conference on computer vision (ECCV).Google Scholar
  49. Sattler, T., Weyand, T., Leibe, B., & Kobbelt, L. (2012). Image retrieval for image-based localization revisited. In: Proceedings of the British Machine Vision Conference (BMVC).Google Scholar
  50. Se, S., Lowe, D.G., Little, J.J. (2005). Vision-based global localization and mapping for mobile robots. IEEE Transactions on Robotics (T-RO), 21, 364–375.Google Scholar
  51. Sivic, J., & Zisserman, A. (2003). Video google: A text retrieval approach to object matching in videos. In: IEEE international conference on computer vision (ICCV).Google Scholar
  52. Smith, M., Baldwin, I., Churchill, W., Paul, R., Newman, P. (2009). The new college vision and laser data set. The International Journal of Robotics Research.Google Scholar
  53. Stewenius, H., Engels, C., & Nistér, D. (2006). Recent developments on direct relative orientation. ISPRS Journal of Photogrammetry and Remote Sensing, 60, 284–294.CrossRefGoogle Scholar
  54. Stewénius, H., Gunderson, S. H., & Pilet, J. (2012). Size matters: Exhaustive geometric verification for image retrieval. In: Proceedings of the European conference on computer vision (ECCV).Google Scholar
  55. Stumm, E., Mei, C., & Lacroix, S. (2013). Probabilistic place recognition with covisibility maps. In: Proceedings of the IEEE/RSJ conference on intelligent robots and systems (IROS).Google Scholar
  56. Sunderhauf, N., & Protzel, P. (2011). BRIEF-gist-closing the loop by simple means. In: 2011 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE.Google Scholar
  57. Sunderhauf, N., Shirazi, S., Jacobson, A., Dayoub, F., Pepperell, E., Upcroft, B., et al. (2015). Place recognition with convnet landmarks: Viewpoint-robust, condition-robust, training-free. Proceedings of Robotics: Science and Systems XII.Google Scholar
  58. Svarm, L., Enqvist, O., Oskarsson, M., & Kahl, F. (2014). Accurate localization and pose estimation for large 3D models. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).Google Scholar
  59. Trzcinski, T., Lepetit, V., & Fua, P. (2012). Thick Boundaries in Binary Space and their Influence on Nearest-Neighbor Search. Pattern Recognition Letters, 33, 2173–2180.CrossRefGoogle Scholar
  60. Wang, O., Schroers, C., Zimmer, H., Gross, M., & Sorkine-Hornung, A. (2014). Videosnapping: Interactive synchronization of multiple videos. ACM Transactions on Graphics (TOG), 33, 77.Google Scholar
  61. Wendel, A., Irschara, A., & Bischof, H. (2011). Natural landmarkbased monocular localization for MAVs. In: Proceedings of the IEEE international conference on robotics and automation (ICRA).Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Autonomous System LabETH ZurichZurichSwitzerland

Personalised recommendations