Skip to main content
SpringerLink
Log in

Trajectory-Based Place-Recognition for Efficient Large Scale Localization

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Place recognition is a core competency for any visual simultaneous localization and mapping system. Identifying previously visited places enables the creation of globally accurate maps, robust relocalization, and multi-user mapping. To match one place to another, most state-of-the-art approaches must decide a priori what constitutes a place, often in terms of how many consecutive views should overlap, or how many consecutive images should be considered together. Unfortunately, such threshold dependencies limit their generality to different types of scenes. In this paper, we present a placeless place recognition algorithm using a novel match-density estimation technique that avoids heuristically discretizing the space. Instead, our approach considers place recognition as a problem of continuous matching between image streams, automatically discovering regions of high match density that represent overlapping trajectory segments. The algorithm uses well-studied statistical tests to identify the relevant matching regions which are subsequently passed to an absolute pose algorithm to recover the geometric alignment. We demonstrate the efficiency and accuracy of our methodology on three outdoor sequences, including a comprehensive evaluation against ground-truth from publicly available datasets that shows our approach outperforms several state-of-the-art algorithms for place recognition. Furthermore we compare our overall algorithm to the currently best performing system for global localization and show how we outperform the approach on challenging indoor and outdoor datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. \({{\mathbf{A }}}\) is computed using the singular value decomposition.

  2. The matches in this plot are down sampled by a factor of 10 for viewing convenience.

  3. A video of the algorithm working online can be found at: https://youtu.be/0ls1MDak1C8

  4. All voting-based algorithms use the same number of matches k per query descriptor.

  5. An example where we use this algorithm today is given at: https://youtu.be/bIKmqZjsc90.

References

  • Agarwal, S., Mierle, K., et al. Ceres solver. https://code.google.com/p/ceres-solver/

  • Alahi, A., Ortiz, R., & Vandergheynst, P. (2012). Freak: Fast retina keypoint. In: Proceedings of the European conference on computer vision (ECCV).

  • Arandjelovic, R., & Zisserman, A. (2013). All about VLAD. In: 2013 IEEE conference on computer vision and pattern recognition (CVPR). IEEE.

  • Arth, C., Wagner, D., Klopschitz, M., Irschara, A., & Schmalstieg, D. (2009). Wide area localization on mobile phones. In: Proceedings of the international symposium on mixed and augmented reality (ISMAR).

  • Babenko, A., & Lempitsky, V. (2012). The inverted multi-index. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Bay, H., Tuytelaars, T., & Gool, L. V. (2006). Surf: Speeded up robust features. In: Proceedings of the European conference on computer vision (ECCV).

  • Bay, H., Tuytelaars, T., & Van Gool, L. (2006). Surf: Speeded up robust features. In: IEEE European conference on computer vision (ECCV).

  • Blanco, J. L., Moreno, F.A., & González-Jiménez, J. (2014). The málaga urban dataset: High-rate stereo and lidars in a realistic urban scenario. International Journal of Robotics Research, 33, 207–214. http://www.mrpt.org/MalagaUrbanDataset

  • Bosse, M., & Zlot, R. (2009). Keypoint design and evaluation for place recognition in 2D lidar maps. Robotics and Autonomous Systems, 57, 1211–1224.

    Article  Google Scholar 

  • Cummins, M., & Newman, P. (2011). Appearance-only SLAM at large scale with FAB-MAP 2.0. The International Journal of Robotics Research, 30, 1100–1123.

    Article  Google Scholar 

  • Cummins, M., & Newman, P. (2008). Fab-map: Probabilistic localization and mapping in the space of appearance. The International Journal of Robotics Research, 27, 647–665.

  • Darling, D. A. (1957). The Kolmogorov-Smirnov, Cramer-von Mises tests. The Annals of Mathematical Statistics.

  • Dong, Z., Zhang, G., Jia, J., & Bao, H. (2009). Keyframe-based real-time camera tracking. In: Proceedings of the international conference on computer vision (ICCV).

  • Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 4, 381–395.

    Article  MathSciNet  Google Scholar 

  • Galvez-Lopez, D., & Tardos, J. D. (2012). Bags of binary words for fast place recognition in image sequences. IEEE Transactions on Robotics and Automation, 28, 1188–1197.

    Article  Google Scholar 

  • Ge, T., He, K., Ke, Q., & Sun, J. (2014). Optimized Product Quantization. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 33, 117–128.

    Google Scholar 

  • Glover, A. J., Maddern, W. P., Milford, M. J., & Wyeth, G. F. (2010). FAB-MAP+ RatSLAM: Appearance-based SLAM for multiple times of day. In: IEEE international conference on robotics and automation (ICRA).

  • Grunert, J. (1841). Das pothenotische Problem in erweiterter Gestalt nebst Bemerkungen über seine Anwendungen in der Geodäisie. Grunerts Archiv fiir Mathematik und Physik, 1, 238–248.

    Google Scholar 

  • Hesch, J. A., Kottas, D. G., Bowman, S. L., Roumeliotis, S. I. (2014). Camera-imu-based localization: Observability analysis and consistency improvement. The International Journal of Robotics Research.

  • Hesch, J. A., & Roumeliotis, S. I. (2011). A Direct Least-squares (DLS) solution for PnP. In: Proceedings of the international conference on computer vision (ICCV).

  • Irschara, A., Zach, C., Frahm, J. M., & Bischof, H. (2009). From structure-from-motion point clouds to fast location recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Jegou, H., Douze, M., Schmid, C. Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI).

  • Jegou, H., Douze, M., Schmid, C. (2008). Hamming embedding and weak geometric consistency for large scale image search. In: Proceedings of the european conference on computer vision (ECCV).

  • Johns, E., & Yang, G. Z. (2014). Generative methods for long-term place recognition in dynamic scenes. International Journal of Computer Vision, 106, 297–314.

    Article  MathSciNet  MATH  Google Scholar 

  • Kendall, A., Grimes, M., & Cipolla, R. (2015). Posenet: A convolutional network for real-time 6-dof camera relocalization. In: Proceedings of the IEEE international conference on computer vision.

  • Kneip, L., & Lynen, S. (2013). Direct optimization of frame-to-frame rotation. In: IEEE international conference on computer vision (ICCV).

  • Kuiper, N. (1960). Tests concerning random points on a circle. Proceedings of the Koninklijke Nederlandse Akademie van Wetenschappe.

  • Leutenegger, S., Chli, M., & Siegwart, R. Y. (2011). BRISK: Binary robust invariant scalable keypoints. In: Proceedings of the international conference on computer vision (ICCV).

  • Leutenegger, S., Lynen, S., Bosse, M., Siegwart, R., & Furgale, P. (2014). Keyframe-Based Visual-Inertial SLAM Using Nonlinear Optimization. International Journal of Robotics Research (IJRR), 34, 314–334.

    Article  Google Scholar 

  • Li, Y., Snavely, N., Huttenlocher, D., & Fua, P. (2010). Worldwide pose estimation using 3D point clouds. In: Proceedings of the european conference on computer vision (ECCV).

  • Li, Y., Snavely, N., & Huttenlocher, D. P. (2010). Location recognition using prioritized feature matching. In: Proceedings of the European conference on computer vision (ECCV).

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision (IJCV), 60, 91–110.

    Article  Google Scholar 

  • Lynen, S., Bosse, M., Furgale, P., & Siegwart, R. (2014). Placeless place-recognition. In: 3DV.

  • Maddern, W., Milford, M., & Wyeth, G. (2012). CAT-SLAM: Probabilistic Localisation and Mapping using a Continuous Appearance-based Trajectory. International Journal of Robotics Research (IJRR), 31, 429–451.

    Article  Google Scholar 

  • Mei, C., Sibley, G., & Newman, P. (2010). Closing loops without places. In: Proceedings of the IEEE/RSJ conference on intelligent robots and systems (IROS).

  • Middelberg, S., Sattler, T., Untzelmann, O., & Kobbelt, L. (2014). Scalable 6-DOF localization on mobile devices. In: Proceedings of the European conference on computer vision (ECCV).

  • Milford, M.J., & Wyeth, G.F. (2012). Seqslam: Visual route-based navigation for sunny summer days and stormy winter nights. In: IEEE international conference on robotics and automation (ICRA).

  • Mourikis, A., Trawny, N., Roumeliotis, S., Johnson, A., Ansar, A., & Matthies, L. (2009). Vision-aided inertial navigation for spacecraft entry, descent, and landing. IEEE Transactions on Robotics (T-RO) 25, 264–280.

  • Murphy, L., & Sibley, G. (2014). Incremental unsupervised topological place discovery. In Proceedings of the IEEE international conference on robotics and automation (ICRA).

  • Naseer, T., Spinello, L., Burgard, W., & Stachniss, C. (2014). Robust visual robot localization across seasons using network flows. In: AAAI.

  • Neyman, J., & Pearson, E. S. (1992). On the problem of the most efficient tests of statistical hypotheses. New York: Springer.

    Book  MATH  Google Scholar 

  • Nistér, D. (2003). Preemptive RANSAC for live structure and motion estimation. In: Proceedings of the international conference on computer vision (ICCV).

  • Nistér, D. (2004). An efficient solution to the five-point relative pose problem. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 26, 756–770.

    Article  Google Scholar 

  • Nister, D., & Stewenius, H. (2006). Scalable recognition with a vocabulary tree. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Pepperell, E., Corke, P.I., & Milford, M.J. (2014). All-environment visual place recognition with smart. In: 2014 IEEE international conference on robotics and automation (ICRA). IEEE.

  • Philbin, J., Chum, O., Isard, M., Sivic, J., & Zisserman, A. (2007). Object retrieval with large vocabularies and fast spatial matching. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Sattler, T., Leibe, B., & Kobbelt, L. (2011). Fast image-based localization using Direct 2D-to-3D Matching. In: Proceedings of the international conference on computer vision (ICCV).

  • Sattler, T., Leibe, B., & Kobbelt, L. (2012). Improving image-based localization by active correspondence search. In: Proceedings of the European conference on computer vision (ECCV).

  • Sattler, T., Weyand, T., Leibe, B., & Kobbelt, L. (2012). Image retrieval for image-based localization revisited. In: Proceedings of the British Machine Vision Conference (BMVC).

  • Se, S., Lowe, D.G., Little, J.J. (2005). Vision-based global localization and mapping for mobile robots. IEEE Transactions on Robotics (T-RO), 21, 364–375.

  • Sivic, J., & Zisserman, A. (2003). Video google: A text retrieval approach to object matching in videos. In: IEEE international conference on computer vision (ICCV).

  • Smith, M., Baldwin, I., Churchill, W., Paul, R., Newman, P. (2009). The new college vision and laser data set. The International Journal of Robotics Research.

  • Stewenius, H., Engels, C., & Nistér, D. (2006). Recent developments on direct relative orientation. ISPRS Journal of Photogrammetry and Remote Sensing, 60, 284–294.

    Article  Google Scholar 

  • Stewénius, H., Gunderson, S. H., & Pilet, J. (2012). Size matters: Exhaustive geometric verification for image retrieval. In: Proceedings of the European conference on computer vision (ECCV).

  • Stumm, E., Mei, C., & Lacroix, S. (2013). Probabilistic place recognition with covisibility maps. In: Proceedings of the IEEE/RSJ conference on intelligent robots and systems (IROS).

  • Sunderhauf, N., & Protzel, P. (2011). BRIEF-gist-closing the loop by simple means. In: 2011 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE.

  • Sunderhauf, N., Shirazi, S., Jacobson, A., Dayoub, F., Pepperell, E., Upcroft, B., et al. (2015). Place recognition with convnet landmarks: Viewpoint-robust, condition-robust, training-free. Proceedings of Robotics: Science and Systems XII.

  • Svarm, L., Enqvist, O., Oskarsson, M., & Kahl, F. (2014). Accurate localization and pose estimation for large 3D models. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).

  • Trzcinski, T., Lepetit, V., & Fua, P. (2012). Thick Boundaries in Binary Space and their Influence on Nearest-Neighbor Search. Pattern Recognition Letters, 33, 2173–2180.

    Article  Google Scholar 

  • Wang, O., Schroers, C., Zimmer, H., Gross, M., & Sorkine-Hornung, A. (2014). Videosnapping: Interactive synchronization of multiple videos. ACM Transactions on Graphics (TOG), 33, 77.

    Google Scholar 

  • Wendel, A., Irschara, A., & Bischof, H. (2011). Natural landmarkbased monocular localization for MAVs. In: Proceedings of the IEEE international conference on robotics and automation (ICRA).

Download references

Author information

Authors and Affiliations

  1. Autonomous System Lab, ETH Zurich, Leonhardstr. 21, 8092, Zurich, Switzerland

    Simon Lynen, Michael Bosse & Roland Siegwart

Authors
  1. Simon Lynen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Bosse
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Roland Siegwart
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simon Lynen.

Additional information

Communicated by Lourdes Agapito, Hiroshi Kawasaki, Katsushi Ikeuchi and Martial Hebert.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 4676 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lynen, S., Bosse, M. & Siegwart, R. Trajectory-Based Place-Recognition for Efficient Large Scale Localization. Int J Comput Vis 124, 49–64 (2017). https://doi.org/10.1007/s11263-016-0947-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-016-0947-9

Keywords

Search

Navigation