Machine Vision and Applications

, Volume 28, Issue 8, pp 953–963 | Cite as

Real-time SLAM relocalization with online learning of binary feature indexing

  • Youji Feng
  • Yihong WuEmail author
  • Lixin Fan
Original Paper


A visual simultaneous localization and mapping (SLAM) system usually contains a relocalization module to recover the camera pose after tracking failure. The core of this module is to establish correspondences between map points and key points in the image, which is typically achieved by local image feature matching. Since recently emerged binary features have orders of magnitudes higher extraction speed than traditional features such as scale invariant feature transform, they can be applied to develop a real-time relocalization module once an efficient method of binary feature matching is provided. In this paper, we propose such a method by indexing binary features with hashing. Being different from the popular locality sensitive hashing, the proposed method constructs the hash keys by an online learning process instead of pure randomness. Specifically, the hash keys are trained with the aim of attaining uniform hash buckets and high collision rates of matched feature pairs, which makes the method more efficient on approximate nearest neighbor search. By distributing the online learning into the simultaneous localization and mapping process, we successfully apply the method to SLAM relocalization. Experiments show that camera poses can be recovered in real time even when there are tens of thousands of landmarks in the map.


SLAM relocalization Binary feature indexing Approximate nearest neighbor search 



This work was supported by the National High Technology Research and Development Program of China (2015AA020504), the National Natural Science Foundation of China under Grant No. 61572499, 61421004 and Nokia Research Grant No. LF14011659182.


  1. 1.
    Calonder, M., Lepetit, V., Strecha, C., Fua, P.: Brief: binary robust independent elementary features. In: Proceedings of European Conference Computer Vision, pp. 778–792 (2010)Google Scholar
  2. 2.
    Davison, A., Reid, I., Molton, N., Stasse, O.: Monoslam: real-time single camera slam. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1052–1067 (2007)CrossRefGoogle Scholar
  3. 3.
    Engel, J., Schöps, T., Cremers, D.: Lsd-slam: Large-scale direct monocular slam. In: Proceedings of European Conference on Computer Vision, pp. 834–849. Springer (2014)Google Scholar
  4. 4.
    Feng, Y., Fan, L., Wu, Y.: Online learning of binary feature indexing for real-time slam relocalization. In: Proceedings of Asian Conference on Computer Vision Workshops, pp. 206–217 (2014)Google Scholar
  5. 5.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and auto cartography. Commun. ACM 24(6), 381–395 (1981)CrossRefGoogle Scholar
  6. 6.
    Galvez-Lpez, D., Tardos, J.D.: Bags of binary words for fast place recognition in image sequences. IEEE Trans. Robot. 28(5), 1188–1197 (2012)CrossRefGoogle Scholar
  7. 7.
    Gionis, A., Indyk, P., Motwani, R.: Similarity search in high dimensions via hashing. In: Proceedings of 25th International Conference on Very Large Data Bases, pp. 518–529 (1999)Google Scholar
  8. 8.
    Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: Proceedings of IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 225–234 (2007)Google Scholar
  9. 9.
    Lepetit, V., Moreno-Noguer, F., Fua, P.: Epnp: an accurate o(n) solution to the pnp problem. Int. J. Comput. Vis. 81(2), 155–166 (2009)CrossRefGoogle Scholar
  10. 10.
    Leutenegger, S., Chli, M., Siegwart, R.: Brisk: binary robust invariant scalable keypoints. In: Proceedings of IEEE International Conference on Computer Vision, pp. 2548–2555 (2011)Google Scholar
  11. 11.
    Lim, H., Sinha, S., Cohen, M., Uyttendaele, M.: Real-time image-based 6-dof localization in large-scale environments. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1043–1050 (2012)Google Scholar
  12. 12.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  13. 13.
    Muja, M., Lowe, D.G.: Fast matching of binary features. In: Conference on Computer and Robot Vision, pp. 404–410 (2012)Google Scholar
  14. 14.
    Mur-Artal, R., Montiel, J.M.M., Tards, J.D.: Orb-slam: a versatile and accurate monocular slam system. IEEE Trans. Robot. 31(5), 1147–1163 (2015)CrossRefGoogle Scholar
  15. 15.
    Nister, D., Stewenius, H.: Scalable recognition with a vocabulary tree. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2161–2168 (2006)Google Scholar
  16. 16.
    Ozuysal, M., Calonder, M., Lepetit, V., Fua, P.: Fast keypoint recognition using random ferns. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 448–461 (2010)CrossRefGoogle Scholar
  17. 17.
    Rosten, E., Drummond, T.: Machine learning for high-speed corner detection. In: Proceedings of European Conference on Computer Vision, pp. 430–443 (2006)Google Scholar
  18. 18.
    Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: Orb: an efficient alternative to sift or surf. In: Proceedings of IEEE International Conference on Computer Vision, pp. 2564–2571 (2011)Google Scholar
  19. 19.
    Sattler, T., Leibe, B., Kobbelt, L.: Fast image-based localization using direct 2d-to-3d matching. In: Proceedings of IEEE International Conference on Computer Vision, pp. 667–674 (2011)Google Scholar
  20. 20.
    Silpa-Anan, C., Hartley, R.: Optimised kd-trees for fast image descriptor matching. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  21. 21.
    Straub, J., Hilsenbeck, S., Schroth, G., Huitl, R., Moller, A., Steinbach, E.: Fast relocalization for visual odometry using binary features. In: Proceedings of IEEE International Conference on Image Processing, pp. 2548–2552 (2013)Google Scholar
  22. 22.
    Tola, E., Lepetit, V., Fua, P.: Daisy: an efficient dense descriptor applied to wide-baseline stereo. IEEE Trans. Pattern Anal. Mach. Intell. 32(5), 815–830 (2010)CrossRefGoogle Scholar
  23. 23.
    Trzcinski, T., Lepetit, V., Fua, P.: Thick boundaries in binary space and their influence on nearest-neighbor search. Pattern Recogn. Lett. 33(16), 2173–2180 (2012)CrossRefGoogle Scholar
  24. 24.
    Williams, B., Klein, G., Reid, I.: Automatic relocalization and loop closing for real-time monocular slam. IEEE Trans. Pattern Anal. Mach. Intell. 33(9), 1699–1712 (2011)CrossRefGoogle Scholar
  25. 25.
    Yianilos, P.: Data structures and algorithms for nearest neighbor search in general metric spaces. In: Proceedings of the fourth annual ACM-SIAM symposium on Discrete algorithms, pp. 311–321 (1993)Google Scholar
  26. 26.
    Yunpeng, L., Snavely, N., Huttenlocher, D., Fua, P.: Worldwide pose estimation using 3d point clouds. In: Proceedings of European Conference on Computer Vision, pp. 15–29 (2012)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  1. 1.Chongqing Institute of Green and Intelligent TechnologyChinese Academy of SciencesChongqingChina
  2. 2.National Laboratory of Pattern Recognition, Institute of AutomationChinese Academy of SciencesBeijingChina
  3. 3.Nokia TechnologiesTampereFinland

Personalised recommendations