Skip to main content
SpringerLink
Log in

Slam loop closure detection algorithm based on MSA-SG

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

This paper introduces an innovative method to improve loop closure detection within the domain of Simultaneous Localization And Mapping (SLAM) by integrating a Multi-Scale Attention and Semantic Guidance (MSA-SG) framework. In SLAM systems, accurate loop closure detection is essential for minimizing localization errors over time and ensuring the reliability of the constructed maps in robotics navigation through uncharted environments. Our proposed method utilizes EfficientNet-EA for robust feature extraction and introduces MSA-SG, a novel mechanism that synergizes multiscale attention with semantic guidance to focus on critical semantic features essential for loop closure detection. This approach ensures the prioritization of static environmental landmarks over transient and irrelevant objects, significantly enhancing the accuracy and efficiency of loop closure detection in complex and dynamic settings. Experimental validations on recognized datasets underscore the superiority of our approach, demonstrating marked improvements in precision, recall, and overall SLAM performance. This research highlights the significant benefits of leveraging semantic insights and attentional focus in advancing the capabilities of loop closure detection for SLAM applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data availability

Enquiries about data availability should be directed to the authors.

References

  1. Liu, C., Qin, J., Wang, S., Yu, L., Wang, Y.: Accurate rgb-d slam in dynamic environments based on dynamic visual feature removal. Sci. China Inf. Sci. 65(10), 202206 (2022)

    Article  Google Scholar 

  2. Grisetti, G., Kümmerle, R., Stachniss, C., Burgard, W.: A tutorial on graph-based slam. IEEE Intell. Transp. Syst. Mag. 2(4), 31–43 (2010)

    Article  Google Scholar 

  3. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004). https://doi.org/10.1023/B:VISI.0000029664.99615.94

    Article  Google Scholar 

  4. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput. Vis. Image Underst. 110(3), 346–359 (2008)

    Article  Google Scholar 

  5. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: Orb: An efficient alternative to sift or surf. 2011 International conference on computer vision, 2564–2571 (2011)

  6. Mur-Artal, R., Montiel, J.M.M., Tardos, J.D.: Orb-slam: a versatile and accurate monocular slam system. IEEE Trans. Rob. 31(5), 1147–1163 (2015)

    Article  Google Scholar 

  7. Mur-Artal, R., Tardós, J.D.: Orb-slam2: an open-source slam system for monocular, stereo, and rgb-d cameras. IEEE Trans. Rob. 33(5), 1255–1262 (2017)

    Article  Google Scholar 

  8. Han, F., Wang, H., Huang, G., Zhang, H.: Sequence-based sparse optimization methods for long-term loop closure detection in visual slam. Auton. Robot. 42, 1323–1335 (2018)

    Article  Google Scholar 

  9. Dong, R., Wei, Z.-G., Liu, C., Kan, J.: A novel loop closure detection method using line features. IEEE Access 7, 111245–111256 (2019)

    Article  Google Scholar 

  10. Han, J., Dong, R., Kan, J.: A novel loop closure detection method with the combination of points and lines based on information entropy. J. Field Robot. 38(3), 386–401 (2021)

    Article  Google Scholar 

  11. Chen, Z., Jacobson, A., Sünderhauf, N., Upcroft, B., Liu, L., Shen, C., Reid, I., Milford, M.: Deep learning features at scale for visual place recognition. 2017 IEEE international conference on robotics and automation (ICRA), 3223–3230 (2017)

  12. Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., Sivic, J.: Netvlad: Cnn architecture for weakly supervised place recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, 5297–5307 (2016)

  13. Merrill, N., Huang, G.: Lightweight unsupervised deep loop closure. Robotics: Science and Systems (RSS) (2018)

  14. Hu, M., Li, S., Wu, J., Guo, J., Li, H., Kang, X.: Loop closure detection for visual slam fusing semantic information. 2019 Chinese Control Conference (CCC), 4136–4141 (2019)

  15. Li, D., Shi, X., Long, Q., Liu, S., Yang, W., Wang, F., Wei, Q., Qiao, F.: Dxslam: A robust and efficient visual slam system with deep features. 2020 IEEE/RSJ International conference on intelligent robots and systems (IROS), 4958–4965 (2020)

  16. Sarlin, P.-E., Cadena, C., Siegwart, R., Dymczyk, M.: From coarse to fine: Robust hierarchical localization at large scale. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12716–12725 (2019)

  17. DeTone, D., Malisiewicz, T., Rabinovich, A.: Superpoint: Self-supervised interest point detection and description. Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 224–236 (2018)

  18. Dusmanu, M., Rocco, I., Pajdla, T., Pollefeys, M., Sivic, J., Torii, A., Sattler, T.: D2-net: A trainable cnn for joint description and detection of local features. Proceedings of the ieee/cvf conference on computer vision and pattern recognition, 8092–8101 (2019)

  19. Zhang, K., Li, Z., Ma, J.: Appearance-based loop closure detection via bidirectional manifold representation consensus. 2021 IEEE International Conference on Robotics and Automation (ICRA), 6811–6817 (2021)

  20. Arshad, S., Kim, G.-W.: Role of deep learning in loop closure detection for visual and lidar slam: a survey. Sensors 21(4), 1243 (2021)

    Article  Google Scholar 

  21. Dai, K., Cheng, L., Yang, R., Yan, G.: Loop closure detection using kpca and cnn for visual slam. 2021 IEEE 40th Chinese Control Conference (CCC), 8088–8093 (2021)

  22. Sun, L., Singh, R.P., Kanehiro, F.: Visual slam framework based on segmentation with the improvement of loop closure detection in dynamic environments. J. Robot. Mech. 33(6), 1385–1397 (2021)

    Article  Google Scholar 

  23. Ying, T., Yan, H., Li, Z., Shi, K., Feng, X.: Loop closure detection based on image covariance matrix matching for visual slam. Int. J. Control Autom. Syst. 19, 3708–3719 (2021)

    Article  Google Scholar 

  24. Chen, Y., Zhong, Y., Wang, W., Peng, H.: Fast and robust loop-closure detection using deep neural networks and matrix transformation for a visual slam system. J. Electron. Imaging 31(6), 061816–061816 (2022)

    Article  Google Scholar 

  25. Islam, M.T., Hasib, K.M., Rahman, M.M., Tusher, A.N., Alam, M.S., Islam, M.R.: Convolutional auto-encoder and independent component analysis based automatic place recognition for moving robot in invariant season condition. Human-Centric Intell. Syst. 3(1), 13–24 (2023)

    Article  Google Scholar 

  26. Zhou, D., Luo, Y., Zhang, Q., Xu, Y., Chen, D., Zhang, X.: A lightweight neural network for loop closure detection in indoor visual slam. Int. J. Comput. Intell. Syst. 16(1), 49 (2023)

    Article  Google Scholar 

  27. Iegawa, F.N., Botelho, W.T., Santos, T.d., Pimentel, E.P., Yamamoto, F.S.: Loop closure detection in visual slam based on convolutional neural network. In: International Conference on Information Technology-New Generations, pp. 3–10 (2023). International Conference on Information Technology-New Generations

  28. Zhong, Q., Fang, X.: A bigbigan-based loop closure detection algorithm for indoor visual slam. J. Electric. Comput. Eng. (2021). https://doi.org/10.1155/2021/9978022

    Article  Google Scholar 

  29. Xie, H., Chen, W., Wang, J.: Hierarchical forest based fast online loop closure for low-latency consistent visual-inertial slam. Robot. Auton. Syst. 151, 104035 (2022)

    Article  Google Scholar 

  30. Yuan, Z., Xu, K., Zhou, X., Deng, B., Ma, Y.: Svg-loop: semantic-visual-geometric information-based loop closure detection. Remote Sensing 13(17), 3520 (2021)

    Article  Google Scholar 

  31. Islam, R., Habibullah, H.: A semantically aware place recognition system for loop closure of a visual slam system. In: 2021 4th International Conference on Mechatronics, Robotics and Automation (ICMRA), pp. 117–121 (2021). IEEE 2021 4th International Conference on Mechatronics, Robotics and Automation (ICMRA)

  32. Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR International conference on machine learning

  33. Yuan, Z.-W., Zhang, J.: Feature extraction and image retrieval based on alexnet. Eighth International Conference on Digital Image Processing (ICDIP 2016) 10033, 65–69 (2016). SPIE

Download references

Acknowledgements

In the process of completing this paper, I would like to express my deepest gratitude to all those who have supported and assisted me. First and foremost, I would like to thank my supervisor, Professor Yanli Liu and Professor Heng Zhang, for their meticulous guidance and selfless support. Throughout the entire research process, the expertise and invaluable advice of my supervisor played a crucial role in my research work. I am grateful to Yawei Li and all members of the laboratory team for their cooperation and support in the research project. Their discussions and collaboration greatly expanded my research perspective. I also want to express my gratitude to my family and friends, who have provided unwavering support and understanding throughout my academic journey. Their encouragement has been a driving force for my continuous progress. Additionally, special thanks to National Natural Science Foundation of China for providing financial support for this research. Without this funding, I would not have been able to complete this research work. Finally, thanks to everyone who contributed to this research, making the successful completion of this study possible.

Funding

This work was supported in part by the National Natural Science Foundation of China under Grant No. 61963017; in part by Shanghai Science and Technology Program, China, under Grant No. 23010501000; in part by Shanghai Educational Science Research Project, China, under Grant No. C2022056; in part by Humanities and Social Sciences of Ministry of Education Planning Fund, China, under Grant No. 22YJAZHA145.

Author information

Author notes

  1. Heng Zhang, Yihong Zhang, Yanli Liu, Neal Naixue Xiong and Yawei Li have contributed equally to this work.

Authors and Affiliations

  1. School of Electronic Information Engineering, Shanghai Dianji University, Shui Hua Road, Shanghai, 201306, China

    Heng Zhang, Yihong Zhang, Yanli Liu & Yawei Li

  2. Department of Mathematics and Computer Science, Sul Ross State University, 400 North Harrison, Alpine, 79832, USA

    Neal Naixue Xiong

  3. School of Electronic Information Engineering, Hunan University of Science, Technology, Tao Yuan Road, Xiangtan, 411201, China

    Neal Naixue Xiong

Authors
  1. Heng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yihong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanli Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Neal Naixue Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yawei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanli Liu.

Ethics declarations

Conflict of interest

The authors declare no potential Conflict of interest. This research has obtained ethics approval from Shanghai Dianji University and strictly adheres to ethical standards and guiding principles. Detailed research information has been provided to all individuals participating in the study, and written consent has been obtained from them prior to their involvement in the research. We have obtained publication consent from all relevant individuals and strictly adhere to publication ethical standards. The availability of data and materials used in this paper will be provided upon request to ensure the verifiability and reproducibility of the research. Please contact the authors for further information. We currently do not make the research code publicly available. However, interested researchers may contact the authors for specific inquiries regarding the code used in this study

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, H., Zhang, Y., Liu, Y. et al. Slam loop closure detection algorithm based on MSA-SG. Cluster Comput (2024). https://doi.org/10.1007/s10586-024-04406-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10586-024-04406-6

Keywords

Search

Navigation