Skip to main content
SpringerLink
Log in

Semantic SLAM for Mobile Robot with Human-in-the-Loop

  • Conference paper
  • First Online:
Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom 2022)

Abstract

Mobile robots are an important participant in today’s modern life, and have huge commercial application prospects in the fields of unmanned security inspection, logistics, express delivery, cleaning and medical disinfection. Since LiDAR is not affected by ambient light and can operate in a dark environment, localization and navigation based on LiDAR point clouds have become one of the basic modules of mobile robots. However, compared with traditional binocular vision images, the sparse, disordered and noisy point cloud poses a challenge to efficient and stable feature extraction. This makes the LiDAR-based SLAM have more significant cumulative errors, and poor consistency of the final map, which affects tasks such as positioning based on the prior point cloud map. In order to alleviate the above problems and improve the positioning accuracy, a semantic SLAM with human-in-the-loop is proposed. First, the interactive SLAM is introduced to optimize the point cloud pose to obtain a highly consistent point cloud map; then the point cloud segmentation model is trained by artificial semantic annotation to obtain the semantic information of a single frame of point cloud; finally, the positioning accuracy is optimized based on the point cloud semantics. The proposed system is validated on the local platform in an underground garage, without involving GPS or expensive measuring equipment.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/jbehley/point_labeler.

References

  1. Luo, Y., Binbin, S., Zheng, X.: Trends and challenges for population and health during population aging-china, 2015–2050. China CDC Weekly 3(28), 593 (2021)

    Article  Google Scholar 

  2. Yang, G., et al.: Homecare robotic systems for healthcare 4.0: visions and enabling technologies. IEEE J. Biomed. Health Inform. 24(9), 2535–2549 (2020)

    Article  Google Scholar 

  3. Zhang, J., Singh, S.: Loam: lidar odometry and mapping in real-time. In: Robotics: Science and Systems, Berkeley, CA, vol. 2, no. 9, pp. 1–9 (2014)

    Google Scholar 

  4. Song, C.K., Uchanski, M., Karl Hedrick, J.: Vehicle speed estimation using accelerometer and wheel speed measurements. Soc. of Automotive Engineers (2002)

    Google Scholar 

  5. Barbieri, L., Brambilla, M., Trabattoni, A., Mervic, S., Nicoli, M.: UWB localization in a smart factory: augmentation methods and experimental assessment. IEEE Trans. Instrum. Meas. 70, 1–18 (2021)

    Google Scholar 

  6. Wang, L., et al.: Initial assessment of the LEO based navigation signal augmentation system from Luojia-1a satellite. Sensors 18(11), 3919 (2018)

    Article  Google Scholar 

  7. Li, Y., He, L., Zhang, X., Zhu, L., Zhang, H., Guan, Y.: Multi-sensor fusion localization of indoor mobile robot. In: 2019 IEEE International Conference on Real-time Computing and Robotics (RCAR), pp. 481–486. IEEE (2019)

    Google Scholar 

  8. Campos, C., Elvira, R., Gómez Rodríguez, J.J., Montiel, J.M.M., Tardós, J.D.: Orb-slam3: an accurate open-source library for visual, visual-inertial, and multimap slam. IEEE Trans. Robot. 37(6), 1874–1890 (2021)

    Article  Google Scholar 

  9. Shan, T., Englot, B.: Lego-loam: lightweight and ground-optimized lidar odometry and mapping on variable terrain. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4758–4765. IEEE (2018)

    Google Scholar 

  10. Cui, J., Niu, J., Ouyang, Z., He, Y., Liu, D.: ACSC: automatic calibration for non-repetitive scanning solid-state lidar and camera systems. arXiv preprint arXiv:2011.08516 (2020)

  11. Chen, X., Milioto, A., Palazzolo, E., Giguère, P., Behley, J., Stachniss, C.: Suma++: efficient lidar-based semantic slam. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4530–4537. IEEE (2019)

    Google Scholar 

  12. Koide, K., Miura, J., Yokozuka, M., Oishi, S., Banno, A.: Interactive 3d graph slam for map correction. IEEE Robot. Autom. Lett. 6(1), 40–47 (2020)

    Article  Google Scholar 

  13. Behley, J., et al.: Semantickitti: a dataset for semantic scene understanding of lidar sequences. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 9297–9307 (2019)

    Google Scholar 

  14. Zhang, Y., et al.: PolarNet: an improved grid representation for online lidar point clouds semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9601–9610 (2020)

    Google Scholar 

  15. Zhang, L., Wei, L., Shen, P., Wei, W., Zhu, G., Song, J.: Semantic slam based on object detection and improved octomap. IEEE Access 6, 75545–75559 (2018)

    Article  Google Scholar 

  16. Mur-Artal, R., Tardós, J.D.: Orb-slam2: an open-source slam system for monocular, stereo, and RGB-D cameras. IEEE Trans. Robot. 33(5), 1255–1262 (2017)

    Article  Google Scholar 

  17. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

    Google Scholar 

  18. Wang, Z., Zhang, Q., Li, J., Zhang, S., Liu, J.: A computationally efficient semantic slam solution for dynamic scenes. Remote Sen. 11(11), 1363 (2019)

    Article  Google Scholar 

  19. Kang, X., Yuan, S.: Robust data association for object-level semantic slam. arXiv preprint arXiv:1909.13493 (2019)

  20. Long, X., Zhang, W., Zhao, B.: Pspnet-slam: a semantic slam detect dynamic object by pyramid scene parsing network. IEEE Access (2020)

    Google Scholar 

  21. Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2881–2890 (2017)

    Google Scholar 

  22. Zhao, Z., Mao, Y., Ding, Y., Ren, P., Zheng, N.: Visual-based semantic slam with landmarks for large-scale outdoor environment. In: 2019 2nd China Symposium on Cognitive Computing and Hybrid Intelligence (CCHI), pp. 149–154. IEEE (2019)

    Google Scholar 

  23. Rosinol, A., Abate, M., Chang, Y., Carlone, L.: Kimera: an open-source library for real-time metric-semantic localization and mapping. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 1689–1696. IEEE (2020)

    Google Scholar 

  24. Fan, Y., et al.: Semantic slam with more accurate point cloud map in dynamic environments. IEEE Access 8, 112237–112252 (2020)

    Article  Google Scholar 

  25. Mahe, H., Marraud, D., Comport, A.I.: Real-time RGB-D semantic keyframe slam based on image segmentation learning from industrial cad models. In: 2019 19th International Conference on Advanced Robotics (ICAR), pp.s 147–154. IEEE (2019)

    Google Scholar 

  26. Nicholson, L., Milford, M., Sünderhauf, N.: Quadricslam: constrained dual quadrics from object detections as landmarks in semantic slam. IEEE Robot. Autom. Lett. (RA-L) (2018)

    Google Scholar 

  27. Li, R., Wang, S., Dongbing, G.: Ongoing evolution of visual slam from geometry to deep learning: challenges and opportunities. Cogn. Comput. 10(6), 875–889 (2018)

    Article  Google Scholar 

  28. Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263–7271 (2017)

    Google Scholar 

  29. Lehtonen, M., Ostojic, D., Ilic, A., Michahelles, F.: Securing RFID systems by detecting tag cloning. In: Tokuda, H., Beigl, M., Friday, A., Brush, A.J.B., Tobe, Y. (eds.) Pervasive 2009. LNCS, vol. 5538, pp. 291–308. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-01516-8_20

    Chapter  Google Scholar 

  30. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)

    Google Scholar 

  31. Dong, X., Niu, J., Cui, J., Fu, Z., Ouyang, Z.: Fast segmentation-based object tracking model for autonomous vehicles. In: Qiu, M. (ed.) ICA3PP 2020. LNCS, vol. 12453, pp. 259–273. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60239-0_18

    Chapter  Google Scholar 

  32. Graeter, J., Wilczynski, A., Lauer, M.: Limo: lidar-monocular visual odometry. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7872–7879. IEEE (2018)

    Google Scholar 

  33. Jian, R., et al.: A semantic segmentation based lidar SLAM system towards dynamic environments. In: Yu, H., Liu, J., Liu, L., Ju, Z., Liu, Y., Zhou, D. (eds.) ICIRA 2019. LNCS (LNAI), vol. 11742, pp. 582–590. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-27535-8_52

    Chapter  Google Scholar 

  34. Zhao, Z., Zhang, W., Jianfeng, G., Yang, J., Huang, K.: Lidar mapping optimization based on lightweight semantic segmentation. IEEE Trans. Intell. Veh. 4(3), 353–362 (2019)

    Article  Google Scholar 

  35. Milioto, A., Vizzo, I., Behley, J., Stachniss, C.: Rangenet++: fast and accurate lidar semantic segmentation. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4213–4220. IEEE (2019)

    Google Scholar 

  36. Chen, X.: et al.: Overlapnet: loop closing for lidar-based SLAM. In: Proceedings of the Robotics: Science and Systems (RSS), Freiburg, Germany, pp. 12–16 (2020)

    Google Scholar 

  37. Sun, L., Yan, Z., Zaganidis, A., Zhao, C., Duckett, T.: Recurrent-octomap: learning state-based map refinement for long-term semantic mapping with 3-d-lidar data. IEEE Robot. Autom. Lett. 3(4), 3749–3756 (2018)

    Article  Google Scholar 

  38. Pan, Y., Gao, B., Mei, J., Geng, S., Li, C., Zhao, H.: Semanticposs: a point cloud dataset with large quantity of dynamic instances. arXiv preprint arXiv:2002.09147 (2020)

  39. Hu, Q., et al.: Randla-net: efficient semantic segmentation of large-scale point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11108–11117 (2020)

    Google Scholar 

  40. Zhu, X., et al.: Cylindrical and asymmetrical 3D convolution networks for lidar segmentation. arXiv preprint arXiv:2011.10033 (2020)

  41. Chen, X., Vizzo, I., Läbe, T., Behley, J., Stachniss, C.: Range image-based lidar localization for autonomous vehicles. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 5802–5808. IEEE (2021)

    Google Scholar 

  42. Razlaw, J., Droeschel, D., Holz, D., Behnke, S.: Evaluation of registration methods for sparse 3d laser scans. In: 2015 European Conference on Mobile Robots (ECMR), pp. 1–7. IEEE (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Beihang Hangzhou Innovation Institute Yuhang, Beihang University, Yuhang, Hangzhou, 311100, Zhejiang, China

    Zhenchao Ouyang & Jiahe Cui

  2. School of Computer Science and Engineering, Beihang University, Beijing, 100191, China

    Zhenchao Ouyang, Changjie Zhang & Jiahe Cui

Authors
  1. Zhenchao Ouyang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Changjie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiahe Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenchao Ouyang .

Editor information

Editors and Affiliations

  1. Shanghai University, Shanghai, China

    Honghao Gao

  2. Xi’an Jiaotong-Liverpool University, Suzhou, China

    Xinheng Wang

  3. Zhejiang University City College, Hangzhou, China

    Wei Wei

  4. London South Bank University, London, UK

    Tasos Dagiuklas

Rights and permissions

Reprints and permissions

Copyright information

© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ouyang, Z., Zhang, C., Cui, J. (2022). Semantic SLAM for Mobile Robot with Human-in-the-Loop. In: Gao, H., Wang, X., Wei, W., Dagiuklas, T. (eds) Collaborative Computing: Networking, Applications and Worksharing. CollaborateCom 2022. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 461. Springer, Cham. https://doi.org/10.1007/978-3-031-24386-8_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-24386-8_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-24385-1

  • Online ISBN: 978-3-031-24386-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation