Skip to main content
SpringerLink
Log in

SAGE: Semantic Annotation of Georeferenced Environments

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

  • 6 Altmetric

Abstract

The emergence of portable 3D mapping systems are revolutionizing the way we generate digital 3D models of environments. These systems are human-centric and require the user to hold or carry the device while continuously walking and mapping an environment. In this paper, we adapt this unique coexistence of man and machines to propose SAGE (Semantic Annotation of Georeferenced Environments). SAGE consists of a portable 3D mobile mapping system and a smartphone that enables the user to assign semantic content to georeferenced 3D point clouds while scanning a scene. The proposed system contains several components including touchless speech acquisition, background noise adaptation, real time audio and vibrotactile feedback, automatic speech recognition, distributed clock synchronization, 3D annotation localization, user interaction, and interactive visualization. The most crucial advantage of SAGE technology is that it can be used to infer dynamic activities within an environment. Such activities are difficult to be identified with existing post-processing semantic annotation techniques. The capability of SAGE leads to many promising applications such as intelligent scene classification, place recognition and navigational aid tasks. We conduct several experiments to demonstrate the effectiveness of the proposed system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Anguelov, D., Taskarf, B., Chatalbashev, V., Koller, D., Gupta, D., Heitz, G., Ng, A.: Discriminative learning of markov random fields for segmentation of 3D scan data. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition volume 2, pages 169–176. IEEE (2005)

  2. Ben Hmida, H., Cruz, C., Boochs, F., Nicolle, C.: From 3D Point Clouds To Semantic Objects An Ontology-Based Detection Approach. ArXiv e-prints (2013)

  3. Bok, Y., Jeong, Y., Choi, D.G., bok Kweon, I.S.: Capturing village-level heritages with a hand-held camera-laser fusion sensor. Int. J. Comput. Vis. 94(1), 36–53 (2011)

    Article  Google Scholar 

  4. Bosse, M., Zlot, R.: Place recognition using keypoint voting in large 3D lidar datasets. In: IEEE International Conference on Robotics and Automation (2013)

  5. Bosse, M., Zlot, R., Zebedee, P.F.: Design of a spring-mounted 3D range sensor with application to mobile mapping. IEEE Trans. Robot. 28(5), 1104–1119 (2012)

    Article  Google Scholar 

  6. Dorninger, P., Nothegger, C.: 3D segmentation of unstructured point clouds for building modelling. Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci. 35(3/W49A), 191–196 (2007)

    Google Scholar 

  7. Du, H., Henry, P., Ren, X., Cheng, M., Goldman, D.B., Seitz, S.M., Fox, D.: Interactive 3D modeling of indoor environments with a consumer depth camera. In: Proceedings of the 13th international conference on Ubiquitous computing pages 75–84. ACM (2011)

  8. Henry, P., Krainin, M, Herbst, E., Ren, X., Fox, D.: RGB-D mapping: Using kinect-style depth cameras for dense 3D modeling of indoor environments. Int. J. Robot. Res. 31(5), 647–663 (2012)

    Article  Google Scholar 

  9. Lai, K., Bo, L., Ren, X., Fox, D.: Sparse distance learning for object recognition combining rgb and depth information. In: IEEE International Conference on Robotics and Automation, pages 4007–4013. IEEE (2011)

  10. Lai, K., Bo, L., Ren, X., Fox, D.: Detection-based object labeling in 3D scenes. In: IEEE International Conference on Robotics and Automation, pp. 1330–1337 (2012)

  11. Lai, K., Fox, D.: Object recognition in 3D point clouds using web data and domain adaptation. Int. J. Robot. Res. 29(8), 1019–1037 (2010)

    Article  Google Scholar 

  12. Liu, T., Carlberg, M., Chen, G., Chen, J., Kua, J., Zakhor, A.: Indoor localization and visualization using a human-operated backpack system. In: International Conference on Indoor Positioning and Indoor Navigation (2010)

  13. Moghadam, P., Bosse, M., Zlot, R.: Line-based extrinsic calibration of range and image sensors. In: IEEE International Conference on Robotics and Automation, pp. 3685–3691 (2013)

  14. Nüchter, A., Hertzberg, J.: Towards semantic maps for mobile robots. Robot. Auton. Syst. 56 (11), 915–926 (2008)

    Article  Google Scholar 

  15. Teichman, A., Levinson, J., Thrun, S.: Towards 3D object recognition via classification of arbitrary object tracks. In: IEEE International Conference on Robotics and Automation, pages 4034–4041. IEEE (2011)

  16. Vidas, S., Moghadam, P.: HeatWave: a handheld 3D thermography system for energy auditing. Energy Build. 66, 445–460 (2013)

    Article  Google Scholar 

  17. Vidas, S., Moghadam, P., Bosse, M.: 3D thermal mapping of building interiors using an RGB-D and thermal camera. In: IEEE International Conference on Robotics and Automation, pp. 3670–3676 (2013)

  18. Wolf, D.F., Sukhatme, G.S.: Semantic mapping using mobile robots. IEEE Trans. Robot. 24(2), 245–258 (2008)

    Article  Google Scholar 

  19. Xiong, X., Huber, D.: Using context to create semantic 3D models of indoor environments. In Proc. BMVC (2010)

  20. Li, Z., Liu, Z., Xia, C.H.: Clock synchronization algorithms for network measurements. In: INFOCOM 2002. Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies., volume 1, pages 160–169. IEEE (2002)

  21. Zhang, Y., Luo, C., Liu, J.: Walk&sketch: Create floor plans with an RGB-D camera. In: UbiComp ’12 Proceedings of the 2012 ACM Conference on Ubiquitous Computing, pages 461–470. ACM (2012)

Download references

Author information

Authors and Affiliations

  1. Autonomous Systems, CSIRO, 1 Technology Court, Pullenvale, Brisbane, QLD, 4069, Australia

    Peyman Moghadam, Benjamin Evans & Elliot Duff

Authors
  1. Peyman Moghadam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benjamin Evans
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Elliot Duff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peyman Moghadam.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moghadam, P., Evans, B. & Duff, E. SAGE: Semantic Annotation of Georeferenced Environments. J Intell Robot Syst 83, 635–648 (2016). https://doi.org/10.1007/s10846-015-0302-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-015-0302-3

Keywords

Search

Navigation