Skip to main content

Online Semantic Mapping of Urban Environments

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNAI,volume 7463)

Abstract

In this paper we present an integrated approach for efficient online 3D semantic map building of urban environments and the subsequent extraction of qualitative spatial relationships between the different objects in the scene. We split this process into three stages, where we combine a state of the art image segmentation and classification algorithm with an online clustering algorithm to obtain a coherent representation of the environment. Finally, a graph representation is extracted which can then be used for spatial reasoning and human robot interaction. We present first results from data collected by a mobile robot which operates in city areas.

Keywords

  • Point Cloud
  • Mobile Robot
  • Urban Environment
  • Spatial Relation
  • Conditional Random Field

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (Canada)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bazeille, S., Filliat, D.: Incremental topo-metric slam using vision and robot odometry. In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 4067–4073. IEEE (2011)

    Google Scholar 

  2. Doh, N.L., Lee, K., Chung, W.K., Cho, H.: Simultaneous localisation and mapping algorithm for topological maps with dynamics. Control Theory & Applications, IET 3(9), 1249–1260 (2009)

    CrossRef  Google Scholar 

  3. Elfes, A.: Using occupancy grids for mobile robot perception and navigation. Computer 22(6), 46–57 (1989)

    CrossRef  Google Scholar 

  4. Thrun, S., Bücken, A.: Integrating grid-based and topological maps for mobile robot navigation. In: Proceedings of the National Conference on Artificial Intelligence, pp. 944–951 (1996)

    Google Scholar 

  5. Zender, H., Martínez Mozos, O., Jensfelt, P., Kruijff, G.J.M., Burgard, W.: Conceptual spatial representations for indoor mobile robots. Robotics and Autonomous Systems 56(6), 493–502 (2008)

    CrossRef  Google Scholar 

  6. Galindo, C., Fernández-Madrigal, J.A., González, J., Saffiotti, A.: Robot task planning using semantic maps. Robotics and Autonomous Systems 56, 955–966 (2008)

    CrossRef  Google Scholar 

  7. Skubic, M., Perzanowski, D., Blisard, S., Schultz, A., Adams, W., Bugajska, M., Brock, D.: Spatial language for human-robot dialogs. IEEE Systems, Man, and Cybernetics, Part C: Applications and Reviews 34, 154–167 (2004)

    Google Scholar 

  8. Vasudevan, S., Siegwart, R.: Bayesian space conceptualization and place classification for semantic maps in mobile robotics. Robotics and Autonomous Systems 56(6), 522–537 (2008)

    CrossRef  Google Scholar 

  9. Pronobis, A., Mozos, O.M., Caputo, B., Jensfelt, P.: Multi-modal semantic place classification. The International Journal of Robotics Research 29, 298–320 (2010)

    CrossRef  Google Scholar 

  10. Nüchter, A., Hertzberg, J.: Towards semantic maps for mobile robots. Robotics and Autonomous Systems 56(11), 915–926 (2008)

    CrossRef  Google Scholar 

  11. Rusu, R.B., Marton, Z.C., Blodow, N., Holzbach, A., Beetz, M.: Model-based and learned semantic object labeling in 3d point cloud maps of kitchen environments. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2009, pp. 3601–3608. IEEE (2009)

    Google Scholar 

  12. Wolf, D.F., Sukhatme, G.S.: Semantic mapping using mobile robots. IEEE Transactions on Robotics 24(2), 245–258 (2008)

    CrossRef  Google Scholar 

  13. Douillard, B., Fox, D., Ramos, F., Durrant-Whyte, H.: Classification and semantic mapping of urban environments. International Journal of Robotics Research 30(1), 5–32 (2011)

    CrossRef  Google Scholar 

  14. Kuipers, B.: The spatial semantic hierarchy. Artificial Intelligence 119(1-2), 191–233 (2000)

    CrossRef  MathSciNet  MATH  Google Scholar 

  15. He, X., Zemel, R.S., Carreira-Perpinán, M.A.: Multiscale conditional random fields for image labeling. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 2, pp. 695–702. IEEE (2004)

    Google Scholar 

  16. Sturgess, P., Alahari, K., Ladicky, L., Torr, P.H.S.: Combining appearance and structure from motion features for road scene understanding. In: BMVC (2009)

    Google Scholar 

  17. Posner, I., Cummins, M., Newman, P.: A generative framework for fast urban labeling using spatial and temporal context. Auton. Robots 26(2-3), 153–170 (2009)

    CrossRef  Google Scholar 

  18. Ladicky, L., Sturgess, P., Russell, C., Sengupta, S., Bastanlar, Y., Clocksin, W., Torr, P.: Joint optimisation for object class segmentation and dense stereo reconstruction (2010)

    Google Scholar 

  19. Geiger, A., Lauer, M., Urtasun, R.: A generative model for 3d urban scene understanding from movable platforms. In: CVPR (2011)

    Google Scholar 

  20. Geiger, A., Wojek, C., Urtasun, R., Geiger, A., Lauer, M., Urtasun, R., Geiger, A., Ziegler, J., Stiller, C., Lenz, P., et al.: Joint 3d estimation of objects and scene layout. In: Neural Information Processing Systems

    Google Scholar 

  21. Jain, A.K., Murty, M.N., Flynn, P.J.: Data clustering: a review. ACM Computing Surveys (CSUR) 31(3), 264–323 (1999)

    CrossRef  Google Scholar 

  22. Xu, R., Wunsch, D., et al.: Survey of clustering algorithms. IEEE Transactions on Neural Networks 16(3), 645–678 (2005)

    CrossRef  Google Scholar 

  23. Rusu, R., Marton, Z., Blodow, N., Holzbach, A., Beetz, M.: Model-based and learned semantic object labeling in 3D point cloud maps of kitchen environments. In: IROS (2009)

    Google Scholar 

  24. Wolf, D., Howard, A., Sukhatme, G.S.: Towards geometric 3d mapping of outdoor environments using mobile robots. In: IROS. IEEE (2005)

    Google Scholar 

  25. Valencia, R., Teniente, E.H., Trulls, E., Andrade-Cetto, J.: 3D mapping for urban service robots. In: IROS, IEEE (2009)

    Google Scholar 

  26. Marton, Z.C., Rusu, R.B., Beetz, M.: On fast surface reconstruction methods for large and noisy datasets. In: ICRA (2009)

    Google Scholar 

  27. Klasing, K., Wollherr, D., Buss, M.: Realtime segmentation of range data using continuous nearest neighbors. In: ICRA (2009)

    Google Scholar 

  28. Zlatanova, S., Rahman, A.A., Shi, W.: Topological models and frameworks for 3D spatial objects. In: Computers & Geosciences, pp. 419–428 (2004)

    Google Scholar 

  29. Jungert, E.: Qualitative spatial reasoning for determination of object relations using symbolic interval projections. In: Proceedings 1993 IEEE Symposium on Visual Languages, pp. 83–87 (August 1993)

    Google Scholar 

  30. Li, C., Lu, J., Yin, C., Ma, L.: Qualitative spatial representation and reasoning in 3d space. In: Intelligent Computation Technology and Automation, ICICTA 2009, vol. 1, pp. 653–657 (2009)

    Google Scholar 

  31. Skubic, M., Perzanowski, D., Schultz, A., Adams, W.: Using spatial language in a human-robot dialog. In: Proceedings of the 2002 International Conference on IEEE Robotics and Automation, ICRA 2002, vol. 4, pp. 4143–4148. IEEE (2002)

    Google Scholar 

  32. Moratz, R., Tenbrink, T., Bateman, J.A., Fischer, K.: Spatial Knowledge Representation for Human-Robot Interaction. In: Freksa, C., Brauer, W., Habel, C., Wender, K.F. (eds.) Spatial Cognition III. LNCS (LNAI), vol. 2685, pp. 263–286. Springer, Heidelberg (2003)

    CrossRef  Google Scholar 

  33. Bloch, I., Saffiotti, A.: On the representation of fuzzy spatial relations in robot maps. In: Intelligent Systems for Information Processing, pp. 47–57 (2002)

    Google Scholar 

  34. Bauer, A., Gonsior, B., Wollherr, D., Buss, M.: Heuristic rules for human-robot interaction based on principles from linguistics - asking for directions. In: AISB Convention - Symposium on New Frontiers in Human-Robot Interaction (2009)

    Google Scholar 

  35. Sj, K., Aydemir, A., Moerwald, T., Zhou, K., Jensfelt, P.: Mechanical support as a spatial abstraction for mobile robots. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4894–4900. IEEE (2010)

    Google Scholar 

  36. Sj, K., Pronobis, A., Jensfelt, P.: Functional topological relations for qualitative spatial representation. In: The 15th International Conference on Advanced Robotics (2011)

    Google Scholar 

  37. Tenorth, M.: Knowledge Processing for Autonomous Robots. PhD thesis, Technische Universitaet Muenchen (2011)

    Google Scholar 

  38. Lafferty, J., McCallum, A., Pereira, F.C.N.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data (2001)

    Google Scholar 

  39. Kumar, S., Hebert, M.: Discriminative random fields: A discriminative framework for contextual interaction in classification. In: Proceedings of the Ninth IEEE International Conference on Computer Vision, pp. 1150–1157. IEEE (2003)

    Google Scholar 

  40. Vishwanathan, S.V.N., Schraudolph, N.N., Schmidt, M.W., Murphy, K.P.: Accelerated training of conditional random fields with stochastic gradient methods. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 969–976. ACM (2006)

    Google Scholar 

  41. Wainwright, M.J., Jaakkola, T.S., Willsky, A.S.: Tree-based reparameterization framework for analysis of sum-product and related algorithms. IEEE Transactions on Information Theory 49(5), 1120–1146 (2003)

    CrossRef  MathSciNet  MATH  Google Scholar 

  42. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8), 888–905 (2000)

    CrossRef  Google Scholar 

  43. Levinshtein, A., Stere, A., Kutulakos, K.N., Fleet, D.J., Dickinson, S.J., Siddiqi, K.: Turbopixels: Fast superpixels using geometric flows. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(12), 2290–2297 (2009)

    CrossRef  Google Scholar 

  44. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: Slic superpixels. Technical Report 149300 EPFL (June 2010)

    Google Scholar 

  45. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)

    CrossRef  Google Scholar 

  46. Bhatia, A., Snyder, W.E., Bilbro, G.: Stacked integral image. In: 2010 IEEE International Conference on Robotics and Automation (ICRA), pp. 1530–1535. IEEE (2010)

    Google Scholar 

  47. Zhang, C., Wang, L., Yang, R.: Semantic Segmentation of Urban Scenes Using Dense Depth Maps. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 708–721. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  48. Spiliopoulou, M., Ntoutsi, I., Theodoridis, Y., Schult, R.: Monic: modeling and monitoring cluster transitions. In: Proceedings of the 12th SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 706–711. ACM (2006)

    Google Scholar 

  49. Wurm, K.M., Hornung, A., Bennewitz, M., Stachniss, C., Burgard, W.: Octomap: A probabilistic, flexible, and compact 3d map representation for robotic systems. In: Proc. of the ICRA 2010 Workshop on Best Practice in 3D Perception and Modeling for Mobile Manipulation (2010)

    Google Scholar 

  50. del Pobil, A.P., Escrig, M.T., Jaen, J.A.: An attempt towards a general representation paradigm for spatial reasoning. In: International Conference on Systems, Man and Cybernetics, ‘Systems Engineering in the Service of Humans’, Conference Proceedings, vol. 1, pp. 215–220 (October 1993)

    Google Scholar 

  51. Brostow, G.J., Fauqueur, J., Cipolla, R.: Semantic object classes in video: A high-definition ground truth database. Pattern Recognition Letters (2008)

    Google Scholar 

  52. Geiger, A., Ziegler, J., Stiller, C.: Stereoscan: Dense 3d reconstruction in real-time. In: IEEE Intelligent Vehicles Symposium, Baden-Baden, Germany (2011)

    Google Scholar 

  53. Bauer, A., Klasing, K., Xu, T., Sosnowski, S., Lidoris, G., Muhlbauer, Q., Zhang, T., Rohrmuller, F., Wollherr, D., Kuhnlenz, K., et al.: The autonomous city explorer project. In: IEEE International Conference on Robotics and Automation, ICRA 2009, pp. 1595–1596. IEEE (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Mitsou, N. et al. (2012). Online Semantic Mapping of Urban Environments. In: Stachniss, C., Schill, K., Uttal, D. (eds) Spatial Cognition VIII. Spatial Cognition 2012. Lecture Notes in Computer Science(), vol 7463. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32732-2_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-32732-2_4

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-32731-5

  • Online ISBN: 978-3-642-32732-2

  • eBook Packages: Computer ScienceComputer Science (R0)