Skip to main content
SpringerLink
Log in

Map evaluation using matched topology graphs

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

Mapping is an important task for mobile robots. The assessment of the quality of maps in a simple, efficient and automated way is not trivial and an ongoing research topic. Here, a new approach for the evaluation of 2D grid maps is presented. This structure-based method makes use of a topology graph, i.e., a topological representation that includes abstracted local metric information. It is shown how the topology graph is constructed from a Voronoi diagram that is pruned and simplified such that only high level topological information remains to concentrate on larger, topologically distinctive places. Several methods for computing the similarity of vertices in two topology graphs, i.e., for performing a place-recognition, are presented. Based on the similarities, it is shown how subgraph-isomorphisms can be efficiently computed and two topology graphs can be matched. The match between the graphs is then used to calculate a number of standard map evaluation attributes like coverage, global accuracy, relative accuracy, consistency, and brokenness. Experiments with robot generated maps are used to highlight the capabilities of the proposed approach and to evaluate the performance of the underlying algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  • Aurenhammer, F. (1991). Voronoi diagramsã survey of a fundamental geometric data structure. ACM Computing Surveys, 23, 345–405. doi:10.1145/116873.116880.

    Article  Google Scholar 

  • Bai, X., & Latecki, L. (2008). Path similarity skeleton graph matching. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 30(7), 1282–1292. doi:10.1109/TPAMI.2007.70769.

    Article  Google Scholar 

  • Bai, X., Latecki, L., & yu Liu, W. (2007). Skeleton pruning by contour partitioning with discrete curve evolution. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 29(3), 449–462. doi:10.1109/TPAMI.2007.59.

    Article  Google Scholar 

  • Balaguer, B., Balakirsky, S., Carpin, S., & Visser, A. (2009). Evaluating maps produced by urban search and rescue robots: Lessons learned from robocup. Autonomous Robots, 27, 449–464.

    Article  Google Scholar 

  • Besl, P. J., & McKay, N. D. (1992). A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239–256.

    Article  Google Scholar 

  • Birk, A. (2010). A quantitative assessment of structural errors in grid maps. Autonomous Robots, 28, 187–196.

    Article  Google Scholar 

  • Birk, A., Wiggerich, B., Bülow, H., Pfingsthorn, M., & Schwertfeger, S. (2011). Safety, security, and rescue missions with an unmanned aerial vehicle (UAV): Aerial mosaicking and autonomous flight at the 2009 european land robots trials (elrob) and the 2010 response robot evaluation exercises (rree). Journal of Intelligent and Robotic Systems, 64(1), 57–76.

    Article  Google Scholar 

  • Chen, Q. S., Defrise, M., & Deconinck, F. (1994). Symmetric phase-only matched filtering of fourier-mellin transforms for image registration and recognition. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 16(12), 1156–1168.

    Article  Google Scholar 

  • Edelsbrunner, H., Kirkpatrick, D., & Seidel, R. (1983). On the shape of a set of points in the plane. Information Theory, IEEE Transactions on, 29(4), 551–559. doi:10.1109/TIT.1983.1056714.

    Article  MathSciNet  MATH  Google Scholar 

  • Fortin, S. (1996). The graph isomorphism problem, Technical report.

  • Handa, A., Whelan, T., McDonald, J. & Davison, A. (2014) A benchmark for rgb-d visual odometry, 3D reconstruction and slam. In: Robotics and automation (ICRA), 2014 IEEE international conference on, pp. 1524–1531 . doi:10.1109/ICRA.2014.6907054

  • Horn, B. K. P. (1987). Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America, 4(4), 629–642.

    Article  MathSciNet  Google Scholar 

  • Karavelas, M. (2011). 2D Voronoi diagram adaptor. In: CGAL user and reference manual, 3.8 edn. CGAL Editorial Board. http://www.cgal.org/Manual/3.8/doc_html/cgal_manual/packages.html#Pkg:VoronoiDiagramAdaptor2. Accessed 12 April 2012.

  • Klein, R. (1988). Abstract voronoi diagrams and their applications. In H. Noltemeier (Ed.), Computational geometry and its applications., Lecture notes in computer science Berlin: Springer.

    Google Scholar 

  • Kümmerle, R., Steder, B., Dornhege, C., Ruhnke, M., Grisetti, G., Stachniss, C., et al. (2009). On measuring the accuracy of slam algorithms. Autonomous Robots, 27(4), 387–407.

    Article  Google Scholar 

  • Lakaemper, R., & Adluru, N. (2009). Using virtual scans for improved mapping and evaluation. Autonomous Robots, 27(4), 431–448. doi:10.1007/s10514-009-9149-4.

    Article  Google Scholar 

  • Lee, D.C. (1996). The map-building and exploration strategies of a simple sonar-equipped mobile robot: An experimental, quantitative evaluation, distinguished dissertations in computer science, Cambridge University Press.

  • Leonard, J. J., & Durrant-Whyte, H. F. (1992). Directed sonar sensing for mobile robot navigation. New York: Springer.

    Book  MATH  Google Scholar 

  • Pellenz, J. & Paulus, D. (2008). Mapping and map scoring at the roboCupRescue competition. Quantitative performance evaluation of navigation solutions for mobile robots (RSS 2008, Workshop CD)

  • Pomerleau, F., Liu, M., Colas, F., & Siegwart, R. (2012). Challenging data sets for point cloud registration algorithms. The International Journal of Robotics Research, 31(14), 1705–1711.

    Article  Google Scholar 

  • Schwertfeger, S.: Robocuprescue interleague mapping challenge (2010). http://robotics.jacobs-university.de/mappingChallenge/. Accessed 16 Mar 2014.

  • Schwertfeger, S. (2012). Robotic mapping in the real world: Performance evaluation and system integration. Ph.D. thesis, Department of Electrical Engeneering and Computer Science, Jacobs University Bremen. http://www.jacobs-university.de/phd/files/phd20120706_Schwertfeger. Accessed 16 Mar 2014.

  • Schwertfeger, S. & Birk, A. (2013) Evaluation of map quality by matching and scoring high-level, topological map structures. In: Robotics and Automation (ICRA), 2013 IEEE International Conference on

  • Schwertfeger, S., Bülow, H. & Birk, A.(2010) On the effects of sampling resolution in improved Fourier mellin based registration for underwater mapping. In 7th international symposium on intelligent autonomous vehicles (IAV 2010), IFAC.

  • Schwertfeger, S., Jacoff, A., Pellenz, J. & Birk, A. (2011) Using a fiducial map metric for assessing map quality in the context of robocup rescue. In: International workshop on safety, security, and rescue robotics (SSRR), IEEE Press.

  • Schwertfeger, S., Jacoff, A., Scrapper, C., Pellenz, J. & Kleiner, A. (2010). Evaluation of maps using fixed shapes: The fiducial map metric. In Proceedings of PerMIS

  • Scrapper, C., Madhavan, R. & Balakirsky, S. (2007). Stable navigation solutions for robots in complex environments. In: IEEE international workshop on safety, security and rescue robotics (SSRR), pp. 1–6.

  • Siddiqi, K., Shokoufandeh, A., Dickenson, S. & Zucker, S. (1998) Shock graphs and shape matching. In: Computer vision, 1998. Sixth international conference on, pp. 222–229. doi:10.1109/ICCV.1998.710722

  • Sturm, J., Engelhard, N., Endres, F., Burgard, W. & Cremers, D. (2012). A benchmark for the evaluation of rgb-d slam systems. In Intelligent robots and systems (IROS), 2012 IEEE/RSJ international conference on, pp. 573–580. doi:10.1109/IROS.2012.6385773

  • Texas A&M University College Station, T. (2010) National Institute of Standards and Technology: NIST response robot evaluation exercise #6. http://nist.gov/el/isd/disaster_city_6.cfm. Accessed 30 Aug 2015.

  • Thrun, S. (2002). Robotic mapping: A survey. In G. Lakemeyer & B. Nebel (Eds.), Exploring artificial intelligence in the new millenium. San Diego: Morgan Kaufmann.

  • Torsello, A., & Hancock, E. (2001). A skeletal measure of 2D shape similarity. In C. Arcelli, L. Cordella, & G. di Baja (Eds.), Visual form 2001 (Vol. 2059, pp. 260–271)., Lecture notes in computer science Berlin: Springer.

    Chapter  Google Scholar 

  • Varsadan, I., Birk, A., & Pfingsthorn, M. (2009). Determining map quality through an image similarity metric. In L. Locchi, H. Matsubara, A. Weitzenfeld, & C. Zhou (Eds.), RoboCup 2008: Robot World Cup XII (pp. 355–365)., Lecture notes in artificial intelligence New York: Springer.

    Chapter  Google Scholar 

  • Voronoi, G. (1908). Nouvelles applications des paramètres continus à la théorie des formes quadratiques. Premier mémoire. Sur quelques propriétés des formes quadratiques positives parfaites. Journal Für Die Reine und Angewandte Mathematik, 133, 97–102. doi:10.1515/crll.1908.133.97.

  • Wagan, A.I., Godil, A. & Li, X. (2008). Map quality assessment. In: PerMIS ’08: Proceedings of the 8th workshop on performance metrics for intelligent systems, pp. 278–282. ACM, New York, doi:10.1145/1774674.1774718

  • Wulf, O., Nüchter, A., Hertzberg, J. & Wagner, B. (2007). Ground truth evaluation of large urban 6D slam. pp. 650–657. doi:10.1109/IROS.2007.4399026

  • Zemlyachenko, V. N., Korneenko, N. M., & Tyshkevich, R. I. (1985). Graph isomorphism problem. Journal of Mathematical Sciences, 29(4), 1426–1481. doi:10.1007/BF02104746.

    MATH  Google Scholar 

Download references

Acknowledgments

The authors would like to thank all RoboCup Rescue teams that agreed to have their maps published in the context of this research: CASualty: University of New South Wales, Australia and Resko: University of Koblenz, Germany.

Author information

Authors and Affiliations

  1. School of Information Science and Technology, ShanghaiTech University, 200031, Shanghai, China

    Sören Schwertfeger

  2. Department of Electrical Engineering and Computer Science, Jacobs University Bremen, 28759, Bremen, Germany

    Andreas Birk

Authors
  1. Sören Schwertfeger
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andreas Birk
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sören Schwertfeger.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Schwertfeger, S., Birk, A. Map evaluation using matched topology graphs. Auton Robot 40, 761–787 (2016). https://doi.org/10.1007/s10514-015-9493-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-015-9493-5

Keywords

Search

Navigation