Autonomous Robots

, Volume 34, Issue 3, pp 133–148 | Cite as

Comparing ICP variants on real-world data sets

Open-source library and experimental protocol
  • François Pomerleau
  • Francis Colas
  • Roland Siegwart
  • Stéphane Magnenat


Many modern sensors used for mapping produce 3D point clouds, which are typically registered together using the iterative closest point (ICP) algorithm. Because ICP has many variants whose performances depend on the environment and the sensor, hundreds of variations have been published. However, no comparison frameworks are available, leading to an arduous selection of an appropriate variant for particular experimental conditions. The first contribution of this paper consists of a protocol that allows for a comparison between ICP variants, taking into account a broad range of inputs. The second contribution is an open-source ICP library, which is fast enough to be usable in multiple real-world applications, while being modular enough to ease comparison of multiple solutions. This paper presents two examples of these field applications. The last contribution is the comparison of two baseline ICP variants using data sets that cover a rich variety of environments. Besides demonstrating the need for improved ICP methods for natural, unstructured and information-deprived environments, these baseline variants also provide a solid basis to which novel solutions could be compared. The combination of our protocol, software, and baseline results demonstrate convincingly how open-source software can push forward the research in mapping and navigation.


Experimental protocol Iterative closest point Registration Open-source SLAM Mapping 



This work was supported by the EU FP7 IP projects Natural Human-Robot Cooperation in Dynamic Environments (ICT-247870) and myCopter (FP7-AAT-2010-RTD-1). F. Pomerleau was supported by a fellowship from the Fonds québécois de recherche sur la nature et les technologies (FQRNT).


  1. Amigoni, F., Reggiani, M., & Schiaffonati, V. (2009). An insightful comparison between experiments in mobile robotics and in science. Autonomous Robots, 27(4), 313–325.CrossRefGoogle Scholar
  2. Armesto, L., Minguez, J., & Montesano, L. (2010). A generalization of the metric-based iterative closest point technique for 3D scan matching. In Proceedings of the IEEE international conference on robotics and automation (pp. 1367–1372).Google Scholar
  3. Arya, S., & Mount, D. (1993). Approximate nearest neighbor queries in fixed dimensions. In Proceedings of the 4th annual ACM-SIAM symposium on discrete algorithms (pp. 271–280).Google Scholar
  4. Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., & Szeliski, R. (2007). A database and evaluation methodology for optical flow. International Journal of Computer Vision, 92, 1–31.Google Scholar
  5. Ben-Kiki, O., Evans, C., & Ingerson, B. (2009). YAML ain’t markup language (\(\text{ YAML}^{\rm TM}\)) version 1.2.
  6. Besl, P., & McKay, H. (1992). A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239–256.Google Scholar
  7. Chen, Y., & Medioni, G. (1991). Object modeling by registration of multiple range images. In Proceedings of the IEEE international conference on robotics and automation (pp. 2724–2729). New York: IEEE Computer Society Press.Google Scholar
  8. Chetverikov, D., Svirko, D., Stepanov, D., & Krsek, P. (2002). The trimmed iterative closest point algorithm. In Proceedings of the 16th international conference on pattern recognition (pp. 545–548).Google Scholar
  9. Elseberg, J., Magnenat, S., Siegwart, R., & Nüchter, A. (2012). Comparison of nearest-neighbor-search strategies and implementations for efficient shape registration. Journal of Software Engineering for Robotics, 3(1), 2–12.Google Scholar
  10. Geiger, A., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? The KITTI vision benchmark suite. In Computer vision and pattern recognition (CVPR).Google Scholar
  11. Hugli, H., & Schutz, C. (1997). Geometric matching of 3D objects: Assessing the range of successful initial configurations. In Proceedings of the international conference on recent advances in 3-D digital imaging and modeling (pp. 101–106).Google Scholar
  12. Huynh, D. Q. (2009). Metrics for 3D rotations: Comparison and analysis. Journal of Mathematical Imaging and Vision, 35(2), 155–164.MathSciNetCrossRefGoogle Scholar
  13. Jian, B., & Vemuri, B. C. (2011). Robust point set registration using Gaussian mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(8), 1633–1645.CrossRefGoogle Scholar
  14. Magnusson, M., Lilienthal, A., & Duckett, T. (2007). Scan registration for autonomous mining vehicles using 3D-NDT. Journal of Field Robotics, 24(10), 803–827.CrossRefGoogle Scholar
  15. Magnusson, M., Nüchter, A., Lorken, C., Lilienthal, A., & Hertzberg, J. (2009). Evaluation of 3D registration reliability and speed—A comparison of ICP and NDT. In Proceedings of the IEEE international conference on robotics and automation (pp. 3907–3912).Google Scholar
  16. May, S., Droeschel, D., Holz, D., Fuchs, S., Malis, E., Nüchter, A., Hertzberg, J. (2009). Three-dimensional mapping with time-of-flight cameras. Journal of Field Robotics, 26(11–12), 934–965.Google Scholar
  17. Pathak, K., Borrmann, D., Elseberg, J., Vaskevicius, N., Birk, A., & Nüchter, A. (2010). Evaluation of the robustness of planar-patches based 3D-registration using marker-based ground-truth in an outdoor urban scenario. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 5725–5730).Google Scholar
  18. Pomerleau, F., Liu, M., Colas, F., & Siegwart, R. (2012). Challenging data sets for point cloud registration algorithms. The International Journal of Robotics Research.Google Scholar
  19. Pomerleau, F., Magnenat, S., Colas, F., Liu, M., & Siegwart, R. (2011) Tracking a depth camera: Parameter exploration for fast ICP. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 3824–3829).Google Scholar
  20. Rusinkiewicz, S., & Levoy, M. (2001). Efficient variants of the ICP algorithm. In Proceedings of the third international conference on 3-D digital imaging and modeling (pp. 145–152).Google Scholar
  21. Rusu, R., & Cousins, S. (2011). 3D is here: Point cloud library (PCL). In Proceedings of the IEEE international conference on robotics and automation (pp. 1–4).Google Scholar
  22. Scharstein, D., & Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1), 7–42.zbMATHCrossRefGoogle Scholar
  23. Schroeder, W., Martin, K., & Lorensen, B. (2006). Visualization toolkit: An object-oriented approach to 3D graphics (4th ed.). Clifton Park, NY: Kitware.Google Scholar
  24. Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In Proceedings of the IEEE computer society conference on computer vision and pattern recognition (pp. 519–528).Google Scholar
  25. Tong, C. H., Barfoot, T. D., & Dupuis, É. (2012). Three-dimensional SLAM for mapping planetary work site environments. Journal of Field Robotics, 29, 381–412.Google Scholar
  26. Wulf, O., Nüchter, A., Hertzberg, J., & Wagner, B. (2008). Benchmarking urban six-degree-of-freedom simultaneous localization and mapping. Journal of Field Robotics, 25(3), 148–163.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • François Pomerleau
    • 1
  • Francis Colas
    • 1
  • Roland Siegwart
    • 1
  • Stéphane Magnenat
    • 1
  1. 1.Autonomous System LabETH ZurichZurichSwitzerland

Personalised recommendations