Skip to main content
Log in

Comparing ICP variants on real-world data sets

Open-source library and experimental protocol

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

Many modern sensors used for mapping produce 3D point clouds, which are typically registered together using the iterative closest point (ICP) algorithm. Because ICP has many variants whose performances depend on the environment and the sensor, hundreds of variations have been published. However, no comparison frameworks are available, leading to an arduous selection of an appropriate variant for particular experimental conditions. The first contribution of this paper consists of a protocol that allows for a comparison between ICP variants, taking into account a broad range of inputs. The second contribution is an open-source ICP library, which is fast enough to be usable in multiple real-world applications, while being modular enough to ease comparison of multiple solutions. This paper presents two examples of these field applications. The last contribution is the comparison of two baseline ICP variants using data sets that cover a rich variety of environments. Besides demonstrating the need for improved ICP methods for natural, unstructured and information-deprived environments, these baseline variants also provide a solid basis to which novel solutions could be compared. The combination of our protocol, software, and baseline results demonstrate convincingly how open-source software can push forward the research in mapping and navigation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. http://github.com/ethz-asl/libnabo, version 1.0.1.

  2. We are in contact with PCL developers to integrate parts of our work into it.

  3. http://github.com/ethz-asl/libpointmatcher, version 1.0.0 at time of submission of this paper.

  4. http://projects.asl.ethz.ch/datasets/doku.php?id=laserregistration:laserregistration.

  5. The semantics of \(\epsilon \) has been changed since Pomerleau et al. (2011) to be compatible with other open-source implementations.

References

  • Amigoni, F., Reggiani, M., & Schiaffonati, V. (2009). An insightful comparison between experiments in mobile robotics and in science. Autonomous Robots, 27(4), 313–325.

    Article  Google Scholar 

  • Armesto, L., Minguez, J., & Montesano, L. (2010). A generalization of the metric-based iterative closest point technique for 3D scan matching. In Proceedings of the IEEE international conference on robotics and automation (pp. 1367–1372).

  • Arya, S., & Mount, D. (1993). Approximate nearest neighbor queries in fixed dimensions. In Proceedings of the 4th annual ACM-SIAM symposium on discrete algorithms (pp. 271–280).

  • Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., & Szeliski, R. (2007). A database and evaluation methodology for optical flow. International Journal of Computer Vision, 92, 1–31.

    Google Scholar 

  • Ben-Kiki, O., Evans, C., & Ingerson, B. (2009). YAML ain’t markup language (\(\text{ YAML}^{\rm TM}\)) version 1.2. http://www.yaml.org/spec/1.2/spec.html.

  • Besl, P., & McKay, H. (1992). A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239–256.

    Google Scholar 

  • Chen, Y., & Medioni, G. (1991). Object modeling by registration of multiple range images. In Proceedings of the IEEE international conference on robotics and automation (pp. 2724–2729). New York: IEEE Computer Society Press.

  • Chetverikov, D., Svirko, D., Stepanov, D., & Krsek, P. (2002). The trimmed iterative closest point algorithm. In Proceedings of the 16th international conference on pattern recognition (pp. 545–548).

  • Elseberg, J., Magnenat, S., Siegwart, R., & Nüchter, A. (2012). Comparison of nearest-neighbor-search strategies and implementations for efficient shape registration. Journal of Software Engineering for Robotics, 3(1), 2–12.

    Google Scholar 

  • Geiger, A., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? The KITTI vision benchmark suite. In Computer vision and pattern recognition (CVPR).

  • Hugli, H., & Schutz, C. (1997). Geometric matching of 3D objects: Assessing the range of successful initial configurations. In Proceedings of the international conference on recent advances in 3-D digital imaging and modeling (pp. 101–106).

  • Huynh, D. Q. (2009). Metrics for 3D rotations: Comparison and analysis. Journal of Mathematical Imaging and Vision, 35(2), 155–164.

    Article  MathSciNet  Google Scholar 

  • Jian, B., & Vemuri, B. C. (2011). Robust point set registration using Gaussian mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(8), 1633–1645.

    Article  Google Scholar 

  • Magnusson, M., Lilienthal, A., & Duckett, T. (2007). Scan registration for autonomous mining vehicles using 3D-NDT. Journal of Field Robotics, 24(10), 803–827.

    Article  Google Scholar 

  • Magnusson, M., Nüchter, A., Lorken, C., Lilienthal, A., & Hertzberg, J. (2009). Evaluation of 3D registration reliability and speed—A comparison of ICP and NDT. In Proceedings of the IEEE international conference on robotics and automation (pp. 3907–3912).

  • May, S., Droeschel, D., Holz, D., Fuchs, S., Malis, E., Nüchter, A., Hertzberg, J. (2009). Three-dimensional mapping with time-of-flight cameras. Journal of Field Robotics, 26(11–12), 934–965.

    Google Scholar 

  • Pathak, K., Borrmann, D., Elseberg, J., Vaskevicius, N., Birk, A., & Nüchter, A. (2010). Evaluation of the robustness of planar-patches based 3D-registration using marker-based ground-truth in an outdoor urban scenario. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 5725–5730).

  • Pomerleau, F., Liu, M., Colas, F., & Siegwart, R. (2012). Challenging data sets for point cloud registration algorithms. The International Journal of Robotics Research.

  • Pomerleau, F., Magnenat, S., Colas, F., Liu, M., & Siegwart, R. (2011) Tracking a depth camera: Parameter exploration for fast ICP. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 3824–3829).

  • Rusinkiewicz, S., & Levoy, M. (2001). Efficient variants of the ICP algorithm. In Proceedings of the third international conference on 3-D digital imaging and modeling (pp. 145–152).

  • Rusu, R., & Cousins, S. (2011). 3D is here: Point cloud library (PCL). In Proceedings of the IEEE international conference on robotics and automation (pp. 1–4).

  • Scharstein, D., & Szeliski, R. (2002). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1), 7–42.

    Article  MATH  Google Scholar 

  • Schroeder, W., Martin, K., & Lorensen, B. (2006). Visualization toolkit: An object-oriented approach to 3D graphics (4th ed.). Clifton Park, NY: Kitware.

    Google Scholar 

  • Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In Proceedings of the IEEE computer society conference on computer vision and pattern recognition (pp. 519–528).

  • Tong, C. H., Barfoot, T. D., & Dupuis, É. (2012). Three-dimensional SLAM for mapping planetary work site environments. Journal of Field Robotics, 29, 381–412.

  • Wulf, O., Nüchter, A., Hertzberg, J., & Wagner, B. (2008). Benchmarking urban six-degree-of-freedom simultaneous localization and mapping. Journal of Field Robotics, 25(3), 148–163.

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the EU FP7 IP projects Natural Human-Robot Cooperation in Dynamic Environments (ICT-247870) and myCopter (FP7-AAT-2010-RTD-1). F. Pomerleau was supported by a fellowship from the Fonds québécois de recherche sur la nature et les technologies (FQRNT).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to François Pomerleau.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pomerleau, F., Colas, F., Siegwart, R. et al. Comparing ICP variants on real-world data sets. Auton Robot 34, 133–148 (2013). https://doi.org/10.1007/s10514-013-9327-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-013-9327-2

Keywords

Navigation