Simultaneous Entire Shape Registration of Multiple Depth Images Using Depth Difference and Shape Silhouette

  • Takuya Ushinohama
  • Yosuke Sawai
  • Satoshi Ono
  • Hiroshi KawasakiEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9004)


This paper proposes a method for simultaneous global registration of multiple depth images which are obtained from multiple viewpoints. Unlike the previous method, the proposed method fully utilizes a silhouette-based cost function taking out-of-view and non-overlapping regions into account as well as depth differences at overlapping areas. With the combination of the above cost functions and a recent powerful meta-heuristics named self-adaptive Differential Evolution, it realizes the entire shape reconstruction from relatively small number (three or four) of depth images, which do not involve enough overlapping regions for Iterative Closest Point even if they are prealigned. In addition, to allow the technique to be applicable not only to time-of-flight sensors, but also projector-camera systems, which has deficient silhouette by occlusions, we propose a simple solution based on color-based silhouette. Experimental results show that the proposed method can reconstruct the entire shape only from three depth images of both synthetic and real data. The influence of noises and inaccurate silhouettes is also evaluated.


Differential Evolution Depth Image Iterative Close Point Iterative Close Point Depth Difference 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

Supplementary material (wmv 8,975 KB)


  1. 1.
    Besl, P.J., McKay, N.D.: A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14, 239–256 (1992)CrossRefGoogle Scholar
  2. 2.
    NEUGEBAUER, P.: Geometrical cloning of 3d objects via simultaneous registration of multiple range image. In: Proceedings of the International Conference on Shape Modeling and Applications (1997)Google Scholar
  3. 3.
    Li, H., Hartley, R.: The 3d–3d registration problem revisited. In: Proceedings of the International Conference on Computer Vision, pp. 1–8 (2007)Google Scholar
  4. 4.
    Yang, J., Li, H., Jia, Y.: Go-icp: Solving 3d registration efficiently and globally optimally. In: IEEE International Conference on Computer Vision, pp. 1457–1464 (2013)Google Scholar
  5. 5.
    Wang, R., Choi, J., Medioni, G.: 3d modeling from wide baseline range scans using contour coherence. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4018–4025 (2014)Google Scholar
  6. 6.
    Johnson, A., Hebert, M.: Using spin images for efficient object recognition in cluttered 3d scenes. IEEE Trans. Pattern Anal. Mach. Intell. 21, 433–449 (1999)CrossRefGoogle Scholar
  7. 7.
    Santamaría, J., Cordón, O., Damas, S.: A comparative study of state-of-the-art evolutionary image registration methods for 3d modeling. Comput. Vis. Image Underst. 115, 1340–1354 (2011)CrossRefGoogle Scholar
  8. 8.
    Silva, L., Bellon, O.R.P., Boyer, K.: Precision range image registration using a robust surface interpenetration measure and enhanced genetic algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 27, 762–776 (2005)CrossRefGoogle Scholar
  9. 9.
    Brunnstrom, K., Stoddart, A.J.: Genetic algorithms for free-form surface matching. In: Proceedings of the International Conference on Pattern Recognition. vol. 4, pp. 689–693 (1996)Google Scholar
  10. 10.
    Salti, S., Tombari, F., Di Stefano, L.: A performance evaluation of 3d keypoint detectors. In: International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission, pp. 236–243 (2011)Google Scholar
  11. 11.
    He, R., Narayana, P.A.: Global optimization of mutual information: application to three-dimensional retrospective registration of magnetic resonance images. Comput. Med. Imaging Graph. 26, 277–292 (2002)CrossRefGoogle Scholar
  12. 12.
    Brest, J., Greiner, S., Boskovic, B., Mernik, M., Zumer, V.: Self-adapting control parameters in differential evolution: A comparative study on numerical benchmark problems. Trans. Evol. Comput. 10, 646–657 (2006)CrossRefGoogle Scholar
  13. 13.
    Rusu, R., Blodow, N., Beetz, M.: Fast point feature histograms (FPFH) for 3d registration. In: International Conference on Robotics and Automation, pp. 3212–3217 (2009)Google Scholar
  14. 14.
    Rusu, R., Blodow, N., Marton, Z., Beetz, M.: Aligning point cloud views using persistent feature histograms. In: International Conference on Intelligent Robots and Systems, pp. 3384–3391 (2008)Google Scholar
  15. 15.
    Furukawa, R., Kawasaki, H.: Uncalibrated multiple image stereo system with arbitrarily movable camera and projector for wide range scanning. In: IEEE Conference on 3DIM, pp. 302–309 (2005)Google Scholar
  16. 16.
    Zhang, Z.: Microsoft kinect sensor and its effect. MultiMedia 19, 4–10 (2012)CrossRefGoogle Scholar
  17. 17.
    Mesa Imaging AG.: SwissRanger SR-4000 (2011).
  18. 18.
    Storn, R., Price, K.: Differential evolution a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11, 341–359 (1997)CrossRefzbMATHMathSciNetGoogle Scholar
  19. 19.
    Das, S., Suganthan, P.N.: Differential evolution: A survey of the state-of-the-art. IEEE Trans. Evol. Comput. 15, 4–31 (2011)CrossRefGoogle Scholar
  20. 20.
    Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evol. Comput. 9, 159–195 (2001)CrossRefGoogle Scholar
  21. 21.
    Li, C., Yang, S., Nguyen, T.T., Yu, E.L., Yao, X., Jin, Y., Beyer, H.-G., Suganthan, P.N.: Benchmark generator for cec’ 2009 competition on dynamic optimization (2008)Google Scholar
  22. 22.
    Goldberg, D.E.: Genetic Algorithms in Search, Optimization, and Machine Learning. Addison Wesley, Reading (1989)zbMATHGoogle Scholar
  23. 23.
    Auger, A., Hansen, N.: A restart cma evolution strategy with increasing population size. Congr. Evolut. Comput. 2, 1769–1776 (2005)Google Scholar
  24. 24.
    Brockhoff, D., Auger, A., Hansen, N.: On the effect of mirroring in the IPOP active CMA-ES on the noiseless BBOB testbed. In: Proceedings of the Annual Conference on Genetic and Evolutionary Computation, pp. 277–284 (2012)Google Scholar
  25. 25.
    Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S., Davis, J., Ginsberg, J., Shade, J., Fulk, D.: The Digital Michelangelo Project: 3D scanning of large statues. Proc. ACM SIGGRAPH 2000, 131–144 (2000)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Takuya Ushinohama
    • 1
  • Yosuke Sawai
    • 1
  • Satoshi Ono
    • 1
  • Hiroshi Kawasaki
    • 1
    Email author
  1. 1.Department of Information Science and Biomedical EngineeringGraduate School of Science and Engineering, Kagoshima UniversityKagoshimaJapan

Personalised recommendations