Multimedia Tools and Applications

, Volume 76, Issue 3, pp 4381–4403 | Cite as

Development of an automatic 3D human head scanning-printing system

  • Longyu Zhang
  • Bote Han
  • Haiwei Dong
  • Abdulmotaleb El Saddik
Article

Abstract

Three-dimensional (3D) technologies have been developing rapidly recent years, and have influenced industrial, medical, cultural, and many other fields. In this paper, we introduce an automatic 3D human head scanning-printing system, which provides a complete pipeline to scan, reconstruct, select, and finally print out physical 3D human heads. To enhance the accuracy of our system, we developed a consumer-grade composite sensor (including a gyroscope, an accelerometer, a digital compass, and a Kinect v2 depth sensor) as our sensing device. This sensing device is then mounted on a robot, which automatically rotates around the human subject with approximate 1-meter radius, to capture the full-view information. The data streams are further processed and fused into a 3D model of the subject using a tablet located on the robot. In addition, an automatic selection method, based on our specific system configurations, is proposed to select the head portion. We evaluated the accuracy of the proposed system by comparing our generated 3D head models, from both standard human head model and real human subjects, with the ones reconstructed from FastSCAN and Cyberware commercial laser scanning systems through computing and visualizing Hausdorff distances. Computational cost is also provided to further assess our proposed system.

Keywords

3D reconstruction RGB-D sensor Motion sensor Reconstruction accuracy evaluation 3D printing 

References

  1. 1.
    Abreu-de Souza M, Robson S, Hebden JC, Gibson AP, Sauret V (2006) The photogrammetric determination of head surface shape and alignment for the optical tomography of newborn infants. In: Proceedings of international society for photogrammetry and remote sensingGoogle Scholar
  2. 2.
    Allen B, Curless B, Popović Z. (2003) The space of human body shapes: reconstruction and parameterization from range scans. ACM Trans Graph 22:587–594CrossRefGoogle Scholar
  3. 3.
    Ball RM (2008) Sizechina: the world shapes up. Innovation:158–161Google Scholar
  4. 4.
    Barmpoutis A (2013) Tensor body: real-time reconstruction of the human body and avatar synthesis from RGB-D. IEEE Trans Cybern 43:1347–1356CrossRefGoogle Scholar
  5. 5.
    Besl PJ, McKay HD (1992) A method for registration of 3D shapes. IEEE Trans Pattern Anal Mach Intell 14:239–256CrossRefGoogle Scholar
  6. 6.
    Cappelletto E, Zanuttigh P, Cortelazzo G (2016) 3D scanning of cultural heritage with consumer depth cameras. Multimedia Tools Appl 75:3631–3654CrossRefGoogle Scholar
  7. 7.
    Chen Y, Medioni G (1991) Object modeling by registration of multiple range images. In: Proceedings of IEEE international conference on robotics and automation, pp 2724–2729Google Scholar
  8. 8.
    Chen C, Hung YP, Cheng JB (1998) A fast automatic method for registration of partially-overlapping range images. In: Proceedings of the 6th international conference on computer vision, pp 242–248Google Scholar
  9. 9.
    Chen C, Liu M, Zhang B, Han J, Jiang J, Liu H (2016) 3D action recognition using multi-temporal depth motion maps and fisher vector. In: Proceedings of the 25th international joint conference on artificial intelligence, pp 3331–3337Google Scholar
  10. 10.
    Dickerson MT, Drysdale RLS, McElfresh SA, Welzl E (1994) Fast greedy triangulation algorithms. In: Proceedings of the 10th annual symposium on computational geometry, pp 211–220Google Scholar
  11. 11.
    Feldmar J, Ayache N (1994) Rigid, affine and locally affine registration of free-form surfaces. Int J Comput Vis 18:99–119CrossRefGoogle Scholar
  12. 12.
    Cignoni P, Rocchini C, Scopigno R (1996) Metro: measuring error on simplified surfaces. Technical report, Paris, FranceGoogle Scholar
  13. 13.
    Curless B, Levoy M (1996) A volumetric method for building complex models from range images. In: Proceedings of the 23rd annual conference on computer graphics and interactive techniques, pp 303– 312Google Scholar
  14. 14.
    Figueroa N, Dong H, El Saddik A (2015) A combined approach toward consistent reconstructions of indoor spaces based on 6D RGB-D odometry and KinectFusion. ACM Trans Intell Syst Technol 6(14):1–10CrossRefGoogle Scholar
  15. 15.
    Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24:381–395MathSciNetCrossRefGoogle Scholar
  16. 16.
    Friess M, Bradtmiller B (2003) 3D head models for protective helmet development. Technical report, SAE Technical PaperGoogle Scholar
  17. 17.
    Han J, Shao L, Xu D, Shotton J (2013) Enhanced computer vision with microsoft kinect sensor: a review. IEEE Trans Cybern 43:1318–1334CrossRefGoogle Scholar
  18. 18.
    Hennessy RJ, Kinsella A, Waddington JL (2002) 3D laser surface scanning and geometric morphometric analysis of craniofacial shape as an index of cerebro-craniofacial morphogenesis: initial application to sexual dimorphism. Biol Psychiatry 51:507–514CrossRefGoogle Scholar
  19. 19.
    Hu Y, Duan F, Yin B, Zhou M, Sun Y, Wu Z, Geng G (2013) A hierarchical dense deformable model for 3D face reconstruction from skull. Multimedia Tools Appl 64:345–364CrossRefGoogle Scholar
  20. 20.
    Hull CW (1986) Apparatus for production of three-dimensional objects by stereolithography, US4575330Google Scholar
  21. 21.
    Izadi S, Kim D, Hilliges O, Molyneaux D, Newcombe R, Kohli P, Shotton J, Hodges S, Freeman D, Davison A, Fitzgibbon A (2011) KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In: Proceedings of the 24th annual ACM symposium on user interface software and technology, pp 559–568Google Scholar
  22. 22.
    Jian M, Dong J (2011) Capture and fusion of 3D surface texture. Multimedia Tools Appl 53:237–251CrossRefGoogle Scholar
  23. 23.
    Kalman R (1960) A new approach to linear filtering and prediction problems. J Basic Eng 82:35–45CrossRefGoogle Scholar
  24. 24.
    Kan P, Ferko A (2010) Automatic image-based 3D head modeling with a parameterized model based on a hierarchical tree of facial features. In: Proceedings of the 14th Central European seminar on computer graphics, pp 191–198Google Scholar
  25. 25.
    Kazhdan M, Bolitho M, Hoppe H (2006) Poisson surface reconstruction. In: Proceedings of the 4th eurographics symposium on geometry processing, pp 61–70Google Scholar
  26. 26.
    Kolb A, Barth E, Koch R, Larsen R (2009) Time-of-flight sensors in computer graphics. In: Proceedings of Eurographics, pp 119–134Google Scholar
  27. 27.
    Lee WS, Magnenat-Thalmann N (2000) Fast head modeling for animation. Image Vis Comput 18:355–364CrossRefGoogle Scholar
  28. 28.
    Lorensen WE, Cline HE (1987) Marching cubes: a high resolution 3D surface construction algorithm. Comput Graph 21:163–169CrossRefGoogle Scholar
  29. 29.
    Luximon Y, Ball RM, Justice L (2009) The 3D Chinese head and face modeling. Comput Aided Des 44:40–47CrossRefGoogle Scholar
  30. 30.
    Masuda T (2002) Object shape modelling from multiple range images by matching signed distance fields. In: Proceedings of the 1st international symposium on 3D data processing visualization and transmission, pp 439–448Google Scholar
  31. 31.
    Merchan P, Adan A, Salamanca S, Dominguez V, Chacon R (2012) Geometric and color data fusion for outdoor 3D models. Sensors 12:6893–6919CrossRefGoogle Scholar
  32. 32.
    Newcombe RA, Izadi S, Hilliges O, Molyneaux D, Kim D, Davison AJ, Kohli P, Shotton J, Hodges S, Fitzgibbon A (2011) KinectFusion: real-time dense surface mapping and tracking. In: Proceedings of the 10th IEEE international symposium on mixed and augmented reality, pp 127–136Google Scholar
  33. 33.
    Nishino K, Ikeuchi K (2002) Robust simultaneous registration of multiple range images. In: Proceedings of the 5th Asian conference on computer vision, pp 454–461Google Scholar
  34. 34.
    Park JH, Shin YD, Bae JH, Baeg MH (2012) Spatial uncertainty model for visual features using a Kinect sensor. Sensors 12:8640–8662CrossRefGoogle Scholar
  35. 35.
    Pulli K (1999) Multiview registration for large data sets. In: Proceedings of the 2nd international conference on 3D digital imaging and modeling, pp 160–168Google Scholar
  36. 36.
    Rau JY, Yeh PC (2012) A semi-automatic image-based close range 3D modeling pipeline using a multi-camera configuration. Sensors 12:11271–11293CrossRefGoogle Scholar
  37. 37.
    Remondino F, Spera MG, Nocerino E, Menna F, Nex F (2014) State of the art in high density image matching. Photogramm Rec 29:144–166CrossRefGoogle Scholar
  38. 38.
    Rockafellar RT, Wets RJB (2005) Variational analysis. Springer-VerlagGoogle Scholar
  39. 39.
    Rusinkiewicz S, Levoy M (2001) Efficient variants of the ICP algorithm. In: Proceedings of IEEE 3rd international conference on 3D digital imaging and modeling, pp 145–152Google Scholar
  40. 40.
    Rusu RB, Cousins S (2011) 3D is here: point cloud library (PCL). In: Proceedings of IEEE international conference on robotics and automation, pp 1–4Google Scholar
  41. 41.
    Shum H, Ho E, Jiang Y, Takagi S (2013) Real-time posture reconstruction for Microsoft Kinect. IEEE Trans Cybern 43:1357–1369CrossRefGoogle Scholar
  42. 42.
    Su X, Chen W (2001) Fourier transform profilometry: a review. Opt Lasers Eng 35:263–284CrossRefGoogle Scholar
  43. 43.
    Sun Y, Dong J, Jian M, Qi L (2015) Fast 3D face reconstruction based on uncalibrated photometric stereo. Multimedia Tools Appl 74:3635–3650CrossRefGoogle Scholar
  44. 44.
    Weise T, Wismer T, Leibe B, Van-Gool L (2009) In-hand scanning with online loop closure. In: Proceedings of IEEE 12th international conference on computer vision workshops, pp 1630–1637Google Scholar
  45. 45.
    Yang QX (2012) Recursive bilateral filtering. In: Proceedings of the 12th European conference on computer vision, pp 399–413Google Scholar
  46. 46.
    Yang L, Zhang L, Dong H, Alelaiwi A, El Saddik A (2015) Evaluating and improving the depth accuracy of Kinect for Windows v2. IEEE Sensors J 15:4275–4285CrossRefGoogle Scholar
  47. 47.
    Yang L, Yang B, Dong H, El-Saddik A (2016) 3D markerless tracking of human gait by geometric trilateration of multiple Kinects. IEEE Syst JGoogle Scholar
  48. 48.
    žbontar K., Mihelj M, Podobnik B, Povše F., Munih M (2013) Dynamic symmetrical pattern projection based laser triangulation sensor for precise surface position measurement of various material types. Appl Opt 52:2750–2760CrossRefGoogle Scholar
  49. 49.
    Zhang Z (1994) Iterative point matching for registration of free-form curves and surfaces. Int J Comput Vis 13:119–152CrossRefGoogle Scholar
  50. 50.
    Zhang L, Dong H, El-Saddik A (2015) From 3D sensing to printing: a survey. ACM Trans Multimed Comput Commun Appl 12(27):1–27:23Google Scholar
  51. 51.
    Zhang B, Perina A, Murino V, Del Bue A (2015) Sparse representation classification with manifold constraints transfer. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 4557–4565Google Scholar
  52. 52.
    Zhang B, Perina A, Li Z, Murino V, Liu J, Ji R (2016) Bounding multiple Gaussians uncertainty with aplication to object tracking. Int J Comput Vis 118(3):364–379MathSciNetCrossRefGoogle Scholar
  53. 53.
    Zhang B, Li Z, Perina A, Del Bue A, Murino V, Liu J (2016) Adaptive local movement modelling for robust object tracking. IEEE Trans Circuits Syst Video TechnolGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Multimedia Computing Research Laboratory (MCRLab), School of Electrical Engineering and Computer ScienceUniversity of OttawaOttawaCanada

Personalised recommendations