Journal of Mathematical Imaging and Vision

, Volume 47, Issue 1–2, pp 124–137 | Cite as

3D Deformable Super-Resolution for Multi-Camera 3D Face Scanning

  • Karima Ouji
  • Mohsen Ardabilian
  • Liming Chen
  • Faouzi Ghorbel


Low-cost and high-accuracy 3D face measurement is becoming increasingly important in many computer vision applications including face recognition, facial animation, games, orthodontics and aesthetic surgery. In most cases fringe projection based systems are used to overcome the relatively uniform appearance of skin. These systems employ a structured light camera/projector device and require explicit user cooperation and controlled lighting conditions. In this paper, we propose a 3D acquisition solution with a 3D space-time non-rigid super-resolution capability, using three calibrated cameras coupled with a non calibrated projector device, which is particularly suited to 3D face scanning, i.e. rapid, easily movable and robust to ambient lighting variation. The proposed solution is a hybrid stereovision and phase-shifting approach, using two shifted patterns and a texture image, which not only takes advantage of stereovision and structured light, but also overcomes their weaknesses. The super-resolution scheme involves a shape+texture 3D non-rigid registration for 3D artifacts correction in the presence of small non-rigid deformations as facial expressions.


Stereovision Phase-shifting Space-time Multi-camera Super-resolution Non-rigid registration 


  1. 1.
    Adaskevicius, R., Vasiliauskas, A.: Three-dimensional determination of dental occlusion and facial structures using soft tissue cephalometric analysis. J. Electron. Electron. Eng. 5(121), 93–96 (2010) Google Scholar
  2. 2.
    Aliaga, D.G., Xu, Y.: Photogeometric structured light: a self-calibrating and multi-viewpoint framework for accurate 3D modeling. In: IEEE Conference on Computer Vision and Pattern Recognition (2008) Google Scholar
  3. 3.
    Aliaga, D.G., Xu, Y.: A self-calibrating method for photogeometric acquisition of 3D objects. IEEE Trans. Pattern Anal. Mach. Intell. 32(4), 747–754 (2010) CrossRefGoogle Scholar
  4. 4.
    Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford Univ. Press, Oxford (1995) Google Scholar
  5. 5.
    Cox, I., Hingorani, S., Rao, S.: A maximum likelihood stereo algorithm. J. Comput. Vis. Image Underst. 63, 542–567 (1996) CrossRefGoogle Scholar
  6. 6.
    Cui, Y., Stricker, D.: In: 3D Body Scanning with One Kinect. Conference on 3D Body Scanning Technologies, Lugano, Switzerland (2011) Google Scholar
  7. 7.
    Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc., Ser. B Methodol. 39, 1–38 (1977) MathSciNetzbMATHGoogle Scholar
  8. 8.
    Farsiu, S., Robinson, D., Elad, M., Milanfar, P.: Fast and robust multi-frame super-resolution. IEEE Trans. Image Process (2004) Google Scholar
  9. 9.
    Han, X., Huang, P.: Combined stereovision and phase shifting method: a new approach for 3-D shape measurement. In: Proc. of SPIE Optical Measurement Systems for Industrial Inspection VI, vol. 7389 (2009) Google Scholar
  10. 10.
    Kil, Y., Mederos, Y., Amenta, N.: Laser scanner super-resolution. In: Eurographics Symposium on Point-Based Graphics (2006) Google Scholar
  11. 11.
    Myronenko, A., Song, X.: Point set registration: coherent point drift. IEEE Trans. Pattern Anal. Mach. Intell. 32, 2262–2275 (2010) CrossRefGoogle Scholar
  12. 12.
    Myronenko, A., Song, X., Carreira-Perpinan, M.A.: Non-rigid point set registration: coherent point drift. In: NIPS Conference (2007) Google Scholar
  13. 13.
    Ouji, K., Ardabilian, M., Chen, L., Ghorbel, F.: In: Pattern Analysis for an Automatic and Low-Cost 3D Face Acquisition Technique. IEEE Advanced Concepts for Intelligent Vision Systems Conference (ACIVS), Bordeaux, France (2009) Google Scholar
  14. 14.
    Rajagopalan, A., Bhavsar, A., Wallhoff, F., Rigoll, G.: Resolution Enhancement of PMD Range Maps. Lecture Notes in Computer Science, vol. 5096, pp. 304–313. Springer, Berlin (2008) Google Scholar
  15. 15.
    Salvi, J., Pagès, J., Batlle, J.: Pattern codification strategies in structured light systems. J. Pattern Recognit. 37, 827–849 (2004) zbMATHCrossRefGoogle Scholar
  16. 16.
    Schuon, S., Theobalt, C., Davis, J., Thrun, S.: In: LidarBoost: Depth Superresolution for ToF 3D Shape Scanning. CVPR Conference (2009) Google Scholar
  17. 17.
    Tikhonov, A.N., Arsenin, V.I.: Solutions of Ill-Posed Problems. Winston and Sons, Washington (1977) zbMATHGoogle Scholar
  18. 18.
    Weise, T., Leibe, B., Van Gool, L.: Fast 3D scanning with automatic motion compensation. In: IEEE Conference on Computer Vision and Pattern Recognition (2007) Google Scholar
  19. 19.
    Yin, L., Wei, X., Sun, Y., Wang, J., Rosato, M.J.: A 3D facial expression database for facial behavior research. In: IEEE International Conference on Automatic Face and Gesture Recognition, Washington, DC, USA (2006) Google Scholar
  20. 20.
    Yin, L., Chen, X., Sun, Y., Worm, T., Reale, M.: A high-resolution 3D dynamic facial expression database. In: IEEE International Conference on Automatic Face and Gesture Recognition, Amsterdam, The Netherlands (2008) Google Scholar
  21. 21.
    Zhang, Z.: Flexible camera calibration by viewing a plane from unknown orientations. In: ICCV Conference (1999) Google Scholar
  22. 22.
    Zhang, S., Huang, P.S.: High-resolution, real-time three-dimensional shape measurement. J. Opt. Eng. 45, 123601 (2006) CrossRefGoogle Scholar
  23. 23.
    Zhang, S., Yau, S.: Absolute phase-assisted three-dimensional data registration for a dual-camera structured light system. J. Appl. Opt. 47, 3134–3142 (2008) CrossRefGoogle Scholar
  24. 24.
    Zhang, L., Curless, B., Seitz, S.M.: Rapid shape acquisition using color structured light and multipass dynamic programming. In: 3DPVT Conference (2002) Google Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  • Karima Ouji
    • 1
  • Mohsen Ardabilian
    • 1
  • Liming Chen
    • 1
  • Faouzi Ghorbel
    • 2
  1. 1.LIRIS, Lyon Research Center for Images and Intelligent Information SystemsEcole Centrale de LyonEcullyFrance
  2. 2.University of ManoubaManoubaTunisia

Personalised recommendations