Reconstructing a Whole Face Image from a Partially Damaged or Occluded Image by Multiple Matching

  • Bon-Woo Hwang
  • Seong-Whan Lee
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4642)


The problem we address in this paper is, given a facial image that is partially occluded or damaged by noise, to reconstruct a whole face. A key process for the reconstruction is to obtain the correspondences between the input image and the reference face. We present a method that matches an input image with multiple example images that are generated from a morphable face model. From the matched feature points, shape and texture of the full face are inferred by the non-iterative data completion algorithm. Compared with single matching with the particular “reference face”, this multiple matching method increases the robustness of the matching. The experimental results of applying the algorithm to face images that are contaminated by Gaussian noise and those which are partially occluded show that the reconstructed faces are plausible and similar to the original ones.


Face reconstruction morphable face model SIFT feature data completion 


  1. 1.
    Bicego, M., Lagorio, A., Grosso, E., Tistarelli, M.: On the Use of SIFT Features for Face Authentication. In: Proc. of IEEE Conference on Computer Vision and Pattern Recognition Workshops, June 2006, p. 35 (2006)Google Scholar
  2. 2.
    Blanz, V., Vetter, T.: Morphable Model for the Synthesis of 3D Faces. In: Proc. of SIGGRAPH 1999, Los Angeles, USA, August 1999, pp. 187–194 (1999)Google Scholar
  3. 3.
    Blanz, V., Mehl, A., Vetter, T., Seidel, H.-P.: A Statistical Method for Robust 3D Surface Reconstruction from Sparse Data. In: Proc. of International Symposium on 3D Data Processing, Visualization and Transmission, Thessaloniki, Greece, September 2004, pp. 293–300 (2004)Google Scholar
  4. 4.
    Everson, R., Sirovich, L.: The Karhunen-Loeve Transform for Incomplete Data. Journal of the Optical Society of America A 12(8), 1657–1664 (1995)CrossRefGoogle Scholar
  5. 5.
    Hwang, B.-W., Blanz, V., Vetter, T., Song, H.-H., Lee, S.-W.: Face Reconstruction from a Small Number of Feature Points. In: Proc. of International Conference on Pattern Recognition, Barcelona, Spain, September 2000, vol. 2, pp. 842–845 (2000)Google Scholar
  6. 6.
    Hwang, B.-W., Lee, S.-W.: Reconstruction of Partially Damaged Face Images Based on a Morphable Face Model. IEEE Transaction on Pattern Analysis and Machine Intelligence 25(3), 365–372 (2003)CrossRefGoogle Scholar
  7. 7.
    Jones, M.J., Poggio, T.: Multidimensional Morphable Models: A Framework for Representing and Matching Object Classes. International Journal of Computer Vision 29(2), 107–131 (1998)CrossRefGoogle Scholar
  8. 8.
    Lowe, D.G.: Object Recognition from Local Scale-Invariant Features. In: Proc. of International Conference on Computer Vision, Corfu, Greece, September 1999, pp. 1150–1157 (1999)Google Scholar
  9. 9.
    Lowe, D.G.: Local Feature View Clustering for 3D Object Recognition. In: Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, USA, December 2001, pp. 682–688 (2001)Google Scholar
  10. 10.
    Lowe, D.G.: Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  11. 11.
    Mikolajczyk, K., Schmid, C.: A Performance Evaluation of Local Descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(10), 1615–1630 (2005)CrossRefGoogle Scholar
  12. 12.
    Turk, M., Pentland, A.: Eigenfaces for Recognition. Journal of Cognitive Neuroscience 3(1), 71–86 (1991)CrossRefGoogle Scholar
  13. 13.
    Vetter, T., Troje, N.E.: Separation of Texture and Shape in Images of Faces for Image Coding and Synthesis. Journal of the Optical Society of America A 14(9), 2152–2161 (1997)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Bon-Woo Hwang
    • 1
  • Seong-Whan Lee
    • 2
  1. 1.The Robotics Institute, Carnegie Mellon University, 5000 Forbes Ave., Pittsburgh, PA 15213USA
  2. 2.Center for Artificial Vision Research, Korea University, Anam-dong, Seongbuk-gu, Seoul 136-713Korea

Personalised recommendations