Skip to main content
SpringerLink
Log in

2D vs. 3D Deformable Face Models: Representational Power, Construction, and Real-Time Fitting

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Model-based face analysis is a general paradigm with applications that include face recognition, expression recognition, lip-reading, head pose estimation, and gaze estimation. A face model is first constructed from a collection of training data, either 2D images or 3D range scans. The face model is then fit to the input image(s) and the model parameters used in whatever the application is. Most existing face models can be classified as either 2D (e.g. Active Appearance Models) or 3D (e.g. Morphable Models). In this paper we compare 2D and 3D face models along three axes: (1) representational power, (2) construction, and (3) real-time fitting. For each axis in turn, we outline the differences that result from using a 2D or a 3D face model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Baker, S., Gross, R., and Matthews, I. 2003. Lucas-Kanade 20 years on: A unifying framework: Part 3. Technical Report CMU-RI-TR-03-35, Carnegie Mellon University Robotics Institute.

  • Baker, S., Gross, R., and Matthews. I. 2004. Lucas-Kanade 20 years on: Part 4. Technical Report CMU-RI-TR-04-14, Carnegie Mellon University Robotics Institute.

  • Baker, S. and Matthews, I. 2001. Equivalence and efficiency of image alignment algorithms. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1, pp. 1090–1097.

  • Baker, S. and Matthews, I. 2004. Lucas-Kanade 20 years on: A unifying framework. International Journal of Computer Vision, 56:221–255.

    Article  Google Scholar 

  • Baker, S., Patil, R., Cheung, K.M., and Matthews, I. 2004. Lucas-Kanade 20 years on: Part 5. Technical Report CMU-RI-TR-04-64, Carnegie Mellon University Robotics Institute.

  • Blanz, V., Basso, C., Poggio, T., and Vetter, T. 2003. Reanimating faces in images and video. In Proceedings of EUROGRAPHICS.

  • Blanz, V. and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. In Computer Graphics, Annual Conference Series (SIGGRAPH), pp. 187–194.

  • Brand, M. 2001. Morphable 3D models from video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 456–463.

  • Bregler, C., Hertzmann, A., and Biermann, H. 2000. Recovering non-rigid 3D shape from image streams. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.

  • Cootes, T., Edwards, G., and Taylor, C. 2001 Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6):681–685.

    Article  Google Scholar 

  • Gross, R., Matthews, I., and Baker, S. 2004. Constructing and fitting active appearance models with occlusion. In Proceedings of the IEEE Workshop on Face Processing in Video, pp. 72–79.

  • Hager, G. and Belhumeur, P. 1998. Efficient region tracking with parametric models of geometry and illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(10):1025–1039.

    Article  Google Scholar 

  • Ishikawa, T., Baker, S., Matthews, I., and Kanade, T. 2004. Passive driver gaze tracking with active appearance models. In Proceedings of the 11th World Congress on Intelligent Transportation Systems, URL: http://www.ri.cmu.edu/pubs/pub_4705.html.

  • Kanade, T. 1973. Picture Processing System by Computer Complex and Recognition of Human Faces. PhD thesis, Kyoto University, November.

  • Koterba, S., Baker, S., Matthews, I., Hu, C., Xiao, J., Cohn, J., and Kanade. T. 2005. Multi-view AAM fitting and camera calibration. In Proceedings of the IEEE International Conference on Computer Vision, pp. 511–518.

  • Lanitis, A., Taylor, C., and Cootes, T. 1997. Automatic interpretation and coding of face images using flexible models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):742–756.

    Article  Google Scholar 

  • Lucas, B. and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674–679.

  • Matthews, I. and Baker, S. 2004. Active Appearance Models revisited. International Journal of Computer Vision, 60(2):135–164.

    Article  Google Scholar 

  • Matthews, I., Cootes, T., Bangham, A., Cox, S., and Harvery, R. 2002. Extraction of visual features for lipreading. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(2):198–213.

    Article  Google Scholar 

  • Matthews, I., Xiao, I., and Baker, S. 2006. On the dimensionality of deformable face models. Technical Report CMU-RI-TR-06-12, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA.

    Google Scholar 

  • Romdhani, S. and Vetter, T. 2003. Efficient, robust and accurate fitting of a 3D morphable model. In Proceedings of the International Conference on Computer Vision, 1:59–55.

    Article  Google Scholar 

  • Rowley, H.A., Baluja, S., and Kanade, T. 1998. Neural network-based face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(1):23–38.

    Article  Google Scholar 

  • Shum, H.Y. and Szeliski, R. 2000. Construction of panoramic image mosaics with global and local alignment. International Journal of Computer Vision, 16(1):63–84.

    Google Scholar 

  • Tian, Y., Kanade, T., and Cohn, J. 2001. Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(2):97–115.

    Article  Google Scholar 

  • Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision, 9(2):137–154.

    Article  Google Scholar 

  • Torresani, L., Yang, D., Alexander, G., and Bregler, C. 2001. Tracking and modeling non-rigid objects with rank constraints. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1:493–500.

    Google Scholar 

  • Xiao, J., Baker, S., Matthews, I., and Kanade, T. 2004a. Real-time combined 2D Plus; 3D Active Appearance Models}. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 535–542.

  • Xiao, J., Chai, J., and Kanade, T. 2004b. A closed-form solution to non-rigid shape and motion recovery. In Proceedings of the European Conference on Computer Vision, Vol. 4, pp. 573–587.

  • Xiao, J., Chai, J., and Kanade, T. 2006. A closed-form solution to non-rigid shape and motion recovery. International Journal of Computer Vision, 67(2):233–246.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The Robotics Institute, Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA, 15213, USA

    Iain Matthews

  2. Epson Palo Alto Laboratory, Epson Research & Development, 2580 Orchard Parkway, Suite 225, San Jose, CA, 95131, USA

    Jing Xiao

  3. Microsoft Research, Microsoft Corporation, One Microsoft Way, Redmond, WA, 98052-6339, USA

    Simon Baker

Authors
  1. Iain Matthews
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jing Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Simon Baker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simon Baker.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Matthews, I., Xiao, J. & Baker, S. 2D vs. 3D Deformable Face Models: Representational Power, Construction, and Real-Time Fitting. Int J Comput Vis 75, 93–113 (2007). https://doi.org/10.1007/s11263-007-0043-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-007-0043-2

Keywords

Search

Navigation