Advertisement

International Journal of Computer Vision

, Volume 75, Issue 1, pp 93–113 | Cite as

2D vs. 3D Deformable Face Models: Representational Power, Construction, and Real-Time Fitting

  • Iain Matthews
  • Jing Xiao
  • Simon Baker
Article

Abstract

Model-based face analysis is a general paradigm with applications that include face recognition, expression recognition, lip-reading, head pose estimation, and gaze estimation. A face model is first constructed from a collection of training data, either 2D images or 3D range scans. The face model is then fit to the input image(s) and the model parameters used in whatever the application is. Most existing face models can be classified as either 2D (e.g. Active Appearance Models) or 3D (e.g. Morphable Models). In this paper we compare 2D and 3D face models along three axes: (1) representational power, (2) construction, and (3) real-time fitting. For each axis in turn, we outline the differences that result from using a 2D or a 3D face model.

Keywords

Model-based face analysis 2D Active Appearance Models 3D Morphable Models representational power model construction non-rigid structure-from-motion factorization real-time fitting the inverse compositional algorithm constrained fitting 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Baker, S., Gross, R., and Matthews, I. 2003. Lucas-Kanade 20 years on: A unifying framework: Part 3. Technical Report CMU-RI-TR-03-35, Carnegie Mellon University Robotics Institute.Google Scholar
  2. Baker, S., Gross, R., and Matthews. I. 2004. Lucas-Kanade 20 years on: Part 4. Technical Report CMU-RI-TR-04-14, Carnegie Mellon University Robotics Institute.Google Scholar
  3. Baker, S. and Matthews, I. 2001. Equivalence and efficiency of image alignment algorithms. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1, pp. 1090–1097.Google Scholar
  4. Baker, S. and Matthews, I. 2004. Lucas-Kanade 20 years on: A unifying framework. International Journal of Computer Vision, 56:221–255.CrossRefGoogle Scholar
  5. Baker, S., Patil, R., Cheung, K.M., and Matthews, I. 2004. Lucas-Kanade 20 years on: Part 5. Technical Report CMU-RI-TR-04-64, Carnegie Mellon University Robotics Institute.Google Scholar
  6. Blanz, V., Basso, C., Poggio, T., and Vetter, T. 2003. Reanimating faces in images and video. In Proceedings of EUROGRAPHICS.Google Scholar
  7. Blanz, V. and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. In Computer Graphics, Annual Conference Series (SIGGRAPH), pp. 187–194.Google Scholar
  8. Brand, M. 2001. Morphable 3D models from video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 456–463.Google Scholar
  9. Bregler, C., Hertzmann, A., and Biermann, H. 2000. Recovering non-rigid 3D shape from image streams. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
  10. Cootes, T., Edwards, G., and Taylor, C. 2001 Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6):681–685.CrossRefGoogle Scholar
  11. Gross, R., Matthews, I., and Baker, S. 2004. Constructing and fitting active appearance models with occlusion. In Proceedings of the IEEE Workshop on Face Processing in Video, pp. 72–79.Google Scholar
  12. Hager, G. and Belhumeur, P. 1998. Efficient region tracking with parametric models of geometry and illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(10):1025–1039.CrossRefGoogle Scholar
  13. Ishikawa, T., Baker, S., Matthews, I., and Kanade, T. 2004. Passive driver gaze tracking with active appearance models. In Proceedings of the 11th World Congress on Intelligent Transportation Systems, URL: http://www.ri.cmu.edu/pubs/pub_4705.html.Google Scholar
  14. Kanade, T. 1973. Picture Processing System by Computer Complex and Recognition of Human Faces. PhD thesis, Kyoto University, November.Google Scholar
  15. Koterba, S., Baker, S., Matthews, I., Hu, C., Xiao, J., Cohn, J., and Kanade. T. 2005. Multi-view AAM fitting and camera calibration. In Proceedings of the IEEE International Conference on Computer Vision, pp. 511–518.Google Scholar
  16. Lanitis, A., Taylor, C., and Cootes, T. 1997. Automatic interpretation and coding of face images using flexible models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):742–756.CrossRefGoogle Scholar
  17. Lucas, B. and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proceedings of the International Joint Conference on Artificial Intelligence, pp. 674–679.Google Scholar
  18. Matthews, I. and Baker, S. 2004. Active Appearance Models revisited. International Journal of Computer Vision, 60(2):135–164.CrossRefGoogle Scholar
  19. Matthews, I., Cootes, T., Bangham, A., Cox, S., and Harvery, R. 2002. Extraction of visual features for lipreading. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(2):198–213.CrossRefGoogle Scholar
  20. Matthews, I., Xiao, I., and Baker, S. 2006. On the dimensionality of deformable face models. Technical Report CMU-RI-TR-06-12, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA.Google Scholar
  21. Romdhani, S. and Vetter, T. 2003. Efficient, robust and accurate fitting of a 3D morphable model. In Proceedings of the International Conference on Computer Vision, 1:59–55.CrossRefGoogle Scholar
  22. Rowley, H.A., Baluja, S., and Kanade, T. 1998. Neural network-based face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(1):23–38.CrossRefGoogle Scholar
  23. Shum, H.Y. and Szeliski, R. 2000. Construction of panoramic image mosaics with global and local alignment. International Journal of Computer Vision, 16(1):63–84.Google Scholar
  24. Tian, Y., Kanade, T., and Cohn, J. 2001. Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(2):97–115.CrossRefGoogle Scholar
  25. Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision, 9(2):137–154.CrossRefGoogle Scholar
  26. Torresani, L., Yang, D., Alexander, G., and Bregler, C. 2001. Tracking and modeling non-rigid objects with rank constraints. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1:493–500.Google Scholar
  27. Xiao, J., Baker, S., Matthews, I., and Kanade, T. 2004a. Real-time combined 2D Plus; 3D Active Appearance Models}. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 535–542.Google Scholar
  28. Xiao, J., Chai, J., and Kanade, T. 2004b. A closed-form solution to non-rigid shape and motion recovery. In Proceedings of the European Conference on Computer Vision, Vol. 4, pp. 573–587.Google Scholar
  29. Xiao, J., Chai, J., and Kanade, T. 2006. A closed-form solution to non-rigid shape and motion recovery. International Journal of Computer Vision, 67(2):233–246.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.The Robotics InstituteCarnegie Mellon UniversityPittsburghUSA
  2. 2.Epson Palo Alto Laboratory, Epson Research & DevelopmentSan JoseUSA
  3. 3.Microsoft ResearchMicrosoft CorporationRedmondUSA

Personalised recommendations