Markov Chain Monte Carlo for Automated Face Image Analysis
We present a novel fully probabilistic method to interpret a single face image with the 3D Morphable Model. The new method is based on Bayesian inference and makes use of unreliable image-based information. Rather than searching a single optimal solution, we infer the posterior distribution of the model parameters given the target image. The method is a stochastic sampling algorithm with a propose-and-verify architecture based on the Metropolis–Hastings algorithm. The stochastic method can robustly integrate unreliable information and therefore does not rely on feed-forward initialization. The integrative concept is based on two ideas, a separation of proposal moves and their verification with the model (Data-Driven Markov Chain Monte Carlo), and filtering with the Metropolis acceptance rule. It does not need gradients and is less prone to local optima than standard fitters. We also introduce a new collective likelihood which models the average difference between the model and the target image rather than individual pixel differences. The average value shows a natural tendency towards a normal distribution, even when the individual pixel-wise difference is not Gaussian. We employ the new fitting method to calculate posterior models of 3D face reconstructions from single real-world images. A direct application of the algorithm with the 3D Morphable Model leads us to a fully automatic face recognition system with competitive performance on the Multi-PIE database without any database adaptation.
KeywordsFace image analysis Markov chain Monte Carlo Model fitting Morphable Model Generative models Top-down and bottom-up integration
- Blanz, V., & Vetter, T. (1999). A morphable model for the synthesis of 3d faces. In SIGGRAPH ’99: Proceedings of the 26th annual conference on computer graphics and interactive techniques (pp. 187–194). New York: ACM Press/Addison-Wesley. doi: 10.1145/311535.311556.
- Chib, S., & Greenberg, E. (1995). Understanding the Metropolis–Hastings algorithm. The American Statistician, 49(4), 327–335.Google Scholar
- Gonick, L., & Smith, W. (1993). Cartoon guide to statistics. New York: HarperCollins.Google Scholar
- Huang, G.B., Ramesh, M., Berg, T., & Learned-Miller, E. (2007). Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Tech. Rep. 07-49, University of Massachusetts, Amherst.Google Scholar
- Köstinger, M., Wohlhart, P., Roth, P. M., & Bischof, H. (2011). Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization. In 2011 IEEE international conference on computer vision workshops (ICCV workshops) (pp. 2144–2151).Google Scholar
- Kulkarni, T. D., Kohli, P., Tenenbaum, J. B., & Mansinghka, V. (2015). Picture: A probabilistic programming language for scene perception. In The IEEE conference on computer vision and pattern recognition (CVPR).Google Scholar
- Liu, C., Shum, H. Y., & Zhang, C. (2002). Hierarchical shape modeling for automatic face localization. In Computer Vision—ECCV 2002 (pp. 687–703). Heidelberg: Springer.Google Scholar
- Lüthi, M., Blanc, R., Albrecht, T., Gass, T., Goksel, O., Buchler, P., et al. (2012). Statismo—A framework for PCA based statistical models. The Insight Journal, 1, 1–18.Google Scholar
- Paysan, P., Knothe, R., Amberg, B., Romdhani, S., & Vetter, T. (2009). A 3D face model for pose and illumination invariant face recognition. In Advanced video and signal based surveillance, 2009 (pp. 296–301).Google Scholar
- Rauschert, I., & Collins, R. T. (2012). A generative model for simultaneous estimation of human body shape and pixel-level segmentation. In Computer Vision—ECCV 2012 (pp. 704–717). Heidelberg: Springer.Google Scholar
- Robert, C. P., & Casella, G. (2004). Monte Carlo statistical methods (Vol. 319). Citeseer.Google Scholar
- Romdhani, S., & Vetter, T. (2003). Efficient, robust and accurate fitting of a 3D morphable model. In Proceedings of ninth IEEE international conference on computer vision, 2003 (pp. 59–66).Google Scholar
- Romdhani, S., & Vetter, T. (2005). Estimating 3D shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior. In IEEE Computer Society conference on computer vision and pattern recognition, 2005 (CVPR 2005) (Vol. 2, pp. 986–993). doi: 10.1109/CVPR.2005.145.
- Schönborn, S., Egger, B., Forster, A., & Vetter, T. (2015). Background modeling for generative image models. Computer Vision and Image Understanding, 136, 117–127. doi: 10.1016/j.cviu.2015.01.008.
- Schönborn, S., Forster, A., Egger, B., & Vetter, T. (2013). A Monte Carlo strategy to integrate detection and model-based face analysis. In J. Weickert, M. Hein, & B. Schiele (Eds.), Pattern recognition. Lecture notes in computer science (Vol. 8142, pp. 101–110). Berlin: Springer.Google Scholar
- Wojek, C., Roth, S., Schindler, K., & Schiele, B. (2010). Monocular 3d scene modeling and inference: Understanding multi-object traffic scenes. In Computer Vision—ECCV 2010 (pp. 467–481). Heidelberg: Springer.Google Scholar
- Xiong, X., & De La Torre, F. (2013). Supervised descent method and its applications to face alignment. In 2013 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 532–539). doi: 10.1109/CVPR.2013.75.
- Yin, L., Wei, X., Sun, Y., Wang, J., & Rosato, M. (2006). A 3D facial expression database for facial behavior research. In 7th International conference on automatic face and gesture recognition, 2006 (FGR 2006) (pp. 211–216). doi: 10.1109/FGR.2006.6.
- Zhu, X., Yan, J., Yi, D., Lei, Z., & Li, S. (2015). Discriminative 3d morphable model fitting. In Proceedings of 11th IEEE international conference on automatic face and gesture recognition FG2015, Ljubljana, Slovenia.Google Scholar
- Zivanov, J., Forster, A., Schönborn, S., & Vetter, T. (2013). Human face shape analysis under spherical harmonics illumination considering self occlusion. In 6th International conference on biometrics, ICB-2013, Madrid.Google Scholar