Advertisement

Joint Unsupervised Face Alignment and Behaviour Analysis

  • Lazaros Zafeiriou
  • Epameinondas Antonakos
  • Stefanos Zafeiriou
  • Maja Pantic
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8692)

Abstract

The predominant strategy for facial expressions analysis and temporal analysis of facial events is the following: a generic facial landmarks tracker, usually trained on thousands of carefully annotated examples, is applied to track the landmark points, and then analysis is performed using mostly the shape and more rarely the facial texture. This paper challenges the above framework by showing that it is feasible to perform joint landmarks localization (i.e. spatial alignment) and temporal analysis of behavioural sequence with the use of a simple face detector and a simple shape model. To do so, we propose a new component analysis technique, which we call Autoregressive Component Analysis (ARCA), and we show how the parameters of a motion model can be jointly retrieved. The method does not require the use of any sophisticated landmark tracking methodology and simply employs pixel intensities for the texture representation.

Keywords

Face alignment time series alignment slow feature analysis 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

978-3-319-10593-2_12_MOESM1_ESM.pdf (1.3 mb)
Electronic Supplementary Material(1,335 KB)
978-3-319-10593-2_12_MOESM2_ESM.mpg (3.1 mb)
Electronic Supplementary Material(3,194 KB)

References

  1. 1.
    Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 31(1), 39–58 (2009)CrossRefGoogle Scholar
  2. 2.
    Zhou, F., De la Torre, F.: Canonical time warping for alignment of human behavior. In: Conference on Neural Information Processing Systems (NIPS), pp. 2286–2294 (2009)Google Scholar
  3. 3.
    Nicolaou, M.A., Pavlovic, V., Pantic, M.: Dynamic probabilistic cca for analysis of affective behaviour. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VII. LNCS, vol. 7578, pp. 98–111. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  4. 4.
    Tzimiropoulos, G., Alabort-i-Medina, J., Zafeiriou, S., Pantic, M.: Generic active appearance models revisited. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012, Part III. LNCS, vol. 7726, pp. 650–663. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  5. 5.
    Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: IEEE Proceedings of Int’l Conf. on Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  6. 6.
    Asthana, A., Zafeiriou, S., Cheng, S., Pantic, M.: Robust discriminative response map fitting with constrained local models. In: IEEE Proceedings of Int’l Conf. on Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  7. 7.
    Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-pie. Image and Vision Computing (JIVC) 28(5), 807–813 (2010)CrossRefGoogle Scholar
  8. 8.
    Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: A semi-automatic methodology for facial landmark annotation. In: IEEE Proceedings of Int’l Conf. on Computer Vision and Pattern Recognition Workshop (CVPR-W 2013), 5th Workshop on Analysis and Modeling of Faces and Gestures (AMFG 2013), Portland Oregon, USA (June 2013)Google Scholar
  9. 9.
    Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: The first facial landmark localization challenge. In: IEEE Proceedings of Int’l Conf. on Computer Vision Workshop (ICCV-W 2013), 300 Faces in-the-Wild Challenge (300-W), Sydney, Australia (December 2013)Google Scholar
  10. 10.
    Zhou, F., De la Torre, F., Cohn, J.F.: Unsupervised discovery of facial events. In: IEEE Proceedings of Int’l Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 2574–2581. IEEE (2010)Google Scholar
  11. 11.
    Zhou, F., De la Torre, F., Hodgins, J.K.: Hierarchical aligned cluster analysis for temporal clustering of human motion. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 35(3), 582–596 (2013)CrossRefGoogle Scholar
  12. 12.
    Antonakos, E., Pitsikalis, V., Rodomagoulakis, I., Maragos, P.: Unsupervised classification of extreme facial events using active appearance models tracking for sign language videos. In: IEEE Proceedings of Int’l Conf. on Image Processing (ICIP), Orlando, FL, USA (October 2012)Google Scholar
  13. 13.
    Zhang, W., Shan, S., Chen, X., Gao, W.: Local gabor binary patterns based on mutual information for face recognition. International Journal of Image and Graphics 7(04), 777–793 (2007)CrossRefGoogle Scholar
  14. 14.
    Ha, S.W., Moon, Y.H.: Multiple object tracking using sift features and location matching. International Journal of Smart Home 5(4) (2011)Google Scholar
  15. 15.
    Zafeiriou, L., Nicolaou, M.A., Zafeiriou, S., Nikitidis, S., Pantic, M.: Learning slow features for behaviour analysis. In: IEEE Proceedings of Int’l Conf. on Computer Vision (ICCV) (November 2013)Google Scholar
  16. 16.
    Zhou, F., De la Torre, F.: Generalized time warping for multi-modal alignment of human motion. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)Google Scholar
  17. 17.
    Rue, H., Held, L.: Gaussian Markov random fields: theory and applications. CRC Press (2004)Google Scholar
  18. 18.
    Peng, Y., Ganesh, A., Wright, J., Xu, W., Ma, Y.: Rasl: Robust alignment by sparse and low-rank decomposition for linearly correlated images. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 34(11), 2233–2246 (2012)CrossRefGoogle Scholar
  19. 19.
    Zhao, C., Cham, W.K., Wang, X.: Joint face alignment with a generic deformable face model. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 561–568. IEEE (2011)Google Scholar
  20. 20.
    Sagonas, C., Panagakis, Y., Zafeiriou, S., Pantic, M.: Raps: Robust and efficient automatic construction of person-specific deformable models. In: Proceedings of IEEE Int’l Conf. on Computer Vision and Pattern Recognition (CVPR 2014) (June 2014)Google Scholar
  21. 21.
    De la Torre, F., Black, M.J.: Robust parameterized component analysis: theory and applications to 2d facial appearance models. Computer Vision and Image Understanding 91(1), 53–71 (2003)CrossRefGoogle Scholar
  22. 22.
    De la Torre, F., Nguyen, M.H.: Parameterized kernel principal component analysis: Theory and applications to supervised and unsupervised image alignment. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8. IEEE (2008)Google Scholar
  23. 23.
    Cheng, X., Fookes, C., Sridharan, S., Saragih, J., Lucey, S.: Deformable face ensemble alignment with robust grouped-l1 anchors. In: 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG 2013), pp. 1–7 (2013)Google Scholar
  24. 24.
    Cheng, X., Sridharan, S., Saragih, J., Lucey, S.: Rank minimization across appearance and shape for aam ensemble fitting. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 577–584. IEEE (2013)Google Scholar
  25. 25.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE Proceedings of Int’l Conf. on Computer Vision and Pattern Recognition (CVPR) (2001)Google Scholar
  26. 26.
    Lampert, C.H., Blaschko, M.B., Hofmann, T.: Efficient subwindow search: A branch and bound framework for object localization. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 31(12), 2129–2142 (2009)CrossRefGoogle Scholar
  27. 27.
    Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: IEEE Proceedings of Int’l Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 2879–2886 (2012)Google Scholar
  28. 28.
    Orozco, J., Martinez, B., Pantic, M.: Empirical analysis of cascade deformable models for multi-view face detection. In: IEEE Proceedings of Int’l Conf. on Image Processing (ICIP) (2013)Google Scholar
  29. 29.
    Jiang, T., Jurie, F., Schmid, C.: Learning shape prior models for object matching. In: IEEE Proceedings of Int’l Conf. on Computer Vision and Pattern Recognition (CVPR) (2009)Google Scholar
  30. 30.
    Kokkinos, I., Yuille, A.: Unsupervised learning of object deformation models. In: IEEE Proceedings of Int’l Conf. on Computer Vision (ICCV) (2007)Google Scholar
  31. 31.
    Yang, J., Frangi, A.F., Yang, J.Y., Zhang, D., Jin, Z.: Kpca plus lda: a complete kernel fisher discriminant framework for feature extraction and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 27(2), 230–244 (2005)CrossRefGoogle Scholar
  32. 32.
    Zou, H., Hastie, T., Tibshirani, R.: Sparse principal component analysis. Journal of Computational and Graphical Statistics 15(2), 265–286 (2006)CrossRefMathSciNetGoogle Scholar
  33. 33.
    Baker, S., Matthews, I.: Lucas-kanade 20 years on: A unifying framework. International Journal of Computer Vision (IJCV) 56(3), 221–255 (2004)CrossRefGoogle Scholar
  34. 34.
    Jolliffe, I.: Principal component analysis. Wiley Online Library (2005)Google Scholar
  35. 35.
    Welling, M.: Fisher linear discriminant analysis. Department of Computer Science. University of Toronto 3 (2005)Google Scholar
  36. 36.
    Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)CrossRefGoogle Scholar
  37. 37.
    He, X., Niyogi, P.: Locality preserving projections. In: NIPS, vol. 16, pp. 234–241 (2003)Google Scholar
  38. 38.
    Roweis, S., Ghahramani, Z.: A unifying review of linear gaussian models. Neural Computation 11(2), 305–345 (1999)CrossRefGoogle Scholar
  39. 39.
    Wiskott, L., Sejnowski, T.J.: Slow feature analysis: Unsupervised learning of invariances. Neural Computation 14(4), 715–770 (2002)CrossRefzbMATHGoogle Scholar
  40. 40.
    Valstar, M.F., Pantic, M.: Induced disgust, happiness and surprise: an addition to the mmi facial expression database. In: Proceedings of Int’l Conf. on Language Resources and Evaluation (LREC), Workshop on EMOTION, Malta (May 2010)Google Scholar
  41. 41.
    Valstar, M.F., Pantic, M.: Mmi facial expression database, http://www.mmifacedb.com/
  42. 42.
    Dibeklioglu, H., Salah, A.A., Gevers, T.: Uva-nemo smile database, http://www.uva-nemo.org/
  43. 43.
    Belhumeur, P.N., Jacobs, D.W., Kriegman, D.J., Kumar, N.: Localizing parts of faces using a consensus of exemplars. In: IEEE Proceedings of Int’l Conf. on Computer Vision and Pattern Recognition (CVPR) (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Lazaros Zafeiriou
    • 1
  • Epameinondas Antonakos
    • 1
  • Stefanos Zafeiriou
    • 1
  • Maja Pantic
    • 1
  1. 1.Computing DepartmentImperial College LondonUK

Personalised recommendations