Abstract
Inter-subject variability plays an important role in the performance of facial expression recognition. Therefore, several methods have been developed to bring the performance of a person-independent system closer to that of a person-dependent one. These techniques need different samples from a new person to increase the generalization ability. We have proposed a new approach to address this problem. It employs the person’s neutral samples as prior knowledge and a synthesis method based on the subspace learning to generate virtual expression samples. These samples have been incorporated in learning process to learn the style of the new person. We have also enriched the training data set by virtual samples created for each person in this set. Compared with previous studies, the results showed that our approach can perform the task of facial expression recognition effectively with better robustness for corrupted data.
Similar content being viewed by others
References
Cohen, I., Sebe, N., Garg, A., et al.: Facial expression recognition from video sequences: temporal and static modeling. Comput. Vis. Image Underst. 91, 160–187 (2003)
Maronidis, A., Bolis, D., Tefas, A., et al.: Improving subspace learning for facial expression recognition using person dependent and geometrically enriched training sets. Neural Netw. 24(8), 814–823 (2011)
Zhang, X.-Y., Liu, C.-L.: Style transfer matrix learning for writer adaptation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 393–400 (2011)
Xuan, T.W.: Rapid Speaker Adaptation by Variable Reference Model Subspace. Université De Rennes, Rennes (2008)
Niyogi, P., Girosi, F., Poggio, T.: Incorporating prior information in machine learning by creating virtual examples. Proc. IEEE 86(11), 2196–2209 (1998)
Jabid, M.H.K.T., Chae, O.: Robust facial expression recognition based on local directional pattern. ETRI J. 32(5), 784–794 (2010)
Cavalin, P.R., Sabourin, R., Suen, C.Y., et al.: Evaluation of incremental learning algorithms for HMM in the recognition of alphanumeric characters. Pattern Recognit. 42(12), 3241–3253 (2009)
Leggetter, C.J., Woodland, P.C.: Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov models. Comput. Speech Lang. 9(2), 171–185 (1995)
Kuhn, R., Junqua, J.-C., Nguyen, P., et al.: Rapid speaker adaptation in eigenvoice space. IEEE Trans. Speech Audio Process. 8(6), 695–707 (2000)
Kienzle, W., Chellapilla, K.: Personalized handwriting recognition via biased regularization. In: Proceedings of the 23rd International Conference on Machine learning, Pittsburgh, Pennsylvania, pp. 457–464 (2006)
Zhang, X.-Y., Liu, C.-L.: Writer adaptation with style transfer mapping. IEEE Trans. Pattern Anal. Mach. Intell. PP(99), 1 (2012)
Patel, N., Zaveri, M.: 3D facial model reconstruction, expressions synthesis and animation using single frontal face image. Signal Image Video Process. 7(5), 889–897 (2013). 2013/09/01
Yu, H., Zhang, J.: Mean value coordinates-based caricature and expression synthesis. Signal Image Video Process. 7(5), 899–910 (2013). 2013/09/01
Sharma, A., Anamika, D.: Facial expression recognition using virtual neutral image synthesis. In: National Conference on Computer Vision Pattern Recognition Image Processing and Graphics, Jaipur, India (2010)
Senechal, T., Bailly, K., Prevost, L.: Impact of action unit detection in automatic emotion recognition. In: Pattern Analysis and Applications, pp. 1–17. Springer, London (2012)
Mohammadzade, H., Hatzinakos, D.: An expression transformation for improving the recognition of expression-variant faces from one sample image per person. In: Fourth IEEE International Conference on Biometrics: Theory Applications and Systems (BTAS), pp. 1–6 (2010)
Zhao, X., Zhang, S.: Facial expression recognition using local binary patterns and discriminant kernel locally linear embedding. EURASIP J. Adv. Signal Process. 2012(1), 20 (2012)
Shan, C., Gong, S., McOwan, P.: Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis. Comput. 27(6), 803–816 (2009)
Mohammadzade, H., Hatzinakos, D.: Projection into expression subspaces for face recognition from single sample per person. IEEE Trans. Affect. Comput. 4(1), 69–82 (2013)
Bartlett, M.S., Littlewort, G., Frank, M. et al.: Fully automatic facial action recognition in spontaneous behavior. In: 7th International Conference on Automatic Face and Gesture Recognition, 2006. FGR 2006., pp. 223–230 (2006)
Ji, Y., Idrissi, K.: Automatic facial expression recognition based on spatiotemporal descriptors. Pattern Recognit. Lett. 33(10), 1373–1380 (2012)
Dornaika, F., Lazkano, E., Sierra, B.: Improving dynamic facial expression recognition with feature subset selection. Pattern Recognit. Lett. 32(5), 740–748 (2011)
Zhang, Y., Ji, Q.: Active and dynamic information fusion for facial expression understanding from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 699–714 (2005)
Schmid, K., Marx, D., Samal, A.: Computation of a face attractiveness index based on neoclassical canons, symmetry, and golden ratios. Pattern Recognit. 41(8), 2710–2717 (2008)
Lucey, P., Cohn, J.F., Kanade, T. et al.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 94–101 (2010)
Savran, L.S.A., Bilge, M.T.: Comparative evaluation of 3D vs. 2D modality for automatic detection of facial action units. Pattern Recognit. 45(2), 767–782 (2012)
Gu, W., Ã, C.X., Venkatesh, Y.V., et al.: Facial expression recognition using radial encoding of local Gabor features and classifier synthesis. Pattern Recognit. 45(1), 80–91 (2012)
Acknowledgments
The authors gratefully acknowledge partial funding from the Research Center of Intelligent Signal Processing.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix 1: Golden ratios features [24]
Mid-eye distance to inter-ocular distance
Mid-eye distance to nose width
Mouth width to inter-ocular distance
Lips–chin distance to inter-ocular distance
Lips–chin distance to nose width
Inter-ocular distance to eye fissure width
Inter-ocular distance to lip height
Nose width to eye fissure width
Nose width to lip height
Eye fissure width to nose–mouth distance
Lip height to nose–mouth distance
Length of face to width of face
Nose–chin distance to lips–chin distance
Nose width to nose–mouth distance
Mouth width to nose width
Appendix 2: neoclassic rules features [24]
Nose length \(=\) ear length
Inter-ocular distance \(=\) nose width
Inter-ocular distance \(=\) right or left eye fissure width
Mouth width \(= 1.5 \times \) nose width
Face width \(=4 \times \) nose width
Rights and permissions
About this article
Cite this article
Mohammadian, A., Aghaeinia, H. & Towhidkhah, F. Incorporating prior knowledge from the new person into recognition of facial expression. SIViP 10, 235–242 (2016). https://doi.org/10.1007/s11760-014-0732-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-014-0732-6