Multimedia Tools and Applications

, Volume 75, Issue 2, pp 709–731 | Cite as

Facial expression recognition based on a mlp neural network using constructive training algorithm

  • Hayet Boughrara
  • Mohamed Chtourou
  • Chokri Ben Amar
  • Liming Chen
Article

Abstract

This paper presents a constructive training algorithm for Multi Layer Perceptron (MLP) applied to facial expression recognition applications. The developed algorithm is composed by a single hidden-layer using a given number of neurons and a small number of training patterns. When the Mean Square Error MSE on the Training Data TD is not reduced to a predefined value, the number of hidden neurons grows during the neural network learning. Input patterns are trained incrementally until all patterns of TD are presented and learned. The proposed MLP constructive training algorithm seeks to find synthesis parameters as the number of patterns corresponding for subsets of each class to be presented initially in the training step, the initial number of hidden neurons, the number of iterations during the training step as well as the MSE predefined value. The suggested algorithm is developed in order to classify a facial expression. For the feature extraction stage, a biological vision-based facial description, namely Perceived Facial Images PFI has been applied to extract features from human face images. To evaluate, the proposed approach is tested on three databases which are the GEMEP FERA 2011, the Cohn-Kanade facial expression and the facial expression recognition FER-2013 databases. Compared to the fixed MLP architecture and the literature review, experimental results clearly demonstrate the efficiency of the proposed algorithm.

Keywords

Facial expression recognition Constructive training algorithm MLP Back-propagation Feature extraction Perceived facial images PCA 

References

  1. 1.
    Banziger T, Scherer KR (2010) Introducing the geneva multimodal emotion portrayal (gemep) corpus, Blueprint for Affective Computing, A Sourcebook, Series in affective science, chapter 6.1 pp 271–294. Oxford University Press, OxfordGoogle Scholar
  2. 2.
    Bejani M, Gharavian D, Charkari NM (2012) method, Audiovisual emotion recognition using ANOVA feature selection networks, multi-classifier neural Neural Computer & Application. Springer-Verlag LondonGoogle Scholar
  3. 3.
    Boughrara H, Chen L, Ben Amar C, Chtourou M (2012) Face Recognition under varying Facial Expression Based on Perceived Facial Images And Local Feature Matching, International Conference on Information Technology and e-Service, pp 1–6, 2nd edition. Sousse TunisiaGoogle Scholar
  4. 4.
    Boughrara H, Chtourou M, Ben Amar C (2012) MLP Neural Network Based Face Recognition System Using Constructive Training algorithm, International Conference on Multimedia Computing and System ICMCS, pp 233–238, 3rd edition. Tangier MoroccoGoogle Scholar
  5. 5.
    Boughrara H, Chen L, Ben Amar C, Chtourou M (2013) Facial Expression Recognition Based on Perceived Facial Images and Local Feature Matching, International Conference on Image Analysis and Processing, ICIAP, Part II, LNCS 8157. Springer-Verlag Berlin, Heidelberg, pp 591–600Google Scholar
  6. 6.
    Chakrabarti D, Dutta D (2013) Facial Expression Recognition Using Eigenspaces Procedia Technology vol 10, pp 755–761Google Scholar
  7. 7.
    Chumkamon S, Hayashi E (2013) Facial expression recognition using constrained local models and Hidden Markov models with consciousness-based architecture IEEE/SICE International Symposium on System Integration (SII), pp 382–387Google Scholar
  8. 8.
    Dahmane M, Meunier J (2011) Emotion recognition using dynamic gridbased hog features IEEE Int’l Conf. Automatic Face and Gesture Analysis, pp 884–888Google Scholar
  9. 9.
    Danisman T, Bilasco IM, Martinet J, Djeraba C (2013) Intelligent pixels of interest selection with application to facial expression recognition using multilayer perceptron. Signal Process 93(6):1547–1556CrossRefGoogle Scholar
  10. 10.
    Danisman T, Bilasco IM, Martinet J, Djeraba C (2013) Intelligent pixels of interest selection with application to facial expression recognition using multilayer perceptron. Signal Process 93(6):1547–1556CrossRefGoogle Scholar
  11. 11.
    Danisman T, Bilasco IM, Martinet J, Djeraba C (2013) Intelligent pixels of interest selection with application to facial expression recognition using multilayer perceptron. Signal Process 93(6):1547– 1556CrossRefGoogle Scholar
  12. 12.
    Dhall A, Asthana A, Goecke R, Gedeon T (2011) phog Emotion recognition using, features, lpq IEEE Int’l Conf Automatic Face and Gesture Analysis, pp 878–883Google Scholar
  13. 13.
    Edelman S, Intrator N, Poggio T (1997) Complex cells and object recognition, unpublished manuscript. http://kybele.psych.cornell.edu/edelman/archive.html
  14. 14.
    Ekman P (1992) Facial expressions of emotion: an old controversy and new findings. Philos Trans: Biol Sci 335:63–69CrossRefGoogle Scholar
  15. 15.
    Eskil MT, Benli KS (2014) Facial expression recognition based on anatomy Computer Vision and Image Understanding vol 119, pp 1–14Google Scholar
  16. 16.
    Fan W, Bouguila N (2013) Face detection and facial expression recognition using simultaneous clustering and feature selection via an expectation propagation statistical learning framework Multimed Tools ApplGoogle Scholar
  17. 17.
    Fang H, Parthalïn NM, Aubrey AJ, Tam G KL, Borgoa R, Rosin PL, Grant PW, Marshall D, Chend M (2014) Facial expression recognition in dynamic sequences: An integrated approach. Pattern Recognit 47(3):1271–1281CrossRefGoogle Scholar
  18. 18.
    Farajzadeh N, Pan G, Wu Z (2013) Facial expression recognition based on meta probability codes, Pattern Analysis Application. Springer-Verlag LondonGoogle Scholar
  19. 19.
    Gizatdinova Y, Surakka V (2006) Feature-based detection of facial landmarks from neutral and expressive facial images. IEEE Trans Pattern Anal Mach Intell 28:135139CrossRefGoogle Scholar
  20. 20.
    Gu W, Xiang C, Venkatesh YV, Huang D, Lin H (2012) Facial expression recognition using radial encoding of local Gabor features and classifier synthesis. Pattern Recognit 45:80–91CrossRefGoogle Scholar
  21. 21.
    Honggui H, Junfei Q. (2009) A Novel Pruning Algorithm for Self-organizing Neural Network, Proceedings of International Joint Conference on Neural Networks, Atlanta, Georgia USA, pp 1245–1250Google Scholar
  22. 22.
    Huang D, Ben Soltana W, Ardabilian M, Wang YH, Chen L (2011) Textured 3D face recognition using biological vision-based facial representation and optimized weighted sum fusion Proc. CVPR Workshop on Biometrics, Colorado Springs, CO USAGoogle Scholar
  23. 23.
    Huang D (2011) Robust Face Recognition Based on Three Dimentional Data,thesis informatic, LRIS. university of LyonGoogle Scholar
  24. 24.
    Jaimes A, Sebe N (2007) Multimodal human-computer interaction: a survey. Comput Vis Image Underst 108:116–134CrossRefGoogle Scholar
  25. 25.
    Kaburlasos VG, Papadakis SE, Papakostas GA (2013) Lattice computing extension of the FAM neural classifier for human facial expression recognition. IEEE Trans Neural Netw Learn Syst 24(10):1526–1538CrossRefGoogle Scholar
  26. 26.
    Kanade T, Cohn J, Tian YL (2000) Comprehensive database for facial expression analysis, International Conference on Automatic Face and Gesture Recognition, 4th edition, pp 46–53Google Scholar
  27. 27.
    Khan RA, Meyer A, Konik H, Bouakaz S (2013) Framework for reliable real-time facial expression recognition for low resolution images. Pattern Recognit Lett 34:11591168CrossRefGoogle Scholar
  28. 28.
    Lajevardi SM, Hussain ZM (2012) Automatic facial expression recognition: feature extraction and selection. SIViP 6:159–169CrossRefGoogle Scholar
  29. 29.
    Lee SH, Kim H, Man Ro Y, Plataniotis KN (2013) Using color texture sparsity for facial expression recognition, 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp 1–6Google Scholar
  30. 30.
    HC, Wu CY, Lin TM (2013) Facial Expression Recognition Using Image Processing Techniques and Neural Networks Advances in Intelligent Systems & Applications. Springer-Verlag Berlin Heidelberg, pp 259–267Google Scholar
  31. 31.
    Lee C, Huang S, Shih C (2010) Facial affect recognition using regularized discriminant analysis-based algorithms, EURASIP J Adv Signal Process vol 10Google Scholar
  32. 32.
    Lienhart R, Maydt J (2001) An extended set of haar-like features for rapid object detection. Int Conf Image Process 1:900–903CrossRefGoogle Scholar
  33. 33.
    Liu S, Ruan Q, Wang C, An G (2012) Tensor rank one differential graph preserving analysis for facial expression recognition. Image Vis Comput 30:535–545CrossRefGoogle Scholar
  34. 34.
    Liu S, Ruan Q, Wang C, An G (2012) Tensor rank one differential graph preserving analysis for facial expression recognition. Image Vis Comput 30:535–545CrossRefGoogle Scholar
  35. 35.
    Liu D, Tsu-Shuan C, Zhang Y (2002) A constructive algorithm for feedforward neural networks with incremental training. Trans Circ Syst-Fundam Theory Appl 49:1876–1879CrossRefGoogle Scholar
  36. 36.
    Lippmann RP (1989) Pattern classification using neural networks, IEEE Communications Magazine, vol 27, pp 47–50 59–64Google Scholar
  37. 37.
    Littlewort G, Whitehill J, Wu T, Butko N, Ruvolo P, Movellan J, Bartlett M (2011) The motion in emotion-a cert based approach to the fera emotion challenge IEEE Int’l Conf. Automatic Face and Gesture Analysis, pp 897–902Google Scholar
  38. 38.
    Liu S, Ruan Q, Wang C, An G (2012) Tensor rank one differential graph preserving analysis for facial expression recognition. Image Vis Comput 30:535–545CrossRefGoogle Scholar
  39. 39.
    Long F, Wu T, Movellan JR, Bartlett MS, Littlewort G (2012) Learning spatiotemporal features by using independent component analysis with application to facial expression recognition. Neurocomputing 93:126132CrossRefGoogle Scholar
  40. 40.
    Ma L, Khorasani K (2004) Facial expression recognition using constructive feedforward neural networks. IEEE Trans Syst Man Cybern B: Cybern 34:1588–1595CrossRefGoogle Scholar
  41. 41.
    Masmoudi S, Frikha M, Chtourou M, Hamida A, Efficient MLP (2011) constructive training algorithm using a neuron recruiting approach for isolated word recognition system. Int J Speech Technol 14:1–10CrossRefGoogle Scholar
  42. 42.
    Meng H, Romera-Paredes B, Berthouze N (2011) Emotion recognition by two view svm 2k classifier on dynamic facial expression features IEEE Int’l Conf. Automatic Face and Gesture Analysis, pp 854–859Google Scholar
  43. 43.
    Majumder A, Behera L, Subramanian VK (2014) Emotion recognition from geometric facial features using self-organizing map. Pattern Recognit 47 (3):1282–1293CrossRefGoogle Scholar
  44. 44.
    Mohseni S, Kordy HM, Ahmadi R (2013) Facial expression recognition using DCT features and neural network based decision tree, 55th International Symposium ELMAR, pp 361–364Google Scholar
  45. 45.
    Niese R, Al-Hamadi A, Farag A, Neumann H, Michaelis B (2012) Facial expression recognition based on geometric and optical flow features in colour image sequences. IET Comput Vis 6(2):79–89CrossRefMathSciNetGoogle Scholar
  46. 46.
    Owusu E, Zhan Y, Mao QR (2014) A neural-AdaBoost based facial expression recognition system. Expert Syst Appl 41(7):3383–3390CrossRefGoogle Scholar
  47. 47.
    Pratama M, Anavatti SG, Angelov PP, Lughofer E (2014) PANFIS: A novel incremental learning machine. IEEE Trans Neural Netw Learn Syst 25(1):55–68CrossRefGoogle Scholar
  48. 48.
    Puma-Villanueva WJ, dos Santos EP, Von Zuben FJ (2012) A constructive algorithm to synthesize arbitrarily connected feedforward neural networks. Neurocomputing 75(1):14–32CrossRefGoogle Scholar
  49. 49.
    Sadeghi H, Raie A-A, Mohammadi M-R (2013) Facial expression recognition using geometric normalization and appearance representation, 8th Iranian Conference on Machine Vision and Image Processing (MVIP), pp 159–163Google Scholar
  50. 50.
    Sharma SK, Chandra P (2010) Constructive neural networks: A Review. Int J Eng Sci Technol 2(12):7847–7855Google Scholar
  51. 51.
    Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: A comprehensive study. Image Vis Comput 27:803–816CrossRefGoogle Scholar
  52. 52.
    Srivastava R, Roy S, Yan S, Sim T (2011) Accumulated motion images for facial expression recognition in videos IEEE Int’l Conf. Automatic Face and Gesture Analysis, pp 903–908Google Scholar
  53. 53.
    Sridhar SS, Ponnavaikko M (2011) Improved adaptive learning algorithm for constructive neural networks. Int J Comput Electr Eng 3 (1):1793–8163Google Scholar
  54. 54.
  55. 55.
    Tang Y (2013) Machines, Deep learning using linear support vector Workshop on Challenges in Representation Learning ICMLGoogle Scholar
  56. 56.
    Tariq U, Lin K-H, Li Z, Zhou X, Wang Z, Le V, Huang T, Lv X, Han T (2011) Emotion recognition from an ensemble of features IEEE Int’l Conf. Automatic Face and Gesture Analysis, pp 872–877Google Scholar
  57. 57.
    Tian Y, Kanade T, Cohn J (2005) Facial expression analysis Handbook of Face Recognition. Springer. (Chapter 11)Google Scholar
  58. 58.
    Ionescu RT, Popescu M, Grozea C (2013) Local learning to improve bag of visual words model for facial expression recognition. Workshop on Challenges in Representation Learning ICMLGoogle Scholar
  59. 59.
    Valstar MF, Jiang B, Mehu M, Pantic M, Scherer K (2011) The First Facial Expression Recognition and Analysis Challenge, 9th IEEE conference on Automatic Face and Gesture Recognition. Santa Barbara, CaliforniaGoogle Scholar
  60. 60.
    Villegas M, Paredes R (2011) Dimensionality reduction by minimizing nearest-neighbor classification error. Pattern Recognit Lett 32:633–639CrossRefGoogle Scholar
  61. 61.
    Viola P , Jones M (2004) Robust real-time face detection. Int J Comput Vis 57(2):137154CrossRefGoogle Scholar
  62. 62.
    Wan S, Aggarwal JK (2014) Spontaneous facial expression recognition: A robust metric learning approach. Pattern Recognit 47(5):1859–1868CrossRefGoogle Scholar
  63. 63.
    Wang Z, Ruan Q, An G (2012) Facial expression recognition based on tensor local linear discriminant analysis IEEE 11th International Conference on Signal Processing (ICSP), pp 1226–1229Google Scholar
  64. 64.
    Wang S, Liu Z, Wang Z, Wu G, Shen P, He S, Wang X (2013) Analyses of a multimodal spontaneous facial expression database. IEEE Trans Affect Comput 4(1):34–46CrossRefGoogle Scholar
  65. 65.
    Luo Y, Wu C (2013) Facial expression recognition based on fusion feature of PCA and LBP with SVM. Optik - Int J Light Electron Opt 124(17):2767–2770CrossRefGoogle Scholar
  66. 66.
    Yang P, Liu Q, Metaxas DN (2010) Exploring facial expressions with compositional features. IEEE Conf Comput Vis Pattern Recognit:2638–2644Google Scholar
  67. 67.
    Yang S, Bhanu B (2011) Facial expression recognition using emotion avatar image IEEE Int’l Conf. Automatic Face and Gesture Analysis, pp 866–871Google Scholar
  68. 68.
    Zhang Z, Qiao J (2010) A Node Pruning Algorithm for Feedforward Neural Network Based on Neural Complexity, International Conference on Intelligent Control and Information Processing Dalian, China, pp 406–410Google Scholar
  69. 69.
    Zhao X, Zhang S (2012) Facial expression recognition using local binary patterns and discriminant kernel locally linear embedding. EURASIP J Adv Signal Process 10:1687–6180Google Scholar
  70. 70.
    Zavaschi THH, Britto Jr AS, Oliveira LES, Koerich AL (2013) Fusion of feature sets and classifiers for facial expression recognition. Expert Syst Appl 40(2):646–655CrossRefGoogle Scholar
  71. 71.
    Zhen W, Zilu Y (2012) Facial Expression Recognition Based on Local Phase Quantization and Sparse Representation Eighth International Conference on Natural Computation (ICNC), pp 222–225Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Hayet Boughrara
    • 1
  • Mohamed Chtourou
    • 1
  • Chokri Ben Amar
    • 2
  • Liming Chen
    • 3
  1. 1.Control and Energy Management LaboratoryUniversity of SfaxSfaxTunisia
  2. 2.Research Groups on Intelligent MachinesUniversity of SfaxSfaxTunisia
  3. 3.Laboratoire d’InfoRmatique en Image et Systémes d’informationUniversity of LyonEcullyFrance

Personalised recommendations