A modified fuzzy histogram of optical flow for emotion classification

  • P. RagupathyEmail author
  • P. Vivekanandan
Original Research


Human beings tend to express various emotions based on the activities. The criticality of facial expression has been recognized widely in the social interaction along with the social intellect. Human perception is subjective in nature and this makes the classification of emotion an extremely challenging problem. The mood, personality, age, and environment have a major influence on the perception of emotion. Facial expressions are a key to emotion; various studies devote on emotion classification based on facial expression. For the identification of these emotions, there is a mixture of models that make use of the feature representation that are gradient based, a mixture of various dynamic textures along with contextual information. In this work, histogram of optical flow (HOF) was used for the extraction of features and a neural network for bringing about an improvement to the accuracy of classification. With the availability of big data analytics, there has been a major increase in the power of computation in terms of analysing live video data, huge number of images and faster processing which is critical for emotion classification. The work has investigated efficacy of the flow of HOF and proposed a modified fuzzy histogram of optical flow. For choosing optimal rules in fuzzy system, heuristic method namely, charged system search was used. The results have proved that there has been a significant improvement to the methodology proposed.


Emotion recognition Neural network (NN) Histogram of optical flow (HOF) Fuzzy and charged system search (CSS) 


Supplementary material

12652_2019_1607_MOESM1_ESM.docx (20 kb)
Supplementary material 1 (DOCX 19 kb)


  1. Chen S, Tian Y, Liu Q, Metaxas DN (2013) Recognizing expressions from face and body gesture by temporal normalized motion and appearance features. Image Vis Comput 31(2):175–185CrossRefGoogle Scholar
  2. Gaidhane VH, Hote YV, Singh V (2016) Emotion recognition using eigen values and Levenberg–Marquardt algorithm-based classifier. Sadhana 41(4):415–423MathSciNetzbMATHGoogle Scholar
  3. Garg A, Bajaj R (2015) Facial expression recognition & classification using hybridization of ICA, GA, and neural network for human-computer interaction. J Netw Commun Emerg Technol (JNCET) 2(1):49–57Google Scholar
  4. Gharavian D, Bejani M, Sheikhan M (2017) Audio-visual emotion recognition using FCBF feature selection method and particle swarm optimization for fuzzy ARTMAP neural networks. Multim Tools Appl 76(2):2331–2352CrossRefGoogle Scholar
  5. Happy SL, Routray A (2017) Fuzzy histogram of optical flow orientations for micro-expression recognition. IEEE Trans Affect Comput 10(3):394–406CrossRefGoogle Scholar
  6. Kanade T, Tian Y, Cohn JF (2000) Comprehensive database for facial expression analysis. EEE international conference on automatic face and gesture recognition (Cat. No. PR00580), pp 46–53Google Scholar
  7. Kaveh A, Laknejadi K (2011) A novel hybrid charge system search and particle swarm optimization method for multi-objective optimization. Exp Syst Appl 38(12):15475–15488CrossRefGoogle Scholar
  8. Kaveh A, Talatahari S (2010) A novel heuristic optimization method: charged system search. Acta Mech 213(3):267–289CrossRefGoogle Scholar
  9. Kaveh A, Talatahari S (2011) Hybrid charged system search and particle swarm optimization for engineering design problems. Eng Comput 28(4):423–440CrossRefGoogle Scholar
  10. Matiko JW, Beeby SP, Tudor J (2014) Fuzzy logic based emotion classification. IEEE international conference on acoustics, speech and signal processing, pp 4389–4393Google Scholar
  11. Merghani W, Davison AK, Yap MH (2018) A review on facial micro-expressions analysis: datasets, features and metrics. IEEE J. arXiv:1805.02397(submitted)
  12. Mistry K, Zhang L, Neoh SC, Lim CP, Fielding B (2017) A micro-GA embedded PSO feature selection approach to intelligent facial emotion recognition. IEEE Trans Cybern 47(6):1496–1509CrossRefGoogle Scholar
  13. Perš J, Sulić V, Kristan M, Perše M, Polanec K, Kovačič S (2010) Histograms of optical flow for efficient representation of body motion. Pattern Recogn Lett 31(11):1369–1376CrossRefGoogle Scholar
  14. Prasad S, Kumar DV (2017) Hybrid fuzzy charged system search algorithm based state estimation in distribution networks. Eng Sci Technol Int J 20(3):922–933MathSciNetCrossRefGoogle Scholar
  15. Sathyanarayana S, Satzoda RK, Sathyanarayana S, Thambipillai S (2018) Vision-based patient monitoring: a comprehensive review of algorithms and technologies. J Ambient Intell Hum Comput 9(2):225–251CrossRefGoogle Scholar
  16. Singh L, Singh S, Aggarwal N (2019) Improved TOPSIS method for peak frame selection in audio-video human emotion recognition. Multim Tools Appl 78(5):6277–6308CrossRefGoogle Scholar
  17. Zhao X, Zhang S (2016) A review on facial expression recognition: feature extraction and classification. IETE Tech Rev 33(5):505–517CrossRefGoogle Scholar
  18. Zhao Y, Wang X, Goubran M, Whalen T, Petriu EM (2013) Human emotion and cognition recognition from body language of the head using soft computing techniques. J Ambient Intell Hum Comput 4(1):121–140CrossRefGoogle Scholar
  19. Zhao L, Wang Z, Zhang G (2017) Facial expression recognition from video sequences based on spatial-temporal motion local binary pattern and Gabor multi-orientation fusion histogram. Math Prob Eng 2017:1–12Google Scholar
  20. Zhao J, Mao X, Zhang J (2018) Learning deep facial expression features from image and optical flow sequences using 3D CNN. Visual Comput 34(10):1461–1475CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringPark College of Engineering and TechnologyCoimbatoreIndia

Personalised recommendations