Multimedia Tools and Applications

, Volume 76, Issue 8, pp 10881–10892 | Cite as

User emotion recognition from a larger pool of social network data using active learning

  • Ghulam Muhammad
  • Mohammed F. Alhamid


The use of social networks has grown exponentially in recent years. The large amount of data available in these networks can be effectively utilized in many machine learning applications. This paper proposes a framework of an emotion recognition system that fetches huge amount of face images from the social networks into a cloud. In the cloud, the problem of the unlabeled facial images is handled by applying an active learning approach. For the feature extraction, an interlaced derivative pattern is used, while for a base classifier, an extreme learning machine is utilized. Once the emotion is recognized in the cloud, it can be shared with the end users to meet their interests. Several experiments were performed using some publicly available databases and heterogeneous images from the social networks. Experimental results showed that the proposed framework may effectively be used in the emotion recognition.


Social networks Cloud computing Emotion recognition Active learning Extreme learning machine 



The authors extend their appreciation to the Deanship of Scientific Research at King Saud University, Riyadh, Saudi Arabia for funding this work through the research group project no. RGP-1436-023


  1. 1.
    Averbeck BB, Bobina T, Evansa S, Shergill SS (2012) Emotion recognition and oxytocin in patients with schizophrenia. Psychol Med 42(02):259–266CrossRefGoogle Scholar
  2. 2.
    Bettadapura V (2012) Face expression recognition and analysis: the state of the art. College of Computing, Georgia Institute of Technology. Available at:
  3. 3.
    Domes G, Kumbier E, Heinrichs M, Herpertz SC (2014) Oxytocin promotes facial emotion recognition and Amygdala reactivity in adults with Asperger syndrome. Neuropsychopharmacology 39:698–706CrossRefGoogle Scholar
  4. 4.
    Fang Q, Xu C, Sang J, Hossain MS, Muhammad G (2015) Word-of-mouth understanding: entity-centric multimodal aspect opinion mining in social media. IEEE Trans Multimed 17(12):2281–2296CrossRefGoogle Scholar
  5. 5.
    Freytag A, Rodner E, Bodesheim P, Denzler J (2013) Labeling examples that matter: relevance-based active learning with Gaussian processes. Proc GCPR, 282–291Google Scholar
  6. 6.
    Fu Y, Li B, Zhu X, Zhang C (2014) Active learning without knowing individual instance labels: a pairwise label homogeneity query approach. IEEE Trans Knowl Data Eng 26(4):808–822CrossRefGoogle Scholar
  7. 7.
    Haque MM, Holder LB, Skinner MK, Cook DJ (2013) Generalized query-based active learning to identify differentially methylated regions in DNA. IEEE/ACM Trans Comput Biol Bioinf 10(3):632–644CrossRefGoogle Scholar
  8. 8.
    Hossain MS, Muhammad G (2015) Audio-visual emotion recognition using multi-directional regression and Ridgelet transform. J Multimodal User InterfGoogle Scholar
  9. 9.
    Hossain MS, Muhammad G, Alhamid MF, Song B, Al-Mutib K (2016) Audio-visual emotion recognition using big data towards 5G. Mob Netw Appl. doi: 10.1007/s11036-016-0685-9 Google Scholar
  10. 10.
    Hossain MS, Muhammad G, Song B, Hassan M, Alelaiwi A, Alamri A (2015) Audio-visual emotion-aware cloud gaming framework. IEEE Trans Circuits Syst Video Technol 25(12):2105–2118CrossRefGoogle Scholar
  11. 11.
    Huang G-B, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B Cybern 42(2):513–529CrossRefGoogle Scholar
  12. 12.
    Jiang D, Cui Y, Zhang X, Fan P, Ganzalez I, Sahli H (2010) Audio visual emotion recognition based on triple-stream dynamic bayesian network models. In: D’Mello S, et al LNCS 6974, p 609–618Google Scholar
  13. 13.
    Kanade T, Cohn J, Tian Y (2000) Comprehensive database for facial expression analysis. Proc. IEEE international conference on face and gesture recognition, 46–53Google Scholar
  14. 14.
    Lawrence K, Ruth C, Skuse D (2015) Age, gender, and puberty influence the development of facial emotion recognition. Front Psychol 6:761CrossRefGoogle Scholar
  15. 15.
    Liu K-H, Xu C-G (2009) A genetic programming-based approach to the classification of multiclass microarray datasets. Bioinformatics 25(3):331–337CrossRefGoogle Scholar
  16. 16.
    Majumder A, Behera L, Subramanian VK (2014) Emotion recognition from geometric facial features using self-organizing map. Pattern Recogn 47(3):1282–1293CrossRefGoogle Scholar
  17. 17.
    Mansoorizadeh M, Charkari NM (2010) Multimodal information fusion application to human emotion recognition from face and speech. Multimed Tools Appl 49(2):277–297CrossRefGoogle Scholar
  18. 18.
    Martin O, Kotsia I, Macq B, Pitas I (2006) The eNTERFACE’05 audiovisual emotion database. Proc. ICDEW’2006, 8, Atlanta, GAGoogle Scholar
  19. 19.
    Muhammad G (2015) Automatic speech recognition using interlaced derivative pattern for cloud based healthcare system. Clust Comput 18(2):795–802CrossRefGoogle Scholar
  20. 20.
    Muhammad G, Melhem M (2014) Pathological voice detection and binary classification using MPEG-7 audio features. Biomed Signal Process Control 11:1–9. doi: 10.1016/j.bspc.2014.02.001 CrossRefGoogle Scholar
  21. 21.
    Qian S, Zhang T, Xu C, Hossain MS (2015) Social event classification via boosted multi-modal supervised latent Dirichlet allocation. ACM Trans Multimedia Comput Commun Appl (ACM TOMM.) 11(2) Article 1, Article. 27: 27.1–27.22Google Scholar
  22. 22.
    Sachse M, Schlitt S, Hainz D, Ciaramidaro A, Walter H, Poustka F, Bölte S, Freitag CM (2014) Facial emotion recognition in paranoid schizophrenia and autism spectrum disorder. Schizophr Res 159(2–3):509–514CrossRefGoogle Scholar
  23. 23.
    Senechal T, Rapp V, Salam H, Seguier R, Bailly K, Prevost L (2012) Facial action recognition combining heterogeneous features via Multikernel learning. IEEE Trans Syst Man Cybern B Cybern 42(4):993–1005CrossRefGoogle Scholar
  24. 24.
    Shobeirinejad A, Gao Y (2010) Gender classification using interlaced derivative patterns. Proceedings of the 20th International Conference on Pattern Recognition (ICPR), 1509–1512Google Scholar
  25. 25.
    Sourati J, Erdogmus D, Dy JG, Brooks DH (2013) Accelerated learning-based interactive image segmentation using Pairwise constraints. IEEE Trans Image Process 23(7):3057–3070MathSciNetCrossRefGoogle Scholar
  26. 26.
    Viola P, Jones M (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154CrossRefGoogle Scholar
  27. 27.
    Yang X, Zhang T, Xu C, Hossain MS (2015) Automatic visual concept learning for social event understanding. IEEE Trans Multimed 17(3):46–58CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Department of Computer Engineering, College of Computer and Information SciencesKing Saud UniversityRiyadhSaudi Arabia
  2. 2.Department of Software Engineering, College of Computer and Information SciencesKing Saud UniversityRiyadhSaudi Arabia

Personalised recommendations