Skip to main content

Advertisement

SpringerLink
Log in

Hyperparameter optimization in CNN for learning-centered emotion recognition for intelligent tutoring systems

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

An intelligent tutoring system is used as an efficient self-learning tutor, where decisions are based on the affective state of the user. These detected emotions are what experts call basic emotions and the best-known recognition technique is the recognition of facial expressions. A convolutional neural network (CNN) can be used to identify emotions through facial gestures with very high precision. One problem with convolutional networks, however, is the high number of hyperparameters to define, which can range from a hundred to a thousand. This problem is usually solved by an expert experience combined with trial and error optimization. In this work, we propose a methodology using genetic algorithms for the optimization of hyperparameters of a CNN, used to identify the affective state of a person. In addition, we present the optimized network embedded into an intelligent tutoring system running on a mobile phone. The training process of the CNN was carried out on a PC with a GPU and the trained neural network was embedded into a mobile environment. The results show an improvement of 8% (from 74 to 82%) with genetic algorithms compared to a previous work that utilized a trial and error method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: large-scale machine learning on heterogeneous systems. Software available from tensorflow.org

  • Arevalillo-Herraez M, Arnau D, Ferri FJ, Santos OC (2017) Gui-driven intelligent tutoring system with affective support to help learning the algebraic method. In: 2017 IEEE international conference on systems, man, and cybernetics (SMC), IEEE, pp 2867–2872

  • Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13:281–305

    MathSciNet  MATH  Google Scholar 

  • Bhakre SK, Bang A (2016) Emotion recognition on the basis of audio signal using Naive Bayes classifier. In: 2016 international conference on advances in computing, communications and informatics (ICACCI), IEEE, pp 2363–2367

  • Bradski G (2000) The OpenCV Library. Dr. Dobb’s J Softw Tools 120:122–125

    Google Scholar 

  • Burkert P, Trier F, Afzal MZ, Dengel A, Liwicki M (2015) DeXpression: deep convolutional neural network for expression recognition

  • Calvo RA, D’Mello S (2010) Affect detection: an interdisciplinary review of models, methods, and their applications. IEEE Trans Affect Comput 1(1):18–37

    Article  Google Scholar 

  • Chakraborty R, Kopparapu SK (2016) Validating “Is ECC-ANN combination equivalent to DNN?” for speech emotion recognition. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC), IEEE, pp 004311–004316

  • Deng W, Zhao H, Yang X, Xiong J, Sun M, Li B (2017a) Study on an improved adaptive PSO algorithm for solving multi-objective gate assignment. Appl Soft Comput 59:288–302

    Article  Google Scholar 

  • Deng W, Zhao H, Zou L, Li G, Yang X, Wu D (2017b) A novel collaborative optimization algorithm in solving complex optimization problems. Soft Comput 21(15):4387–4398

    Article  Google Scholar 

  • Deng W, Zhang S, Zhao H, Yang X (2018) A novel fault diagnosis method based on integrating empirical wavelet transform and fuzzy entropy for motor bearing. IEEE Access 6:35042–35056

    Article  Google Scholar 

  • Deng W, Xu J, Zhao H (2019) An improved ant colony optimization algorithm based on hybrid strategies for scheduling problem. IEEE Access 7:20281–20292

    Article  Google Scholar 

  • Ding H, Zhou SK, Chellappa R (2016) FaceNet2ExpNet: regularizing a deep face recognition net for expression recognition

  • Dinh HN, Van der Baan M (2019) A grid-search approach for 4D pressure-saturation discrimination. Geophysics 84(4):IM47–IM62

    Article  Google Scholar 

  • Ekman P (1992) An argument for basic emotions. Cognit Emot 6(3–4):169–200

    Article  Google Scholar 

  • Fahmi A (2018) Expected values of aggregation operators on cubic triangular fuzzy number and its application to multi-criteria decision making problems. Eng Math 2(1):1

    Article  MathSciNet  Google Scholar 

  • Fahmi A, Abdullah S, Amin F, Ali A (2018) Weighted average rating (War) method for solving group decision making problem using triangular cubic fuzzy hybrid aggregation (Tcfha) operator. Technical report 1, Punjab University

  • Fahmi A, Amin F (2019) Precursor selection for sol–gel synthesis of titanium carbide nanopowders by a new hesitant CUBIC fuzzy multi-attribute group decision-making model. New Math Nat Comput 15(01):145–167

    Article  MathSciNet  Google Scholar 

  • Floreano D, Dürr P, Mattiussi C (2008) Neuroevolution: from architectures to learning. Evol Intell 1(1):47–62

    Article  Google Scholar 

  • Ghali MA, Ayyad AA, Abu-Naser SS, Laban MA (2018) An intelligent tutoring system for teaching english grammar. Int J Acad Eng Res 2(2):1–6

    Google Scholar 

  • Ghayoumi M, Bansal AK (2016) Unifying geometric features and facial action units for improved performance of facial expression analysis

  • González-Hernández F, Zatarain-Cabada R, Barrón-Estrada ML, Rodríguez-Rangel H (2018) Recognition of learning-centered emotions using a convolutional neural network. J Intell Fuzzy Syst 34(5):3325–3336

    Article  Google Scholar 

  • Graesser AC, Hu X, Nye BD, VanLehn K, Kumar R, Heffernan C, Heffernan N, Woolf B, Olney AM, Rus V, Andrasik F, Pavlik P, Cai Z, Wetzel J, Morgan B, Hampton AJ, Lippert AM, Wang L, Cheng Q, Vinson JE, Kelly CN, McGlown C, Majmudar CA, Morshed B, Baer W (2018) ElectronixTutor: an intelligent tutoring system with multiple learning resources for electronics. Int J STEM Educ 5(1):15

    Article  Google Scholar 

  • Griffith H, Griffith A (2017) Integration of an Intelligent Tutoring Software within an accelerated Engineering mathematics course. In: 2017 IEEE integrated STEM education conference (ISEC), IEEE, pp 131–134

  • Kumar GAR, Kumar RK, Sanyal G (2017) Facial emotion analysis using deep convolution neural network. In: 2017 international conference on signal processing and communication (ICSPC), IEEE, pp 369–374

  • Lin Hao-Chiang Koong, Su SH, Chao CJ, Hsieh CY, Tsai SC (2016) Construction of multi-mode affective learning system: taking affective design as an example. Educ Technol Soc 19(2):132–147

    Google Scholar 

  • Linnenbrink-Garcia L, Pekrun R (2011) Students’ emotions and academic engagement: introduction to the special issue. Contemp Educ Psychol 36(1):1–3

    Article  Google Scholar 

  • McCartin-Lim M, Woolf B, McGregor A (2018) Connect the dots to prove it. In: Proceedings of the 49th ACM technical symposium on computer science education- SIGCSE ’18. ACM Press, New York, USA, pp 533–538

  • Miikkulainen R, Liang J, Meyerson E, Rawal A, Fink D, Francon O, Raju B, Shahrzad H, Navruzyan A, Duffy N, Hodjat B (2017) Evolving deep neural networks. Technical report, Thee University of Texas at Austin

  • Montana DJ, Davis L (1989) Training feedforward neural networks using genetic algorithms

  • Parkkinen J, Jaward MH, Parthiban R, Kamarol SKA (2016) Spatiotemporal feature extraction for facial expression recognition. IET Image Process 10(7):534–541

    Article  Google Scholar 

  • Piho L, Tjahjadi T (2018) A mutual information based adaptive windowing of informative EEG for emotion recognition. IEEE Transactions on Affective Computing pp 1–1

  • Pu X, Fan K, Chen X, Ji L, Zhou Z (2015) Facial expression recognition from image sequences using twofold random forest classifier. Neurocomputing 168:1173–1180

    Article  Google Scholar 

  • Salmam FZ, Madani A, Kissi M (2016) Facial expression recognition using decision trees. In: 2016 13th international conference on computer graphics, imaging and visualization (CGiV), IEEE, pp 125–130

  • Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis Comput 27(6):803–816

    Article  Google Scholar 

  • Snoek J, Larochelle H, Adams RP (2012) Practical bayesian optimization of machine learning algorithms. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems. Curran Associates, Inc., pp 2951–2959. http://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf

  • Suebnukarn S, Haddawy P (2004) A collaborative intelligent tutoring system for medical problem-based learning. In: Proceedings of the 9th international conference on intelligent user interface-IUI ’04. ACM Press, New York, USA, p 14

  • Thompson N, McGill TJ (2017) Genetics with jean: the design, development and evaluation of an affective tutoring system. Educ Technol Res Dev 65(2):279–299

    Article  Google Scholar 

  • Tu F, Yin S, Ouyang P, Tang S, Liu L, Wei S (2017) Deep convolutional neural network architecture with reconfigurable computation patterns. IEEE Trans Very Large Scale Integr (VLSI) Syst 25(8):2220–2233

    Article  Google Scholar 

  • Wang C-H, Lin H-CK (2018) Constructing an affective tutoring system for designing course learning and evaluation. J Educ Comput Res 55(8):1111–1128

    Article  Google Scholar 

  • Wiggins JB, Grafsgaard JF, Boyer KE, Wiebe EN, Lester JC (2017) Do you think you can? the influence of student self-efficacy on the effectiveness of tutorial dialogue for computer science. Int J Artif Intell Educ 27(1):130–153

    Article  Google Scholar 

  • Wixon M, Arroyo I, Muldner K, Burleson W, Lozano C, Rai D, Woolf B (2014) The opportunities and limitations of scaling up sensor-free affect detection. In: Proceedings of the 7th international conference on educational data mining. EDM, London, UK, pp 145–152

  • Xu X, Quan C, Ren F (2015) Facial expression recognition based on Gabor Wavelet transform and histogram of oriented gradients. In: 2015 IEEE international conference on mechatronics and automation (ICMA), IEEE, pp 2117–2122

  • Yu Z, Zhang C (2015) Image based static facial expression recognition with multiple deep network learning. In: Proceedings of the 2015 ACM on international conference on multimodal interaction-ICMI ’15, ACM Press, New York, USA, pp 435–442

  • Zhang L, Tjondronegoro D, Chandran V, Eggink J (2016) Towards robust automatic affective classification of images using facial expressions for practical applications. Multimed Tools Appl 75(8):4669–4695

    Article  Google Scholar 

  • Zeng Zhihong, Pantic M, Roisman G, Huang T (2009) A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans Pattern Anal Mach Intell 31(1):39–58

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division de Estudios de Posgrado e Investigacion, Tecnológico Nacional de México Campus Culiacán, Juan de Dios Batiz 310 pte. Col Guadalupe C.P.80220, Culiacàn, Mexico

    Ramon Zatarain Cabada, Hector Rodriguez Rangel, Maria Lucia Barron Estrada & Hector Manuel Cardenas Lopez

Authors
  1. Ramon Zatarain Cabada
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hector Rodriguez Rangel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maria Lucia Barron Estrada
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hector Manuel Cardenas Lopez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hector Rodriguez Rangel.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest regarding the publication of this paper.

Human and animal rights

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by V. Loia.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zatarain Cabada, R., Rodriguez Rangel, H., Barron Estrada, M.L. et al. Hyperparameter optimization in CNN for learning-centered emotion recognition for intelligent tutoring systems. Soft Comput 24, 7593–7602 (2020). https://doi.org/10.1007/s00500-019-04387-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-019-04387-4

Keywords

Search

Navigation