Hyperparameter optimization in CNN for learning-centered emotion recognition for intelligent tutoring systems

  • Ramon Zatarain Cabada
  • Hector Rodriguez RangelEmail author
  • Maria Lucia Barron Estrada
  • Hector Manuel Cardenas Lopez
Methodologies and Application


An intelligent tutoring system is used as an efficient self-learning tutor, where decisions are based on the affective state of the user. These detected emotions are what experts call basic emotions and the best-known recognition technique is the recognition of facial expressions. A convolutional neural network (CNN) can be used to identify emotions through facial gestures with very high precision. One problem with convolutional networks, however, is the high number of hyperparameters to define, which can range from a hundred to a thousand. This problem is usually solved by an expert experience combined with trial and error optimization. In this work, we propose a methodology using genetic algorithms for the optimization of hyperparameters of a CNN, used to identify the affective state of a person. In addition, we present the optimized network embedded into an intelligent tutoring system running on a mobile phone. The training process of the CNN was carried out on a PC with a GPU and the trained neural network was embedded into a mobile environment. The results show an improvement of 8% (from 74 to 82%) with genetic algorithms compared to a previous work that utilized a trial and error method.


Genetic algorithm Convolutional neural networks Intelligent tutoring systems 


Compliance with ethical standards

Conflict of interest

The authors declare that there is no conflict of interest regarding the publication of this paper.

Human and animal rights

This article does not contain any studies with human participants or animals performed by any of the authors.


  1. Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: large-scale machine learning on heterogeneous systems. Software available from tensorflow.orgGoogle Scholar
  2. Arevalillo-Herraez M, Arnau D, Ferri FJ, Santos OC (2017) Gui-driven intelligent tutoring system with affective support to help learning the algebraic method. In: 2017 IEEE international conference on systems, man, and cybernetics (SMC), IEEE, pp 2867–2872Google Scholar
  3. Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13:281–305MathSciNetzbMATHGoogle Scholar
  4. Bhakre SK, Bang A (2016) Emotion recognition on the basis of audio signal using Naive Bayes classifier. In: 2016 international conference on advances in computing, communications and informatics (ICACCI), IEEE, pp 2363–2367Google Scholar
  5. Bradski G (2000) The OpenCV Library. Dr. Dobb’s J Softw Tools 120:122–125Google Scholar
  6. Burkert P, Trier F, Afzal MZ, Dengel A, Liwicki M (2015) DeXpression: deep convolutional neural network for expression recognitionGoogle Scholar
  7. Calvo RA, D’Mello S (2010) Affect detection: an interdisciplinary review of models, methods, and their applications. IEEE Trans Affect Comput 1(1):18–37CrossRefGoogle Scholar
  8. Chakraborty R, Kopparapu SK (2016) Validating “Is ECC-ANN combination equivalent to DNN?” for speech emotion recognition. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC), IEEE, pp 004311–004316Google Scholar
  9. Deng W, Zhao H, Yang X, Xiong J, Sun M, Li B (2017a) Study on an improved adaptive PSO algorithm for solving multi-objective gate assignment. Appl Soft Comput 59:288–302CrossRefGoogle Scholar
  10. Deng W, Zhao H, Zou L, Li G, Yang X, Wu D (2017b) A novel collaborative optimization algorithm in solving complex optimization problems. Soft Comput 21(15):4387–4398CrossRefGoogle Scholar
  11. Deng W, Zhang S, Zhao H, Yang X (2018) A novel fault diagnosis method based on integrating empirical wavelet transform and fuzzy entropy for motor bearing. IEEE Access 6:35042–35056CrossRefGoogle Scholar
  12. Deng W, Xu J, Zhao H (2019) An improved ant colony optimization algorithm based on hybrid strategies for scheduling problem. IEEE Access 7:20281–20292CrossRefGoogle Scholar
  13. Ding H, Zhou SK, Chellappa R (2016) FaceNet2ExpNet: regularizing a deep face recognition net for expression recognitionGoogle Scholar
  14. Dinh HN, Van der Baan M (2019) A grid-search approach for 4D pressure-saturation discrimination. Geophysics 84(4):IM47–IM62CrossRefGoogle Scholar
  15. Ekman P (1992) An argument for basic emotions. Cognit Emot 6(3–4):169–200CrossRefGoogle Scholar
  16. Fahmi A (2018) Expected values of aggregation operators on cubic triangular fuzzy number and its application to multi-criteria decision making problems. Eng Math 2(1):1MathSciNetCrossRefGoogle Scholar
  17. Fahmi A, Abdullah S, Amin F, Ali A (2018) Weighted average rating (War) method for solving group decision making problem using triangular cubic fuzzy hybrid aggregation (Tcfha) operator. Technical report 1, Punjab UniversityGoogle Scholar
  18. Fahmi A, Amin F (2019) Precursor selection for sol–gel synthesis of titanium carbide nanopowders by a new hesitant CUBIC fuzzy multi-attribute group decision-making model. New Math Nat Comput 15(01):145–167MathSciNetCrossRefGoogle Scholar
  19. Floreano D, Dürr P, Mattiussi C (2008) Neuroevolution: from architectures to learning. Evol Intell 1(1):47–62CrossRefGoogle Scholar
  20. Ghali MA, Ayyad AA, Abu-Naser SS, Laban MA (2018) An intelligent tutoring system for teaching english grammar. Int J Acad Eng Res 2(2):1–6Google Scholar
  21. Ghayoumi M, Bansal AK (2016) Unifying geometric features and facial action units for improved performance of facial expression analysisGoogle Scholar
  22. González-Hernández F, Zatarain-Cabada R, Barrón-Estrada ML, Rodríguez-Rangel H (2018) Recognition of learning-centered emotions using a convolutional neural network. J Intell Fuzzy Syst 34(5):3325–3336CrossRefGoogle Scholar
  23. Graesser AC, Hu X, Nye BD, VanLehn K, Kumar R, Heffernan C, Heffernan N, Woolf B, Olney AM, Rus V, Andrasik F, Pavlik P, Cai Z, Wetzel J, Morgan B, Hampton AJ, Lippert AM, Wang L, Cheng Q, Vinson JE, Kelly CN, McGlown C, Majmudar CA, Morshed B, Baer W (2018) ElectronixTutor: an intelligent tutoring system with multiple learning resources for electronics. Int J STEM Educ 5(1):15CrossRefGoogle Scholar
  24. Griffith H, Griffith A (2017) Integration of an Intelligent Tutoring Software within an accelerated Engineering mathematics course. In: 2017 IEEE integrated STEM education conference (ISEC), IEEE, pp 131–134Google Scholar
  25. Kumar GAR, Kumar RK, Sanyal G (2017) Facial emotion analysis using deep convolution neural network. In: 2017 international conference on signal processing and communication (ICSPC), IEEE, pp 369–374Google Scholar
  26. Lin Hao-Chiang Koong, Su SH, Chao CJ, Hsieh CY, Tsai SC (2016) Construction of multi-mode affective learning system: taking affective design as an example. Educ Technol Soc 19(2):132–147Google Scholar
  27. Linnenbrink-Garcia L, Pekrun R (2011) Students’ emotions and academic engagement: introduction to the special issue. Contemp Educ Psychol 36(1):1–3CrossRefGoogle Scholar
  28. McCartin-Lim M, Woolf B, McGregor A (2018) Connect the dots to prove it. In: Proceedings of the 49th ACM technical symposium on computer science education- SIGCSE ’18. ACM Press, New York, USA, pp 533–538Google Scholar
  29. Miikkulainen R, Liang J, Meyerson E, Rawal A, Fink D, Francon O, Raju B, Shahrzad H, Navruzyan A, Duffy N, Hodjat B (2017) Evolving deep neural networks. Technical report, Thee University of Texas at AustinGoogle Scholar
  30. Montana DJ, Davis L (1989) Training feedforward neural networks using genetic algorithmsGoogle Scholar
  31. Parkkinen J, Jaward MH, Parthiban R, Kamarol SKA (2016) Spatiotemporal feature extraction for facial expression recognition. IET Image Process 10(7):534–541CrossRefGoogle Scholar
  32. Piho L, Tjahjadi T (2018) A mutual information based adaptive windowing of informative EEG for emotion recognition. IEEE Transactions on Affective Computing pp 1–1Google Scholar
  33. Pu X, Fan K, Chen X, Ji L, Zhou Z (2015) Facial expression recognition from image sequences using twofold random forest classifier. Neurocomputing 168:1173–1180CrossRefGoogle Scholar
  34. Salmam FZ, Madani A, Kissi M (2016) Facial expression recognition using decision trees. In: 2016 13th international conference on computer graphics, imaging and visualization (CGiV), IEEE, pp 125–130Google Scholar
  35. Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis Comput 27(6):803–816CrossRefGoogle Scholar
  36. Snoek J, Larochelle H, Adams RP (2012) Practical bayesian optimization of machine learning algorithms. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems. Curran Associates, Inc., pp 2951–2959.
  37. Suebnukarn S, Haddawy P (2004) A collaborative intelligent tutoring system for medical problem-based learning. In: Proceedings of the 9th international conference on intelligent user interface-IUI ’04. ACM Press, New York, USA, p 14Google Scholar
  38. Thompson N, McGill TJ (2017) Genetics with jean: the design, development and evaluation of an affective tutoring system. Educ Technol Res Dev 65(2):279–299CrossRefGoogle Scholar
  39. Tu F, Yin S, Ouyang P, Tang S, Liu L, Wei S (2017) Deep convolutional neural network architecture with reconfigurable computation patterns. IEEE Trans Very Large Scale Integr (VLSI) Syst 25(8):2220–2233CrossRefGoogle Scholar
  40. Wang C-H, Lin H-CK (2018) Constructing an affective tutoring system for designing course learning and evaluation. J Educ Comput Res 55(8):1111–1128CrossRefGoogle Scholar
  41. Wiggins JB, Grafsgaard JF, Boyer KE, Wiebe EN, Lester JC (2017) Do you think you can? the influence of student self-efficacy on the effectiveness of tutorial dialogue for computer science. Int J Artif Intell Educ 27(1):130–153CrossRefGoogle Scholar
  42. Wixon M, Arroyo I, Muldner K, Burleson W, Lozano C, Rai D, Woolf B (2014) The opportunities and limitations of scaling up sensor-free affect detection. In: Proceedings of the 7th international conference on educational data mining. EDM, London, UK, pp 145–152Google Scholar
  43. Xu X, Quan C, Ren F (2015) Facial expression recognition based on Gabor Wavelet transform and histogram of oriented gradients. In: 2015 IEEE international conference on mechatronics and automation (ICMA), IEEE, pp 2117–2122Google Scholar
  44. Yu Z, Zhang C (2015) Image based static facial expression recognition with multiple deep network learning. In: Proceedings of the 2015 ACM on international conference on multimodal interaction-ICMI ’15, ACM Press, New York, USA, pp 435–442Google Scholar
  45. Zhang L, Tjondronegoro D, Chandran V, Eggink J (2016) Towards robust automatic affective classification of images using facial expressions for practical applications. Multimed Tools Appl 75(8):4669–4695CrossRefGoogle Scholar
  46. Zeng Zhihong, Pantic M, Roisman G, Huang T (2009) A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans Pattern Anal Mach Intell 31(1):39–58CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Division de Estudios de Posgrado e InvestigacionTecnológico Nacional de México Campus CuliacánCuliacànMexico

Personalised recommendations