Evaluation of Classifiers for Emotion Detection While Performing Physical and Visual Tasks: Tower of Hanoi and IAPS

  • Shahnawaz QureshiEmail author
  • Johan Hagelbäck
  • Syed Muhammad Zeeshan IqbalEmail author
  • Hamad Javaid
  • Craig A. Lindley
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 868)


With the advancement in robot technology, smart human-robot interaction is of increasing importance for allowing the more excellent use of robots integrated into human environments and activities. If a robot can identify emotions and intentions of a human interacting with it, interactions with humans can potentially become more natural and effective. However, mechanisms of perception and empathy used by humans to achieve this understanding may not be suitable or adequate for use within robots. Electroencephalography (EEG) can be used for recording signals revealing emotions and motivations from a human brain. This study aimed to evaluate different machine learning techniques to classify EEG data associated with specific affective/emotional states. For experimental purposes, we used visual (IAPS) and physical (Tower of Hanoi) tasks to record human emotional states in the form of EEG data. The obtained EEG data processed, formatted and evaluated using various machine learning techniques to find out which method can most accurately classify EEG data according to associated affective/emotional states. The experiment confirms the choice of a method for improving the accuracy of results. According to the results, Support Vector Machine was the first, and Regression Tree was the second best method for classifying EEG data associated with specific affective/emotional states with accuracies up to 70.00% and 60.00%, respectively. In both tasks, SVM was better in performance than RT.


K-Nearest Neighbor (KNN) Regression Tree (RT) Bayesian Network (BNT) Support Vector Machine (SVM) Artificial Neural Networks (ANN) Tower of Hanoi (ToH) Cognitive psychology; Human Computer Interaction (HCI) Electroencephalography (EEG) 


  1. 1.
    Bethel, C.L., Salomon, K., Murphy, R.R., Burke, J.L.: Survey of psychophysiology measurements applied to human-robot interaction. In: The 16th IEEE International Symposium on Robot and Human interactive Communication, RO-MAN 2007, pp. 732–737. IEEE (2007)Google Scholar
  2. 2.
    Hagelbäck, J., Hilborn, O., Jerčić, P., Johansson, S.J., Lindley, C.A., Svensson, J., Wen, W.: Psychophysiological Interaction and Empathic Cognition for Human-Robot Cooperative Work (PsyIntEC), pp. 283–299. Springer, Cham (2014)Google Scholar
  3. 3.
    Liu, Y., Sourina, O., Nguyen, M.K.: Real-time EEG-based human emotion recognition and visualization. In: International Conference on Cyberworlds (CW), pp. 262–269, October 2010Google Scholar
  4. 4.
    Lin, Y.-P., Wang, C.-H., Wu, T.-L., Jeng, S.-K., Chen, J.-H.: EEG-based emotion recognition in music listening: a comparison of schemes for multiclass support vector machine. In: International Conference on Acoustics, Speech and Signal Processing, ICASSP 2009, pp. 489–492. IEEE (2009)Google Scholar
  5. 5.
    Bos, D.O.: EEG-based emotion recognition. Influ. Vis. Audit. Stimuli 56(3), 1–17 (2006)Google Scholar
  6. 6.
    Horlings, R., Datcu, D., Rothkrantz, L.J.: Emotion recognition using brain activity. In: Proceedings of the 9th International Conference on Computer Systems and Technologies and Workshop for Ph.D. Students in Computing, p. 6. ACM (2008)Google Scholar
  7. 7.
    Murugappan, M., Rizon, M., Nagarajan, R., Yaacob, S., Zunaidi, I., Hazry, D.: Lifting scheme for human emotion recognition using EEG. In: International Symposium on Information Technology, ITSim 2008, vol. 2, pp. 1–7. IEEE (2008)Google Scholar
  8. 8.
    Schaaff, K.: EEG-based emotion recognition. Ph.D. dissertation, Ph.D. thesis, Universitat Karlsruhe (TH) (2008)Google Scholar
  9. 9.
    Li, M., Chai, Q., Kaixiang, T., Wahab, A., Abut, H.: EEG emotion recognition system. In: In-vehicle Corpus and Signal Processing for Driver Behavior, pp. 125–135. Springer, Boston (2009)Google Scholar
  10. 10.
    Fedele, P., Gioia, M., Giannini, F., Rufa, A.: Results of a 3 year study of a BCI-based communicator for patients with severe disabilities (2016)Google Scholar
  11. 11.
    Sørensen, A.S., Nielsen, J., Maagaard, J., Nielsen, J.L., Rasmussen, G., Day, D.: Natural kinesthetic interaction and social relations between training-robots and their users. In: Workshop on Advances and Challenges on the Development, Testing and Assessment of Assistive and Rehabilitation Robots: Experiences from Engineering and Human Science Research, ICRA 2017, vol. 1, p. 44 (2017)Google Scholar
  12. 12.
    Fisher, R.A.: The use of multiple measurements in taxonomic problems. Ann. Eugen. 7(2), 179–188 (1936)CrossRefGoogle Scholar
  13. 13.
    Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32(2), 407–499 (2004)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Jaakkola, T., Jordan, M.: A variational approach to bayesian logistic regression models and their extensions. In: Sixth International Workshop on Artificial Intelligence and Statistics. Citeseer (1997)Google Scholar
  15. 15.
    Wilson, G.F., Russell, C., Monnin, J., Estepp, J., Christensen, J.: How does day-to-day variability in psychophysiological data affect classifier accuracy? In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 54(3), pp. 264–268 (2010). SAGE PublicationsGoogle Scholar
  16. 16.
    Millan, J.R., Renkens, F., Mouriño, J., Gerstner, W.: Noninvasive brain-actuated control of a mobile robot by human EEG. IEEE Trans. Biomed. Eng. 51(6), 1026–1033 (2004)CrossRefGoogle Scholar
  17. 17.
    Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23(10), 1175–1191 (2001)CrossRefGoogle Scholar
  18. 18.
    Nasoz, F., Alvarez, K., Lisetti, C.L., Finkelstein, N.: Emotion recognition from physiological signals using wireless sensors for presence technologies. Cogn., Technol. Work. 6(1), 4–14 (2004)CrossRefGoogle Scholar
  19. 19.
    Conati, C.: Probabilistic assessment of user’s emotions in educational games. Appl. Artif. Intell. 16(7–8), 555–575 (2002)CrossRefGoogle Scholar
  20. 20.
    Wilson, G.F., Russell, C.A.: Real-time assessment of mental workload using psychophysiological measures and artificial neural networks. Hum. Factors: J. Hum. Factors Ergon. Soc. 45(4), 635–644 (2003)CrossRefGoogle Scholar
  21. 21.
    Rani, P., Liu, C., Sarkar, N., Vanman, E.: An empirical study of machine learning techniques for affect recognition in human-robot interaction. Pattern Anal. Appl. 9(1), 58–69 (2006)CrossRefGoogle Scholar
  22. 22.
    Lourens, S., Zhang, Y., Long, J.D., Paulsen, J.S.: Analysis of longitudinal censored semicontinuous data with application to the study of executive dysfunction: the towers task. Stat. Methods Med. Res. (2014). Scholar
  23. 23.
    Zillmer, E., Culbertson, W.C.: Tower of London, Drexel University(TOLDX). Multi-Health System, Chicago, IL (2001)Google Scholar
  24. 24.
    Kirk, U.K.M., Kemp, S.: NEPSY a Development Neuropsychological Assessment Subtest Administration. The Psychological Corporation (1998)Google Scholar
  25. 25.
    Sahakian, B.J., Morris, R.G., Evenden, J.L., Heald, A., Levy, R., Philpot, M., Robbins, T.W.: A comparative study of visuospatial memory and learning in Alzheimer-type dementia and Parkinson’s disease. Brain 111(3), 695–718 (1988)CrossRefGoogle Scholar
  26. 26.
    Ruiz-Díaz, M., Hernández-González, M., Guevara, M.A., Amezcua, C., Ågmo, A.: Prefrontal EEG correlation during tower of Hanoi and WCST performance: effect of emotional visual stimuli. J. Sex. Med. 9(10), 2631–2640 (2012)CrossRefGoogle Scholar
  27. 27.
    Sohaib, A.T., Qureshi, S., Hagelbäck, J., Hilborn, O., Jerčić, P.: Evaluating classifiers for emotion recognition using EEG. In: Foundations of Augmented Cognition, pp. 492–501. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  28. 28.
    Chen, G., Hou, R.: A new machine double-layer learning method and its application in non-linear time series forecasting. In: International Conference on Mechatronics and Automation, pp. 795–799. IEEE (2007)Google Scholar
  29. 29.
    Vapnik, V.N.: An overview of statistical learning theory. IEEE Trans. Neural Netw. 10(5), 988–999 (1999)CrossRefGoogle Scholar
  30. 30.
    Burges, C.J.: A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Discov. 2(2), 121–167 (1998)CrossRefGoogle Scholar
  31. 31.
    Sasaki, M., et al.: EEG data classification with several mental tasks. In: IEEE International Conference on Systems, Man and Cybernetics, vol. 6, p. 4. IEEE (2002)Google Scholar
  32. 32.
    Breiman, L.: Classification and regression trees (1984)Google Scholar
  33. 33.
    Downey, S., Russell, M.: A decision tree approach to task-independent speech recognition. In: Proceedings-Institute of Acoustics. vol. 14, p. 181 (1992)Google Scholar
  34. 34.
    Brown, L.E., Tsamardinos, I., Aliferis, C.F.: A novel algorithm for scalable and accurate Bayesian network learning. Med. Info. 11(Pt 1), 711–715 (2004)Google Scholar
  35. 35.
    Ben-Gal, I.: Bayesian networks. Encyclopedia of Statistics in Quality and Reliability (2007)Google Scholar
  36. 36.
    Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Mach. Learn. 29(2–3), 131–163 (1997)CrossRefGoogle Scholar
  37. 37.
    Parvin, H., Alizadeh, H., Minaei-Bidgoli, B.: MKNN: Modified K-Nearest Neighbor. In: Proceedings of the World Congress on Engineering and Computer Science, pp. 831–834. Citeseer (2008)Google Scholar
  38. 38.
    Dasarathy, B.V.: Nearest neighbor \(\{\)NN\(\}\) norms: \(\{\)NN\(\}\) pattern classification techniques (1991)Google Scholar
  39. 39.
    Beyer, K., Goldstein, J., Ramakrishnan, R., Shaft, U.: When is nearest neighbor meaningful? In: International Conference on Database Theory, pp. 217–235. Springer, Heidelberg (1999)Google Scholar
  40. 40.
    Mauss, I.B., Robinson, M.D.: Measures of emotion: a review. Cogn. Emot. 23(2), 209–237 (2009)CrossRefGoogle Scholar
  41. 41.
    Plutchik, R.: Emotions and Life: Perspectives from Psychology, Biology, and Evolution. American Psychological Association (2003)Google Scholar
  42. 42.
    Russell, J.A.: Affective space is bipolar. J. Pers. Soc. Psychol. 37(3), 345 (1979)CrossRefGoogle Scholar
  43. 43.
    Chanel, G.: Emotion assessment for affective computing based on brain and peripheral signals. Ph.D. dissertation, University of Geneva (2009)Google Scholar
  44. 44.
    Wu, Y., Ianakiev, K., Govindaraju, V.: Improved k-nearest neighbor classification. Pattern Recognit. 35(10), 2311–2318 (2002)CrossRefGoogle Scholar
  45. 45.
    Speybroeck, N.: Classification and regression trees. Int. J. Public Health 57(1), 243–246 (2012)CrossRefGoogle Scholar
  46. 46.
    Nykopp, T.: Statistical modelling issues for the adaptive brain interface (2001)Google Scholar
  47. 47.
    Posner, J., Russell, J.A., Peterson, B.S.: The circumplex model of affect: an integrative approach to affective neuroscience, cognitive development, and psychopathology. Dev. Psychopathol. 17(03), 715–734 (2005)CrossRefGoogle Scholar
  48. 48.
    Simon, H.A.: The functional equivalence of problem solving skills. Cogn. Psychol. 7(2), 268–288 (1975)CrossRefGoogle Scholar
  49. 49.
    Goel, V., Grafman, J.: Are the frontal lobes implicated in planning functions? Interpreting data from the Tower of Hanoi. Neuropsychology 33(5), 623–642 (1995)CrossRefGoogle Scholar
  50. 50.
    Egan, D.E., Greeno, J.G.: Theory of rule induction: knowledge acquired in concept learning, serial pattern learning, and problem solving (1974)Google Scholar
  51. 51.
    Hayes, J.R., Simon, H.A.: Understanding written problem instructions (1974)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Shahnawaz Qureshi
    • 1
    Email author
  • Johan Hagelbäck
    • 2
  • Syed Muhammad Zeeshan Iqbal
    • 3
    Email author
  • Hamad Javaid
    • 4
  • Craig A. Lindley
    • 5
  1. 1.Department of Computer SciencePrince of Songkla UniversityHatyaiThailand
  2. 2.Department of Computer ScienceLinnaeus UniversityVäxjöSweden
  3. 3.BrightWareRiyadhSaudi Arabia
  4. 4.Jinnah International HospitalAbbottabadPakistan
  5. 5.Intelligent Sensing LaboratoryCSIROCanberraAustralia

Personalised recommendations