Modeling Negative Affect Detector of Novice Programming Students Using Keyboard Dynamics and Mouse Behavior

  • Larry VeaEmail author
  • Ma. Mercedes Rodrigo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10004)


We developed affective models for detecting negative affective states, particularly boredom, confusion, and frustration, among novice programming students learning C++, using keyboard dynamics and/or mouse behavior. The keystroke dynamics are already sufficient to model negative affect detector. However, adding mouse behavior, specifically the distance it travelled along the x-axis, slightly improved the model’s performance. The idle time and typing error are the most notable features that predominantly influence the detection of negative affect. The idle time has the greatest influence in detecting high and fair boredom, while typing error comes before the idle time for low boredom. Conversely, typing error has the highest influence in detecting high and fair confusion, while idle time comes before typing error for low confusion. Though typing error is also the primary indicator of high and fair frustrations, other features are still needed before it is acknowledged as such. Lastly, there is a very slim chance to detect low frustration.


Affect Model Novice programmer Keyboard dynamics Mouse behavior 


  1. 1.
    Affect. Encyclopedia of Mental Disorders.
  2. 2.
  3. 3.
    Carlos, C.M., Delos Santos, J.E., Fournier, G., Vea, L.: Towards the development of an intelligent agent for novice programmers through face expression recognition. In: Proceedings of the 13th Philippine Computing Science Congress, pp. 101–106 (2013)Google Scholar
  4. 4.
    Picard R.W., The Medial Lab – Affective Computing Group: Affective computing.
  5. 5.
    Rodrigo, M.M.T., Baker, R.S., Jadud, M.C., Amarra, A.C.M., Dy, T., Espejo-Lahoz, M.B.V., Lim, S.A., Pascua, S.A., Sugay, J.O., Tabanao, E.S.: Affective and behavioral predictors of novice programmer achievement. In: Proceedings of the 14th Annual ACM SIGCSE Conference on Innovation and Technology in Computer Science Education (ITiCSE 2009), vol. 41, no. 3, pp. 156–160 (2009).
  6. 6.
    Khanna, P., Sasikumar, M.: Recognising emotions from keyboard stroke pattern. Int. J. Comput. Appl. 11(9), 1–5 (2010)Google Scholar
  7. 7.
    Salmeron-Majadas, S., Santos, O.C., Boticario, J.G.: Exploring indicators from keyboard and mouse interactions to predict the user affective state. In: Educational Data Mining (2014)Google Scholar
  8. 8.
    Epp, C., Lippold, M., Mandryk, R.L.: Identifying emotional states using keystroke dynamics. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 715–724 (2011)Google Scholar
  9. 9.
    Tsui, W.H., Lee, P., Hsiao, T.C.: The effect of emotion on keystroke: an experimental study using facial feedback hypothesis. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 2870–2873 (2013)Google Scholar
  10. 10.
    Schuller, B., Rigoll, G., Lang, M.: Emotion recognition in the manual interaction with graphical user interfaces. In: Proceedings of the 2004 IEEE International Conference on Multimedia and Expo (ICME 2004), vol. 2, pp. 1215–1218 (2004).
  11. 11.
    Tsoulouhas, G., Georgiou, D., Karakos, A.: Detection of learner’s affective state based on mouse movements. J. Comput. 3(11), 9–18 (2011)Google Scholar
  12. 12.
    Felipe, D.A.M., Gutierrez, K.I.N., Quiros, E.C.M., Vea, L.A.: Towards the development of intelligent agent for novice C/C++ programmers through affective analysis of event logs. Proc. Int. MultiConf. Eng. Comput. Sci. 1, 511–518 (2012)Google Scholar
  13. 13.
    Lee, D.: Detecting confusion among novice programmers using BlueJ compile logs. Master’s thesis. Ateneo de Manila University, Quezon City (2011)Google Scholar
  14. 14.
    Dragon, T., Arroyo, I., Woolf, B.P., Burleson, W., el Kaliouby, R., Eydgahi, H.: Viewing student affect and learning through classroom observation and physical sensors. In: Woolf, B.P., Aïmeur, E., Nkambou, R., Lajoie, S. (eds.) ITS 2008. LNCS, vol. 5091, pp. 29–39. Springer, Heidelberg (2008). doi: 10.1007/978-3-540-69132-7_8 CrossRefGoogle Scholar
  15. 15.
    Viera, A.J., Garrett, J.M.: Understanding interobserver agreement: the kappa statistic. Fam. Med. 37(5), 360–363 (2005)Google Scholar
  16. 16.
    Bixler, R., D’Mello, S.: Detecting boredom and engagement during writing with keystroke analysis, task appraisals, and stable traits. In: Proceedings of the International Conference on Intelligent User Interfaces, pp. 225–234 (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Mapua Institute of TechnologyMakati CityPhilippines
  2. 2.Ateneo de Manila UniversityQuezon CityPhilippines

Personalised recommendations