The Affective Experience of Novice Computer Programmers

  • Nigel BoschEmail author
  • Sidney D’Mello


Novice students (N = 99) participated in a lab study in which they learned the fundamentals of computer programming in Python using a self-paced computerized learning environment involving a 25-min scaffolded learning phase and a 10-min unscaffolded fadeout phase. Students provided affect judgments at approximately 100 points (every 15 s) over the course of viewing videos of their faces and computer screens recorded during the learning session. The results indicated that engagement, confusion, frustration, boredom, and curiosity were the most frequent affective states, while anxiety, happiness, anger, surprise, disgust, sadness, and fear were rare. Confusion + frustration and curiosity + engagement were identified as two frequently co-occurring pairs of affective states. An analysis of affect dynamics indicated that there were reciprocal transitions between engagement and confusion, confusion and frustration, and one way transitions between frustration and boredom and boredom and engagement. Considering interaction events in tandem with affect revealed that constructing code was the central activity that preceded and followed each affective state. Further, confusion and frustration followed errors and preceded hint usage, while curiosity and engagement followed reading or coding. An analysis of affect-learning relationships after partialling out control variables (e.g., scholastic aptitude, hint usage) indicated that boredom (r = −.149) and frustration (r = −.218) were negative correlated with learning while transitions between confusion → frustration (r = .103), frustration → confusion (r = .105), and boredom → engagement (r = .282) were positively correlated with learning. Implications of the results to theory on affect incidence and dynamics and on the design of affect-aware learning environments are discussed.


Affect Computer science education Intelligent tutoring systems 



This research was supported by the National Science Foundation (NSF) (ITR 0325428, HCC 0834847, DRL 1235958) and the Bill & Melinda Gates Foundation. Any opinions, findings and conclusions, or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of NSF.


  1. Alspaugh, C. A. (1972). Identification of some components of computer programming aptitude. Journal for Research in Mathematics Education, 3(2), 89–98.CrossRefGoogle Scholar
  2. Baker, R., Rodrigo, M. M. T., & Xolocotzin, U. E. (2007). The dynamics of affective transitions in simulation problem-solving environments. In A. C. R. Paiva, R. Prada, & R. W. Picard (Eds.), Affective computing and intelligent interaction (pp. 666–677). Berlin: Springer.CrossRefGoogle Scholar
  3. Blignaut, P., & Naude, A. (2008). The influence of temperament style on a student’s choice of and performance in a computer programming course. Computers in Human Behavior, 24(3), 1010–1020.CrossRefGoogle Scholar
  4. Bosch, N., & D’Mello, S. (2013). Sequential patterns of affective states of novice programmers. In E. Walker & C. K. Looi (Eds.), Proceedings of the First Workshop on AI-supported Education for Computer Science (AIEDCS 2013) (pp. 1–10).Google Scholar
  5. Bosch, N., D’Mello, S., & Mills, C. (2013). What emotions do novices experience during their first computer programming learning session? In H. C. Lane, K. Yacef, J. Mostow, & P. Pavlik (Eds.), Proceedings of the 16th International Conference on Artificial Intelligence in Education (AIED 2013) (pp. 11–20). Berlin Heidelberg: Springer.Google Scholar
  6. Bosch, N., Chen, H., Baker, R., Shute, V., & D’Mello, S. (2015). Accuracy vs. availability heuristic in multimodal affect detection in the wild. In Proceedings of the 17th International Conference on Multimodal Interaction. New York, NY: ACM.Google Scholar
  7. Burleson, W., & Picard, R. W. (2004). Affective agents: Sustaining motivation to learn through failure and a state of stuck. In Social and Emotional Intelligence in Learning Environments Workshop In Conjunction with the 7th International Conference on Intelligent Tutoring Systems, Maceio-Alagoas, Brasil.Google Scholar
  8. Calvo, R. A., & D’Mello, S. (2010). Affect detection: an interdisciplinary review of models, methods, and their applications. IEEE Transactions on Affective Computing, 1(1), 18–37.CrossRefGoogle Scholar
  9. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale: Erlbaum.zbMATHGoogle Scholar
  10. Cole, J. S., & Gonyea, R. M. (2010). Accuracy of self-reported SAT and ACT test scores: implications for research. Research in Higher Education, 51(4), 305–319.CrossRefGoogle Scholar
  11. Craig, S., Graesser, A., Sullins, J., & Gholson, B. (2004). Affect and learning: an exploratory look into the role of affect in learning with AutoTutor. Journal of Educational Media, 29(3), 241–250.CrossRefGoogle Scholar
  12. Craig, S., D’Mello, S., Witherspoon, A., & Graesser, A. (2008). Emote aloud during learning with AutoTutor: applying the Facial Action Coding System to cognitive–affective states during learning. Cognition & Emotion, 22(5), 777–788.CrossRefGoogle Scholar
  13. Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New York: Harper and Row.Google Scholar
  14. D’Mello, S. (2013). A selective meta-analysis on the relative incidence of discrete affective states during learning with technology. Journal of Educational Psychology, 105(4), 1082–1099.CrossRefGoogle Scholar
  15. D’Mello, S., & Graesser, A. (2010). Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features. User Modeling and User-Adapted Interaction, 20(2), 147–187.CrossRefGoogle Scholar
  16. D’Mello, S., & Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2), 145–157.CrossRefGoogle Scholar
  17. D’Mello, S., & Graesser, A. (2014a). Confusion. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), International handbook of emotions in education (pp. 289–310). New York: Routledge.Google Scholar
  18. D’Mello, S., & Graesser, A. (2014b). Inducing and tracking confusion and cognitive disequilibrium with breakdown scenarios. Acta Psychologica, 151, 106–116.CrossRefGoogle Scholar
  19. D’Mello, S., & Graesser, A. C. (2015). Feeling, thinking, and computing with affect-aware learning technologies. In R. A. Calvo, S. D’Mello, J. Gratch, & A. Kappas (Eds.), Handbook of affective computing (pp. 419–434). New York: Oxford University Press.Google Scholar
  20. D’Mello, S., & Kory, J. (2012). Consistent but modest: a meta-analysis on unimodal and multimodal affect detection accuracies from 30 studies. In Proceedings of the 14th ACM international conference on Multimodal interaction (pp. 31–38). New York, NY, USA: ACM.Google Scholar
  21. D’Mello, S., Taylor, R. S., & Graesser, A. (2007). Monitoring affective trajectories during complex learning. In Proceedings of the 29th annual meeting of the cognitive science society (pp. 203–208). Austin, TX: Cognitive Science Society.Google Scholar
  22. D’Mello, S., Person, N. K., & Lehman, B. (2009). Antecedent-consequent relationships and cyclical patterns between affective states and problem solving outcomes. In V. Dimitrova, R. Mizoguchi, B. du Boulay, & A. Graesser (Eds.), Proceedings of the 14th International Conference on Artificial Intelligence in Education (pp. 57–64). Amsterdam: IOS Press.Google Scholar
  23. D’Mello, S., Blanchard, N., Baker, R., Ocumpaugh, J., & Brawner, K. (2014a). I feel your pain: A selective review of affect-sensitive instructional strategies. In R. Sottilare, A. Graesser, X. Hu, & B. Goldberg (Eds.), Design recommendations for intelligent tutoring systems - volume 2: Instructional management (pp. 35–48). Orlando: U.S. Army Research Laboratory.Google Scholar
  24. D’Mello, S., Lehman, B., Pekrun, R., & Graesser, A. (2014b). Confusion can be beneficial for learning. Learning and Instruction, 29(1), 153–170.CrossRefGoogle Scholar
  25. Graesser, A., McDaniel, B., Chipman, P., Witherspoon, A., D’Mello, S., & Gholson, B. (2006). Detection of emotions during learning with AutoTutor. In R. Sun & N. Miyake (Eds.), Proceedings of the 28th Annual Meetings of the Cognitive Science Society (pp. 285–290). Austin, TX: Cognitive Science Society.Google Scholar
  26. Grafsgaard, J. F., Fulton, R. M., Boyer, K. E., Wiebe, E. N., & Lester, J. C. (2012). Multimodal analysis of the implicit affective channel in computer-mediated textual communication. In Proceedings of the 14th ACM international conference on Multimodal interaction (pp. 145–152). New York, NY, USA: ACM.Google Scholar
  27. Harley, J. M., Bouchet, F., & Azevedo, R. (2012). Measuring learners’ co-occurring emotional responses during their interaction with a pedagogical agent in MetaTutor. In S. A. Cerri, W. J. Clancey, G. Papadourakis, & K. Panourgia (Eds.), Intelligent tutoring systems (pp. 40–45). Berlin: Springer.CrossRefGoogle Scholar
  28. Haungs, M., Clark, C., Clements, J., & Janzen, D. (2012). Improving first-year success and retention through interest-based CS0 courses. In Proceedings of the 43rd ACM technical symposium on Computer Science Education (pp. 589–594). New York, NY, USA: ACM.Google Scholar
  29. Hosseini, R., Vihavainen, A., & Brusilovsky, P. (2014). Exploring problem solving paths in a Java programming course. In M. Coles & G. Ollis (Eds.), Psychology of Programming Interest Group Annual Conference 2014 (pp. 65–76).Google Scholar
  30. Inventado, P. S., Legaspi, R., Cabredo, R., & Numao, M. (2012). Student learning behavior in an unsupervised learning environment. In Proceedings of the 20th International Conference on Computers in Education (pp. 730–737). Singapore: National Institute of Education.Google Scholar
  31. Jadud, M. C. (2005). A first look at novice compilation behavior using BlueJ. Computer Science Education, 15(1), 25–40.CrossRefGoogle Scholar
  32. Khan, I. A., Hierons, R. M., & Brinkman, W. P. (2007). Mood independent programming. In Proceedings of the 14th European Conference on Cognitive Ergonomics: Invent! Explore! (pp. 28–31). New York, NY, USA: ACM.Google Scholar
  33. Larson, R. W., & Richards, M. H. (1991). Boredom in the middle school years: blaming schools versus blaming students. American Journal of Education, 99(4), 418–443.CrossRefGoogle Scholar
  34. Law, K. M. Y., Lee, V. C. S., & Yu, Y. T. (2010). Learning motivation in e-learning facilitated computer programming courses. Computers & Education, 55(1), 218–228.CrossRefGoogle Scholar
  35. Lee, D. M. C., Rodrigo, M. M. T., Baker, R., Sugay, J. O., & Coronel, A. (2011). Exploring the relationship between novice programmer confusion and achievement. In S. D’Mello, A. Graesser, B. Schuller, & J. C. Martin (Eds.), Affective computing and intelligent interaction (pp. 175–184). Berlin: Springer.CrossRefGoogle Scholar
  36. McQuiggan, S. W., Robison, J. L., & Lester, J. C. (2008). Affective transitions in narrative-centered learning environments. In B. P. Woolf, E. Aïmeur, R. Nkambou, & S. Lajoie (Eds.), Intelligent tutoring systems (pp. 490–499). Berlin: Springer.CrossRefGoogle Scholar
  37. Min, W., Mott, B., & Lester, J. (2014). Adaptive scaffolding in an intelligent game-based learning environment for computer science. In Proceedings of the Workshop on AI-supported Education for Computer Science (AIEDCS) at the 12th International Conference on Intelligent Tutoring Systems (pp. 41–50).Google Scholar
  38. Ocumpaugh, J., Baker, R., & Rodrigo, M. M. T. (2015). Baker Rodrigo Ocumpaugh Monitoring Protocol (BROMP) 2.0 Technical and Training Manual. In Technical Report. New York, NY: Teachers College, Columbia University. Manila, Philippines: Ateneo Laboratory for the Learning Sciences.Google Scholar
  39. Pekrun, R., & Linnenbrink-Garcia, L. (Eds.). (2014). International handbook of emotions in education. New York: Routledge.Google Scholar
  40. Pekrun, R., & Stephens, E. J. (2012). Academic emotions. In K. R. Harris, S. Graham, T. Urdan, S. Graham, J. M. Royer, & M. Zeidner (Eds.), APA educational psychology handbook, Vol 2: Individual differences and cultural and contextual factors (pp. 3–31). Washington, DC: American Psychological Association.CrossRefGoogle Scholar
  41. Porayska-Pomsta, K., Mavrikis, M., D’Mello, S., Conati, C., & Baker, R. (2013). Knowledge elicitation methods for affect modelling in education. International Journal of Artificial Intelligence in Education, 22(3), 107–140.Google Scholar
  42. Rodrigo, M. M. T., & Baker, R. (2009). Coarse-grained detection of student frustration in an introductory programming course. In Proceedings of the Fifth International Workshop on Computing Education Research (pp. 75–80). New York, NY, USA: ACM.Google Scholar
  43. Rodrigo, M. M. T., Baker, R., Jadud, M. C., Amarra, A. C. M., Dy, T., Espejo-Lahoz, M. B. V., … Tabanao, E. S. (2009a). Affective and behavioral predictors of novice programmer achievement. SIGCSE Bulletin, 41(3), 156–160.Google Scholar
  44. Rodrigo, M. M. T., Baker, R., Sugay, J. O., & Tabanao, E. S. (2009b). Monitoring novice programmer affect and behaviors to identify learning bottlenecks. In Philippine Computing Society Congress 2009 Research-in-Progress Section. Dumaguete City.Google Scholar
  45. Rosenberg, E. L., & Ekman, P. (1994). Coherence between expressive and experiential systems in emotion. Cognition & Emotion, 8(3), 201–229.CrossRefGoogle Scholar
  46. Shute, V. J., & Kyllonen, P. C. (1990). Modeling Individual Differences in Programming Skill Acquisition (Technical Paper No. AFHRL-TP-90-76) (p. 34). Brooks AFB, TX: Air Force Human Resources Laboratory.Google Scholar
  47. Tan, P.-N., Kumar, V., & Srivastava, J. (2002). Selecting the Right Interestingness Measure for Association Patterns. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 32–41). New York, NY, USA: ACM.Google Scholar
  48. VanLehn, K., Siler, S., Murray, C., Yamauchi, T., & Baggett, W. B. (2003). Why do only some events cause learning during human tutoring? Cognition and Instruction, 21(3), 209–249.CrossRefGoogle Scholar
  49. Weber, G., & Brusilovsky, P. (2001). ELM-ART: an adaptive versatile system for web-based instruction. International Journal of Artificial Intelligence in Education (IJAIED), 12, 351–384.Google Scholar

Copyright information

© International Artificial Intelligence in Education Society 2015

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of Notre DameNotre DameUSA
  2. 2.Department of PsychologyUniversity of Notre DameNotre DameUSA

Personalised recommendations