Journal of Science Education and Technology

, Volume 25, Issue 6, pp 877–887 | Cite as

Middle School Engagement with Mathematics Software and Later Interest and Self-Efficacy for STEM Careers

  • Jaclyn Ocumpaugh
  • Maria Ofelia San Pedro
  • Huei-yi Lai
  • Ryan S. Baker
  • Fred Borgen


Research suggests that trajectories toward careers in science, technology, engineering, and mathematics (STEM) emerge early and are influenced by multiple factors. This paper presents a longitudinal study, which uses data from 76 high school students to explore how a student’s vocational self-efficacy and interest are related to his or her middle school behavioral and affective engagement. Measures of vocational self-efficacy and interest are drawn from STEM-related scales in CAPAExplore, while measures of middle school performance and engagement in mathematics are drawn from several previously validated automated indicators extracted from logs of student interaction with ASSISTments, an online learning platform. Results indicate that vocational self-efficacy correlates negatively with confusion, but positively with engaged concentration and carelessness. Interest, which also correlates negatively with confusion, correlates positively with correctness and carelessness. Other disengaged behaviors, such as gaming the system, were not correlated with vocational self-efficacy or interest, despite previous studies indicating that they are associated with future college attendance. We discuss implications for these findings, which have the potential to assist educators or counselors in developing strategies to sustain students’ interest in STEM-related careers.


STEM Affect Engagement Career self-efficacy Career interest 


  1. Baker RSJD, Ocumpaugh J (2014) Interaction-based affect detection in educational software. In: Calvo RA, D’Mello SK, Gratch J, Kappas A (eds) The oxford handbook of affective computing. Oxford University Press, OxfordGoogle Scholar
  2. Baker RSJD, Rossi LM (2013) Assessing the disengaged behavior of learners. In: Sottilare R, Graesser A, Hu X, Holden H (eds) Design recommendations for intelligent tutoring systems—volume 1—learner modeling. U.S. Army Research Lab, Orlando, pp 155–166Google Scholar
  3. Baker RS, Yacef K (2009) The state of educational data mining in 2009: a review and future visions. J Educ Data Min 1(1):3–17Google Scholar
  4. Baker RS, Corbett AT, Koedinger KR (2004) Detecting student misuse of intelligent tutoring systems. In: Proceedings of the international conference on intelligent tutoring systems, pp 531–540Google Scholar
  5. Baker RSJD, D’Mello SK, Rodrigo MMT, Graesser AC (2010) Better to be frustrated than bored: the incidence, persistence, and impact of learners’ cognitive-affective states during interactions with three different computer-based learning environments. Int J Hum Comput Stud 68(4):223–241CrossRefGoogle Scholar
  6. Bandura A (1997) Self-efficacy: the exercise of control. Freeman, New YorkGoogle Scholar
  7. Betz NE, Borgen FH (2006) Manual for the career confidence inventory. CAPA, AmesGoogle Scholar
  8. Betz NE, Borgen FH (2009) Comparative effectiveness of CAPA and FOCUS online career assessment systems with undecided college students. J Career Assess 17(4):351–366CrossRefGoogle Scholar
  9. Betz NE, Borgen FH (2010) The CAPA integrative online system for college major exploration. J Career Assess 18:317–327CrossRefGoogle Scholar
  10. Betz NE, Borgen FH (2015) CAPA confidence inventory manual: foundations, research, and interpretation. CAPA, AmesGoogle Scholar
  11. Betz NE, Hackett G (1981) The relationship of career-related self-efficacy expectations to perceived career options in college women and men. J Couns Psychol 28(5):399Google Scholar
  12. Blustein DL (2011) A relational theory of working. J Vocat Behav 79(1):1–17CrossRefGoogle Scholar
  13. Blustein DL, Kozan S, Connors-Kellgren A, Rand B (2015) Social class and career intervention. In: Hartung PJ, Savickas ML, Walsh WB (eds) APA handbook of career intervention, vol 1, foundations. American Psychological Association, Washington, DCGoogle Scholar
  14. Borgen FH, Betz NE (2011) Integrating self through personality, interests, and self-efficacy. In: Hartung PJ, Subich LM (eds) Developing self in work and career: concepts, cases, and contexts. American Psychological Association, Washington, DC, pp 141–160CrossRefGoogle Scholar
  15. Cantrell P, Ewing-Taylor J (2009) Exploring STEM career options through collaborative high school seminars. J Eng Educ 98(3):295–303CrossRefGoogle Scholar
  16. Clements K (1982) Careless errors made by sixth-grade children on written mathematical tasks. J Res Math Educ 13:136–144CrossRefGoogle Scholar
  17. Cocea M, Hershkovitz A, Baker RSJD (2009) The impact of off-task and gaming behaviors on learning: immediate or aggregate? In: Proceedings of the 14th international conference on artificial intelligence in education, pp 507–514Google Scholar
  18. Cook C, Heath F, Thompson RL (2000) A meta-analysis of response rates in web-or internet-based surveys. Educ Psychol Meas 60(6):821–836CrossRefGoogle Scholar
  19. Corbett AT, Anderson JR (1995) Knowledge tracing: modeling the acquisition of procedural knowledge. User Model User Adapt Interact 4(4):253–278CrossRefGoogle Scholar
  20. Craig S, Graesser A, Sullins J, Gholson B (2004) Affect and learning: an exploratory look into the role of affect in learning with AutoTutor. J Educ Media 29(3):241–250CrossRefGoogle Scholar
  21. Csikszentmihalyi M (1990) Flow and the psychology of discovery and invention. Harper Collins, New YorkGoogle Scholar
  22. D’Mello S (2013) A selective meta-analysis on the relative incidence of discrete affective states during learning with technology. J Educ Psychol 105(4):1082CrossRefGoogle Scholar
  23. D’Mello S, Graesser A (2012) Dynamics of affective states during complex learning. Learn Instr 22(2):145–157CrossRefGoogle Scholar
  24. D’Mello S, Blanchard N, Baker R, Ocumpaugh J, Brawner K (2014) I feel your pain: A selective review of affect-sensitive instructional strategies. In: Sottilare R, Graesser A, Hu X, Goldberg B (eds) Design recommendations for intelligent tutoring systems: Volume 2—Instructional Management. U.S. Army Research Laboratory, Orlando, FL, pp 35–48Google Scholar
  25. D’Mello SK, Craig SD, Witherspoon A, Mcdaniel B, Graesser A (2008) Automatic detection of learner’s affect from conversational cues. User Model User Adapt Interact 18(1–2):45–80CrossRefGoogle Scholar
  26. Doddannara L, Gowda S, Baker RSJD, Gowda S, de Carvalho AMJB (2013) Exploring the relationships between design, students’ affective states, and disengaged behaviors within an ITS. In: Proceedings of the 16th international conference on artificial intelligence & education, pp 31–40Google Scholar
  27. Fancsali SE (2013) Data-driven causal modeling of “gaming the system” and off-task behavior in cognitive tutor algebra. In: Proceedings of the NIPS workshop on data driven education, Las Vegas, NV, USA. December 10, 2013Google Scholar
  28. Feng M, Heffernan NT (2005) Informing teachers live about student learning: reporting in the ASSISTments system. Technol Instr Cognit Learn 3:1–14Google Scholar
  29. Fredricks JA, Blumenfeld PC, Paris A (2004) School engagement: potential of the concept: state of the evidence. Rev Educ Res 74:59–119Google Scholar
  30. Gong Y, Beck JE, Heffernan NT (2010) Comparing knowledge tracing and performance factor analysis by using multiple model fitting procedures. In: International conference on intelligent tutoring systems, pp 35–44Google Scholar
  31. Goodman L (1990) Time and learning in the special education classroom. SUNY Press, AlbanyGoogle Scholar
  32. Hackett G, Betz NE (1981) A self-efficacy approach to the career development of women. J Vocat Behav 18(3):326–339Google Scholar
  33. Hayden K, Ouyang Y, Scinski L, Olszewski B, Bielefeldt T (2011) Increasing student interest and attitudes in STEM: professional development and activities to engage and inspire learners. Contemp Issues Technol Teach Educ 11(1):47–69Google Scholar
  34. Holland JL (1959) A theory of vocational choice. J Couns Psychol 6:35–45CrossRefGoogle Scholar
  35. Holland JL (1997) Making vocational choices: a theory of vocational personalities and work environments, 3rd edn. Psychological Assessment Resources, OdessaGoogle Scholar
  36. Karweit N, Slavin RE (1982) Time-on-task: issues of timing, sampling, and definition. J Educ Psychol 74(6):844CrossRefGoogle Scholar
  37. Koedinger KR, Corbett A (2006) Cognitive tutors. In: Sawywer RK (ed) The Cambridge handbook of the learning sciences. Cambridge University Press, Cambridge, pp 61–77Google Scholar
  38. Krosnick J (1999) Survey research. Annu Rev Psychol 50:537–567CrossRefGoogle Scholar
  39. Larson LM (2012) Worklife across the lifespan. In: Altmaier EM, Hansen JC (eds) The oxford handbook of counseling psychology. Oxford University Press, New York, pp 128–178Google Scholar
  40. Lee DM, Rodrigo MM, Baker RSJd, Sugay J, Coronel A (2011) Exploring the relationship between novice programmer confusion and achievement. In: Proceedings of Affective Computing and Intelligent Interaction 2011, pp 175–184Google Scholar
  41. Lent RW, Brown SD (2006) On conceptualizing and assessing social cognitive constructs in career research: a measurement guide. J Career Assess 14(1):12–35CrossRefGoogle Scholar
  42. Lent RW, Brown SD, Hackett G (1994) Toward a unifying social cognitive theory of career and academic interest, choice, and performance. J Vocat Behav 45(1):79–122CrossRefGoogle Scholar
  43. Liu Z, Pataranutaporn V, Ocumpaugh J, Baker RSJd (2013) Sequences of frustration and confusion, and learning. In: Proceedings of the 6th international conference on educational data mining, pp 114–120Google Scholar
  44. National Science Foundation (2013) Science and engineering degrees: 1966–2010. Detailed Statistical Tables NSF 13-327. National Science Foundation, ArlingtonGoogle Scholar
  45. Ocumpaugh J, Baker RSJD, Rodrigo MMT (2012) Baker–Rodrigo observation method protocol (BROMP) 1.0. Training manual version 1.0. Technical report. EdLab, Ateneo Laboratory for the Learning Sciences, New York, ManilaGoogle Scholar
  46. Ocumpaugh J, Baker R, Gowda S, Heffernan N, Heffernan C (2014) Population validity for educational data mining models: a case study in affect detection. Br J Educ Technol 45(3):487–501CrossRefGoogle Scholar
  47. Ocumpaugh J, Baker RS, Rodrigo MMT (2015) Baker Rodrigo ocumpaugh monitoring protocol (BROMP) 2.0 technical and training manual. Technical report. Teachers College, Columbia University, Ateneo Laboratory for the Learning Sciences, New York, ManilaGoogle Scholar
  48. Pardos ZA, Gowda SM, Baker RSJd, Heffernan NT (2011) Ensembling predictions of student post-test scores for an intelligent tutoring system. In: Proceedings of the 4th international conference on educational data mining, pp 189–198Google Scholar
  49. Pardos ZA, Baker RS, San Pedro MO, Gowda SM, Gowda SM (2013) Affective states and state tests: Investigating how affect throughout the school year predicts end of year learning outcomes. In: Proceedings of the third international conference on learning analytics and knowledge, pp 117–124Google Scholar
  50. Pardos ZA, Baker RS, San Pedro MOCZ, Gowda SM (2014) Affective states and state tests: investigating how affect and engagement during the school year predict end of year learning outcomes. J Learn Anal 1(1):107–128Google Scholar
  51. Pekrun R (2011) Emotions as drivers of learning and cognitive development. In: Calvo RA, D’Mello SK (eds) New perspectives on affect and learning technologies. Springer, New York, pp 23–39CrossRefGoogle Scholar
  52. Pekrun R, Goetz T, Titz W, Perry RP (2002) Academic emotions in students’ self-regulated learning and achievement: a program of qualitative and quantitative research. Educ Psychol 37(2):91–105CrossRefGoogle Scholar
  53. Pekrun R, Goetz T, Daniels LM, Stupnisky RH, Perry RP (2010) Boredom in achievement settings: exploring control–value antecedents and performance outcomes of a neglected emotion. J Educ Psychol 102(3):531CrossRefGoogle Scholar
  54. Perneger TV (1998) What’s wrong with Bonferroni adjustments. Br Med J 316(7139):1236–1238CrossRefGoogle Scholar
  55. Razzaq L, Feng M, Nuzzo-Jones G, Heffernan NT, Koedinger KR, Junker B, Ritter S, Knight A, Aniszczyk C, Choksey S, Livak T, Mercado E, Turner TE, Upalekar R, Walonoski JA, Macasek MA, Rasmussen KP (2005) The Assistment project: blending assessment and assisting. In: Proceedings of the 12th annual conference on artificial intelligence in education, pp 555–562Google Scholar
  56. Reschly AL, Christenson SL (2012) Jingle, jangle, and conceptual haziness: evolution and future directions of the engagement construct. In: Christenson SL, Reschly AL, Wylie C (eds) Handbook of research on student engagement. Springer, New York, pp 3–19CrossRefGoogle Scholar
  57. Rodrigo MMT, Baker RS, Jadud MC, Amarra ACM, Dy T, Espejo-Lahoz MBV, Lim SAL, Pascua SAMS, Sugay JO, Tabanao ES (2009) Affective and behavioral predictors of novice programmer achievement. ACM SIGCSE Bull 41(3):156–160CrossRefGoogle Scholar
  58. Rottinghaus PJ, Eshelman AJ (2015) Integrative approaches to career intervention. In: Hartung PJ, Savickas ML, Walsh WB (eds) APA handbook of career intervention, vol 2: applications. American Psychological Association, Washington, DC, pp 25–39CrossRefGoogle Scholar
  59. San Pedro MOCZ, Baker RS, Rodrigo MMT (2011a) Detecting carelessness through contextual estimation of slip probabilities among students using an intelligent tutor for mathematics. In: Proceedings of the international conference on artificial intelligence in education, pp 304–311Google Scholar
  60. San Pedro MOCZ, Rodrigo MMT, Baker RS (2011b) The relationship between carelessness and affect in a Cognitive Tutor. In: Proceedings of the international conference on affective computing and intelligent interaction, pp 306–315Google Scholar
  61. San Pedro MOZ, Baker RS, Bowers AJ, Heffernan NT (2013) Predicting college enrollment from student interaction with an intelligent tutoring system in middle school. In: Proceedings of the 6th international conference on educational data mining, pp 177–184Google Scholar
  62. San Pedro MOZ, Ocumpaugh JL, Baker RS, Heffernan NT (2014) Predicting STEM and non-STEM college major enrollment from middle school interaction with mathematics educational software. In: Proceedings of the 7th international conference on educational data mining, pp 276–279Google Scholar
  63. Schiefele U, Csikszentmihalyi M (1995) Motivation and ability as factors in mathematics experience and achievement. J Res Math Educ 26(2):163–181Google Scholar
  64. Storey JD (2002) A direct approach to false discovery rates. J R Stat Soc B 64:479–498CrossRefGoogle Scholar
  65. Thomas CR, Gadbois SA (2007) Academic self-handicapping: the role of self-concept clarity & students’ learning strategies. Br J Educ Psychol 77(1):101–119CrossRefGoogle Scholar
  66. Turner S, Lapan RT (2002) Career self-efficacy and perceptions of parent support in adolescent career development. Career Dev Q 51(1):44–55Google Scholar
  67. Urdan T (2004) Predictors of academic self-handicapping and achievement: examining achievement goals, classroom goal structures, and culture. J Educ Psychol 96(2):251CrossRefGoogle Scholar
  68. Urdan T, Midgley C (2001) Academic self-handicapping: what we know, what more there is to learn. Educ Psychol Rev 13(2):115–138CrossRefGoogle Scholar
  69. Wang X (2013) Why students choose STEM majors motivation, high school learning, and postsecondary context of support. Am Educ Res J 50(5):1081–1121CrossRefGoogle Scholar
  70. Wentzel KR (1993) Does being good make the grade? Social behavior and academic competence in middle school. J Educ Psychol 85(2):357–364CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Jaclyn Ocumpaugh
    • 1
  • Maria Ofelia San Pedro
    • 2
  • Huei-yi Lai
    • 3
  • Ryan S. Baker
    • 1
  • Fred Borgen
    • 4
  1. 1.Penn Center for Learning Analytics, Graduate School of EducationUniversity of PennsylvaniaPhiladelphiaUSA
  2. 2.ACTIowa CityUSA
  3. 3.Department of Human Development, Teachers CollegeColumbia UniversityNew YorkUSA
  4. 4.Iowa State UniversityAmesUSA

Personalised recommendations