Advertisement

Technology, Knowledge and Learning

, Volume 22, Issue 3, pp 243–270 | Cite as

An Integrated Look at Middle School Engagement and Learning in Digital Environments as Precursors to College Attendance

  • Maria Ofelia Z. San Pedro
  • Ryan S. Baker
  • Neil T. Heffernan
Original Research
First Online:

Abstract

Middle school is an important phase in the academic trajectory, which plays a major role in the path to successful post-secondary outcomes such as going to college. Despite this, research on factors leading to college-going choices do not yet utilize the extensive fine-grained data now becoming available on middle school learning and engagement. This paper uses interaction-based data-mined assessments of student behavior, academic emotions and knowledge from a middle school online learning environment, and evaluates their relationships with different outcomes in high school and college. The data-mined measures of student behavior, emotions, and knowledge are used in three analyses: (1) to develop a prediction model of college attendance; (2) to evaluate their relationships to intermediate outcomes on the path to college attendance such as math and science course-taking during high school; (3) to develop an overall path model between the educational experiences students have during middle school, their high school experiences, and their eventual college attendance. This gives a richer picture of the cognitive and non-cognitive mechanisms that students experience throughout varied phases in their years in school, and how they may be related to one another. Such understanding may provide educators with information about students’ trajectories within the college pipeline.

Keywords

Post-secondary outcomes Middle school learning Academic emotion Engagement Educational technology Educational data mining Learning analytics 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

This research was supported by Grants NSF #DRL-1031398, NSF #SBE-0836012, and Grant #OPP1048577 from the Bill and Melinda Gates Foundation.

References

  1. Aleven, V., Mclaren, B., Roll, I., & Koedinger, K. (2006). Toward meta-cognitive tutoring: A model of help seeking with a Cognitive Tutor. International Journal of Artificial Intelligence in Education, 16(2), 101–128.Google Scholar
  2. Arnold, K. E., & Pistilli, M. D. (2012, April). Course signals at Purdue: Using learning analytics to increase student success. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 267–270). ACM.Google Scholar
  3. Arroyo, I., Ferguson, K., Johns, J., Dragon, T., Meheranian, H., Fisher, D., et al. (2007) Repairing disengagement with non-invasive interventions. In Proceedings of AIED 2007 (pp. 195–202).Google Scholar
  4. Baker, R., & Siemens, G. (2014). Educational data mining and learning analytics. In K. Sawyer (Ed.), Cambridge handbook of the learning sciences (2nd ed., pp. 253–274).Google Scholar
  5. Baker, R., Walonoski, J., Heffernan, N., Roll, I., Corbett, A., & Koedinger, K. (2008b). Why students engage in “gaming the system” behavior in interactive learning environments. Journal of Interactive Learning Research, 19(2), 185–224.Google Scholar
  6. Baker, R. S., Corbett, A. T., & Aleven, V. (2008a). More accurate student modeling through contextual estimation of slip and guess probabilities in bayesian knowledge tracing. In ITS 2008 (pp. 406–415).Google Scholar
  7. Baker, R. S., Corbett, A. T., Gowda, S. M., Wagner, A. Z., MacLaren, B. A., Kauffman, L. R., et al. (2010a). Contextual slip and prediction of student performance after use of an intelligent tutor. In Proceedings of UMAP 2010 (pp. 52–63).Google Scholar
  8. Baker, R. S., Corbett, A. T., Koedinger, K. R., Evenson, S. E., & Beck, J. (2006). Adapting to when students game an intelligent tutoring system. In Proceedings ITS 2006 (pp. 392–401).Google Scholar
  9. Baker, R. S., Corbett, A. T., Koedinger, K. R., & Wagner, A. Z. (2004). Off-task behavior in the cognitive tutor classroom: When students game the system. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 383–390).Google Scholar
  10. Baker, R. S., D'Mello, S. K., Rodrigo, M. M. T., & Graesser, A. C. (2010b). Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive–affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies, 68(4), 223–241.CrossRefGoogle Scholar
  11. Balfanz, R. (2009). Putting middle grades students on the graduation path: A policy and practice brief. Everyone Graduates Center & Talent Development Middle Grades Program.Google Scholar
  12. Bandura, A. (1986). Social foundations of thought and action (pp. 5–107). Englewood Cliffs, NJ: Prentice Hall.Google Scholar
  13. Bowers, A. J. (2010). Grades and graduation: A longitudinal risk perspective to identify student dropouts. The Journal of Educational Research, 103(3), 191–207.CrossRefGoogle Scholar
  14. Cabrera, A. F. (1994). Logistic regression analysis in higher education: An applied perspective. Higher Education: Handbook of Theory and Research, 10, 225–256.Google Scholar
  15. Cabrera, A. F., & La Nasa, S. M. (2000). Understanding the college-choice process. New Directions for Institutional Research, 2000(107), 5–22.CrossRefGoogle Scholar
  16. Cabrera, A. F., La Nasa, S. M., & Burkum, K. R. (2001). Pathways to a four-year degree: The higher education story of one generation. Center for the Study of Higher Education.Google Scholar
  17. Camblin, S. (2003). The middle grades: Putting all students on track for college. Honolulu, HI: Pacific Resources for Education and Learning.Google Scholar
  18. Canfield, W. (2001). ALEKS: A Web-based intelligent tutoring system. Mathematics and Computer Education, 35(2), 152–158.Google Scholar
  19. Carnevale, A. P., & Rose, S. J. (2003). Socioeconomic status, race/ethnicity, and selective college admissions. New York: Century Foundation.Google Scholar
  20. Clarke-Midura, J., & Dede, C. (2010). Assessment, technology, and change. Journal of Research on Technology in Education, 42(3), 309–328.CrossRefGoogle Scholar
  21. Clements, M. A. (1982). Careless errors made by sixth-grade children on written mathematical tasks. Journal for Research in Mathematics Education, 13, 136–144.CrossRefGoogle Scholar
  22. Cocea, M., Hershkovitz, A., & Baker, R. S. (2009). The impact of off-task and gaming behaviors on learning: Immediate or aggregate? In Proceedings of AIED 2009 (pp. 507–514).Google Scholar
  23. Cohen, J. (1960). A coefficient of agreement for nominal scale. Educational and Psychological Measurement, 20, 37–46.CrossRefGoogle Scholar
  24. Conley, D. T. (2007). Redefining college readiness. Eugene: Educational Policy Improvement Center.Google Scholar
  25. Conley, D. T. (2008). College knowledge: What it really takes for students to succeed and what we can do to get them ready. Hoboken: Wiley.Google Scholar
  26. Conley, D., Lombardi, A., Seburn, M., & McGaughy, C. (2009). Formative assessment for college readiness: Measuring skill and growth in five key cognitive strategies associated with postsecondary success. Paper presented at the annual conference of the American Educational Research Association, San Diego, CA.Google Scholar
  27. Corbett, A. T., & Anderson, J. R. (1995). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4), 253–278.CrossRefGoogle Scholar
  28. Craig, S., Graesser, A., Sullins, J., & Gholson, B. (2004). Affect and learning: An exploratory look into the role of affect in learning with AutoTutor. Journal of Educational Media, 29(3), 241–250.CrossRefGoogle Scholar
  29. Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New York: Harper-Row.Google Scholar
  30. D’Mello, S. K., Craig, S. D., Witherspoon, A., Mcdaniel, B., & Graesser, A. (2008). Automatic detection of learner’s affect from conversational cues. User Modeling and User-Adapted Interaction, 18(1–2), 45–80.CrossRefGoogle Scholar
  31. D’Mello, S. K., Lehman, B., Pekrun, R., & Graesser, A. C. (2014). Confusion can be beneficial for learning. Learning & Instruction, 29(1), 153–170.CrossRefGoogle Scholar
  32. DeFalco, J. A., Baker, R. S., Paquette, L., Georgoulas, V., Rowe, J., Mott, B., et al. (2015). Motivational feedback designs for frustration in a simulation-based combat medic training environment. In Generalized Intelligent Framework for Tutoring (GIFT) users symposium (p. 81).Google Scholar
  33. Drummond, J., & Litman, D. (2010). In the zone: Towards detecting student zoning out using supervised machine learning. In Intelligent tutoring systems (pp. 306–308).Google Scholar
  34. Eccles, J. S., & Jacobs, J. E. (1986). Social forces shape math attitudes and performance. Signs, 11(2), 367–380.CrossRefGoogle Scholar
  35. Eccles, J. S., Vida, M. N., & Barber, B. (2004). The relation of early adolescents’ college plans and both academic ability and task-value beliefs to subsequent college enrollment. The Journal of Early Adolescence, 24(1), 63–77.CrossRefGoogle Scholar
  36. Fancsali, S. (2014, July). Causal discovery with models: Behavior, affect, and learning in cognitive tutor algebra. In Educational data mining 2014.Google Scholar
  37. Farrington, C. A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T. S., Johnson, D. W., et al. (2012). Teaching adolescents to become learners: The role of noncognitive factors in shaping school performance—A critical literature review. Consortium on Chicago School Research.Google Scholar
  38. Feng, S., D’Mello, S., & Graesser, A. C. (2013). Mind wandering while reading easy and difficult texts. Psychonomic Bulletin & Review, 20(3), 586–592.CrossRefGoogle Scholar
  39. Feng, M., Heffernan, N., & Koedinger, K. (2009). Addressing the assessment challenge with an online system that tutors as it assesses. User Modeling and User-Adapted Interaction, 19(3), 243–266.CrossRefGoogle Scholar
  40. Finkelstein, N., Fong, A., Tiffany-Morales, J., Shields, P., & Huang, M. (2012). College bound in middle school & high school? How Math course sequences matter. Center for the Future of Teaching and Learning at WestEd. Google Scholar
  41. Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of educational research, 74(1), 59–109.CrossRefGoogle Scholar
  42. Griffith, A. L., & Rothstein, D. S. (2009). Can’t get there from here: The decision to apply to a selective college. Economics of Education Review, 28(5), 620–628.CrossRefGoogle Scholar
  43. Hanley, J., & McNeil, B. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143, 29–36.CrossRefGoogle Scholar
  44. Hawn, A. (2015, March). The bridge report: Bringing learning analytics to low-income, urban schools. In Proceedings of the fifth international conference on learning analytics and knowledge (pp. 410–411). ACM.Google Scholar
  45. Hershkovitz, A., Baker, R. S., Gowda, S. M., & Corbett, A. T. (2013). Predicting future learning better using quantitative analysis of moment-by-moment learning. In EDM (Vol. 13, pp. 74–81).Google Scholar
  46. Hosmer, D. W., Jr., Lemeshow, S., & Sturdivant, R. X. (2013). Applied logistic regression (Vol. 398). Hoboken: Wiley.CrossRefGoogle Scholar
  47. Hossler, D., Braxton, J., & Coopersmith, G. (1989). Understanding student college choice. Higher education: Handbook of theory and research, 5, 231–288.Google Scholar
  48. Karweit, N., & Slavin, R. E. (1982). Time-on-task—Issues of timing, sampling, and definition. Journal of Educational Psychology, 74(6), 844–851.CrossRefGoogle Scholar
  49. Kellam, S. G., Ling, X., Merisca, R., Brown, C. H., & Ialongo, N. (1998). The effect of the level of aggression in the first grade classroom on the course and malleability of aggressive behavior into middle school. Development and Psychopathology, 10, 165–185.CrossRefGoogle Scholar
  50. Khajah, M., Lindsey, R. V., & Mozer, M. C. (2016). How deep is knowledge tracing? In T. Barnes, M. Chi, & M. Feng (Eds.), Proceedings of the ninth international conference on educational data mining (pp. 94–101). Educational Data Mining Society Press.Google Scholar
  51. Koedinger, K. R., & Corbett, A. T. (2006). Cognitive tutors: Technology bringing learning science to the classroom. In K. Sawyer (Ed.), The Cambridge handbook of the learning sciences. Cambridge: Cambridge University Press.Google Scholar
  52. Kort, B., Reilly, R., Picard, R. (2001). An affective model of interplay between emotions and learning: reengineering educational pedagogy—Building a learning companion. In Proceedings IEEE ICALT 2001 (pp. 43–48).Google Scholar
  53. Lehman, B., D’Mello, S., & Graesser, A. (2012). Interventions to regulate confusion during learning. In Intelligent tutoring systems (pp. 576–578). Berlin/Heidelberg: Springer.Google Scholar
  54. Lent, R. W., Brown, S. D., & Hackett, G. (1994). Toward a unifying social cognitive theory of career and academic interest, choice and performance. Journal of Vocational Behavior, 45(1), 79–122.CrossRefGoogle Scholar
  55. Lent, R. W., Brown, S. D., & Hackett, G. (2000). Contextual supports and barriers to career choice: A social cognitive analysis. Journal of Counseling Psychology, 47(1), 36–49.CrossRefGoogle Scholar
  56. Madigan, T. (1997). Science proficiency and course taking in high school: The relationship of science course-taking patterns to increases in science proficiency between 8th and 12th grades. Washington, DC: National Center for Education Statistics (ED).Google Scholar
  57. McQuiggan, S. W., Mott, B. W., & Lester, J. C. (2008). Modeling self-efficacy in intelligent tutoring systems: An inductive approach. User Modeling and User-Adapted Interaction, 18(1–2), 81–123.CrossRefGoogle Scholar
  58. Milliron, M. D., Malcolm, L., & Kil, D. (2014). Insight and action analytics: Three case studies to consider. Research & Practice in Assessment, 9, 70–89.Google Scholar
  59. Núñez, A. M., & Bowers, A. J. (2011). Exploring what leads high school students to enroll in hispanic-serving institutions a multilevel analysis. American Educational Research Journal, 48(6), 1286–1313.CrossRefGoogle Scholar
  60. Ocumpaugh, J., Baker, R., Gowda, S., Heffernan, N., & Heffernan, C. (2014). Population validity for educational data mining models: A case study in affect detection. British Journal of Educational Technology, 45(3), 487–501.CrossRefGoogle Scholar
  61. Ocumpaugh, J., Baker, R. S., Rodrigo, M. M. T., Salvi, A. van Velsen, M., Aghababyan, A., et al. (2015). HART: The human affect recording tool. In Proceedings of SIGDOC 2015.Google Scholar
  62. Pardos, Z. A., Baker, R. S., San Pedro, M. O., Gowda, S. M., & Gowda, S. M. (2013). Affective states and state tests: Investigating how affect throughout the school year predicts end of year learning outcomes. In Proceedings of LAK 2013 (pp. 117–124).Google Scholar
  63. Pekrun, R., Goetz, T., Daniels, L. M., Stupnisky, R. H., & Perry, R. P. (2010). Boredom in achievement settings: Exploring control-value antecedents and performance outcomes of a neglected emotion. Journal of Educational Psychology, 102(3), 531.CrossRefGoogle Scholar
  64. Pekrun, R., & Linnenbrink-Garcia, L. (2012). Academic emotions and student engagement. In Handbook of research on student engagement (pp. 259–282).Google Scholar
  65. Razzaq, L., Feng, M., Nuzzo-Jones, G., & Rasmussen, K. P. (2005). The Assistment project: Blending assessment and assisting. In Proceedings of AIED 2015 (pp. 555–562).Google Scholar
  66. Reinke, W. M., & Herman, K. C. (2002). Creating school environments that deter antisocial behaviours in youth. Psychology in the Schools, 39, 549–559.CrossRefGoogle Scholar
  67. Rock, D. A., Owings, J. A., & Lee, R. (1994). Changes in math proficiency between 8th and 10th grades. US Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics.Google Scholar
  68. Roderick, M., Coca, V., & Nagaoka, J. (2011). Potholes on the road to college high school effects in shaping urban students’ participation in college application, four-year college enrollment, and college match. Sociology of Education, 84(3), 178–211.CrossRefGoogle Scholar
  69. Roderick, M., Nagaoka, J., & Coca, V. (2009). College readiness for all: The challenge for urban high schools. The Future of Children, 19(1), 185–210.CrossRefGoogle Scholar
  70. Rottinghaus, P. J., & Eshelman, A. J. (2015). Integrative approaches to career intervention. In APA handbook of career intervention, Vol 2: Applications (pp. 25–39).Google Scholar
  71. Rowe, J. P., McQuiggan, S. W., Robison, J. L., & Lester, J. C. (2009). Off-task behavior in narrative-centered learning environments. In AIED 2009 (pp. 99–106).Google Scholar
  72. Rozin, P., & Cohen, A. B. (2003). High frequency of facial expressions corresponding to confusion, concentration, and worry in an analysis of naturally occurring facial expressions of Americans. Emotion, 3(1), 68.CrossRefGoogle Scholar
  73. Sabourin, J., Rowe, J., Mott, B., & Lester, J. (2011). When off-task is on-task: The affective role of off-task behavior in narrative-centered learning environments. In Proceedings of artificial intelligence in education 2011 (pp. 534–536).Google Scholar
  74. San Pedro, M. O., Baker, R. S., Bowers, A. J., & Heffernan, N. T. (2013). Predicting college enrollment from student interaction with an intelligent tutoring system in middle school. In Proceedings of EDM 2013 (pp. 177–184).Google Scholar
  75. San Pedro, M. O., Baker, R. S., & Rodrigo, M. M. T. (2011). Detecting carelessness through contextual estimation of slip probabilities among students using an intelligent tutor for mathematics. In Proceedings of AIED 2011 (pp. 304–311).Google Scholar
  76. Tze, V. M., Daniels, L. M., & Klassen, R. M. (2016). Evaluating the relationship between boredom and academic outcomes: A meta-analysis. Educational Psychology Review, 28(1), 119–144.CrossRefGoogle Scholar
  77. Vaessen, B. E., Prins, F. J., & Jeuring, J. (2014). University students’ achievement goals and help-seeking strategies in an intelligent tutoring system. Computers & Education, 72, 196–208.CrossRefGoogle Scholar
  78. Wentzel, K. R. (1993). Does being good make the grade? Social behavior and academic competence in middle school. Journal of Educational Psychology, 85(2), 357–364.CrossRefGoogle Scholar
  79. Witten, I. H., & Frank, E. (2005). Data mining: Practical machine learning tools and techniques. San Francisco: Morgan Kaufmann.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2017

Authors and Affiliations

  1. 1.Research on Assessment and LearningACT, Inc.Iowa CityUSA
  2. 2.Teaching, Learning, and Leadership DivisionUniversity of PennsylvaniaPhiladelphiaUSA
  3. 3.Department of Computer ScienceWorcester Polytechnic InstituteWorcesterUSA

Personalised recommendations

Cite article

Buy options