A Bayesian Network Approach for Modeling the Influence of Contextual Variables on Scientific Problem Solving
A challenge for intelligent tutoring is to develop methodologies for transforming streams of performance data into insights and models about underlying learning mechanisms. Such modeling at different points in time could provide evidence of a student’s changing understanding of a task, and given sufficient detail, could extend our understanding of how gender, prior achievement, classroom practices and other student/contextual characteristics differentially influence performance and participation in complex problem-solving environments. If the models had predictive properties, they could also provide a framework for directing feedback to improve learning.
In this paper we describe the causal relationships between students’ problem-solving effectiveness (i.e. reaching a correct solution) and strategy (i.e. approach) and multiple contextual variables including experience, gender, classroom environment, and task difficulty. Performances of the IMMEX problem set Hazmat (n ~ 33,000) were first modeled by Item Response Theory analysis to provide a measure of effectiveness and then by self-organizing artificial neural networks and hidden Markov modeling to provide measures of strategic efficiency. Correlation findings were then used to link the variables into a Bayesian network representation. Sensitivity analysis indicated that whether a problem was solved or not was most likely influenced by findings related to the problem under investigation and the classroom environment while strategic approaches were most influenced by the actions taken, the classroom environment and the number of problems previously performed. Subsequent testing with unknown performances indicated that the strategic approaches were most easily predicted (17% error rate), whereas whether the problem was solved was more difficult (32% error rate).
KeywordsBayesian Network Item Response Theory Classroom Environment Hide Markov Model State Entropy Reduction
Unable to display preview. Download preview PDF.
- 1.Arroyo, I., Woolf, B.: Inferring learning and attitudes from a Bayesian Network of log file data. In: Looi, C.-K., et al. (eds.) Artificial Intelligence in Education. IOS Press, Amsterdam (2005)Google Scholar
- 3.Brown, A., Palincsar, A.: Guided, Cooperative Learning and Individual Knowledge Acquisition. In: Resnick, L. (ed.) Knowing, learning, and instruction: Essays in honor of Robert Glaser. Lawrence Earlbaum Associates, NJ (1989)Google Scholar
- 4.Case, E.L.: The effects of collaborative grouping on problem solving in first-year chemistry. Ph.D. thesis, Clemson University (2004)Google Scholar
- 7.Fennema, E., Carpenter, T., Jacobs, V., Franke, M., Levi, L.: Gender differences in mathematical thinking. Educational Researcher 27, 6–11 (1998)Google Scholar
- 9.Gerosa, L., Giordani, A., Ronchetti, M., Soller, A., Stevens, R.: Symmetric Synchronous Collaborative Navigation. In: A cura di Isaias, P., Karmakar, N. (eds.) IADIS International Conference WWW/Internet 2004, vol. 1, pp. 748–754. IADIS Press, Lisboa (2004)Google Scholar
- 14.Lepper, M.R., Woolverton, M., Mumme, D., Gurtner, J.: Motivational Techniques of Expert Human Tutors: Lessons for the Design of Computer-Based Tutors. In: Lajoie, S.P., Derry, S.J. (eds.) Computers as cognitive tools, pp. 75–105. Erlbaum, Hillsdale (1993)Google Scholar
- 15.Linacre, J.M.: WINSTEPS Rasch measurement computer program. Chicago. Winsteps.com (2004)Google Scholar
- 21.Murphy, K.: Hidden Markov Model (HMM) Toolbox for Matlab (2004), Available online at: http://www.cs.ubc.ca/~murphyk/Software/HMM/hmm.html
- 22.O’Regan, K.: Emotion and e-Learning. Journal of Asynchronous Learning Network 7(3), 78–92 (2003)Google Scholar
- 23.Olson, A., Loucks-Horsley, S. (eds.): Inquiry and the National Science Education Standards: A guide for teaching and learning. National Academy Press, Washington (2000)Google Scholar
- 26.Soller, A., Lesgold, A.: A Computational Approach to Analyzing Online Knowledge Sharing Interaction. In: Proceedings of Artificial Intelligence in Education, Sydney Australia, pp. 253–260 (2003)Google Scholar
- 27.Soller, A., Wiebe, J., Lesgold, A.: A Machine Learning Approach to Assessing Knowledge Sharing During Collaborative Learning Activities. In: Proceedings of Computer Support for Collaborative Learning, Boulder, CO, pp. 128–137 (2002)Google Scholar
- 30.Stevens, R.H., Wang, P., Lopo, A.: Artificial Neural Networks Can Distinguish Novice and Expert Strategies During Complex Problem Solving. JAMIA 3(2), 131–138 (1996)Google Scholar
- 35.Stevens, R.H., Soller.: Implementing a Layered Analytic Approach for Real-Time Modeling of Student’s Scientific Understanding. In: Proceedings of the Artificial Intelligence in Education Conference (2005)Google Scholar
- 37.Topping, K.J., Ehly, S. (eds.): Peer-assisted learning. Lawrence Erlbaum, Mahwah (1998)Google Scholar
- 38.Underdahl, J., Palacio-Cayetano, J., Stevens, R.: Practice makes perfect: assessing and enhancing knowledge and problem-solving skills with IMMEX software. Learning and Leading with Technology 28, 26–31 (2001)Google Scholar
- 40.VanLehn, K., Ohlsson, S., Nason, R.: Applications of Simulated Students: An Exploration. Journal of Artificial Intelligence in Education 5(2), 135–175 (1994)Google Scholar
- 41.Vendlinski, T., Stevens, R.: A Markov Model Analysis of Problem-Solving Progress and Transfer. Journal of Technology, Learning, and Assessment 1(3) (2002)Google Scholar
- 42.Webb, N.: Testing a Theoretical Model of Student Interaction and Learning in Small Groups. In: Hertz-Lazarowitz, R., Miller, N. (eds.) Interaction in Cooperative Groups: The Theoretical Anatomy of Group Learning, pp. 102–119. Cambridge University Press, New York (1992)Google Scholar