A Bayesian Network Approach for Modeling the Influence of Contextual Variables on Scientific Problem Solving

  • Ronald H. Stevens
  • Vandana Thadani
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4053)


A challenge for intelligent tutoring is to develop methodologies for transforming streams of performance data into insights and models about underlying learning mechanisms. Such modeling at different points in time could provide evidence of a student’s changing understanding of a task, and given sufficient detail, could extend our understanding of how gender, prior achievement, classroom practices and other student/contextual characteristics differentially influence performance and participation in complex problem-solving environments. If the models had predictive properties, they could also provide a framework for directing feedback to improve learning.

In this paper we describe the causal relationships between students’ problem-solving effectiveness (i.e. reaching a correct solution) and strategy (i.e. approach) and multiple contextual variables including experience, gender, classroom environment, and task difficulty. Performances of the IMMEX problem set Hazmat (n ~ 33,000) were first modeled by Item Response Theory analysis to provide a measure of effectiveness and then by self-organizing artificial neural networks and hidden Markov modeling to provide measures of strategic efficiency. Correlation findings were then used to link the variables into a Bayesian network representation. Sensitivity analysis indicated that whether a problem was solved or not was most likely influenced by findings related to the problem under investigation and the classroom environment while strategic approaches were most influenced by the actions taken, the classroom environment and the number of problems previously performed. Subsequent testing with unknown performances indicated that the strategic approaches were most easily predicted (17% error rate), whereas whether the problem was solved was more difficult (32% error rate).


Bayesian Network Item Response Theory Classroom Environment Hide Markov Model State Entropy Reduction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arroyo, I., Woolf, B.: Inferring learning and attitudes from a Bayesian Network of log file data. In: Looi, C.-K., et al. (eds.) Artificial Intelligence in Education. IOS Press, Amsterdam (2005)Google Scholar
  2. 2.
    Alexander, P.A.: The Development of Expertise: The Journey from Acclimation to Proficiency. Educational Researcher 32(8), 10–14 (2003)CrossRefGoogle Scholar
  3. 3.
    Brown, A., Palincsar, A.: Guided, Cooperative Learning and Individual Knowledge Acquisition. In: Resnick, L. (ed.) Knowing, learning, and instruction: Essays in honor of Robert Glaser. Lawrence Earlbaum Associates, NJ (1989)Google Scholar
  4. 4.
    Case, E.L.: The effects of collaborative grouping on problem solving in first-year chemistry. Ph.D. thesis, Clemson University (2004)Google Scholar
  5. 5.
    Chung, G.K.W.K., de Vries, L.F., Cheak, A.M., Stevens, R.H., Bewley, W.L.: Cognitive Process Validation of an Online Problem Solving Assessment. Computers and Human Behavior 18, 669 (2002)CrossRefGoogle Scholar
  6. 6.
    Ericsson, K.A.: Deliberate Practice and the Acquisition and Maintenance of Expert Performance in Medicine and Related Domains. Academic Medicine 79(10), S70–S81 (2004)CrossRefGoogle Scholar
  7. 7.
    Fennema, E., Carpenter, T., Jacobs, V., Franke, M., Levi, L.: Gender differences in mathematical thinking. Educational Researcher 27, 6–11 (1998)Google Scholar
  8. 8.
    Gallagher, A.M., De Lisi, R.: Gender differences in scholastic aptitude test mathematics problem solving among high ability students. Journal of Educational Psychology 86(2), 204–214 (1994)CrossRefGoogle Scholar
  9. 9.
    Gerosa, L., Giordani, A., Ronchetti, M., Soller, A., Stevens, R.: Symmetric Synchronous Collaborative Navigation. In: A cura di Isaias, P., Karmakar, N. (eds.) IADIS International Conference WWW/Internet 2004, vol. 1, pp. 748–754. IADIS Press, Lisboa (2004)Google Scholar
  10. 10.
    Haider, H., Frensch, P.A.: The role of information reduction in skill acquisition. Cognitive Psychology 30, 304–337 (1996)CrossRefGoogle Scholar
  11. 11.
    Hiebert, J., Stigler, J.: A Proposal for Improving Classroom Teaching: Lessons from the TIMSS Video Study. The Elementary School Journal 101, 3–21 (2000)CrossRefGoogle Scholar
  12. 12.
    Kohonen, T.: Self Organizing Maps, 3rd extended edit. Springer, Heidelberg (2001)MATHGoogle Scholar
  13. 13.
    Lajoie, S.P.: Transitions and Trajectories for Studies of Expertise. Educational Researcher 32, 21–25 (2003)CrossRefGoogle Scholar
  14. 14.
    Lepper, M.R., Woolverton, M., Mumme, D., Gurtner, J.: Motivational Techniques of Expert Human Tutors: Lessons for the Design of Computer-Based Tutors. In: Lajoie, S.P., Derry, S.J. (eds.) Computers as cognitive tools, pp. 75–105. Erlbaum, Hillsdale (1993)Google Scholar
  15. 15.
    Linacre, J.M.: WINSTEPS Rasch measurement computer program. Chicago. (2004)Google Scholar
  16. 16.
    Marshall, S.: Schemas in Problem Solving. Cambridge University Press, Cambridge (1995)CrossRefGoogle Scholar
  17. 17.
    Mayer, R.E.: Should There Be A Three-Strikes Rule Against Pure Discovery Learning? American Psychologist 59, 14–19 (2004)CrossRefGoogle Scholar
  18. 18.
    Mayer, R.E.: Cognitive, Metacognitive and Motivational Aspects of Problem Solving. Instructional Science 26, 49–63 (1998)CrossRefGoogle Scholar
  19. 19.
    Mislevy, R.J., Steinberg, L.S., Breyer, F.J., Almond, R.G., Johnson, L.: A cognitive task analysis, with implications for designing a simulation-based assessment system. Computer and Human Behavior 15, 335–374 (1999)CrossRefGoogle Scholar
  20. 20.
    Moreno, R., Duran, R.: Do Multiple Representations Need Explanations? The Role of Verbal Guidance and Individual Differences in Multimedia Mathematics Learning. Journal of Educational Psychology 96, 492–503 (2004)CrossRefGoogle Scholar
  21. 21.
    Murphy, K.: Hidden Markov Model (HMM) Toolbox for Matlab (2004), Available online at:
  22. 22.
    O’Regan, K.: Emotion and e-Learning. Journal of Asynchronous Learning Network 7(3), 78–92 (2003)Google Scholar
  23. 23.
    Olson, A., Loucks-Horsley, S. (eds.): Inquiry and the National Science Education Standards: A guide for teaching and learning. National Academy Press, Washington (2000)Google Scholar
  24. 24.
    Rabiner, L.: A tutorial on Hidden Markov Models and selected applications in speech recognition. Proceedings of IEEE 77, 257–286 (1989)CrossRefGoogle Scholar
  25. 25.
    Schunn, C.D., Anderson, J.R.: The Generality/Specificity of Expertise in Scientific Reasoning. Cognitive Science 13, 337–370 (1999)CrossRefGoogle Scholar
  26. 26.
    Soller, A., Lesgold, A.: A Computational Approach to Analyzing Online Knowledge Sharing Interaction. In: Proceedings of Artificial Intelligence in Education, Sydney Australia, pp. 253–260 (2003)Google Scholar
  27. 27.
    Soller, A., Wiebe, J., Lesgold, A.: A Machine Learning Approach to Assessing Knowledge Sharing During Collaborative Learning Activities. In: Proceedings of Computer Support for Collaborative Learning, Boulder, CO, pp. 128–137 (2002)Google Scholar
  28. 28.
    Stevens, R.H., Palacio-Cayetano, J.: Design and Performance Frameworks for Constructing Problem-solving Simulations. Cell Biology Education 2, 162–179 (2003)CrossRefGoogle Scholar
  29. 29.
    Stevens, R.H., Ikeda, J., Casillas, A., Palacio-Cayetano, J., Clyman, S.: Artificial Neural Network-based Performance Assessments. Computers in Human Behavior 15, 295–314 (1999)CrossRefGoogle Scholar
  30. 30.
    Stevens, R.H., Wang, P., Lopo, A.: Artificial Neural Networks Can Distinguish Novice and Expert Strategies During Complex Problem Solving. JAMIA 3(2), 131–138 (1996)Google Scholar
  31. 31.
    Stevens, R., Johnson, D.F., Soller, A.: Probabilities and Predictions: Modeling the Development of Scientific Competence. Cell Biology Education, The American Society for Cell Biology 4, 42–57 (2005)CrossRefGoogle Scholar
  32. 32.
    Stevens, R., Soller, A., Cooper, M., Sprang, M.: Modeling the Development of Problem Solving Skills in Chemistry with a Web-Based Tutor. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 580–591. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  33. 33.
    Stevens, R.H., Najafi, K.: Artificial Neural Networks as Adjuncts for Assessing Medical Students’ Problem-solving Performances on Computer-based Simulations. Computers and Biomedical Research 26, 172–187 (1993)CrossRefGoogle Scholar
  34. 34.
    Stevens, R.H., Ikeda, J., Casillas, A., Palacio-Cayetano, J., Clyman, S.: Artificial neural network-based performance assessments. Computers in Human Behavior 15, 295–314 (1999)CrossRefGoogle Scholar
  35. 35.
    Stevens, R.H., Soller.: Implementing a Layered Analytic Approach for Real-Time Modeling of Student’s Scientific Understanding. In: Proceedings of the Artificial Intelligence in Education Conference (2005)Google Scholar
  36. 36.
    Sugrue, B.: A theory-based framework for assessing domain-specific problem solving ability. Educational Measurement: Issues and Practice 14(3), 29–36 (1995)CrossRefGoogle Scholar
  37. 37.
    Topping, K.J., Ehly, S. (eds.): Peer-assisted learning. Lawrence Erlbaum, Mahwah (1998)Google Scholar
  38. 38.
    Underdahl, J., Palacio-Cayetano, J., Stevens, R.: Practice makes perfect: assessing and enhancing knowledge and problem-solving skills with IMMEX software. Learning and Leading with Technology 28, 26–31 (2001)Google Scholar
  39. 39.
    VanLehn, K.: Cognitive Skill Acquisition. Annual Review. Psychology. 47, 513–539 (1996)CrossRefGoogle Scholar
  40. 40.
    VanLehn, K., Ohlsson, S., Nason, R.: Applications of Simulated Students: An Exploration. Journal of Artificial Intelligence in Education 5(2), 135–175 (1994)Google Scholar
  41. 41.
    Vendlinski, T., Stevens, R.: A Markov Model Analysis of Problem-Solving Progress and Transfer. Journal of Technology, Learning, and Assessment 1(3) (2002)Google Scholar
  42. 42.
    Webb, N.: Testing a Theoretical Model of Student Interaction and Learning in Small Groups. In: Hertz-Lazarowitz, R., Miller, N. (eds.) Interaction in Cooperative Groups: The Theoretical Anatomy of Group Learning, pp. 102–119. Cambridge University Press, New York (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Ronald H. Stevens
    • 1
  • Vandana Thadani
    • 2
  1. 1.UCLA IMMEX ProjectCulver CityUSA
  2. 2.Dept of PsychologyLoyola Marymount UniversityLos AngelesUSA

Personalised recommendations