Modeling Metacognitive Activities in Medical Problem-Solving with BioWorld

  • Susanne P. Lajoie
  • Eric G. Poitras
  • Tenzin Doleck
  • Amanda Jarrell
Part of the Intelligent Systems Reference Library book series (ISRL, volume 76)


Medical diagnostic reasoning is ill-defined and complex, requiring novice physicians to monitor and control their problem-solving efforts. Self-regulation is critical for effective medical problem-solving, helping individuals progress towards a correct diagnosis through a series of actions that informs subsequent ones. BioWorld is a computer-based learning environment designed to support novices in developing medical diagnostic reasoning as they receive feedback in the context of solving virtual cases. The system provides tools that scaffold learners in their requisite cognitive and metacognitive activities. Novices attain higher levels of competence as the system dynamically assesses their performance against expert solution paths. Dynamic assessment in this system relies on a novice-expert overlay and it is used to develop feedback when novices request help. When help-seeking occurs, help is provided by the tutoring module which applies a set of pre-defined rules based on the context of the learner’s activity. The system also provides cumulative feedback by comparing the novice solution with an expert solution following completion of the case. This chapter covers the essential design guidelines of this scaffolding approach to metacognitive activities in problem-solving within the domain of medical education. Specifically, we review recent advances in modeling metacognition through online measures, including concurrent think-aloud protocols, video-screen captures, and log-file entries. Educational data mining techniques are outlined with the goals of capturing metacognitive activities as they unfold throughout problem solving, and guiding the design of scaffolding tools in order to promote higher levels of competence in novices.


Tools Scaffolding approaches ITS Metacognition Problem-solving Bioworld Medical education Novice-expert overlay Help-seeking 



Artificial Neural Networks


Hidden Markov Models


Multinomial Naïve Bayes


Naïve Bayes


Sequential Minimal Optimization


Technology-Rich Learning Environment


  1. 1.
    Lesgold, A.M.: Problem solving. In: Sternberg, R.J., Smith, E.E. (eds.) The Psychology of Human Thought. Cambridge University Press, Cambridge (1988)Google Scholar
  2. 2.
    Lajoie, S.P.: Developing professional expertise with a cognitive apprenticeship model: Examples from avionics and medicine. In: Ericsson, K.A. (ed.) Development of Professional Expertise: Toward Measurement of Expert Performance and Design of Optimal Learning Environments, pp. 61–83. Cambridge University Press, Cambridge (2009)Google Scholar
  3. 3.
    Lajoie, S., Naismith, L., Poitras, E., Hong, Y., Panesso-Cruz, I., Ranelluci, J., Wiseman, J.: Technology rich tools to support self-regulated learning and performance in medicine. In: Azevedo, R., Aleven, V. (eds.) International Handbook of Metacognition and Learning Technologies. Springer, Amsterdam (2013)Google Scholar
  4. 4.
    Ericsson, K.A., Krampe, RTh, Tesch-Romer, C.: The role of deliberate practice in the acquisition of expert performance. Psychol. Rev. 100(3), 363–406 (1993)CrossRefGoogle Scholar
  5. 5.
    Lajoie, S.P.: Transitions and trajectories for studies of expertise. Educ. Researcher 32, 21–25 (2003)CrossRefGoogle Scholar
  6. 6.
    Alexander, P.A., Dinsmore, D.L., Parkinson, M.M., Winters, F.I.: Self-regulated learning in academic domains. In: Zimmerman, B., Schunk, D. (eds.) Handbook of Self-Regulation of Learning and Performance. Routledge, New York (2011)Google Scholar
  7. 7.
    White, C.B., Gruppen, L.D.: Self-regulated learning in medical education. In: Swanwick, T. (ed.) Understanding Medical Education. Wiley-Blackwell, Sussex (2010)Google Scholar
  8. 8.
    Evensen, D.H., Salisbury-Glennon, J.D., Glenn, J.: A qualitative study of six medical students in a problem-based curriculum: Toward a situated model of self-regulation. J. Educ. Psychol. 93, 76–659 (2001)CrossRefGoogle Scholar
  9. 9.
    Brydges, R., Butler, D.L.: A reflective analysis of medical education research on self-regulation in learning and practice. Med. Educ. 46, 71–79 (2012)CrossRefGoogle Scholar
  10. 10.
    Meijer, J., Veenman, M.V.J., Van Hout-Wolters, B.H.A.M.: Metacognitive activities in text-studying and problem-solving: Development of a taxonomy. Educ. Res. Eval. 12(3), 209–237 (2006)CrossRefGoogle Scholar
  11. 11.
    Lu, J., Lajoie, S.P.: Supporting medical decision making with argumentation tools. Contemp. Educ. Psychol. 33, 425–442 (2008)CrossRefGoogle Scholar
  12. 12.
    Lajoie, S.P., Lu, J.: Supporting collaboration with technology: Does shared cognition lead to co-regulation in medicine? Metacogn. Learn. 7, 45–62 (2012)CrossRefGoogle Scholar
  13. 13.
    Zimmerman, B.J.: Self-regulated learning and academic achievement: An overview. Educ. Psychol. 25(1), 3–17 (1990)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Zimmerman, B.J., Campillo, M.: Motivating self-regulated problem solvers. In: Davidson, J.E., Sternberg, R. (eds.) The Nature of Problem Solving pp. 233–262. Cambridge University Press, New York (2003)Google Scholar
  15. 15.
    Collins, A.: Cognitive apprenticeship. In: Sawyer, K. (ed.) Cambridge Handbook of the Learning Sciences pp. 47–60. Cambridge University Press, New York (2006)Google Scholar
  16. 16.
    Lajoie, S.P.: Aligning theories with technology innovations in education. Br. J. Educ. Psychol.—Monogr. Ser. II (5) Learning through Digital Technologies, 27–38 (2007)Google Scholar
  17. 17.
    Lajoie, S.P.: Cognitive tools for the mind: The promises of technology: Cognitive amplifiers or bionic prosthetics? In: Sternberg, R.J., Preiss, D. (eds.) Intelligence and Technology: Impact of Tools on the Nature and Development of Human Skills, pp. 87–102. Erlbaum, Mahwah (2005)Google Scholar
  18. 18.
    Lajoie, S.P., Azevedo, R.: Teaching and learning in technology-rich environments. In: Alexander, P.A., Winne, P.H. (2nd ed.) Handbook of Educational Psychology pp. 803–821. Lawrence Erlbaum Associates, Mahwah (2006)Google Scholar
  19. 19.
    Goldstein, I.P.: The genetic graph: a representation for the evolution of procedural knowledge. In: Sleeman, D., Brown, J.S. (eds.) Intelligent Tutoring Systems pp. 51–77. Academic Press, London (1982)Google Scholar
  20. 20.
    Shute, V.J., Zapata-Rivera, D.: Adaptive educational systems. In: Adaptive Technologies for Training and Education, pp. 7–27 (2012)Google Scholar
  21. 21.
    Naismith, L., Lajoie, S.P.: Using expert models to provide feedback on clinical reasoning skills. In: Aleven, V., Kay, J., Mostow, J. (eds.) 10th International Conference on Intelligent Tutoring Systems, LNCS, vol. 6095, pp. 44–242. Springer, Berlin (2010)Google Scholar
  22. 22.
    Stevens, R.: Machine Learning Assessment Systems for Modeling Patterns of Student Learning, pp. 349–365. Games and Simulation in Online, Learning (2007)Google Scholar
  23. 23.
    Stevens, R., Beal, C.R., Sprang, M.: Assessing students’ problem solving ability and cognitive regulation with learning trajectories. In: International Handbook of Metacognition and Learning Technologies pp. 409–423. Springer, New York (2013)Google Scholar
  24. 24.
    Lajoie, S.P., Faremo, S., Wiseman, J.: A knowledge-based approach to designing authoring tools: From tutor to author. In: Moore, J.D., Redfield, C., Johnson, L.W. (eds.) Artificial Intelligence in Education: AI-ED in the Wired and Wireless future pp. 77–86. IOS Press, Amsterdam (2001)Google Scholar
  25. 25.
    Järvelä, S.: How does help seeking help?–New prospects in a variety of contexts. Learn. Instr. 21(2), 297–299 (2011)CrossRefGoogle Scholar
  26. 26.
    Newman, R.S.: Adaptive help-seeking: a strategy of self-regulated learning. In: Schunk, D.H., Zimmerman, B.J. (eds.) Self-Regulation of Learning and Performance: Issues and Educational Applications pp. 283–301. Erlbaum, Hillsdale (1994)Google Scholar
  27. 27.
    Karabenick, S.A.: Strategic Help Seeking: Implications for Learning and Teaching. Erlbaum, Mahwah (1998)Google Scholar
  28. 28.
    Huet, N., Escribe, C., Dupeyrat, C., Sakdavong, J.-C.: The influence of achievement goals and perceptions of online help on its actual use in an interactive learning environment. Comput. Hum. Behav. 27, 413–420 (2011)CrossRefGoogle Scholar
  29. 29.
    Aleven, V., Stahl, E., Schworm, S., Fischer, F., Wallace, R.: Help-seeking and help design in interactive learning environments. Rev. Educ. Res. 73(3), 277–320 (2003)CrossRefGoogle Scholar
  30. 30.
    Gräsel, C., Fischer, F., Mandl, H.: The use of additional information in problem oriented learning environments. Learn. Environ. Res. 3, 287–305 (2000)CrossRefGoogle Scholar
  31. 31.
    Newman, R.S.: Children’s help-seeking in the classroom: the role of motivational factors and attitudes. J. Educ. Psychol. 82, 71–80 (1990)CrossRefGoogle Scholar
  32. 32.
    Newman, R.S.: The motivational role of adaptive help seeking in self-regulated learning. In: Motivation and Self-Regulated Learning: Theory, Research, and Applications, 315–337 (2008)Google Scholar
  33. 33.
    Vygotsky, L.S.: Mind in Society: The Development of Higher Psychological Processes. Harvard University Press, Cambridge (1978)Google Scholar
  34. 34.
    Aleven, V.: Help seeking and intelligent tutoring systems: theoretical perspectives and a step towards theoretical integration. In: International Handbook of Metacognition and Learning Technologies pp. 311–335. Springer, New York (2013)Google Scholar
  35. 35.
    Aleven, V., McLaren, B., Roll, I., Koedinger, K.: Toward meta-cognitive tutoring: a model of help seeking with a cognitive tutor. Int. J. Artif. Intell. Educ. 16, 101–128 (2006)Google Scholar
  36. 36.
    Aleven, V., Roll, I., McLaren, B.M., Koedinger, K.R.: Automated, unobtrusive, action-by-action assessment of self-regulation during learning with an intelligent tutoring system. Educ. Psychol. 45(4), 224–233 (2010)CrossRefGoogle Scholar
  37. 37.
    Kinnebrew, J.S., Mack, D.L.C., Biswas, G.: Mining temporally-interesting learning behavior patterns. In: 6th International Conference on Educational Data Mining, Memphis (2013)Google Scholar
  38. 38.
    Quinlan, J.R.: Improved use of continuous attributes in c4.5. J. Artif. Intell. Res. 4(1), 77–90 (1996)zbMATHGoogle Scholar
  39. 39.
    Mierswa, I., Wurst, M., Klinkenberg, R., Scholz, M., Euler, T.: YALE: rapid prototyping for complex data mining tasks. In: Ungar, L., Craven, M., Gunopulos, D., Eliassi-Rad, T. (eds.) 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining KDD-06. ACM, New York (2006)Google Scholar
  40. 40.
    Kellogg, R.T.: Professional writing expertise. In: Ericsson, K.A., Charness, N., Feltovich, P.J., Hoffman, R.R. (eds.) The Cambridge Handbook of Expertise and Expert Performance. Cambridge University Press, New York (2006)Google Scholar
  41. 41.
    Sebastiani, F.: Machine learning in automated text categorization. ACM Comput. Surv. 34(1), 1–47 (2002)CrossRefGoogle Scholar
  42. 42.
    Aggarwal, C.C., Zhai, C.: A survey of text classification algorithms. In: Aggarwal, C.C., Zhai, C. (eds.) Mining Text Data pp. 163–222 Springer (2012)Google Scholar
  43. 43.
    McNamara, D.S.: IIS: A marriage of computational linguistics, psychology, and educational technologies. In: Wilson D., Sutcliffe G. (eds.) 20th International Florida Artificial Intelligence Research Society Conference pp. 15–20. The AAAI Press, Menlo Park (2007)Google Scholar
  44. 44.
    McNamara, D.S., Crossley, S.A., McCarthy, P.M.: Linguistic features of writing quality. Written Communic. 27(1), 57–86 (2010)CrossRefGoogle Scholar
  45. 45.
    Kibriya, A.M., Frank, E., Pfahringer, B., Holmes, G.: Multinomial naïve Bayes for text categorization revisited. In: Webb, G.I., Yu, X. (eds.) Advances in Artificial Intelligence pp. 488–499. Springer, Heidelberg (2004)Google Scholar
  46. 46.
    Platt, J.C.: A fast algorithm for training support vector machines. Technical Report MSR-TR-98-14 (1998)Google Scholar
  47. 47.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: an update. SIGKDD Explor. 11(1), 10–18 (2009)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Susanne P. Lajoie
    • 1
  • Eric G. Poitras
    • 2
  • Tenzin Doleck
    • 1
  • Amanda Jarrell
    • 1
  1. 1.Department of Educational and Counselling PsychologyMcGill UniversityMontrealCanada
  2. 2.Advanced Instructional Systems and Technologies LaboratoryUniversity of Utah Educational PsychologySalt Lake CityUSA

Personalised recommendations