Advertisement

Diagnosing and Evaluating the Acquisition Process of Problem Solving Schemata in the Domain of Functional Programming

Conference paper
Part of the NATO ASI Series book series (volume 125)

Abstract

This paper describes an approach to model students’ knowledge growth from novice to expert within the framework of a help system, ABSYNT, in the domain of functional programming. The help system has expert knowledge about a large solution space. On the other hand, in order to provide learner-centered help there is a model of the student’s actual state of domain knowledge. The model is continuously updated based on the learner’s actions. It distinguishes between newly acquired and improved knowledge. Newly acquired knowledge is represented by augmenting the model with rules from the expert knowledge base. Although they are expert rules, only rules able to explain the student’s action sequences are incorporated in the model. Knowledge improvement is represented by rule composition. This allows the prediction of various knowledge acquisition phenomena, like performance speedup and a decrease of verbalizations.

In this way, the knowledge contained in the model is partially ordered from general rules to more specific schemas for solution fragments to specific cases (= example solutions). The model construction is implemented but not yet actually used for help generation within the help system. This paper focuses on knowledge diagnosis as accomplished by the model, and on an empirical analysis of some of its predictions.

Keywords

knowledge acquisition knowledge optimization schema identification empirical validation of student models analysis of time-based and correction-based data 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Anderson, J. R.: The Architecture of Cognition. Cambridge, MA: Harvard University Press 1983Google Scholar
  2. 2.
    Anderson, J. R.: Knowledge compilation: the general learning mechanism. In: Machine Learning (R. S. Michalski, J. G. Carbonell, & T. M. Mitchell, eds.). Vol. II, pp. 289–310. Los Altos, CA: Morgan Kaufmann 1986Google Scholar
  3. 3.
    Anderson, J. R.: Causal analysis and inductive learning. Proceedings of the 4th Workshop on Machine Learning, Irvine, CA, pp. 288–299 1987Google Scholar
  4. 4.
    Anderson, J. R.: A theory of the origins of human knowledge. Artificial Intelligence, 40, pp. 313–351 (1989)CrossRefGoogle Scholar
  5. 5.
    Anderson, J. R., Boyle, C. F., Farrell, R., & Reiser, B. J.: Cognitive principles in the design of computer tutors. In: Modelling Cognition (P. Morris, ed.), pp. 93–133. New York, NY: Wiley 1987Google Scholar
  6. 6.
    Anderson, J. R., Conrad, F. G., & Corbett, A. T.: Skill acquisition and the LISP tutor. Cognitive Science, 13, pp. 467–505 (1989)CrossRefGoogle Scholar
  7. 7.
    Anderson, J. R. & Swarecki, E.: The automated tutoring of introductory computer programming. Communications of the ACM, 29, pp. 842–849 (1986)CrossRefGoogle Scholar
  8. 8.
    Bauer, F. L. & Goos, G.: Informatik, Vol. 1. 3rd ed. Berlin: Springer-Verlag 1982Google Scholar
  9. 9.
    Brown, J. S. & Burton, R. R.: Diagnosing bugs in a simple procedural skill. In: Intelligent Tutoring Systems (D. Sleeman & J. S. Brown, eds.). pp. 157–183. New York, NY: Academic Press 1982Google Scholar
  10. 10.
    Brown, J. S. & VanLehn, K.: Repair theory: a generative theory of bugs in procedural skills. Cognitive Science, 4, pp. 379–426 (1980)CrossRefGoogle Scholar
  11. 11.
    Carbonell, J. G. & Gil, Y.: Learning by experimentation. Proceedings of the 4th Workshop on Machine Learning, Irvine, CA, pp. 256–266, 1987Google Scholar
  12. 12.
    Chang, S. K.(ed): Principles of Visual Programming Systems. Englewood Cliffs: Prentice Hall 1990Google Scholar
  13. 13.
    Chase, N. G. & Simon, H. A.: Perception in chess. Cognitive Psychology, 4, pp. 55–81 (1973)CrossRefGoogle Scholar
  14. 14.
    Corbett, A. T. & Anderson, J. R.: Student modeling and mastery learning in a computer-based programming tutor. In: Intelligent Tutoring Systems (C. Frasson, G. Gauthier, & G. I. McCalla, eds.). Proceedings of the ITS 92, pp. 413–420, Lecture Notes in Computer Science, Vol. 608, Berlin: Springer-Verlag 1992Google Scholar
  15. 15.
    Egan, D. E. & Greeno, J. G.: Theory of rule induction: knowledge acquired in concept learning, serial pattern learning, and problem solving. In: Knowledge and Cognition (L. W. Gregg, ed.). pp. 43–103. Potomac: Lawrence Erlbaum Associates 1974Google Scholar
  16. 16.
    Elio, R. & Scharf, P. B.: Modeling novice-to-expert shifts in problem solving strategy and knowledge organization. Cognitive Science, 14, 579–639 (1990)CrossRefGoogle Scholar
  17. 17.
    Ericsson, K. A. & Simon, H. A.: Protocol Analysis. Cambridge, MA: MIT Press 1984Google Scholar
  18. 18.
    Frasson, C. & Gauthier, G. (eds): Intelligent Tutoring Systems. Norwood, NJ: Ablex 1990Google Scholar
  19. 19.
    Gollwitzer, P. M.: Action phases and mind sets. In: Handbook of Motivation and Cognition: Foundations of Social Behavior (E. T. Higgins & R. M. Sorrentino, eds.). Vol. 2, pp. 53–92. 1990Google Scholar
  20. 20.
    Goliwitzer, P. M.: Abwägen und planen. Göttingen: Verlag für Psychologie 1991Google Scholar
  21. 21.
    Greer, J., McCalla, G. I., & Mark, M. A.: Incorporating granularity-based recognition into SCENT. In: Proceedings of the 4th International Conference on Artificial Intelligence and Education, Amsterdam (D. Bierman, J. Breuker & J. Sandberg, eds.), pp. 107–115, Amsterdam: IOS 1989Google Scholar
  22. 22.
    Gregory, S.: Parallel logic Programming in PARLOG: The language and its Implementation. Wokingham: Addison-Wesley 1987Google Scholar
  23. 23.
    Gugerty, L. & Olson, G. M.: Comprehension differences in debugging by skilled and novice programmers. In: Empirical Studies of Programmers (E. Soloway & S. Iyengar, eds.). pp. 13–27. Norwood, NJ: Ablex 1986Google Scholar
  24. 24.
    Hogger, Ch. J.: Essentials of Logic Programming. Oxford University Press 1990Google Scholar
  25. 25.
    Huber, P., Jensen, K., & Shapiro, R. M.: Hierarchies in colored petri nets. In: Advances in Petri Nets 1990 (G. Rozenberg, ed.), Lecture Notes in Computer Science, Vol. 483, pp. 373–341, Berlin: Springer-Verlag 1991Google Scholar
  26. 26.
    Kearsley, G.: Online Help Systems. Norwood, NJ: Ablex 1988Google Scholar
  27. 27.
    Kowalski, R’ Logic for Problem Solving. Amsterdam: Elsevier 1979Google Scholar
  28. 28.
    Laird, J. E., Rosenbloom, P. S., & Newell, A.: Universal Subgoaling and Chunking: The Automatic Generation and Learning of Goal Hierarchies. Boston, MA: Kluwer 1986CrossRefGoogle Scholar
  29. 29.
    Laird, J. E., Rosenbloom, P. S., & Newell, A.: SOAR: an architecture for general intelligence, Artificial Intelligence, 33, pp. 1–64 (1987)CrossRefGoogle Scholar
  30. 30.
    Lanzetta, J. T. & Driscoll, J. M.: Effects of uncertainty and importance on information search in decision making. Journal of Personality and Social Psychology, 10, pp. 479–486 (1968)CrossRefGoogle Scholar
  31. 31.
    Lehnen, W. G.: The Process of Question Answering. Hillsdale, NJ: Lawrence Erlbaum Associates 1978Google Scholar
  32. 32.
    Lewis, C.: Composition of productions. In: Production System Models of Learning and Development (D. Klahr, P. Langley, & R. Neches, eds.). pp. 329–358. Cambridge, MA: MIT Press 1987Google Scholar
  33. 33.
    McCalla, G. I., Greer, J., and the SCENT Research Team: Intelligent advising in problem solving domains: the SCENT-3 architecture. Proceedings of the 2nd International Conference on Intelligent Tutoring Systems, Montreal, Quebec (C. Frasson, G. Gauthier & G. McCalla, eds.), pp. 124–131, Lecture Notes in Computer Science, Vol. 608, Berlin: Springer-Verlag 1992Google Scholar
  34. 34.
    McCalla, G. I., Greer, J., & Coulman, R.: Enhancing the robustness of model-based recognition. Proceedings of the 3rd International Workshop on User Modeling, Wadern, Germany (E. Andre, R. Cohen, W. Graf, B. Kass, C. Paris, & W. Wahlster, eds.). pp. 240–248, Kaiserslautern: DFKI 1992Google Scholar
  35. 35.
    Minton, S. & Carbonell, J. G.: Strategies for learning search control rules: an explanation-based approach. Proceedings of the UCAI 1987, pp. 228–235, Milan, Italy 1987Google Scholar
  36. 36.
    Möbus, C.: Toward the design of adaptive instructions and help for knowledge communication with the problem solving monitor ABSYNT. In: Artificial Intelligence in Higher Education, (V. Malik, O. Stepankova, & Z. Zdrahal, eds.), pp. 138–145, Lecture Notes in Computer Science, Vol. 451, Berlin: Springer-Verlag 1990Google Scholar
  37. 37.
    Möbus, C.: The relevance of computational models of knowledge acquisition for the design of helps in the problem solving monitor ABSYNT. Proceedings of the Conference on Advanced Research on Computers in Education (ARCE ‘80), pp. 137–144, (R. Lewis & O. Setsuko, eds.). Amsterdam: North-Holland 1991Google Scholar
  38. 38.
    Möbus, C. & Schröder, O.: Representing semantic knowledge with 2-dimensional rules in the domain of functional programming. In: Visualization in Human-Computer Interaction, 7th Interdisciplinary Workshop in Informatics and Psychology, Schärding. Austria, 1988 (P. Gorny & M. Tauber, eds.). pp. 47–81, Lecture Notes in Computer Science, Vol. 439, Berlin: Springer-Verlag 1990Google Scholar
  39. 39.
    Möbus, C., Schröder, O., & Thole, H. J.: Runtime modeling the novice-expert shift in programming skills on a rule-schema-case continuum. Proceedings of the UCAI Workshop W.4 Agent Modelling for Intelligent Interaction, 12th International Joint Conference on Artificial Intelligence, (J. Kay & A. Quilici, eds.). pp. 137–143, Darling Harbour, Sydney, Australia, 1991Google Scholar
  40. 40.
    Möbus, C., Schröder, O., & Thole, H. J.: Online modelling the novice-expert shift in programming skills on a rule-schema-case partial order. In: Working Notes of the AAA’ Spring Symposium Series: Cognitive Aspects of Knowledge Acquisition, pp. 155–161. Stanford University, 1992Google Scholar
  41. 41.
    Möbus, C. & Thole, H. J.: Tutors, instructions and helps. In: Künstliche Intelligenz KIFS 1987 (Th. Christaller, ed.). pp. 336–385. Informatik-Fachberichte 202, Berlin: Springer-Verlag 1989Google Scholar
  42. 42.
    Möbus, C. & Thole, H. J.: Interactive support for planning visual programs in the problem solving monitor ABSYNT: giving feedback to user hypotheses on the language level. In: Computer Assisted Learning. Proceedings of the 3rd International Conference on Computer-Assisted Learning (ICCAL 90), Hagen, Germany, 1990. (D. H. Norrie & H. W. Six, eds.). pp. 36–49, Lecture Notes in Computer Science, Vol. 438, Berlin: Springer-Verlag 1990Google Scholar
  43. 43.
    Neves, D. M. & Anderson, J. R.: Knowledge compilation: mechanisms for the automatization of cognitive skills. In: Cognitive Skills and their Acquisition (J. R. Anderson, ed.). pp. 57–84. Hillsdale, NJ: Lawrence Erlbaum Associates 1981Google Scholar
  44. 44.
    Newell, A.: The knowledge level. Artificial Intelligence, 18, pp. 87–127 (1982)CrossRefGoogle Scholar
  45. 45.
    Rosenbloom, P. S., Laird, J. E., Newell, A., & McCarl, R.: A preliminary analysis of the SOAR architecture as a basis for general intelligence. Artificial Intelligence, 47, pp. 289–305 (1991)MathSciNetCrossRefGoogle Scholar
  46. 46.
    Rosenbloom, P. S. & Newell, A.: Learning by chunking: a production system model of practice. In: Production System Models of Learning and Development (D. Klahr, P. Langley, & R. Neches, eds.). pp. 221–286. Cambridge, MA: MIT Press 1987Google Scholar
  47. 47.
    Schank, R. C.: Dynamic Memory. London: Cambridge University Press 1982Google Scholar
  48. 48.
    Schröder, O.: A model of the acquisition of rule knowledge with visual helps: the operational knowledge for a functional, visual programming language. In: Computer Assisted Learning, Proceedings of the 3rd International Conference on Computer-Assisted Learning (ICCAL 90), Hagen, Germany 1990 (D. H. Norrie & H. W. Six, eds.). pp. 142–157, Lecture Notes in Computer Science, Vol. 438, Berlin: Springer-Verlag 1990Google Scholar
  49. 49.
    Schröder, O.: Erwerb von regelwissen mit visuellen hilfen: das semantikwissen für eine graphische funktionale programmiersprache. Frankfurt: Lang 1992Google Scholar
  50. 50.
    Self, J. A.: Bypassing the intractable problem of student modeling. In: Intelligent Tutoring Systems: at the Crossroads of Artificial Intelligence and Education (C. Frasson & G. Gauthier, eds.). pp. 107–123. Norwood, NJ: Ablex 1990Google Scholar
  51. 51.
    Self, J. A.: Formal Approaches to Learner Modelling. Technical Report AI-59, Department of Computing, Lancaster University; Lancaster, UK 1991Google Scholar
  52. 52.
    Simon, H. A. & Simon, D. P.: Individual differences in solving physics problems. In: Childrens’ Thinking: What Develops? (R. S. Siegler, ed.). pp. 325–348. Hillsdale, NJ: Lawrence Erlbaum Associates 1978Google Scholar
  53. 53.
    Sleeman, D.: An attempt to understand students’ understanding of basic algebra. Cognitive Science, 8, pp. 387–412 (1984)CrossRefGoogle Scholar
  54. 54.
    Sleeman, D. & Brown, J. S.: Intelligent Tutoring Systems. New York, NY: Academic Press 1982Google Scholar
  55. 55.
    VanLehn, K.: Learning one subprocedure per lesson. Artificial Intelligence, 31, pp. 1–40 (1987)CrossRefGoogle Scholar
  56. 56.
    VanLehn, K.: Toward a theory of impasse-driven learning. In: Learning Issues for Intelligent Tutoring Systems (H. Mandl & A. Lesgold, eds.). pp. 19–41. New York, NY: Springer-Verlag 1988Google Scholar
  57. 57.
    VanLehn, K.: Learning events in the acquisition of three skills. Proceedings of the 11th Conference of the Cognitive Science Society, Ann Arbor, MI, pp. 434–441, 1989Google Scholar
  58. 58.
    VanLehn, K.: Mind Bugs: The Origins of Procedural Misconceptions. Cambridge,MA: MIT Press 1990Google Scholar
  59. 59.
    VanLehn, K.: Two pseudo-students: applications of machine learning to formative evaluation. In: Advanced Research on Computers in Education, (R. Lewis & S. Otsuki, eds.). pp 17–25. Amsterdam: Elsevier/IFIP 1991Google Scholar
  60. 60.
    VanLehn, K.: Rule acquisition events in the discovery of problem solving strategies. Cognitive Science, 15, pp. 1–47 (1991)CrossRefGoogle Scholar
  61. 61.
    Vere, S. A.: Relational production systems. Artificial Intelligence8, pp. 47–68 (1977) Google Scholar
  62. 62.
    Weber, G.: Cognitive diagnosis and episodic modeling in an intelligent LISP tutor. Proceedings of the 2nd International Conference on Intelligent Tutoring Systems, Montreal, Quebec (C. Frasson, G. Gauthier & G. McCalla, eds.), pp. 207–214, Lecture Notes in Computer Science, Vol. 608, Berlin: Springer-Verlag 1992Google Scholar
  63. 63.
    Weber, G.: Explanation-based retrieval in a case-based learning model. Proceedings of the 13th Annual Meeting of the Cognitive Science Society, Chicago, IL, pp. 522–527, Hillsdale, NJ: Lawrence Erlbaum Associates 1991Google Scholar
  64. 64.
    Wenger, E.: Artificial Intelligence and Tutoring Systems. Los Altos, CA 1987Google Scholar
  65. 65.
    Wolff, J. G.: Cognitive development as optimization. In: Computational Models of Learning (L. Bolc, ed.). pp. 161–205. Berlin: Springer-Verlag 1987Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1994

Authors and Affiliations

  1. 1.Department of Computational ScienceUniversity of OldenburgOldenburgGermany

Personalised recommendations