Journal of Automated Reasoning

, Volume 32, Issue 3, pp 187–226 | Cite as

Abductive Theorem Proving for Analyzing Student Explanations to Guide Feedback in Intelligent Tutoring Systems

  • Maxim Makatchev
  • Pamela W. Jordan
  • Kurt VanLehn


The Why2-Atlas tutoring system presents students with qualitative physics questions and encourages them to explain their answers through natural language. Although there are inexpensive techniques for analyzing explanations, we claim that better understanding is necessary for use within tutoring systems. In this paper we motivate and describe how the system creates and uses a deeper proof-based representation of student essays in order to provide students with substantive feedback on their explanations. We describe in detail the abductive reasoner, Tacitus-lite+, that we use within the tutoring system. We also discuss evaluation results for an early version of the Why2-Atlas system and a subsequent evaluation of the theorem-proving module. We conclude with the discussion of work in progress and additional future work for deriving more benefits from a proof-based approach for tutoring applications.

intelligent tutoring systems abductive reasoning qualitative physics 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aleven, V. and Koedinger, K. R. (2000) The need for tutorial dialog to support self-explanation, in Building Dialogue System for Tutorial Applications, Papers of the 2000 AAAI Fall Symposium.Google Scholar
  2. Aleven, V., Popescu, O. and Koedinger, K. (2002) Pilot-testing a tutorial dialogue system that sup-ports self-explanation, in Proceedings of Intelligent Tutoring Systems Conference, LNCS 2363, Springer, pp. 344–354.Google Scholar
  3. Aleven, V., Popescu, O. and Koedinger, K. R. (2001a) Toward tutorial dialog to support self-explanation: Adding natural language understanding to a cognitive tutor, in Proceedings of 10th International Conference on Artificial Intelligence in Education (AI-ED 2001), IOS Press, Amsterdam, pp. 246–255.Google Scholar
  4. Aleven, V., Popescu, O. and Koedinger, K. R. (2001b) A tutorial dialogue system with knowledge-based understanding and classification of student explanations, in Working Notes of 2nd IJCAI Workshop on Knowledge and Reasoning in Practical Dialogue Systems.Google Scholar
  5. Appelt, D. and Pollack, M. (1992) Weighted abduction for plan ascription, User Modeling and User-Adapted Interaction 2(1-2), 1–25CrossRefGoogle Scholar
  6. Austin, J. L. (1962) How to Do Things with Words, Oxford University Press, Oxford.Google Scholar
  7. Bacchus, F., Tennenberg, J. and Koomen, J. (1989) A non-reified temporal logic, in J. F. Allen, R. Fikes and E. Sandewall (eds.), KR'89: Principles of Knowledge Representation and Reasoning, San Mateo, California, Morgan Kaufmann, pp. 2–10.Google Scholar
  8. Bylander, T., Allemang, D., Tanner, M. C. and Josephson, J. R. (1991) The computational complexity of abduction, Artificial Intelligence 49(1-3), 25–60.zbMATHMathSciNetCrossRefGoogle Scholar
  9. Charniak, E. (1986) A neat theory of marker passing, in Proceedings of the 5th National Conference on Artificial Intelligence (AAAI'86), pp. 584–588.Google Scholar
  10. Charniak, E. and Shimony, S. E. (1990) Probabilistic semantics for cost-based abduction, in Proceedings of AAAI-90, pp. 106–111.Google Scholar
  11. Charniak, E. and Shimony, S. E. (1994) Cost-based abduction and MAP explanation, Artificial Intelligence 66, 345–374.zbMATHMathSciNetCrossRefGoogle Scholar
  12. Chi, M. T. H. and Ceci, S. J. (1987) Content knowledge: Its role, representation and restructuring in memory development, Advances in Child Development and Behavior 20, 91–142.Google Scholar
  13. Chi, M. T. H., de Leeuw, N., Chiu, M.-H. and LaVancher, C. (1994) Eliciting self-explanations improves understanding, Cognitive Science 18, 439–477.CrossRefGoogle Scholar
  14. Chi, M. T. H., Siler, S. A., Jeong, H., Yamauchi, T. and Hausmann, R. G. (2001) Learning from human tutoring, Cognitive Science 25(4), 471–533.CrossRefGoogle Scholar
  15. Cohn, A. G. (1989) Taxonomic reasoning with many-sorted logics, Artificial Intelligence 3, 89–128.Google Scholar
  16. Conati, C., Gertner, A. and VanLehn, K. (2002) Using Bayesian networks to manage uncertainty in student modeling, J. User Modeling and User-Adapted Interaction 12(4).Google Scholar
  17. de Kleer, J. (1990) Multiple representations of knowledge in a mechanics problem-solver, in D. S. Weld and J. de Kleer (eds.), Readings in Qualitative Reasoning about Physical Systems, San Mateo, California, Morgan Kaufmann, pp. 40–45.Google Scholar
  18. Eiter, T. and Gottlob, G. (1993) The complexity of logic-based abduction, in Symposium on Theoretical Aspects of Computer Science, pp. 70–79.Google Scholar
  19. Eshghi, K. (1993) A tractable class of abduction problems, in Proceedings 13th International Joint Conference on Artificial Intelligence, Chambery, France, pp. 3–8.Google Scholar
  20. Eshghi, K. and Kowalski, R. A. (1989) Abduction compared with negation by failure, in Proceedings of the 6th International Conference on Logic Programming (ICLP '89), pp. 234–254.Google Scholar
  21. Forbus, K., Carney, K., Harris, R. and Sherin, B. (2001) A qualitative modeling environment for middle-school students: A progress report, in QR-01.Google Scholar
  22. Forbus, K. D. (1997) Using qualitative physics to create articulate educational software, IEEE Expert, pp. 32–41.Google Scholar
  23. Frisch, A. M. (1991) The substitutional framework for sorted deduction: Fundamental results on hybrid reasoning, Artificial Intelligence 49(1-3), 161–198.zbMATHMathSciNetCrossRefGoogle Scholar
  24. Graesser, A. C., Wiemer-Hastings, P., Wiemer-Hastings, K., Harter, D., Person, N. and the TRG (2000) Using latent semantic analysis to evaluate the contributions of students in AutoTutor, Interactive Learning Environments 8, 129–148.CrossRefGoogle Scholar
  25. Hake, R. R. (1998) Interactive-engagement versus traditional methods: A six-thousand student survey of mechanics test data for introductory physics students, American Journal of Physics 66(4), 64–74.CrossRefGoogle Scholar
  26. Haugh, B. (1987) Non-standard semantics for the method of temporal arguments, in Proc. of IJCAI'87, pp. 449–454.Google Scholar
  27. Hestenes, D., Wells, M. and Swackhamer, G. (1992) Force concept inventory, The Physics Teacher 30, 141–158.Google Scholar
  28. Hewitt, P. G. (1998) Conceptual Physics, 8th edn, Addison-Wesley.Google Scholar
  29. Hobbs, J., Stickel, M., Appelt, D. and Martin, P. (1993) Interpretation as abduction, Artificial Intelligence 63(1-2), 69–142.CrossRefGoogle Scholar
  30. Hobbs, J., Stickel, M., Martin, P. and Edwards, D. (1988) Interpretation as abduction, in Proc. 26th Annual Meeting of the ACL, Association of Computational Linguistics, pp. 95–103Google Scholar
  31. Horacek, H. (1997) A model for adapting explanations to users' likely inferences, User Modeling and User-Adapted Interaction 7(1), 1–55.MathSciNetCrossRefGoogle Scholar
  32. Jordan, P. and VanLehn, K. (2002) Discourse processing for explanatory essays in tutorial applications, in Proceedings of the 3rd SIGdial Workshop on Discourse and Dialogue.Google Scholar
  33. Jordan, P. W. (2004) Using student explanations as models for adapting tutorial dialogues, in Proceedings of 17th International FLAIRS Conference.Google Scholar
  34. Kakas, A., Kowalski, R. A. and Toni, F. (1998) The role of abduction in logic programming, in D. M. Gabbay, C. J. Hogger and J. A. Robinson (eds.), Handbook of Logic in Artificial Intelligence and Logic Programming, Vol. 5, Oxford University Press, pp. 235–324.Google Scholar
  35. Kaneiwa, K. and Tojo, S. (2001) An order-sorted resolution with implicitly negative sorts, in Proceedings of the 2001 International Conference on Logic Programming (ICLP'01), LNCS 2237, Springer, pp. 300–314.Google Scholar
  36. Keeney, R. and Raiffa, H. (1976) Decisions with Multiple Objectives, Wiley.Google Scholar
  37. Landauer, T. K., Foltz, P. W. and Laham, D. (1998) An introduction to latent semantic analysis, Discourse Processes 25, 259–284.CrossRefGoogle Scholar
  38. Lascarides, A. and Asher, N. (1991) Discourse relations and defeasible knowledge, in 29th Annual Meeting of the Association for Computational Linguistics, pp. 55–62.Google Scholar
  39. Lavoie, B. and Rambow, O. (1997) A fast and portable realizer for text generation systems, in Proceedings of the Fifth Conference on Applied Natural Language Processing Chapter of the Association for Computational Linguistics, Washington, DC, pp. 265–268.Google Scholar
  40. Leake, D. (1995) Abduction, experience, and goals: A model of everyday abductive explanation, J. Experimental and Theoretical Artificial Intelligence 7, 407–428.Google Scholar
  41. McCallum, A. and Nigam, K. (1998) A comparison of event models for naive Bayes text classification, in Proceedings of AAAI/ICML-98 Workshop on Learning for Text Categorization, AAAI Press.Google Scholar
  42. McRoy, S. and Hirst, G. (1995) The repair of speech act misunderstandings by abductive inference, Computational Linguistics 21(4), 435–478.Google Scholar
  43. Murray, R. C. and VanLehn, K. (2000) DT tutor: A dynamic decision-theoretic approach for optimal selection of tutorial actions, in Proceedings of Intelligent Tutoring Systems Conference,LNCS 1839, Springer, pp. 153–162.Google Scholar
  44. Ng, V. and Cardie, C. (2002) Improving machine learning approaches to coreference resolution, in Proceedings of Association for Computational Linguistics 2002.Google Scholar
  45. Paul, G. (1993) Approaches to abductive reasoning-An overview, Artificial Intelligence Review 7(2), 109–152.CrossRefGoogle Scholar
  46. Ploetzner, R. and VanLehn, K. (1997) The acquisition of qualitative physics knowledge during textbook-based physics training, Cognition and Instruction 15(2), 169–205.CrossRefGoogle Scholar
  47. Poole, D. (1993) Probabilistic Horn abduction and Bayesian networks, Artificial Intelligence 64(1), 81–129.zbMATHCrossRefGoogle Scholar
  48. Rayner, M. and Alshawi, H. (1992) Deriving database queries from logical forms by abductive definition expansion, in Proceedings of the Third Conference of Applied Natural Language Processing, Trento, Italy, pp. 1–8.Google Scholar
  49. Rosé, C., Bhembe, D., Roque, A., Siler, S., Srivastava, R. and VanLehn, K. (2002) A hybrid understanding approach for robust selection of tutoring goals, in Proceedings of Intelligent Tutoring Systems Conference, LNCS 2363, Springer, pp. 552–561.Google Scholar
  50. Rosé, C., Jordan, P., Ringenberg, M., Siler, S., VanLehn, K. and Weinstein, A. (2001) Interactive conceptual tutoring in Atlas-Andes, in Proceedings of AI in Education 2001 Conference.Google Scholar
  51. Rosé, C., Roque, A., Bhembe, D. and VanLehn, K. (2002) An efficient incremental architecture for robust interpretation, in Proceedings of Human Language Technology Conference, San Diego, CA.Google Scholar
  52. Santos, Jr., E. and Santos, E. S. (1996) Polynomial solvability of cost-based abduction, Artificial Intelligence 86(1), 157–170MathSciNetCrossRefGoogle Scholar
  53. Schmidt-Schauß, M. (1989) Computational Aspects of an Order-Sorted Logic with Term Declarations, Springer.Google Scholar
  54. Searle, J. R. (1975) Indirect speech acts, in P. Cole and J. Morgan (eds.), Syntax and Seman-tics 3: Speech Acts, Academic Press. Reprinted in S. Davis (ed.), Pragmatics. A Reader, Oxford University Press, 1991.Google Scholar
  55. Selman, B. and Levesque, H. J. (1990) Abductive and default reasoning: A computational core, in Proceedings of AAAI-90, Boston, MA, pp. 343–348.Google Scholar
  56. Slotta, J. D., Chi, M. T. and Joram, E. (1995) Assessing students' misclassifications of physics concepts: An ontological basis for conceptual change, Cognition and Instruction 13(3), 373–400.CrossRefGoogle Scholar
  57. Stickel, M. (1988) A Prolog-like inference system for computing minimum-cost abductive explanations in natural-language interpretation, Technical Report 451, SRI International, 333 Ravenswood Ave., Menlo Park, California.Google Scholar
  58. Strube, M., Rapp, S. and Müller, C. (2002) The influence of minimum edit distance on reference resolution, in Proceedings of Empirical Methods in Natural Language Processing Conference.Google Scholar
  59. Thomason, R. H., Hobbs, J. and Moore, J. D. (1996) Communicative goals, in K. Jokinen, M. May-bury, M. Zock and I. Zukerman (eds.), Proceedings of the ECAI 96 Workshop Gaps and Bridges: New Directions in Planning and Natural Language Generation.Google Scholar
  60. VanLehn, K., Jordan, P., Rosé, C., Bhembe, D., Böttner, M., Gaydos, A., Makatchev, M., Pap-puswamy, U., Ringenberg, M., Roque, A., Siler, S. and Srivastava, R. (2002) The architecture of Why2-Atlas: A coach for qualitative physics essay writing, in Proceedings of Intelligent Tutoring Systems Conference, LNCS 2363, Springer, pp. 158–167.Google Scholar
  61. Wahlster, W. and Kobsa, A. (1989) User models in dialogue systems, in A. Kobsa and W. Wahlster (eds.), User Models in Dialogue Systems, Springer Verlag, Berlin, pp. 4–34.Google Scholar
  62. Walther, C. (1987) A Many-Sorted Calculus Based on Resolution and Paramodulation, Morgan Kaufmann, Los Altos, California.Google Scholar
  63. Weld, D. S. and de Kleer, J. (eds.) (1990) Readings in Qualitative Reasoning about Physical Systems, Morgan Kaufmann, San Mateo, California.Google Scholar
  64. Zukerman, I. and Albrecht, D. W. (2001) Predictive statistical models for user modeling, User Modeling and User-Adpated Interaction 11(1-2), 5–18.zbMATHCrossRefGoogle Scholar
  65. Zukerman, I. and George, S. (2002) A minimum message length approach for argument interpretation, in Proceedings of the 3rd SIGdial Workshop on Discourse and Dialogue.Google Scholar
  66. Zukerman, I. and McConachy, R. (1993) Generating concise discourse that addresses a user's infer-ences, in Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence, Morgan Kaufmann Publishers, Inc., pp. 1202–1207.Google Scholar
  67. Zukerman, I., McConachy, R. and Korb, K. B. (2000) Using argumentation strategies in automated argument generation, in Proceedings of the 1st International Natural Language Generation Conference, pp. 55–62.Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Maxim Makatchev
    • 1
  • Pamela W. Jordan
    • 1
  • Kurt VanLehn
    • 1
  1. 1.Learning Research and Development CenterUniversity of PittsburghPittsburghU.S.A. e-mail

Personalised recommendations