A Natural Language Argumentation Interface for Explanation Generation in Markov Decision Processes
A Markov Decision Process (MDP) policy presents, for each state, an action, which preferably maximizes the expected reward accrual over time. In this paper, we present a novel system that generates, in real time, natural language explanations of the optimal action, recommended by an MDP while the user interacts with the MDP policy. We rely on natural language explanations in order to build trust between the user and the explanation system, leveraging existing research in psychology in order to generate salient explanations for the end user. Our explanation system is designed for portability between domains and uses a combination of domain specific and domain independent techniques. The system automatically extracts implicit knowledge from an MDP model and accompanying policy. This policy-based explanation system can be ported between applications without additional effort by knowledge engineers or model builders. Our system separates domain-specific data from the explanation logic, allowing for a robust system capable of incremental upgrades. Domain-specific explanations are generated through case-based explanation techniques specific to the domain and a knowledge base of concept mappings for our natural language model.
KeywordsOptimal Policy Markov Decision Process Variable Assignment Explanation System Discount Function
Unable to display preview. Download preview PDF.
- 1.Aamodt, A., Plaza, E.: Case-based reasoning: Foundational issues, methodological variations, and system approaches. AI communications 7(1), 39–59 (1994)Google Scholar
- 4.Camara, W.J., Echternacht, G.: The SAT I and high school grades: utility in predicting success in college. RN-10, College Entrance Examination Board, New York (2000)Google Scholar
- 7.Guerin, J.T., Crawford, R., Goldsmith, J.: Constructing dynamic bayes nets using recommendation techniques from collaborative filtering. Tech report, University of Kentucky (2010)Google Scholar
- 8.Hoey, J., St-Aubin, R., Hu, A., Boutilier, C.: SPUDD: Stochastic planning using decision diagrams. In: Proc. UAI, pp. 279–288 (1999)Google Scholar
- 9.Khan, O., Poupart, P., Black, J.: Minimal sufficient explanations for factored Markov decision processes. In: Proc. ICAPS (2009)Google Scholar
- 10.Mathias, K., Williams, D., Cornett, A., Dekhtyar, A., Goldsmith, J.: Factored mdp elicitation and plan display. In: Proc. ISDN. AAAI, Menlo Park (2006)Google Scholar
- 11.Moore, B., Parker, R.: Critical Thinking. McGraw-Hill, New York (2008)Google Scholar
- 14.Nugent, C., Doyle, D., Cunningham, P.: Gaining insight through case-based explanation. JIIS 32, 267–295 (2009)Google Scholar
- 16.Renooij, S.: Qualitative Approaches to Quantifying Probabilistic Networks. Ph.D. thesis, Institute for Information and Computing Sciences, Utrecht University, The Netherlands (2001)Google Scholar
- 17.Sinha, R., Swearingen, K.: The role of transparency in recommender systems. In: CHI 2002 Conference Companion, pp. 830–831 (2002)Google Scholar
- 21.Witteman, C., Renooij, S., Koele, P.: Medicine in words and numbers: A cross-sectional survey comparing probability assessment scales. BMC Med. Informatics and Decision Making 7(13) (2007)Google Scholar