Abstract
Building intelligent assistants has been a long-cherished goal of AI, and many were built and fine-tuned to specific application domains. In recent work, a domain-independent decision-theoretic model of assistance was proposed, where the task is to infer the user’s goal and take actions that minimize the expected cost of the user’s policy. In this paper, we extend this work to domains where the user’s policies have rich relational and hierarchical structure. Our results indicate that relational hierarchies allow succinct encoding of prior knowledge for the assistant, which in turn enables the assistant to start helping the user after a relatively small amount of experience.
Similar content being viewed by others
References
Ambite JL, Barish G, Knoblock CA, Muslea M, Oh J, Minton S (2002) Getting from here to there: interactive planning and agent execution for optimizing travel. In: IAAI, pp 862–869
Bertsekas DP, Tsitsiklis JN (1996) Neuro-dynamic programming. Athena Scientific, Belmont
Bui H, Venkatesh S, West G (2002) Policy recognition in the abstract hidden Markov models. JAIR 17:451–499
Bui HH, Cesari F, Elenius D, House N, Morley D, Myers KM, Natarajan S, Saadati S, Yeh E, Yorke-Smith N (2008) CALO workflow recognition and proactive assistance. In: AAAI-08 AI video competition
Boger J, Poupart P, Hoey J, Boutilier, C, Fernie G, Mihailidis A (2005) A decision-theoretic approach to task assistance for persons with dementia. In: IJCAI, pp 1293–1299
De Raedt L, Frasconi P, Kersting K, Muggleton SH (2010) Probabilistic inductive logic programming, Lecture Notes in Computer Science/Lecture Notes in Artificial Intelligence
Dietterich TG (2000) Hierarchical reinforcement learning with the MAXQ value function decomposition. JAIR 13: 227–303
Domingos P, Richardson M (2006) Markov logic networks. Mach Learn 62(1–2): 107–136
Fern A, Natarajan S, Judah K, Tadepalli P (2007) A decision-theoretic model of assistance. In: IJCAI
Fierens D, Blockeel H, Bruynooghe M, Ramon J (2005) Logical Bayesian networks and their relation to other probabilistic logical models. In: Proceedings of ILP-05
Fine S, Singer Y, Tishby N (1998) The hierarchical hidden Markov model: analysis and applications. Mach Learn 32(1): 41–62
Getoor L, Friedman N, Koller D, Pfeffer A (2001) Learning probabilistic relational models. In: Dzeroski S, Lavrac N (eds) Invited contribution to the book relational data mining. Springer, Berlin
Getoor L, Grant J (2005) PRL: a probabilistic relational language. Mach Learn J 62:7–31
Horvitz E, Breese J, Heckerman D, Hovel D, Rommelse K (1998) The Lumiere project: Bayesian user modeling for inferring the goals and needs of software users. In: UAI, pp 256–265
Hui B, Boutilier C (2006) Who’s asking for help? A Bayesian approach to intelligent assistance. In: IUI, pp 186–193
Jaeger M (1997) Relational Bayesian networks. In: UAI-97
Kearns MJ, Mansour Y, Ng AY (1999) A sparse sampling algorithm for near-optimal planning in large Markov decision processes. In: IJCAI
Kersting K, De Raedt L (2000) Bayesian logic programs. In: ILP
Kim H, Lee S (2004) An intelligent information system for organizing online text documents. KAIS 6(2): 125–149
Lafferty J, McCallum A, Pereira F (2001) Conditional Random Fields,b Probabilistic Models for Segmenting and Labeling Sequence Data. In: Proceedings of the 18th international conference on machine learning
Laskey KB (2008) MEBN: a language for first-order Bayesian knowledge bases. Artif Intell 172(2–3): 140–178
Littman M, Cassandra AR, Kaelbling LP (2005) Learning policies for partially observable environments: scaling up. ICML
Liu S, Duffy AHB, Whitfield RI, Boyle IM (2008) Integration of decision support systems to improve decision support performance. KAIS 22(31): 261–286
Muggleton S (1996) Stochastic logic programs. In: Advances in inductive logic programming
Murphy K, Paskin M (2001) Linear time inference in hierarchical HMMs. In: NIPS
Myers K, Berry P, Blythe J, Conleyn K, Gervasio M, McGuinness D, Morley D, Pfeffer A, Pollack M, Tambe M (2007) An intelligent personal assistant for task and time management.. AI Magazine 28(2): 47–61
Natarajan S, Tadepalli P, Altendorf E, Dietterich TG, Fern A, Restificar A (2005) Learning first-order probabilistic models with combining rules. In: Proceedings of ICML-05
Ngo L, Haddawy P (1995) Probabilistic logic programming and Bayesian networks. In: Proceedings ACSC
Qin B, Xia Y, Prabhakar S (2011) Rule induction for uncertain data. Knowl Inf Syst (KAIS), 1–28. doi:10.1007/s10115-010-0335-7
Russell S, Norvig P (2002) Artificial Intelligence: a modern approach, 2nd edn. Prentice-Hall , New York
Sato T, Kameya Y (2001) Parameter learning of logic programs for symbolic-statistical modeling. J Artif Intell Res 15: 391–454
Varakantham P, Maheswaran R, Tambe M (2005) Exploiting belief bounds: practical POMDPs for personal assistant agents. In: AAMAS
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Natarajan, S., Tadepalli, P. & Fern, A. A relational hierarchical model for decision-theoretic assistance. Knowl Inf Syst 32, 329–349 (2012). https://doi.org/10.1007/s10115-011-0435-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10115-011-0435-z