Knowledge and Information Systems

, Volume 32, Issue 2, pp 329–349 | Cite as

A relational hierarchical model for decision-theoretic assistance

  • Sriraam NatarajanEmail author
  • Prasad Tadepalli
  • Alan Fern
Regular Paper


Building intelligent assistants has been a long-cherished goal of AI, and many were built and fine-tuned to specific application domains. In recent work, a domain-independent decision-theoretic model of assistance was proposed, where the task is to infer the user’s goal and take actions that minimize the expected cost of the user’s policy. In this paper, we extend this work to domains where the user’s policies have rich relational and hierarchical structure. Our results indicate that relational hierarchies allow succinct encoding of prior knowledge for the assistant, which in turn enables the assistant to start helping the user after a relatively small amount of experience.


Intelligent assistants Decision-theory Graphical models Inference 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ambite JL, Barish G, Knoblock CA, Muslea M, Oh J, Minton S (2002) Getting from here to there: interactive planning and agent execution for optimizing travel. In: IAAI, pp 862–869Google Scholar
  2. 2.
    Bertsekas DP, Tsitsiklis JN (1996) Neuro-dynamic programming. Athena Scientific, BelmontzbMATHGoogle Scholar
  3. 3.
    Bui H, Venkatesh S, West G (2002) Policy recognition in the abstract hidden Markov models. JAIR 17:451–499Google Scholar
  4. 4.
    Bui HH, Cesari F, Elenius D, House N, Morley D, Myers KM, Natarajan S, Saadati S, Yeh E, Yorke-Smith N (2008) CALO workflow recognition and proactive assistance. In: AAAI-08 AI video competitionGoogle Scholar
  5. 5.
    Boger J, Poupart P, Hoey J, Boutilier, C, Fernie G, Mihailidis A (2005) A decision-theoretic approach to task assistance for persons with dementia. In: IJCAI, pp 1293–1299Google Scholar
  6. 6.
    De Raedt L, Frasconi P, Kersting K, Muggleton SH (2010) Probabilistic inductive logic programming, Lecture Notes in Computer Science/Lecture Notes in Artificial IntelligenceGoogle Scholar
  7. 7.
    Dietterich TG (2000) Hierarchical reinforcement learning with the MAXQ value function decomposition. JAIR 13: 227–303MathSciNetzbMATHGoogle Scholar
  8. 8.
    Domingos P, Richardson M (2006) Markov logic networks. Mach Learn 62(1–2): 107–136Google Scholar
  9. 9.
    Fern A, Natarajan S, Judah K, Tadepalli P (2007) A decision-theoretic model of assistance. In: IJCAIGoogle Scholar
  10. 10.
    Fierens D, Blockeel H, Bruynooghe M, Ramon J (2005) Logical Bayesian networks and their relation to other probabilistic logical models. In: Proceedings of ILP-05Google Scholar
  11. 11.
    Fine S, Singer Y, Tishby N (1998) The hierarchical hidden Markov model: analysis and applications. Mach Learn 32(1): 41–62CrossRefzbMATHGoogle Scholar
  12. 12.
    Getoor L, Friedman N, Koller D, Pfeffer A (2001) Learning probabilistic relational models. In: Dzeroski S, Lavrac N (eds) Invited contribution to the book relational data mining. Springer, BerlinGoogle Scholar
  13. 13.
    Getoor L, Grant J (2005) PRL: a probabilistic relational language. Mach Learn J 62:7–31Google Scholar
  14. 14.
    Horvitz E, Breese J, Heckerman D, Hovel D, Rommelse K (1998) The Lumiere project: Bayesian user modeling for inferring the goals and needs of software users. In: UAI, pp 256–265Google Scholar
  15. 15.
    Hui B, Boutilier C (2006) Who’s asking for help? A Bayesian approach to intelligent assistance. In: IUI, pp 186–193Google Scholar
  16. 16.
    Jaeger M (1997) Relational Bayesian networks. In: UAI-97Google Scholar
  17. 17.
    Kearns MJ, Mansour Y, Ng AY (1999) A sparse sampling algorithm for near-optimal planning in large Markov decision processes. In: IJCAIGoogle Scholar
  18. 18.
    Kersting K, De Raedt L (2000) Bayesian logic programs. In: ILPGoogle Scholar
  19. 19.
    Kim H, Lee S (2004) An intelligent information system for organizing online text documents. KAIS 6(2): 125–149Google Scholar
  20. 20.
    Lafferty J, McCallum A, Pereira F (2001) Conditional Random Fields,b Probabilistic Models for Segmenting and Labeling Sequence Data. In: Proceedings of the 18th international conference on machine learningGoogle Scholar
  21. 21.
    Laskey KB (2008) MEBN: a language for first-order Bayesian knowledge bases. Artif Intell 172(2–3): 140–178MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Littman M, Cassandra AR, Kaelbling LP (2005) Learning policies for partially observable environments: scaling up. ICMLGoogle Scholar
  23. 23.
    Liu S, Duffy AHB, Whitfield RI, Boyle IM (2008) Integration of decision support systems to improve decision support performance. KAIS 22(31): 261–286Google Scholar
  24. 24.
    Muggleton S (1996) Stochastic logic programs. In: Advances in inductive logic programmingGoogle Scholar
  25. 25.
    Murphy K, Paskin M (2001) Linear time inference in hierarchical HMMs. In: NIPSGoogle Scholar
  26. 26.
    Myers K, Berry P, Blythe J, Conleyn K, Gervasio M, McGuinness D, Morley D, Pfeffer A, Pollack M, Tambe M (2007) An intelligent personal assistant for task and time management.. AI Magazine 28(2): 47–61Google Scholar
  27. 27.
    Natarajan S, Tadepalli P, Altendorf E, Dietterich TG, Fern A, Restificar A (2005) Learning first-order probabilistic models with combining rules. In: Proceedings of ICML-05Google Scholar
  28. 28.
    Ngo L, Haddawy P (1995) Probabilistic logic programming and Bayesian networks. In: Proceedings ACSCGoogle Scholar
  29. 29.
    Qin B, Xia Y, Prabhakar S (2011) Rule induction for uncertain data. Knowl Inf Syst (KAIS), 1–28. doi: 10.1007/s10115-010-0335-7
  30. 30.
    Russell S, Norvig P (2002) Artificial Intelligence: a modern approach, 2nd edn. Prentice-Hall , New YorkGoogle Scholar
  31. 31.
    Sato T, Kameya Y (2001) Parameter learning of logic programs for symbolic-statistical modeling. J Artif Intell Res 15: 391–454MathSciNetzbMATHGoogle Scholar
  32. 32.
    Varakantham P, Maheswaran R, Tambe M (2005) Exploiting belief bounds: practical POMDPs for personal assistant agents. In: AAMASGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Sriraam Natarajan
    • 1
    • 2
    Email author
  • Prasad Tadepalli
    • 1
  • Alan Fern
    • 1
  1. 1.School of EECSOregon State UniversityCorvallisUSA
  2. 2.Wake Forest University School of MedicineWinston-SalemUSA

Personalised recommendations