User Modeling and User-Adapted Interaction

, Volume 22, Issue 3, pp 281–311 | Cite as

Modeling sequences of user actions for statistical goal recognition

  • Marcelo G. Armentano
  • Analía A. Amandi
Original Paper


User goals are of major importance for an interface agent because they serve as a context to define what the user’s focus of attention is at a given moment. The user’s goals should be detected as soon as possible, after observing few user actions, in order to provide the user with timely assistance. In this article, we describe an approach for modeling and recognizing user goals from observed sequences of user actions by using Variable Order Markov models combined with an exponential moving average (EMA) on the prediction probabilities. The validity of our approach has been tested using data collected from real users in the Unix domain. The results obtained show that an interface agent can achieve near 90% average accuracy and over 58% online accuracy in predicting the most probable user goal after each observed action, in a time linear to the number of goals being modeled. We also found that the use of an EMA allows a faster convergence in the actual user goal.


Goal recognition Variable Order Markov models Interface agents User modeling 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Amandi A. et al.: Recognition of user intentions for interface agents with variable order Markov models. In: Houben, G.-J (eds) UMAP 2009 Lecture Notes in Computer Science vol 5535., pp. 173–184. Springer, Heidelberg (2009)Google Scholar
  2. Armentano M.G., Amandi A.A.: Personalized detection of user intentions. Knowl. Based Syst. 24(8), 1169–1180 (2011)CrossRefGoogle Scholar
  3. Armentano M., Godoy D., Amandi A.: Personal assistants: direct manipulation vs. mixed initiative interfaces. Int. J. Hum. Comput. Stud. 64(1), 27–35 (2006)CrossRefGoogle Scholar
  4. Bauer, M.: Acquisition of abstract plan descriptions for plan recognition. In: AAAI ’98/IAAI ’98: Proceedings of the Fifteenth National/Tenth Conference on Artificial Intelligence/Innovative Applications of Artificial Intelligence, Menlo Park, pp. 936–941. American Association for Artificial Intelligence, Menlo Park (1998)Google Scholar
  5. Bauer M.: From interaction data to plan libraries: a clustering approach. In: IJCAI ’99: Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, San Francisco, pp. 962–967. Morgan Kaufmann, San Fransisco (1999)Google Scholar
  6. Begleiter R., El-yaniv R., Yona G.: On prediction using variable order Markov models. J. Artif. Intell. Res. 22, 385–421 (2004)MathSciNetzbMATHGoogle Scholar
  7. Bejerano G., Yona G.: Variations on probabilistic suffix trees: statistical modeling and prediction of protein families. Bioinformatics 1(17), 23–43 (2001)CrossRefGoogle Scholar
  8. Blaylock N.: Towards Tractable Agent-Based Dialogue. Ur csd/ tr880. Computer Science Department, University of Rochester, Rochester (2005)Google Scholar
  9. Blaylock, N., Allen, J.: Corpus-based, statistical goal recognition. In: Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, Acapulco, pp. 1303–1308 (2003)Google Scholar
  10. Blaylock, N., Allen, J.: Recognizing instantiated goals using statistical methods. In: Kaminka, G. (ed.) IJCAI Workshop on Modeling Others from Observations (MOO-2005), Edinburgh, pp. 79–86 (2005)Google Scholar
  11. Brand, M., Oliver, N., Pentland, A.: Coupled hidden Markov models for complex action recognition. In: Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR ’97), Washington, pp. 994–999 (1997)Google Scholar
  12. Bratko A., Cormack G.V., Filipic B., Lynam T.R., Zupan B.: Spam filtering using statistical data compression models. J. Mach. Learn. Res. 7, 2673–2698 (2006)MathSciNetzbMATHGoogle Scholar
  13. Brown, S.M.: A decision theoretic approach for interface agent development. PhD thesis, Wright Patterson AFB. AAI9905125 (1998)Google Scholar
  14. Bui H., Venkatesh S., West G.: Policy recognition in the abstract hidden Markov model. J. Artif. Intell. Res. 17, 451–499 (2002)MathSciNetzbMATHGoogle Scholar
  15. Bui H.H.: A general model for online probabilistic plan recognition. In: Gottlob, G., Walsh, T. (eds) Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI ’03), pp. 1309–1318. Morgan Kaufmann, Acapulco (2003)Google Scholar
  16. Buntine, W.: Theory refinement on bayesian networks. In: Proceedings of the Seventh Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann, San Francisco, pp. 52–60 (1991)Google Scholar
  17. Card S.K., Newell A., Moran T.P.: The Psychology of Human–Computer Interaction. Lawrence Erlbaum, Hillsdale (1983)Google Scholar
  18. Charniak E., Goldman R.: A probabilistic model of plan recognition. In: Proceedings of the Ninth National Conference on Artificial Intelligence, AAAI’91, vol. 1, pp. 160–165. AAAI Press, Menlo Park (1991)Google Scholar
  19. Cooper G., Herskovits E.: A bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9(4), 309–347 (1992)zbMATHGoogle Scholar
  20. Davison, B.D., Hirsh, H.: Predicting sequences of user actions. In: Predicting the Future: AI Approaches to Time Series. AAAI Press, Menlo Park (1998)Google Scholar
  21. Dempster A., Laird N., Rubin D.: Maximum likelihood from incomplete data via the em algorithm. J. Roy. Stat. Soc. 39(1), 1–38 (1977)MathSciNetzbMATHGoogle Scholar
  22. Duong, T.V., Phung, D.Q., Bui, H.H., Venkatesh, S.: Human behavior recognition with generic exponential family duration modeling in the hidden semi-Markov model. In: International Conference on Pattern Recognition, Hong Kong, vol. 3, pp. 202–207 (2006)Google Scholar
  23. Fine S., Singer Y., Tishby N.: The hierarchical hidden Markov model: analysis and applications. Mach. Learn. 32, 41–62 (1998)zbMATHCrossRefGoogle Scholar
  24. Galata A., Johnson N., Hogg D.: Learning variable-length Markov models of behavior. Comput. Vis. Image Underst. 81(3), 398–413 (2001)zbMATHCrossRefGoogle Scholar
  25. Garland, A., Lesh, N.: Learning hierarchical task models by demonstration, Technical Report TR2002-04. Mitsubishi Electric Research Laboratories, Cambridge (2002)Google Scholar
  26. Geib, C.W., Maraist, J., Goldman, R.P.: A new probabilistic plan recognition algorithm based on string rewriting. In: Proceedings of the International Conference on Automated Planning and Scheduling (ICAPS 2008), Sydney, pp. 91–98 (2008)Google Scholar
  27. Goldman, R., Geib, C., Miller, C.: Learning hierarchical task models by defining and refining examples. In: Proceedings of the 15th Annual Conference on Uncertainty in Artificial Intelligence (UAI-99), pp. 245–254. Morgan Kaufmann, San Francisco (1999)Google Scholar
  28. Gorniak P., Poole D.: Building a stochastic dynamic model of application. In: Boutilier, C., Goldszmidt, M. (eds) Sixteenth Conference on Uncertainty in Artificial Intelligence (UAI-2000), pp. 230–237. Stanford University, Morgan Kaufmann, San Francisco (2000)Google Scholar
  29. Hong J.: Goal recognition through goal graph analysis. J. Artif. Intell. Res. 15, 1–30 (2001)zbMATHGoogle Scholar
  30. Horvitz E., Breese J., Heckerman D., Hovel D., Rommelse K.: The Lumière project: Bayesian user modeling for inferring the goals and needs of software users. In: Cooper, G.F., Moral, S. (eds) Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, pp. 256–265. Morgan Kaufmann, San Mateo (1998)Google Scholar
  31. Hu J., Turin W., Brown M.K.: Language modeling using stochastic automata with variable length contexts. Comput. Speech Lang. 11(1), 1–16 (1997)CrossRefGoogle Scholar
  32. Hunter J.S.: The exponentially weighted moving average. J. Qual. Technol. 18(4), 203–209 (1986)Google Scholar
  33. Kautz H.: A formal theory of plan recognition and its implementation. In: Allen, J.F., Kautz, H.A., Pelavin, R., Tenenberg, J. (eds) Reasoning About Plans, pp. 69–125. Morgan Kaufmann, San Mateo (1991)Google Scholar
  34. Kearns, M., Mansour, Y., Ron D., Rubinfeld, R., Schapire, R.E., Sellie, L.: On the learnability of discrete distributions. In: Proceedings of the 26th Annual ACM Symposium on Theory of Computing, pp. 273–282. ACM Press, New York (1994)Google Scholar
  35. Köck M., Paramythis A.: Activity sequence modelling and dynamic clustering for personalized e-learning. User Model. User Adapt. Interact. 21, 51–97 (2011)CrossRefGoogle Scholar
  36. Lesh, N.: Scalable and adaptive goal recognition. PhD thesis, University of Washington, Washington (1998)Google Scholar
  37. Lesh, N., Rich, C., Sidner, C.L.: Using plan recognition in human-computer collaboration. In: Proceedings of the Seventh International Conference on User modeling, Secaucus, pp. 23–32. Springer, New York (1999)Google Scholar
  38. Liao L., Patterson D.J., Fox D., Kautz H.A.: Learning and inferring transportation routines. Artifi. Intell. 171(5–6), 311–331 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  39. Lieberman H.: Your Wish Is My Command: Programming by Example. Morgan Kaufmann, San Francisco (2001)Google Scholar
  40. Madani, O., Bui, H., Yeh, E.: Efficient online learning and prediction of user’s desktop activities. In: Proceedings of IJCAI 2009, Pasadena, vol. 3, pp. 1457–1462 (2009)Google Scholar
  41. Maes P.: Agents that reduce work and information overload. Commun. ACM 37, 30–40 (1994)CrossRefGoogle Scholar
  42. Nguyen, N.T., Phung, D.Q., Venkatesh, S., Bui, H.H.: Learning and detecting activities from movement trajectories using the hierarchical hidden Markov model. In: IEEE Computer Vision and Pattern Recognition or CVPR, pp. 955–960. IEEE Computer Society, Washington (2005)Google Scholar
  43. Nkambou R., Fournier-Viger P., Nguifo E.M.: Learning task models in ill-defined domain using an hybrid knowledge discovery framework. Knowl. Based Syst. 24, 176–185 (2011)CrossRefGoogle Scholar
  44. Nunez-Yanez J.L., Chouliaras V.A.: A configurable statistical lossless compression core based on variable order Markov modeling and arithmetic coding. IEEE Trans. Comput. 54(11), 1345–1359 (2005)CrossRefGoogle Scholar
  45. Oliver, N., Horvitz, E., Garg, A.: Layered representations for human activity recognition. In Proceedings of the 4th IEEE International Conference on Multimodal Interfaces (ICMI 2002), pp. 3–8. IEEE Computer Society, Washington (2002)Google Scholar
  46. Pascal, J.H., Poupart, P., Boutilier, C., Mihailidis, A.: Semi-supervised learning of a POMDP model of patient-caregiver interactions. In: Proceedings of IJCAI Workshop on Modeling Others from Observations, Edinburgh, pp. 101–110 (2005)Google Scholar
  47. Philipose M., Fishkin K.P., Perkowitz M., Patterson D.J., Fox D., Kautz H., Hahnel D.: Inferring activities from interactions with objects. Pervasive Comput. Mag. 3(4), 10–17 (2004)Google Scholar
  48. Pynadath D.V., Wellman M.P.: Probabilistic state-dependent grammars for plan recognition. In Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence (UAI ’00), Stanford, pp. 507–514 (2000)Google Scholar
  49. Rabin M.O.: Probabilistic automata. Inform. Control 6(3), 230–245 (1963)CrossRefGoogle Scholar
  50. Rich C., Sidner C.L., Lesh N.: COLLAGEN: applying collaborative discourse theory to human-computer interaction. AI Mag. 22(4), 15–26 (2001)Google Scholar
  51. Rissanen J.: A universal data compression system. IEEE Trans. Inform. Theory 29(5), 656–663 (1983)MathSciNetzbMATHCrossRefGoogle Scholar
  52. Ron D., Singer Y., Tishby N.: The power of amnesia: learning probabilistic automata with variable memory length. Mach. Learn. 25(2–3), 117–149 (1996)zbMATHCrossRefGoogle Scholar
  53. Shmilovici A., Ben-Gal I.: Using a VOM model for reconstructing potential coding regions in est sequences. Comput. Stat. 22(1), 49–69 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  54. Soller A.: Adaptive support for distributed collaboration. In: Brusilovsky, P., Kobsa, A., Neijl, W. (eds) The Adaptive Web., pp. 507–514. Springer, Berlin (2007)Google Scholar
  55. Spirtes P., Glymour C., Scheines R.: Causation, Prediction, and Search. 2nd edn. The MIT Press, Cambridge (2000)Google Scholar
  56. Stumpf, S., Bao, X., Dragunov, A., Dietterich, T.G., Herlocker, J., Johnsrude, K., Li, L., Shen, J.: Predicting user tasks: I know what you’re doing! In: 20th National Conference on Artificial Intelligence (AAAI-05), Workshop on Human Comprehensible Machine Learning, Pittsburgh (2005)Google Scholar
  57. Whitworth B.: Polite computing. Behav. Inform. Technol. 24(5), 353–363 (2005)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  1. 1.Department of Computer ScienceUNICEN UniversityTandilArgentina
  2. 2.ISISTAN Research Institute, Fac. Cs. Exactas, UNCPBATandilArgentina
  3. 3.CONICET, National Council of Scientific and Technological ResearchTandilArgentina

Personalised recommendations