Skip to main content
SpringerLink
Log in

Focus tree: modeling attentional information in task-oriented human-machine interaction

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

This paper introduces a new model of attentional state in task-oriented human-machine interaction. It integrates three lines of research: (i) neurocognitive understanding of the focus of attention in working memory, (ii) the notion of attention related to the theory of discourse structure in the field of computational linguistics, and (iii) investigation of a corpus that comprises recordings of spontaneous speech-based human-machine interaction. The underlying idea was to make a computationally appropriate representation of attentional information that imitates the function of a focus of attention in human perception. The introduced model addresses both the research questions of storage and processing of attentional information. Finally, the paper illustrates the model for concrete interaction domains, and discusses its implementation within a prototype spoken dialogue system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Awh E, Jonides J (2001) Overlapping mechanisms of attention and spatial working memory. Trends Cogn Sci 5(3):119–126

    Article  Google Scholar 

  2. Atkinson RC, Shiffrin RM (1968) Human memory: A proposed system and its control processes. In: Spence KW, Spence JT (eds) The psychology of learning and motivation: Advances in research and theory, vol 2. Academic Press, New York, pp 89–195

    Google Scholar 

  3. Baddeley AD (1993) Working memory or working attention? In: Baddeley AD, Weiskrantz L (eds) Attention: Selection, awareness, and control. A tribute to Donald Broadbent. Oxford University Press, New York, pp 152–170

    Google Scholar 

  4. Baddeley AD, Hitch GJ (1974) Working memory. In: Bower GH (ed) The psychology of learning and motivation: Advances in research and theory, vol 8. Academic Press, New York, pp 47–89

    Google Scholar 

  5. Baddeley AD, Logie RH (1999) Working memory: The multiple component model. In: Miyake A, Shah P (eds) Models of working memory. Cambridge University Press, New York, pp 28–61

    Google Scholar 

  6. Bledowski C, Rahm B, Rowe BR (2009) What ‘works’ in working memory? Separate systems for selection and updating of critical information. J Neurosci 29:13735–13741

    Article  Google Scholar 

  7. Bledowski C, Kaiser J, Rahm B (2010) Basic operations in working memory: Contributions from functional imaging studies. Behav Brain Res 214(2):172–179

    Article  Google Scholar 

  8. Broadbent DE (1958) Perception and communication. Pergamon Press, New York

    Book  Google Scholar 

  9. Campbell N (2006) On the structure of spoken language. In: Proceedings of the 3rd international conference on speech prosody 2006, Dresden, Germany, 4 pages

    Google Scholar 

  10. Campbell N (2007) Towards conversational speech synthesis; lessons learned from the expressive speech processing project. In: Proceedings of the sixth ISCA workshop on speech synthesis (SSW6), Bonn, Germany, pp 22–27

    Google Scholar 

  11. Cowan N (1988) Evolving conceptions of memory storage selective attention, and their mutual constraints within the information processing system. Psychol Bull 104(2):163–191

    Article  Google Scholar 

  12. Cowan N (2001) The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behav Brain Sci 24:87–185

    Article  Google Scholar 

  13. Cowan N, Fristoe NM, Elliot EM, Brunner RP, Saults JS (2006) Scope of attention, control of attention, and intelligence in children and adults. Mem Cogn 34:1754–1768

    Article  Google Scholar 

  14. Engle RW, Kane MJ (2004) Executive attention, working memory capacity, and a two-factor theory of cognitive control. In: Ross B (ed) The psychology of learning and motivation, vol 44. Elsevier, New York, pp 145–199

    Google Scholar 

  15. Engle RW, Cantor J, Carullo JJ (1992) Individual differences in working memory and comprehension: A test of four hypotheses. J Exp Psychol Learn Mem Cogn 18:972–992

    Article  Google Scholar 

  16. Ericsson KA, Kintsch W (1995) Long-term working memory. Psychol Rev 102:211–245

    Article  Google Scholar 

  17. Gnjatović M (2009) Adaptive dialogue management in human-machine interaction. Verlag Dr Hut, Munich

    Google Scholar 

  18. Gnjatović M, Rösner D (2007) An approach to processing of user’s commands in human-machine interaction. In: Proceedings of the 3rd language and technology conference (LTC’07), Adam Mickiewicz University, Poznan, Poland, pp 152–156

    Google Scholar 

  19. Gnjatović M, Rösner D (2008) Adaptive dialogue management in the NIMITEK prototype system. In: Proceedings of the 4th IEEE tutorial and research workshop perception and interactive technologies for speech-based systems (PIT’08), Lecture notes in computer science, vol 5078. Springer, Berlin, pp 14–25

    Google Scholar 

  20. Gnjatović M, Rösner D (2010) Inducing genuine emotions in simulated speech-based human-machine interaction: The nimitek corpus. IEEE Trans Affect Comput 1:132–144

    Article  Google Scholar 

  21. Gnjatović M, Pekar D, Delić V (2011) Naturalness, adaptation and cooperativeness in spoken dialogue systems. In: Toward autonomous, adaptive, and context-aware multimodal interfaces, theoretical and practical issues, Lecture notes in computer science, vol 6456. Springer, Berlin, pp 298–304

    Chapter  Google Scholar 

  22. Griffin IC, Nobre AC (2003) Orienting attention to locations in internal representations. J Cogn Neurosci 15(8):1176–1194

    Article  Google Scholar 

  23. Grosz B, Sidner C (1986) Attention, intentions, and the structure of discourse. Comput Linguist 12(3):175–204

    Google Scholar 

  24. Halliday M (1994) An introduction to functional grammar, 2nd edn. Edward Arnold, London

    Google Scholar 

  25. Hovy E, McCoy K (1989) Focusing your RST: A step toward generating coherent, multisentential text. In: Proceedings of the 11th annual conference of the cognitive science society, Ann Arbor

    Google Scholar 

  26. Jokinen K (2009) Constructive dialogue modelling: Speech interaction and rational agents. Wiley, New York

    Google Scholar 

  27. Jokinen K, Tanaka H, Yokoo A (1998) Context management with topics for spoken dialogue systems. In: Proceedings of COLING-ACL 1998, pp 631–637

    Google Scholar 

  28. Just MA, Carpenter PA (1992) A capacity theory of comprehension: Individual differences in working memory. Psychol Rev 99:122–149

    Article  Google Scholar 

  29. Kane MJ, Conway ARA, Hambrick DZ, Engle RW (2007) Variation in working memory capacity as variation in executive attention and control. In: Conway ARA, Jarrold C, Kane MJ, Miyake A, Towse JN (eds) Variation in working memory. Oxford University Press, Oxford, pp 21–48

    Google Scholar 

  30. Kari L, Rozenberg G (2008) The many facets of natural computing. Commun ACM 51(10):72–83

    Article  Google Scholar 

  31. Kirschner M (2007) Applying a focus tree model of dialogue context to interactive question answering. In: Proceedings of the ESSLLI’07 student session, Dublin, Ireland, pp 135–147

    Google Scholar 

  32. Lecœuche R, Robertson D, Barry C, Mellish C (2000) Evaluating focus theories for dialogue management. Int J Hum-Comput Stud 52(1):23–76

    Article  Google Scholar 

  33. Li S-T, Tsai F-C (2010) Constructing tree-based knowledge structures from text corpus. Appl Intell 33(1):67–78

    Article  MathSciNet  Google Scholar 

  34. McCoy K, Cheng J (1991) Focus of attention: Constraining what can be said next. In: Paris CL, Swartout WR, Moore WC (eds) Natural language generation in artificial intelligence and computational linguistics. Kluwer Academic, Norwell, pp 103–124

    Google Scholar 

  35. Mendonça M, de Arruda LVR, Neves F (2011) Autonomous navigation system using event driven-fuzzy cognitive maps. Appl Intell. doi:10.1007/s10489-011-0320-1

    Google Scholar 

  36. Miller GA (1956) The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychol Rev 63:81–97

    Article  Google Scholar 

  37. Moeller J-U (1996) Domain related focus-shifting constraints in dialogues with knowledge based systems. In: Adorni G, Zock M (eds) Trends in natural language generation: An artificial intelligence perspective. Springer, Berlin, pp 188–204

    Chapter  Google Scholar 

  38. Mogle JA, Lovett BJ, Stawski RS, Sliwinski MJ (2008) What’s so special about working memory? An examination of the relationship among working memory, secondary memory, and fluid intelligence. Psychol Sci 19:1071–1077

    Article  Google Scholar 

  39. Mohammad Y, Nishida T (2010) Controlling gaze with an embodied interactive control architecture. Appl Intell 32(2):148–163

    Article  Google Scholar 

  40. Moubaiddin A, Obeid N (2009) Partial information basis for agent-based collaborative dialogue. Appl Intell 30(2):142–167

    Article  Google Scholar 

  41. Nobre CA, Coull TJ, Maquet P, Frith DC, Vandenberghe R, Mesulam MM (2004) Orienting attention to locations in perceptual versus mental representations. J Cogn Neurosci 16(3):363–373

    Article  Google Scholar 

  42. Oberauer K (2002) Access to information in working memory: Exploring the focus of attention. J Exp Psychol Learn Mem Cogn 28(3):411–421

    Article  Google Scholar 

  43. Oberauer K, Lange EB (2009) Activation and binding in verbal working memory: A dual-process model for the recognition of nonwords. Cogn Psychol 58(1):102–136

    Article  Google Scholar 

  44. O’Reilly RC, Braver TS, Cohen JD (1999) A biologically-based computational model of working memory. In: Miyake A, Shah P (eds) Models of working memory. Cambridge University Press, New York, pp 375–411

    Google Scholar 

  45. Rahwan I, McBurney P (2007) Argumentation technology. IEEE Intell Syst 22(6):21–23

    Article  Google Scholar 

  46. Roulet E (1992) On the structure of conversation as negotiation. In: Mey JL, Parret H, Verschueren J (eds) (On) Searle on conversation. John Benjamins, Philadelphia, pp 91–99

    Google Scholar 

  47. Salthouse TA (1996) The processing-speed theory of adult age differences in cognition. Psychol Rev 103:403–428

    Article  Google Scholar 

  48. Searle J (1992) Conversation. In: Mey JL, Parret H, Verschueren J (eds) (On) Searle on conversation. John Benjamins, Philadelphia, pp 7–29

    Google Scholar 

  49. Shah P, Miyake A (1999) Models of working memory: An introduction. In: Miyake A, Shah P (eds) Models of working memory: Mechanisms of active maintenance and executive control. Cambridge University Press, New York, pp 1–26

    Chapter  Google Scholar 

  50. Stede M, Schlangen D (2004) Information-seeking chat: Dialogue management by topic structure. In: Proceedings of the 8th workshop on semantics and pragmatics of dialogue, CATALOG 04, Barcelona

    Google Scholar 

  51. Stoltzfus ER, Hasher L, Zacks RT (1996) Working memory and aging: Current status of the inhibitory view. In: Richardson JTE, Engle RW, Hasher L, Logie RH, Stoltzfus ER, Zacks RT (eds) Working memory and human cognition. Oxford University Press, New York, pp 66–88

    Chapter  Google Scholar 

  52. Unsworth N, Engle RW (2007) The nature of individual differences in working memory capacity: Active maintenance in primary memory and controlled search from secondary memory. Psychol Rev 114:104–132

    Article  Google Scholar 

  53. Unsworth N, Spillers GJ (2010) Working memory capacity: Attention control secondary memory, or both? A direct test of the dual-component model. J Mem Lang 62(4):392–406

    Article  Google Scholar 

  54. Wheeler ME, Treisman AM (2002) Binding in short-term visual memory. J Exp Psychol Gen 131:48–64

    Article  Google Scholar 

  55. Xu Y, Ohmoto Y, Okada S, Ueda K, Komatsu T, Okadome T, Kamei K, Sumi Y, Nishida T (2010) Formation conditions of mutual adaptation in human-agent collaborative interaction. Appl Intell. doi:10.1007/s10489-010-0255-y

    Google Scholar 

  56. Zhong N, Liu J, Yao Y (2010) Introduction to brain informatics. Cogn Syst Res 11(1):1–2

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Technical Sciences, University of Novi Sad, Trg Dositeja Obradovića 6, 21000, Novi Sad, Serbia

    Milan Gnjatović & Vlado Delić

  2. Mathematical Institute of the Serbian Academy of Sciences and Arts, Kneza Mihaila 36, 11001, Beograd, p.p. 367, Serbia

    Marko Janev

Authors
  1. Milan Gnjatović
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marko Janev
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vlado Delić
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Milan Gnjatović.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gnjatović, M., Janev, M. & Delić, V. Focus tree: modeling attentional information in task-oriented human-machine interaction. Appl Intell 37, 305–320 (2012). https://doi.org/10.1007/s10489-011-0329-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-011-0329-5

Keywords

Search

Navigation