Human Cognition as a Foundation for the Emerging Egocentric Interaction Paradigm

  • Dipak Surie
  • Thomas Pederson
  • Lars-Erik Janlert
Part of the Studies in Computational Intelligence book series (SCI, volume 396)


This chapter presents an “egocentric interaction paradigm” (EIP) centered on human agents rather than on the notion of user. More specifically, this paradigm is based on perception, action, intention and attention capabilities and limitations of human agents. Traditional and emerging interaction paradigms are typically related to a specific computing environment, devices or human capabilities. The novelty of the proposed approach stems from aiming at developing a comprehensive and integrated theoretical approach, centered on individual human agent. Development in Human-Computer Interaction (HCI) has been closely related to the understanding and utilization of natural human skills and abilities. This work attempts to understand and model a human agent, and in particular their cognitive capabilities in facilitating HCI. The EIP is based on principles like situatedness and embodiment, the physical-virtual equity principle, and the proximity principle. A situative space model built upon our understanding of human cognition is described in detail, followed by our experience in exploring the egocentric interaction paradigm in the easy ADL home.


Action Space Physical Object Human Agent Human Cognition Simon Effect 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anderson, J.R.: Rules of the mind. Erlbaum, Hillsdale (1993)Google Scholar
  2. 2.
    Bardram, J.: From Desktop Task Management to Ubiquitous Activity-Based Computing. In: Integrated Digital Work Environments: Beyond the Desktop Metaphor, pp. 49–78. MIT Press (2007)Google Scholar
  3. 3.
    Bell, C.G., Newell, A.: Computer Structures: Readings and Examples. McGraw-Hill, New York (1971)Google Scholar
  4. 4.
    Blakemore, S.-J., Decety, J.: From the perception of action to the understanding of intention. Nature Reviews, Neuroscience 2, 561–567 (2001)Google Scholar
  5. 5.
    Brass, M., Schmitt, R.M., Spengler, S., Gergely, G.: Investigating action understanding: Inferential processes versus action simulation. Curr. Biol. 17, 2117–2121 (2007)CrossRefGoogle Scholar
  6. 6.
    Brown, J.S., Collins, A., Duguid, P.: Situated cognition and the culture of learning. Educational Researcher 18(1), 32–41 (1989)CrossRefGoogle Scholar
  7. 7.
    Burdea, G., Coffet, P.: Virtual Reality Technology, 2nd edn. Wiley-IEEE Press (2003)Google Scholar
  8. 8.
    Byrne, M.D., Anderson, J.R.: Perception and action. In: Anderson, J.R., Lebiere, C. (eds.) The Atomic Components of Thought, pp. 167–200. Erlbaum, Hillsdale (1998)Google Scholar
  9. 9.
    Card, S.K., Moran, T.P., Newell, A.: The psychology of human-computer interaction. Erlbaum, Hillsdale (1983)Google Scholar
  10. 10.
    Clark, A.: Being There: Putting Brain, Body, and World Together Again. MIT Press, Cambridge (1997)Google Scholar
  11. 11.
    Clark, A., Chalmers, D.: The Extended Mind. Analysis 58, 7–19 (1997)CrossRefGoogle Scholar
  12. 12.
    Dourish, P.: Where The Action Is: The Foundations of Embodied Interaction. MIT Press, Cambridge (2001)Google Scholar
  13. 13.
    Duncan, J., Humphreys, G.W.: Visual search and stimulus similarity. Psychological Review 96, 433–458 (1989)CrossRefGoogle Scholar
  14. 14.
    Ehn, P.: Work-oriented design of computer artifacts. Lawrence Erlbaum Associates, Hillsdale (1990)Google Scholar
  15. 15.
    Elsner, B., Hommel, B., Mentschel, C., Drzezga, A., Prinz, W., Conrad, B., Siebner, H.R.: Linking actions and their perceivable consequences in the human brain. Neuroimage 17, 364–372 (2002)CrossRefGoogle Scholar
  16. 16.
    Fagioli, S., Hommel, B., Schubotz, R.I.: Intentional control of attention: Action planning primes action-related stimulus dimensions. Psychological Research 71 (2005)Google Scholar
  17. 17.
    Fasel, B., Luettin, J.: Automatic facial expression analysis: A survey. Patt. Recogn. 36, 259–275 (2003)CrossRefzbMATHGoogle Scholar
  18. 18.
    Gallese, V., Goldman, A.: Mirror neurons and the simulation theory of mind-reading. Trends Cogn. Sci. 2, 493–501 (1998)CrossRefGoogle Scholar
  19. 19.
    Gershenfeld, N., Krikorian, R., Cohen, D.: The Internet of Things. Scientific American 291(4), 76–81 (2004)CrossRefGoogle Scholar
  20. 20.
    Gibson, J.J.: The ecological approach to visual perception. Houghton Mifflin, Boston (1979)Google Scholar
  21. 21.
    Emile, G.J.: Marshalling passions in training and education. In: Proceedings of the Fourth International Conference on Autonomous Agents, Barcelona, Spain, pp. 325–332 (2000)Google Scholar
  22. 22.
    Haazebroek, P., Hommel, B.: Towards a Computational Model of Perception and Action in Human Computer Interaction. In: Duffy, V.G. (ed.) ICDHM 2009. LNCS, vol. 5620, pp. 247–256. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  23. 23.
    Hollan, J., Hutchins, E., Kirsh, D.: Distributed cognition: toward a new foundation for human-computer interaction research. ACM Trans. Computer-Human Interaction 7(2) (2000)Google Scholar
  24. 24.
    Hommel, B., Elsner, B.: Acquisition, representation, and control of action. In: Morsella, E., Bargh, J.A., Gollwitzer, P.M. (eds.) Oxford Handbook of Human Action, pp. 371–398. Oxford University Press, New York (2009)Google Scholar
  25. 25.
    Hommel, B., Muesseler, J., Aschersleben, G., Prinz, W.: The theory of event coding (TEC): A framework for perception and action planning. Behavioral and Brain Sciences 24, 849–937 (2001)CrossRefGoogle Scholar
  26. 26.
    Hopfinger, J.B., Buonocore, M.H., Mangun, G.R.: The neural mechanisms of top-down attentional control. Nature Neuroscience 3(3), 284–291 (2000)CrossRefGoogle Scholar
  27. 27.
    Horvitz, E.: Principles of mixed-initiative user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 1999, pp. 159–166. ACM, New York (1999)Google Scholar
  28. 28.
    Hudlicka, E.: To feel or not to feel: The role of affect in human-computer interaction. International Journal of Human-Computer Interaction 59, 1–32 (2003)Google Scholar
  29. 29.
    Ishii, H., Ullmer, B.: Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 234–241 (1997)Google Scholar
  30. 30.
    Jacob, R., Girouard, A., Hirshfield, L., Horn, M., Shaer, O., Solovey, E., Zigelbaum, J.: Reality-based interaction: a framework for post-WIMP interfaces. In: Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems, Florence, Italy, pp. 201–210 (2008)Google Scholar
  31. 31.
    Jaimes, A., Sebe, N.: Multimodal human-computer interaction: A sur-vey. Comput. Vis. Image Underst. 108(1-2), 116–134 (2007)CrossRefGoogle Scholar
  32. 32.
    James, W.: The principles of psychology. Dover Publications, New York (1890)CrossRefGoogle Scholar
  33. 33.
    Janlert, L.-E.: Contextual strategies – notes for a theory of context. Technical report UMINF 02.23, Umeå University (2003); ISSN-0348-0542Google Scholar
  34. 34.
    Keysers, C., Gazzola, V.: Integrating simulation and theory of mind: From self to social cognition. Trends Cogn. Sci. 11, 194–196 (2007)CrossRefGoogle Scholar
  35. 35.
    Kieras, D.E., Meyer, D.E.: An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. Human-Computer Interaction 12, 391–438 (1997)CrossRefGoogle Scholar
  36. 36.
    Kirsh, D., Maglio, P.: On Distinguishing Epistemic from Pragmatic Ac-tion. Cognitive Science 18, 513–549 (1994)CrossRefGoogle Scholar
  37. 37.
    Kisacanin, B., Pavlovic, V., Huang, T.S. (eds.): Real-Time Vision for Human-Computer Interaction. Springer, New York (2005)Google Scholar
  38. 38.
    Kotovsky, L., Gentner, D.: Comparison and categorization in the development of relational similarity. Child Development 67, 2797–2822 (1996)CrossRefGoogle Scholar
  39. 39.
    Kuutti, K.: Activity Theory as a potential framework for human-computer interaction research. In: Nardi, B. (ed.) Context and Consciousness. ch. 2, pp. 17–44 (1995)Google Scholar
  40. 40.
    Langley, P., Choi, D.: A unified cognitive architecture for physical agents. In: Proceedings of the Twenty-First National Conference on Artificial Intelligence (2006)Google Scholar
  41. 41.
    Levy, P.: Collective Intelligence: Mankind’s Emerging World in Cyber-space. Perseus Books (1997)Google Scholar
  42. 42.
    Lindsay, P., Norman, D.: Human Information Processing: An Introduction to Psychology (1977)Google Scholar
  43. 43.
    Mack, A., Tang, B., Tuma, R., Kahn, S., Rock, I.: Perceptual organization and attention. Cognitive Psychology 24 (1992)Google Scholar
  44. 44.
    Milgram, P., Takemura, H., Utsumi, A., Kishino, F.: Augmented reality: A class of displays on the reality-virtuality continuum. In: Proc. SPIE Conf. Telemanipulator and Telepresence Technologies, vol. 2351(34), pp. 282–292 (1994)Google Scholar
  45. 45.
    Moore, C.M., Egeth, H.: Perception without Attention: Evidence of Grouping under conditions of Inattention. Journal of Experimental Psychology: Human Perception and Performance 23(2), 339–352 (1997)Google Scholar
  46. 46.
    Nagel, T.: The view from nowhere. Oxford Univ. Press, New York (1986)Google Scholar
  47. 47.
    Newell, A.: Unified theories of cognition. Harvard University Press, Cambridge (1990)Google Scholar
  48. 48.
    Norman, D., Draper, S. (eds.): User centered system design. Erlbaum, Hillsdale (1986)Google Scholar
  49. 49.
    Oudeyer, P.Y.: The production and recognition of emotions in speech: Features and algorithms. Int. J. of Human-Computer Studies 59(1-2), 157–183 (2003)CrossRefGoogle Scholar
  50. 50.
    Parkhurst, D., Law, K., Niebur, E.: Modeling the role of salience in the allocation of overt visual attention. Vision Research 42(1), 107–123 (2002)CrossRefGoogle Scholar
  51. 51.
    Pederson, T.: From Conceptual Links to Causal Relations — Physical- Virtual Artefacts in Mixed-Reality Space, PhD thesis, Dept. of Computing Science, Umeå university, report UMINF-03.14 (2003)Google Scholar
  52. 52.
    Pederson, T., Janlert, L.-E., Surie, D.: Setting the Stage for Mobile Mixed-Reality Computing - A Situative Space Model based on Human Perception. IEEE Pervasive Computing (June 2010)Google Scholar
  53. 53.
    Schmidt, A.: Implicit human computer interaction through context. Personal and Ubiquitous Computing 4(2-3), 191–199 (2000)Google Scholar
  54. 54.
    Schubotz, R.I., von Cramon, D.Y.: Predicting perceptual events activates corresponding motor schemes in lateral premotor cortex: An fMRI study. Neuroimage 15, 787–796 (2002)CrossRefGoogle Scholar
  55. 55.
    Selkar, T., Burlesson, W.: Context-aware design and interaction in com-puter systems. IBM Syst. J. 39(3-4), 880–891 (2000)CrossRefGoogle Scholar
  56. 56.
    Shneiderman, B.: Direct manipulation: a step beyond programming languages. Computer 16, 57–69 (1983)CrossRefGoogle Scholar
  57. 57.
    Simon, J., Rudell, A.: Auditory s-r compatibility: The effect of an irrelevant cue on information processing. Journal of Applied Psychology 51, 300–304 (1967)CrossRefGoogle Scholar
  58. 58.
    Sloman, A.: Varieties of affect and the CogAff architecture schema. In: Proceedings of the AISB 2001 Symposium on Emotion, Cognition, and Affective Computing, York, UK (2001)Google Scholar
  59. 59.
    Surie, D., Jäckel, F., Janlert, L.-E., Pederson, T.: Situative Space Tracking within Smart Environments. In: 6th International Conference on Intelligent Environments, Malaysia (July 2010)Google Scholar
  60. 60.
    Surie, D., Lagriffoul, F., Pederson, T., Sjölie, D.: Activity Recognition Based on Intra and Extra Manipulation of Everyday Objects. In: Ichikawa, H., Cho, W.-D., Chen, Y., Youn, H.Y. (eds.) UCS 2007. LNCS, vol. 4836, pp. 196–210. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  61. 61.
    Surie, D., Laguionie, O., Pederson, T.: Wireless Sensor Networking of Eve-ryday Objects in a Smart Home Environment. In: Proceedings of the 4th International Conference on Intelligent Sensors, Sensor Networks and Information Processing, Sydney, Australia (2008)Google Scholar
  62. 62.
    Surie, D., Pederson, T.: An Activity-Centered Wearable Computing Infrastructure for Intelligent Environment Applications. In: Kuo, T.-W., Sha, E., Guo, M., Yang, L.T., Shao, Z. (eds.) EUC 2007. LNCS, vol. 4808, pp. 456–465. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  63. 63.
    Surie, D., Pederson, T., Janlert, L.-E.: The easy ADL home: A physical-virtual approach to domestic living. Journal of Ambient Intelligence and Smart Environments 2(3), 287–310 (2010)Google Scholar
  64. 64.
    Surie, D., Pederson, T., Lagriffoul, F., Janlert, L.-E., Sjölie, D.: Activity Recognition Using an Egocentric Perspective of Everyday Objects. In: Indulska, J., Ma, J., Yang, L.T., Ungerer, T., Cao, J. (eds.) UIC 2007. LNCS, vol. 4611, pp. 246–257. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  65. 65.
    Thorndike, E.L.: The law of effect. American Journal of Psychology 39, 212–222 (1927)CrossRefGoogle Scholar
  66. 66.
    Turk, M., Robertson, G.: Perceptual User Interfaces. Communications of the ACM (2000)Google Scholar
  67. 67.
    Vertegaal, R.: Attentive user interfaces. Special Issue of Communica-tions of the ACM 46(3) (2003)Google Scholar
  68. 68.
    Weiser, M.: The Computer for the 21st Century. Scientific American, 933–940 (1991)Google Scholar
  69. 69.
    Weiser, M., Brown, J.S.: The coming age of calm technology. In: Beyond Calculation, Copernicus, New York, NY, USA, pp. 75–85 (1997)Google Scholar
  70. 70.
    Wilson, M.: Six views of embodied cognition. Psychonomic Bulletin and Review 9, 625–636 (2002)CrossRefGoogle Scholar
  71. 71.
    Yarbus, A.: Eye Movements and Vision. Plenum Press, New York (1967)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Dipak Surie
    • 1
  • Thomas Pederson
    • 2
  • Lars-Erik Janlert
    • 1
  1. 1.Dept. of Computing ScienceUmeå UniversityUmeaSweden
  2. 2.IT University of CopenhagenCopenhagenDenmark

Personalised recommendations