Journal on Multimodal User Interfaces

, Volume 7, Issue 3, pp 229–245 | Cite as

AGNES: Connecting people in a multimodal way

  • Christian Peter
  • Andreas Kreiner
  • Martin Schröter
  • Hyosun Kim
  • Gerald Bieber
  • Fredrik Öhberg
  • Kei Hoshi
  • Eva L. Waterworth
  • John Waterworth
  • Soledad Ballesteros
Original Paper


Western societies are confronted with a number of challenges caused by the increasing number of older citizens. One important aspect is the need and wish of older people to live as long as possible in their own home and maintain an independent life. As people grew older, their social networks disperse, with friends and families moving to other parts of town, other cities or even countries. Additionally, people become less mobile with age, leading to less active participation in societal life. Combined, this normal, age-related development leads to increased loneliness and social isolation of older people, with negative effects on mental and physical health of those people. In the AGNES project, a home-based system has been developed that allows connecting elderly with their families, friends and other significant people over the Internet. As most older people have limited experience with computers and often special requirements on technology, one focus of AGNES was to develop with the users novel technological means for interacting with their social network. The resulting system uses ambient displays, tangible interfaces and wearable devices providing ubiquitous options for interaction with the network, and secondary sensors for additionally generating carefully chosen information on the person to be relayed to significant persons. Evaluations show that the chosen modalities for interaction are well adopted by the users. Further it was found that use of the AGNES system had positive effects on the mental state of the users, compared to the control group without the technology.


Ambient assistance Tangible interaction Wearable devices Aging Wellbeing Social network 


  1. 1.
    AGNES project web site (2012) Accessed 27 Feb 2012
  2. 2.
    Barklay L (2007) Loneliness increases risk for AD-like dementia. Arch Gen Psychiatry 64:234–240CrossRefGoogle Scholar
  3. 3.
    Cacciopo J, Hughes ME, Waite LJ, Howkley LC, Thisted RA (2006) Loneliness as a specific risk factor of depressive symptoms: cross-sectional and longitudinal analyses. Psychol Aging 21:140–151CrossRefGoogle Scholar
  4. 4.
    Fiske A, Jones RS (2005) Depression. Cognitive changes across lifespan. In: Johnson ML (ed) The Cambridge handbook of age and aging. Cambridge University Press, Cambridge, pp 245–251Google Scholar
  5. 5.
    Fratiglioni L, Wang HK, Mayan M, Winblad B (2000) Influence of social network on occurrence of dementia: a community-based longitudinal study. Lancet 355:1315–1319CrossRefGoogle Scholar
  6. 6.
    Park DC, Minear M (2004) Cognitive aging: new directions for old theories. In: Dixon RA, Backman L, Nilsson LG (eds) New frontiers in cognitive aging. Oxford University Press, New YorkGoogle Scholar
  7. 7.
    Rabbitt P (2005) Cognitive changes across lifespan. In: Johnson ML (ed) The Cambridge handbook of age and aging. Cambridge University Press, Cambridge, pp 190–199Google Scholar
  8. 8.
    Reblin M, Uchino BN (2008) Social and emotional support and its implication for health. Curr Opin Psychiatry 2:201–205CrossRefGoogle Scholar
  9. 9.
    Waterworth EL, Waterworth JA, Ballesteros S, Peter C (2012) Seniors in charge of new ICT innovation. The 5th ACM international conference on pervasive technologies related to assistive environments: PETRA (2012) CD-ROM. ACM, New YorkGoogle Scholar
  10. 10.
    Lepri B, Mana N, Cappelletti A, Pianesi F, Zancanaro M (2010) What is happening now? Detection of activities of daily living from simple visual features. Personal Ubiquitous Comput 4(8):749–766CrossRefGoogle Scholar
  11. 11.
    Patterson DJ et al (2005) Fine-grained activity recognition by aggregating abstract object usage. Proceedings of the 9th IEEE international symposium on wearable computers, IEEE CS Press, pp 44–51Google Scholar
  12. 12.
    Philipose M et al (2004) Inferring activities from interactions with objects. IEEE Pervasive Comput 3(4):50–57CrossRefGoogle Scholar
  13. 13.
    Bao L, Intille S (2004) Activity recognition from user-annotated acceleration data. In: Ferscha A, Mattern F (eds) Pervasive 2004, LNCS 3001. Springer, Berlin, pp 1–17Google Scholar
  14. 14.
    Bieber G, Voskamp J, Urban B (2009) Activity recognition for everyday life on mobile phones, LNCS 5615, Universal access in human-computer interaction, HCII 2009. Springer, San Francisco, USA, 2009, ISBN 3-642-02709-1Google Scholar
  15. 15.
    Choudhury T, Consolvo S, Harrison B, Hightower J, LaMarca A, LeGrand L, Rahimi A, Rea A, Borriello G, Hemmingway B, Klasnja P, Koscher K, Landay J, Wyatt D, Haehnel D (2008) The mobile sensing platform: an embedded activity recognition system. Pervasive Computing IEEE CS, pp 33–41Google Scholar
  16. 16.
    DeVaul RW, Dunn S (2001) Real-Time motion classification for wearable computing applications. Technical report, Media Laboratory, Massachusetts Institute of TechnologyGoogle Scholar
  17. 17.
    Lester J, Choudhury T, Borriello G, Consolvo S, Landay J, Everitt K, Smith I (2005) Sensing and modelling activities to support physical fitness, ubicomp workshop: monitoring, measuring and motivating exercise: ubiquitous computing to support fitnessGoogle Scholar
  18. 18.
    Maurer U et al (2006) Location and activity recognition using eWatch: a wearable sensor platform, ambient intelligence in every day life, LNCS 3864. Springer, Berlin, pp 86–100Google Scholar
  19. 19.
    Lester J, Choudhury T, Borriello G, Consolvo S, Landay J, Everitt K, Smith I (2005) Sensing and modelling activities to support physical fitness, ubicomp workshop: monitoring. ubiquitous computing to support fitness, measuring and motivating exerciseGoogle Scholar
  20. 20.
    Najafi B, Aminian K, Paraschiv-Ionescu A, Loew F, Buela C, Robert P (2003) Ambulatory system for human motion analysis using a kinematic sensor: monitoring of daily physical activity in the elderly. IEEE Trans Biomed Eng 50(6):711–23CrossRefGoogle Scholar
  21. 21.
    Ishii H (2008) Tangible bits: beyond pixels. Proceedings of the 2nd international conference on tangible and embedded interaction. ACM, Bonn, New York, pp xv–xxvGoogle Scholar
  22. 22.
    Ishii H, Ullmer B (1997) Tangible bits: towards seamless interfaces between people, bits and atoms. Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, Atlanta, New York, pp 234–241Google Scholar
  23. 23.
    Ullmer B, Ishii H (2000) Emerging frameworks for tangible user interfaces. IBM Syst J 39(3):915–931CrossRefGoogle Scholar
  24. 24.
    Abowd G, Mynatt E (2000) Charting past, present, and future research in ubiquitous computing. ACM Trans Comput-Hum Interact 7:29–58CrossRefGoogle Scholar
  25. 25.
    Brewer J, Williams A, Dourish P (2007) A handle on what’s going on: combining tangible interfaces and ambient displays for collaborative groups. Tangible and Embedded Interaction, Baton Rouge, LouisianaGoogle Scholar
  26. 26.
    Dahley A, Wisneski C, Ishii H (1998) Water lamp and pinwheels: ambient projection of digital information into architectural space. In: Proceedings of the CHI ’98, pp 269–270Google Scholar
  27. 27.
    Mankoff J, Dey AK, Hsieh G, Kientz J, Lederer S, Ames M (2003) Heuristic evaluation of ambient displays. In: Proceedings of CHI’03 human factors in computing systems, ACM press, pp 169–176Google Scholar
  28. 28.
    Weiser M, Brown JS (1995) Designing calm technology. Internet:
  29. 29.
    Benjamin LT Jr, Hopkins JR, Nation JR (1994) Psychology, 3rd edn. Macmillan College Publishing Company, New York, pp 250–290 Google Scholar
  30. 30.
    Schacter DJ (1987) Implicit memory: history and current status. J Exp Psychol 13(3):501–518Google Scholar
  31. 31.
    World Wide Web Consortium (2012) Multimodal interaction activity web site. Accessed 31 March 2012
  32. 32.
    World Wide Web Consortium (2012) Multimodal architecture and interfaces web site. Accessed 31 March 2012
  33. 33.
    World Wide Web Consortium (2012) EMMA: extensible multimodal annotation markup language web site. Accessed 31 March 2012
  34. 34.
    World Wide Web Consortium (2012) EmotionML: emotion markup language web site. Accessed 31 March 2012
  35. 35.
    Yesavage JA, Brink TL, Rose TL, Lum O, Huang V, Adey M et al (1982) Developement and validation of a geriatric depression screening scale: a preliminary report. J Psychiatr Res 17:37–49Google Scholar
  36. 36.
    Mahoney FI, Barthel D (1965) Functional evaluation: the Barthel Index. Md State Med J 14:56–61Google Scholar
  37. 37.
    Ballesteros S, Peter C, Waterworth EL, Waterworth JA (2012) The role of ICT and networking in healthy aging. In: The 5th ACM international conference on pervasive technologies related to assistive environments: PETRA (2012) CD-ROM. ACM, New YorkGoogle Scholar
  38. 38.
    Folstein MF, Folstein SE, McHugh PR (1975) “Mini-mental state”. A practical method for grading the cognitive state of patients for clinician. J Psychiatr Res 12:189–198Google Scholar
  39. 39.
    Nieboer A, Linderberg S, Boomsma A, van Bruggen A (2005) Dimensions of well-being and their measurement: the SPF-IL scale. Social Indicators Res 73:313–353Google Scholar

Copyright information

© OpenInterface Association 2013

Authors and Affiliations

  • Christian Peter
    • 1
    • 2
    • 3
  • Andreas Kreiner
    • 4
  • Martin Schröter
    • 1
  • Hyosun Kim
    • 1
  • Gerald Bieber
    • 2
  • Fredrik Öhberg
    • 5
  • Kei Hoshi
    • 6
  • Eva L. Waterworth
    • 6
  • John Waterworth
    • 6
  • Soledad Ballesteros
    • 7
  1. 1.Graz University of TechnologyGrazAusria
  2. 2.Fraunhofer IGDRostockGermany
  3. 3.Ambertree Assistance TechnologiesRostockGermany
  4. GmbHLinzAustria
  5. 5.Radiation SciencesBiomedical EngineeringUmeåSweden
  6. 6.Department of InformaticsUmeå UniversityUmeåSweden
  7. 7.Facultad de Psicologia, UNEDMadridSpain

Personalised recommendations