Analysis and Design of Three Multimodal Interactive Systems to Support the Everyday Needs of Children with Cognitive Impairments

  • Stavroula Ntoa
  • Asterios Leonidis
  • Maria Korozi
  • Eleni Papadaki
  • Ilia Adami
  • George Margetis
  • Margherita AntonaEmail author
  • Constantine Stephanidis
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9177)


The autonomy and independence of users with cognitive impairments can be fostered through cognitive technologies. The use of traditional computer interfaces has however proved to be difficult for these users. This paper proposes three innovative systems to train children with cognitive impairments in three fundamental everyday life activities: (a) familiarizing with the home environments, its objects and activities; (b) learning about money and practicing shopping skills; and (c) learning how to prepare and cook simple meals. All three systems feature multimodal interaction and support multimedia output.


Multimodal interactive systems Children with cognitive impairments Card-based interaction Touch Cooking Monetary transactions Learning the home environment 



The authors would like to thank the Rehabilitation Centre for Children with Disabilities in Heraklion, Crete, for the valuable contribution during the requirements elicitation and the evaluation of the produced mockups and interactive prototypes. The graphics design for the “learning to cook” system was carried out by Elias Giannopoulos. The custom IR pointing device has been constructed by Elias Giannopoulos and Thanasis Toutountzis. Finally, the Content Editor and the Student Profiler have been implemented by Nikolaos Louloudakis, Evangelos Poutouris, Parthena Basina and Evropi Stephanidi.


  1. 1.
    Braddock, D., Rizzolo, M.C., Thompson, M., Bell, R.: Emerging technologies and cognitive disability. J. Spec. Educ. Technol. 19(4), 49–56 (2004)Google Scholar
  2. 2.
    Scherer, M.J.: Assistive Technologies and Other Supports for People with Brain Impairment, pp. 175–176. Springer Publishing Company, Heidelberg (2011)Google Scholar
  3. 3.
    Mechling, L.C.: Assistive technology as a self-management tool for prompting students with intellectual disabilities to initiate and complete daily tasks: A literature review. Education and Training in Developmental Disabilities, 252–269 (2007)Google Scholar
  4. 4.
    Lopresti, E.F., Mihailidis, A., Kirsch, N.: Assistive technology for cognitive rehabilitation: State of the art. Neuropsychological Rehabil. 14(1–2), 5–39 (2004)CrossRefGoogle Scholar
  5. 5.
    Lopresti, E.F., Bodine, C., Lewis, C.: Assistive technology for cognition Understanding the Needs of Persons with Disabilities. IEEE Eng. Med. Biol. Mag. 27(2), 29–39 (2008)CrossRefGoogle Scholar
  6. 6.
    Lancioni, G.E., O’Reilly, M.F., Seedhouse, P., Furniss, F., Cunha, B.: Promoting independent task performance by persons with severe developmental disabilities through a new computer-aided system. Behav. Modif. 24(5), 700–718 (2000)CrossRefGoogle Scholar
  7. 7.
    Mechling, L.C., Gast, D.L., Seid, N.H.: Using a personal digital assistant to increase independent task completion by students with autism spectrum disorder. J. Autism Dev. Disord. 39(10), 1420–1434 (2009)CrossRefGoogle Scholar
  8. 8.
    Giroux, S., Bauchet, J., Pigot, H., Lussier-Desrochers, D., Lachappelle, Y.: Pervasive behavior tracking for cognitive assistance. In: Proceedings of the 1st international conference on PErvasive Technologies Related to Assistive Environments, p. 86. ACM, July 2008Google Scholar
  9. 9.
    Langone, J., Clees, T.J., Rieber, L., Matzko, M.: The future of computer-based interactive technology for teaching individuals with moderate to severe disabilities: Issues relating to research and practice. J. Spec. Educ. Technol. 18(1), 5–16 (2003)Google Scholar
  10. 10.
    Ayres, K.M., Langone, J., Boon, R.T., Norman, A.: Computer-based instruction for purchasing skills. Educ. Training Dev. Disabil. 41(3), 253–263 (2006)Google Scholar
  11. 11.
    Hansen, D.L., Morgan, R.L.: Teaching grocery store purchasing skills to students with intellectual disabilities using a computer-based instruction program. Educ. Training Dev. Disabil. 43(4), 431–442 (2008)Google Scholar
  12. 12.
    Lanyi, C.S., Brown, D.J.: Design of serious games for students with intellectual disability. IHCI 10, 44–54 (2010)Google Scholar
  13. 13.
    Korozi, M., Leonidis, A., Margetis, G., Koutlemanis, P., Zabulis, X., Antona, M., Stephanidis, C.: Ambient educational mini-games. In: Proceedings of the International Working Conference on Advanced Visual Interfaces, pp. 802–803. ACM, May 2012Google Scholar
  14. 14.
    Margetis, G., Zabulis, X., Ntoa, S., Koutlemanis, P., Papadaki, E., Antona, M., Stephanidis, C.: Enhancing education through natural interaction with physical paper. Universal Access in the Information Society, 1–21Google Scholar
  15. 15.
    Margetis, G., Zabulis, X., Koutlemanis, P., Antona, M., Stephanidis, C.: Augmented interaction with physical books in an Ambient Intelligence learning environment. Multimed. Tools Appl. 67(2), 473–495 (2013)CrossRefGoogle Scholar
  16. 16.
    Norman, D.A.: The design of Everyday Things, pp. 13–28. Basic books, New York (2002)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Stavroula Ntoa
    • 1
  • Asterios Leonidis
    • 1
  • Maria Korozi
    • 1
  • Eleni Papadaki
    • 1
  • Ilia Adami
    • 1
  • George Margetis
    • 1
  • Margherita Antona
    • 1
    Email author
  • Constantine Stephanidis
    • 1
    • 2
  1. 1.Institute of Computer ScienceFoundation for Research and Technology – Hellas (FORTH)HeraklionGreece
  2. 2.Department of Computer ScienceUniversity of CreteCreteGreece

Personalised recommendations