Cognitive Computation

, Volume 6, Issue 4, pp 775–788 | Cite as

Therapist-Centered Design of a Robot’s Dialogue Behavior

  • Milan GnjatovićEmail author


Significant research effort has already been invested in the field of robot-assisted therapy for children with developmental disorders, and the researchers generally agree that therapists should be involved in the development of assistive robotic tools. However, relatively little attention has been devoted to robots’ capacity to autonomously engage in a natural language dialogue in the context of robot-assisted therapy. This paper focuses on this desideratum. It introduces a programming platform that enables the therapist to design a robot’s dialogue behavior. To the extent that the platform is domain-independent, it enables the therapist to flexibly model (1) the interaction domain and the lexicon, (2) the interaction context, and (3) the robot’s dialogue strategy. To the extent that the platform is therapist-centered, it is motivated by real-life difficulties that therapists encounter while trying to specify a robot’s dialogue behavior and can be used by nontechnical therapists in a user-friendly and intuitive manner. In addition, the platform (4) enables the therapist to test dialogue strategies independently of therapeutic settings, and (5) provides estimated cognitive load placed on the child while trying to process the therapist’s dialogue acts.


Robot-assisted therapy Human–machine interaction Therapist-centered design Context modeling Dialogue behavior modeling Focus tree 



The presented study is performed as part of the projects “Design of Robots as Assistive Technology for the Treatment of Children with Developmental Disorders” (III44008) and “Development of Dialogue Systems for Serbian and Other South Slavic Languages” (TR32035), funded by the Ministry of Education, Science, and Technological Development of the Republic of Serbia. The responsibility for the content of this paper lies with the author.


  1. 1.
    Colton MB, Ricks DJ, Goodrich MA, Dariush B, Fujimura K, Fujiki M. Toward therapist-in-the-loop assistive robotics for children with autism and specific language impairment. In: Proceedings of the AISB 2009 symposium on new frontiers in human–robot interaction, Edinburgh, Scotland; 2009. 5 pp.Google Scholar
  2. 2.
    Dipietro L, Krebs HI, Volpe BT, Stein J, Bever C, Mernoff ST, Fasoli SE, Hogan N. Learning, not adaptation, characterizes stroke motor recovery: evidence from kinematic changes induced by robot-assisted therapy in trained and untrained task in the same workspace. IEEE Trans Neural Syst Rehabil Eng. 2012;20(1):48–57.PubMedCrossRefGoogle Scholar
  3. 3.
    Kwakkel G, Kollen BJ, Krebs HI. Effects of robot-assisted therapy on upper limb recovery after stroke: a systematic review. Neurorehabil Neural Repair. 2008;22(2):111–21.PubMedCentralPubMedCrossRefGoogle Scholar
  4. 4.
    Johnson MJ. Recent trends in robot-assisted therapy environments to improve real-life functional performance after stroke. J NeuroEng Rehabil. 2006;3:29–34.PubMedCentralPubMedCrossRefGoogle Scholar
  5. 5.
    Blázquez MP. Clinical application of robotics in children with cerebral palsy. In: Pons JL, Torricelli D, Pajaro M, editors. Converging clinical and engineering research on neurorehabilitation. Biosystems and biorobotics, vol. 1. Berlin: Springer; 2013. p. 1097–102.CrossRefGoogle Scholar
  6. 6.
    Krebs HI, Ladenheim B, Hippolyte C, Monterroso L, Mast J. Robot-assisted task-specific training in cerebral palsy. Dev Med Child Neurol. 2009;51(4):140–5.PubMedCrossRefGoogle Scholar
  7. 7.
    Belokopytov M, Fridin M. Motivation of children with cerebral palsy during motor involvement by RAC-CP Fun. In: Proceedings of the workshop on motivational aspects of robotics in physical therapy, IEEE/RSJ international conference on intelligent robots and systems. Vilamoura, Algarve, Portugal; 2012. 6 pp.Google Scholar
  8. 8.
    Thill S, Pop CA, Belpaeme T, Ziemke T, Vanderborght B. Robot-assisted therapy for autism spectrum disorders with (partially) autonomous control: challenges and outlook. Paladyn. 2012;3(4):209–17.CrossRefGoogle Scholar
  9. 9.
    Ricks DJ, Colton MB. Trends and considerations in robot-assisted autism therapy. In: Proceedings of the IEEE international conference on robotics and automation (ICRA), Anchorage, AK; 2010. p. 4354–9.Google Scholar
  10. 10.
    Dautenhahn K, Nehaniv CL, Walters ML, Robins B, Kose-Bagci H, Mirza NA, Blow M. KASPAR—a minimally expressive humanoid robot for human-robot interaction research. Special issue on “humanoid robots”. Appl Bionics Biomech. 2009;6(3):369–97.CrossRefGoogle Scholar
  11. 11.
    Vanderborght B, Simut R, Saldien J, Pop C, Rusu AS, Pintea S, Lefeber D, David DO. Using the social robot probo as a social story telling agent for children with ASD. Interact Stud. 2012;13(3):348–72.CrossRefGoogle Scholar
  12. 12.
    Faundez-Zanuy M, Hussain A, Mekyska J, Sesa-Nogueras E, Monte-Moreno E, Esposito A, Chetouani M, Garre-Olmo J, Abel A, Smekal Z, Lopez-de-Ipiña K. Biometric applications related to human beings: there is life beyond security. Cogn Comput. 2013;5(1):136–51.CrossRefGoogle Scholar
  13. 13.
    López-de-Ipiña K, Alonso JB, Solé-Casals J, Barroso N, Henriquez P, Faundez-Zanuy M, Travieso CM, Ecay-Torres M, Martínez-Lage P, Eguiraun H. On automatic diagnosis of Alzheimer’s disease based on spontaneous speech analysis and emotional temperature. Cogn Comput (Online First) 2013.Google Scholar
  14. 14.
    Gómez-Vilda P, Rodellar-Biarge V, Nieto-Lluis V, Muñoz-Mulas C, Mazaira-Fernández LM, Martínez-Olalla R, Álvarez-Marquina A, Ramírez-Calvo C, Fernández-Fernández M. Characterizing neurological disease from voice quality biomechanical analysis. Cogn Comput. 2013;5(4):399–425.CrossRefGoogle Scholar
  15. 15.
    Blanco JL, Hernández LA, Fernández R, Ramos D. Improving automatic detection of obstructive sleep apnea through nonlinear analysis of sustained speech. Cogn Comput. 2013;5(4):458–72.CrossRefGoogle Scholar
  16. 16.
    Bickmore TW, Picard RW. Establishing and maintaining long-term human–computer relationships. ACM Trans Comput Human Interact. 2005;12(2):293–327.CrossRefGoogle Scholar
  17. 17.
    Fasola J, Matarić MJ. Using socially assistive human–robot interaction to motivate physical exercise for older adults. Proc IEEE. 2012;100(8):2512–26.CrossRefGoogle Scholar
  18. 18.
    Barakova EI. Robots for social training of autistic children empowering the therapists in intensive training programs. In: Proceedings of the 2011 world congress on information and communication technologies, WICT; 2011. p. 14–9.Google Scholar
  19. 19.
    Barakova EI, Gillesen JCC, Huskens BEBM, Lourens T. End-user programming architecture facilitates the uptake of robots in social therapies. Robot Autonom Syst. 2013;61:704–13.CrossRefGoogle Scholar
  20. 20.
    Anacleto J, Silvestre R, Filho CS, Santana B, Fels S. Therapist-centred design of NUI based therapies in a neurological care hospital. In: The proceedings of the 2012 IEEE international conference on systems, man, and cybernetics, COEX, Seoul, Korea; 2012. p. 2318–23.Google Scholar
  21. 21.
    Kelleher C, Tam S, May M, Profitt R, Engsberg J. Towards a therapist-centered programming environment for creating rehabilitation games. In: Proceedings of IEEE computer games; 2011. p. 240–7.Google Scholar
  22. 22.
    Gnjatović M, Delić V. End-user design of emotion-adaptive dialogue strategies for therapeutic purposes. In: Bassis S, Esposito A, Morabito FC, editors. Recent research on neural networks models and applications. smart innovation, systems and technologies, vol 26. Proceedings of the 23rd workshop of the Italian Neural Networks Society (SIREN), Springer; 2014. p. 371–8.Google Scholar
  23. 23.
    Gnjatović M, Janev M, Delić V. Focus tree: modeling attentional information in task-oriented human–machine interaction. Appl Intell. 2012;37(3):305–20.CrossRefGoogle Scholar
  24. 24.
    Gnjatović M, Delić V. A cognitively-inspired method for meaning representation in dialogue systems. In: Proceedings of the 3rd IEEE international conference on cognitive infocommunications, Kosice, Slovakia; 2012. p. 383–8.Google Scholar
  25. 25.
    Gnjatović M, Delić V. Toward computational modeling of the comprehension deficit in Broca’s aphasia. In: Proceedings of the 8th conference on language technologies, IS-JT 2012, Ljubljana, Slovenia; 2012. p. 79–84.Google Scholar
  26. 26.
    Gnjatović M, Delić V. Electrophysiologically-inspired evaluation of dialogue act complexity. In: Proceedings of the 4th IEEE international conference on cognitive infocommunications. Budapest, Hungary; 2013, p. 167–72.Google Scholar
  27. 27.
    Grodzinsky Y. The neurology of syntax: language use without Broca’s area. Behav Brain Sci. 2000;23(1):1–21.PubMedCrossRefGoogle Scholar
  28. 28.
    Dronkers NF, Plaisant O, Iba-Zizen MT, Cabanis EA. Paul Broca’s historic cases: high resolution MR imaging of the brains of Leborgne and Lelong. Brain. 2007;130(5):1432–41.PubMedCrossRefGoogle Scholar
  29. 29.
    Caramazza A, Zurif E. Dissociation of algorithmic and heuristic processes in language comprehension: evidence from aphasia. Brain Lang. 1976;3:572–82.PubMedCrossRefGoogle Scholar
  30. 30.
    Oberauer K. Access to information in working memory: exploring the focus of attention. J Exp Psychol Learn Mem Cogn. 2002;28(3):411–21.PubMedCrossRefGoogle Scholar
  31. 31.
    Oberauer K, Lange EB. Activation and binding in verbal working memory: a dual-process model for the recognition of nonwords. Cogn Psychol. 2009;58(1):102–36.PubMedCrossRefGoogle Scholar
  32. 32.
    Cowan N. Evolving conceptions of memory storage selective attention, and their mutual constraints within the information processing system. Psychol Bull. 1988;104(2):163–91.PubMedCrossRefGoogle Scholar
  33. 33.
    Bledowski C, Kaiser J, Rahm B. Basic operations in working memory: contributions from functional imaging studies. Behav Brain Res. 2010;214(2):172–9.PubMedCrossRefGoogle Scholar
  34. 34.
    Brouwer H, Fitz H, Hoeks J. Getting real about semantic illusions: rethinking the functional role of the P600 in language comprehension. Brain Res. 2012;1446:127–43.PubMedCrossRefGoogle Scholar
  35. 35.
    Gibson E. Linguistic complexity: locality of syntactic dependencies. Cognition. 1998;68:1–76.PubMedCrossRefGoogle Scholar
  36. 36.
    Kutas M, Federmeier KD. Electrophysiology reveals semantic memory use in language comprehension. Trends Cogn Sci. 2000;4(12):463–70.PubMedCrossRefGoogle Scholar
  37. 37.
    Rosenbaum P, Paneth N, Leviton A, Goldstein M, Bax M, Damiano D, Dan B, Jacobsson B. A report: the definition and classification of cerebral palsy April 2006. Dev Med Child Neurol Suppl. 2007;109:8–14.PubMedGoogle Scholar
  38. 38.
    Gnjatović M, Tasevski J, Nikolić M, Mišković D, Borovac B, Delić V. Adaptive multimodal interaction with industrial robot. In: Proceedings of the 10th IEEE jubilee international symposium on intelligent systems and informatics. Subotica, Serbia; 2012. p. 329–33.Google Scholar
  39. 39.
    Covington MA, He C, Brown C, Naçi L, Brown J. How complex is that sentence? A proposed revision of the Rosenberg and Abbeduto D-Level Scale. CASPR research report 2006-01. The University of Georgia, Artificial Intelligence Center, Athens, GA; 2006.Google Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Faculty of Technical SciencesUniversity of Novi SadNovi SadSerbia
  2. 2.Graduate School of Computer SciencesMegatrend UniversityBelgradeSerbia

Personalised recommendations