Development and Validation of a Robust Speech Interface for Improved Human-Robot Interaction

  • Amin Atrash
  • Robert Kaplow
  • Julien Villemure
  • Robert West
  • Hiba Yamani
  • Joelle Pineau


Robotics technology has made progress on a number of important issues in the last decade. However many challenges remain when it comes to the development of systems for human-robot interaction. This paper presents a case study featuring a robust dialogue interface for human-robot communication onboard an intelligent wheelchair. Underlying this interface is a sophisticated software architecture which allows the chair to perform real-time, robust tracking of the dialogue state, as well as select appropriate responses using rich probabilistic representations. The paper also examines the question of rigorous validation of complex human-robot interfaces by evaluating the proposed interface in the context of a standardized rehabilitation task domain.


Intelligent wheelchair Dialogue management Service robotics Human-robot interaction 


  1. 1.
    Amori RD (1992) Vocomotionan intelligent voice-control system for powered wheelchairs. In: Proceedings of the RESNA 1992 annual conference, pp 421–423 Google Scholar
  2. 2.
    Beeson P, MacMahon M, Modayil J, Murarka A, Kuipers B, Stankiewicz B (2007) Integrating multiple representations of spatial knowledge for mapping, navigation, and communication. In: AAAI spring symposium series 2007, interaction challenges for intelligent assistants. AAAI Technical Report SS-07-04 Google Scholar
  3. 3.
    Clark JA, Roemer RB (1977) Voice controlled wheelchair. Arch Phys Med Rehabil 58(4):169–175 Google Scholar
  4. 4.
  5. 5.
    Collett THJ, MacDonald BA, Gerkey BP (2005) Player 2.0: Toward a practical robot programming framework. In: Proceedings of the Australasian conference on robotics and automation Google Scholar
  6. 6.
    Doshi F, Roy N (2007) Efficient model learning for dialog management. In: International conference on human-robot interaction, pp 65–72.
  7. 7.
    Fehr L, Langbein E, Skaar SB (2000) Adequacy of power wheelchair control interfaces for persons with severe disabilities: A clinical survey. J Rehabil Res Develop 37:353–360 Google Scholar
  8. 8.
    Felzer T, Freisleben B (2002) Hawcos: the “hands-free” wheelchair control system. In: Proceedings of the fifth international ACM conference on assistive technologies Google Scholar
  9. 9.
    Festival: Festival speech synthesis system (2004).
  10. 10.
    Gulati S, Kuipers B (2008) High performance control for graceful motion of an intelligent wheelchair. In: Proceedings of the IEEE international conference on robotics and automation (ICRA) Google Scholar
  11. 11.
    Hoey J, Poupart P, Boutilier C, Mihailidis A (2005) POMDP models for assistive technology. In: AAAI fall symposium on caring machines: AI in eldercare Google Scholar
  12. 12.
    HTK (2006) HTK 3.4.
  13. 13.
    Katevas NI, Sgouros NM, Tzafestas SG, Papakonstantinou G, Beattie P, Bishop JM, Tsanakas P, Koutsouris D (1997) The autonomous mobile robot scenario: A sensor-aided intelligent navigation system for powered wheelchairs. In: IEEE robotics and automation magazine, pp 60–70 Google Scholar
  14. 14.
    Kilkens OJE, Post MWM, Dallmeijer AJ, Seelen HAM, van der Woude LHV (2003) Wheelchair skills tests: A systematic review. Clin Rehabil 17:418–430 CrossRefGoogle Scholar
  15. 15.
    Kirby RL (2007) Wheelchair Skills Program (WSP), Version 4.1. Wheelchair Skills Test (WST) Manual. Accessed 04/27/2008
  16. 16.
    Kirby RL, Swuste J, Dupuis DJ, MacLeod DA, Monroe R (2002) The Wheelchair Skills Test: A pilot study of a new outcome measure. Arch Phys Med Rehabil 83:10–18 CrossRefGoogle Scholar
  17. 17.
    Kirby RL, Dupuis DJ, MacPhee AH, Coolen AL, Smith C, Best KL, Newton AM, Mountain AD, MacLeod DA, Bonaparte JP (2004) The Wheelchair Skills Test (version 2.4): Measurement properties. Arch Phys Med Rehabil 85:794–804 CrossRefGoogle Scholar
  18. 18.
    Koenig S, Simmons R (1996) Unsupervised learning of probabilistic models for robot navigation. In: International conference on robotics and automation Google Scholar
  19. 19.
    Levine SP, Bell DA, Jaros LA, Simpson RC, Koren Y, Borenstein J (1999) The navchair assistive wheelchair navigation system. IEEE Trans Rehabil Eng 7(4):443–451 CrossRefGoogle Scholar
  20. 20.
    McGuire WR (1999) Voice operated wheelchair using digital signal processing technology. In: Proceedings of the 22nd annual international conference on assistive technology for people with disabilities (RESNA), pp 364–366 Google Scholar
  21. 21.
    Miller GE, Brown TE, Randolph WR (1985) Voice controller for wheelchairs. Med Biol Eng Comput 23(6):597–600 CrossRefGoogle Scholar
  22. 22.
    Montemerlo M, Roy N, Thrun S (2003) Perspectives on standardization in mobile robot programming: The Carnegie Mellon navigation (CARMEN) toolkit. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS), vol 3, pp 2436–2441 Google Scholar
  23. 23.
    Pineau J, Atrash A (2007) SmartWheeler: A robotic wheelchair test-bed for investigating new models of human-robot interaction. In: AAAI spring symposium on multidisciplinary collaboration for socially assistive robotics, pp 59–64 Google Scholar
  24. 24.
    Pineau J, Gordon G, Thrun S (2003) Point-based value iteration: An anytime algorithm for POMDPs. In: Proceedings of the 18th international joint conference on artificial intelligence (IJCAI), pp 1025–1032 Google Scholar
  25. 25.
    Pires G, Nunes U (2002) A wheelchair steered through voice commands and assisted by a reactive fuzzy-logic controller. J Intell Robot Syst 34(3):301–314 zbMATHCrossRefGoogle Scholar
  26. 26.
    Routhier F, Vincent C, Desrosiers J, Nadeau S (2003) Mobility of wheelchair users: A proposed performance assessment framework. Disabil Rehabil 25:19–34 CrossRefGoogle Scholar
  27. 27.
    Roy N, Pineau J, Thrun S (2000) Spoken dialog management using probabilistic reasoning. In: Proceedings of the 38th annual meeting of the association for computational linguistics (ACL) Google Scholar
  28. 28.
    Simpson RC (2005) Smart wheelchairs: A literature review. J Rehabil Res Develop 42(4):423–436 CrossRefGoogle Scholar
  29. 29.
    Simpson RC, LoPresti EF, Cooper RA (2008) How many people would benefit from a smart wheelchair? J Rehabil Res Develop 45:53–72 CrossRefGoogle Scholar
  30. 30.
    Singh S, Litman D, Kearns M, Walker M (2002) Optimizing dialogue management with reinforcement learning: Experiments with the NJFun system. J Artif Intell Res (JAIR) 16:105–133 Google Scholar
  31. 31.
  32. 32.
    Sondik E (1971) The optimal control of partially observable Markov decision processes. PhD thesis, Stanford University Google Scholar
  33. 33.
    Steedman M, Baldridge J (2005) Combinatory categorial grammar. Blackwell Sci, Oxford.
  34. 34.
    Tapus A, Tapus C, Mataric M (2007) Hands-off therapist robot behavior adaptation to user personality for post-stroke rehabilitation therapy. In: International conference on robotics and automation, pp 1547–1553 Google Scholar
  35. 35.
    Tsui K, Yanco H (2007) Simplifying wheelchair mounted robotic arm control with a visual interface. In: AAAI spring symposium on multidisciplinary collaboration for socially assistive robots, pp 97–102 Google Scholar
  36. 36.
    Vertanen K (2006) HTK wall street journal acoustic models.
  37. 37.
    White M (2001) OpenCCG: The OpenNLP CCG library.
  38. 38.
    Williams J, Poupart P, Young S (2005) Partially observable Markov decision processes with continuous observations for dialogue management. In: SigDial workshop on discourse and dialogue, pp 393–422 Google Scholar
  39. 39.
    Wobbrock JO, Myers BA, Aung HH, LoPresti EF (2004) Text entry from power wheelchairs: EdgeWrite for joysticks and touchpads. In: Proceedings of the 6th international ACM SIGACCESS conference on computers and accessibility Google Scholar
  40. 40.
    Zalzal V (2004) Acropolis: Une architecture logicielle évolutive pour plate-formes mobiles autonomes: Principes de fonctionnement. Technical Report, École Polytechnique de Montréal Google Scholar

Copyright information

© Springer Science & Business Media BV 2009

Authors and Affiliations

  • Amin Atrash
    • 1
  • Robert Kaplow
    • 1
  • Julien Villemure
    • 1
  • Robert West
    • 1
  • Hiba Yamani
    • 1
  • Joelle Pineau
    • 1
  1. 1.School of Computer ScienceMcGill UniversityMontrealCanada

Personalised recommendations