Advances in Intelligent Mobility Assistance Robot Integrating Multimodal Sensory Processing

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8515)


Mobility disabilities are prevalent in our ageing society and impede activities important for the independent living of elderly people and their quality of life. The goal of this work is to support human mobility and thus enforce fitness and vitality by developing intelligent robotic platforms designed to provide user-centred and natural support for ambulating in indoor environments. We envision the design of cognitive mobile robotic systems that can monitor and understand specific forms of human activity, in order to deduce what the human needs are, in terms of mobility. The goal is to provide user and context adaptive active support and ambulation assistance to elderly users, and generally to individuals with specific forms of moderate to mild walking impairment.

To achieve such targets, a reliable multimodal action recognition system needs to be developed, that can monitor, analyse and predict the user actions with a high level of accuracy and detail. Different modalities need to be combined into an integrated action recognition system. This paper reports current advances regarding the development and implementation of the first walking assistance robot prototype, which consists of a sensorized and actuated rollator platform. The main thrust of our approach is based on the enhancement of computer vision techniques with modalities that are broadly used in robotics, such as range images and haptic data, as well as on the integration of machine learning and pattern recognition approaches regarding specific verbal and non-verbal (gestural) commands in the envisaged (physical and non-physical) human-robot interaction context.


Hide Markov Model Stance Phase Gait Cycle Double Support Human Action Recognition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Parkinson Disease Foundation. Statistics for parkinson’s disease (2010)Google Scholar
  2. 2.
    Stroke Center. Stroke statistics (2010)Google Scholar
  3. 3.
    USCensus. The elderly population (2010)Google Scholar
  4. 4.
    Machiel Van der Loos, H.F., Reinkensmeyer, D.J.: Rehabilitation and health care robotics. In: Siciliano, B., Khatib, O. (eds.) Springer Handbook of Robotics, pp. 1223–1251. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  5. 5.
    Martins, M.M., Santos, C.P., Frizera-Neto, A., Ceres, R.: Assistive mobility devices focusing on smart walkers: Classification and review. Robotics and Autonomous Systems 60(4), 548–562 (2012)CrossRefGoogle Scholar
  6. 6.
    Ko, C.-H., Agrawal, S.K.: Walk-assist robot: A novel approach to gain selection of a braking controller using differential flatness. In: American Control Conference (ACC), pp. 2799–2804 (2010)Google Scholar
  7. 7.
    Hirata, Y., Hara, A., Kosuge, K.: Motion control of passive intelligent walker using servo brakes. IEEE Transactions on Robotics 23(5), 981–990 (2007)CrossRefGoogle Scholar
  8. 8.
    Kulyukin, V., Kutiyanawala, A., LoPresti, E., Matthews, J., Simpson, R.: Iwalker: Toward a rollator-mounted wayfinding system for the elderly. In: 2008 IEEE International Conference on RFID, pp. 303–311 (2008)Google Scholar
  9. 9.
    Wasson, G., Sheth, P., Alwan, M., Granata, K., Ledoux, A.: Cunjun Huang. User intent in a shared control framework for pedestrian mobility aids. In: Proceedings of the 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003), vol. 3, pp. 2962–2967 (2003)Google Scholar
  10. 10.
    Lee, G., Jung, E.-J., Ohnuma, T., Chong, N.Y., Yi, B.-J.: Jaist robotic walker control based on a two-layered kalman filter. In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 3682–3687 (2011)Google Scholar
  11. 11.
    Graf, B., Hans, M., Schraft, R.D.: Care-o-bot ii—development of a next generation robotic home assistant. Autonomous Robots 16(2), 193–205 (2004)Google Scholar
  12. 12.
    D’Angelo, L.T., Loercher, A., Lueth, T.C.: A new electrically driven walking frame for both passive and active mobility support. In: 2011 IEEE Conference on Automation Science and Engineering (CASE), pp. 816–821 (2011)Google Scholar
  13. 13.
    Ralston, H.J., Todd, F., Inman, V.T.: Human walking. Williams & Wilkins, Baltimore (1981)Google Scholar
  14. 14.
    Perry, J.: Gait Analysis: Normal and Pathological Function. Slack Incorporated (1992)Google Scholar
  15. 15.
    O’Connor, J.C., Vaughan, C.L., Davis, B.L.: Dynamics of Human Gait. Human Kinetics Publishers (1992)Google Scholar
  16. 16.
    Cochran, G.V.B.: A primer of orthopaedic biomechanics. Churchhill Livingstone, New York (1982)Google Scholar
  17. 17.
    Rabiner, L., Juang, B.: An introduction to hidden markov models. IEEE ASSP Magazine 3(1), 4–16 (1986)CrossRefGoogle Scholar
  18. 18.
    Rabiner, L.R.: Readings in speech recognition. chapter A tutorial on hidden Markov models and selected applications in speech recognition, pp. 267–296. Morgan Kaufmann Publishers Inc., San Francisco (1990)Google Scholar
  19. 19.
    Katsamanis, A., Papandreou, G., Maragos, P.: Audiovisual-to-articulatory speech inversion using active appearance models for the face and hidden markov models for the dynamics. In: ICASSP, pp. 2237–2240. IEEE (2008)Google Scholar
  20. 20.
    Yoon, B.-J.: Hidden markov models and their applications in biological sequence analysis. Curr. Genomics 10(6), 402–415 (2009)CrossRefGoogle Scholar
  21. 21.
    Theodorakis, S., Katsamanis, A., Maragos, P.: Product-hmms for automatic sign language recognition. In: ICASSP, pp. 1601–1604. IEEE (2009)Google Scholar
  22. 22.
    Turaga, P., Chellappa, R., Subrahmanian, V.S., Udrea, O.: Machine recognition of human activities: A survey. IEEE Transactions on Circuits and Systems for Video Technology 18(11), 1473–1488 (2008)CrossRefGoogle Scholar
  23. 23.
    Papageorgiou, X., Chalvatzaki, G., Tzafestas, C., Maragos, P.: Hidden markov modeling of human normal gait using laser range finder for a mobility assistance robot. In: Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA 2014) (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Inst. of Communication & Computer SystemsNational Technical Univ. of AthensGreece
  2. 2.INRIA Ecole Centrale ParisFrance
  3. 3.Technische Universität MünchenMunichGermany
  4. 4.ACCREA EngineeringLublinPoland
  5. 5.Institute for Language and Speech ProcessingATHENA RCAthensGreece

Personalised recommendations