Hand grasp is a complex system that plays an important role in the activities of daily living. Upper-limb neuroprostheses aim at restoring lost reaching and grasping functions on people suffering from neural disorders. However, the dimensionality and complexity of the upper-limb makes the neuroprostheses modeling and control challenging. In this work we present preliminary results for checking the feasibility of using a reinforcement learning (RL) approach for achieving grasp functions with a surface multi-field neuroprosthesis for grasping. Grasps from 20 healthy subjects were recorded to build a reference for the RL system and then two different award strategies were tested on simulations based on neuro-fuzzy models of hemiplegic patients. These first results suggest that RL might be a possible solution for obtaining grasp function by means of multi-field neuroprostheses in the near future.


Neuroprostheses Functional electrical stimulation Grasp Reinforcement learning Modeling and control 



Authors would like to thank to Intelligent Control Research Group of UPV/EHU for giving the means of carrying out this work and to the Health Division of Tecnalia for its continuous support.


  1. 1.
    Jones, L.A., Lederman, S.J.: Human Hand Function. Oxford University Press, Oxford (2006)CrossRefGoogle Scholar
  2. 2.
    Schieber, M.H., Santello, M.: Hand function: peripheral and central constraints on performance. J. Appl. Physiol. 96, 2293–2300 (2004)CrossRefGoogle Scholar
  3. 3.
    Ingram, J.N., Kording, K.P., Howard, I.S., Wolpert, D.M.: The statistics of natural hand movements. Exp. Brain Res. 188, 223–236 (2008)CrossRefGoogle Scholar
  4. 4.
    Castiello, U.: The neuroscience of grasping. Nat. Rev. Neurosci. 6, 726–736 (2005)CrossRefGoogle Scholar
  5. 5.
    Roh, J., Rymer, W.Z., Perreault, E.J., Yoo, S.B., Beer, R.F.: Alterations in upper limb muscle synergy structure in chronic stroke survivors. J. Neurophysiol. 109, 768–781 (2013)CrossRefGoogle Scholar
  6. 6.
    Nichols-Larsen, D.S., Clark, P.C., Zeringue, A., Greenspan, A., Blanton, S.: Factors influencing stroke survivors? quality of life during subacute recovery. Stroke 36, 1480–1484 (2005)CrossRefGoogle Scholar
  7. 7.
    Peckham, P.H., Knutson, J.S.: Functional electrical stimulation for neuromuscular applications. Annu. Rev. Biomed. Eng. 7, 327–360 (2005)CrossRefGoogle Scholar
  8. 8.
    Popović, D.B.: Advances in functional electrical stimulation (FES). J. Electromyogr. Kinesiol. 24, 795–802 (2014)CrossRefGoogle Scholar
  9. 9.
    Keller, T., Kuhn, A.: Electrodes for transcutaneous (surface) electrical stimulation. J. Autom. Control. 18, 35–45 (2008)CrossRefGoogle Scholar
  10. 10.
    Westerveld, A.J., Schouten, A.C., Veltink, P.H., van der Kooij, H.: Selectivity and resolution of surface electrical stimulation for grasp and release. IEEE Trans. Neural Syst. Rehabil. Eng. 20, 94–101 (2012)CrossRefGoogle Scholar
  11. 11.
    Rau, G., Disselhorst-Klug, C., Schmidt, R.: Movement biomechanics goes upwards: from the leg to the arm. J. Biomech. 33, 1207–1216 (2000)CrossRefGoogle Scholar
  12. 12.
    Jagodnik, K.M., Blana, D., van den Bogert, A.J., Kirsch, R.F.: An optimized proportional-derivative controller for the human upper extremity with gravity. J. Biomech. 48, 3692–3700 (2015)CrossRefGoogle Scholar
  13. 13.
    Ethier, C., Oby, E.R., Bauman, M.J., Miller, L.E.: Restoration of grasp following paralysis through brain-controlled stimulation of muscles. Nature 485, 368–371 (2012)CrossRefGoogle Scholar
  14. 14.
    Popovic, M.B.: Control of neural prostheses for grasping and reaching. Med. Eng. Phys. 25, 41–50 (2003)CrossRefGoogle Scholar
  15. 15.
    Popovic, D.B., Popovic, M.B.: Automatic determination of the optimal shape of a surface electrode: selective stimulation. J. Neurosci. Meth. 178, 174–181 (2009)CrossRefGoogle Scholar
  16. 16.
    Freeman, C., Rogers, E., Burridge, J.H., Hughes, A., Meadmore, K.: Iterative Learning Control for Electrical Stimulation and Stroke Rehabilitation. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  17. 17.
    Wright, T.W., Glowczewskie, F., Cowin, D., Wheeler, D.L.: Radial nerve excursion and strain at the elbow and wrist associated with upper-extremity motion. J. Hand Surg. 30, 990–996 (2005)CrossRefGoogle Scholar
  18. 18.
    Gillen, G.: Stroke Rehabilitation: A Function-Based Approach. Elsevier Health Sciences, St. Louis (2015)Google Scholar
  19. 19.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT press, Cambridge (1998)Google Scholar
  20. 20.
    Izawa, J., Kondo, T., Ito, K.: Biological arm motion through reinforcement learning. Biol. Cybern. 91, 10–22 (2004)CrossRefzbMATHGoogle Scholar
  21. 21.
    Imatz-Ojanguren, E., Irigoyen, E., Valencia-Blanco, D., Keller, T.: Neuro-fuzzy models for hand movements induced by functional electrical stimulation in able-bodied and hemiplegic subjects. Med. Eng. Phys. (2016). (In Press)Google Scholar
  22. 22.
    Yozbatiran, N., Der-Yeghiaian, L., Cramer, S.C.: A standardized approach to performing the action research arm test. Neurorehab. Neural Repair 22, 78–90 (2008)CrossRefGoogle Scholar
  23. 23.
    Helwig, N.E., Hong, S., Hsiao-Wecksler, E.T., Polk, J.D.: Methods to temporally align gait cycle data. J. Biomech. 44, 561–566 (2011)CrossRefGoogle Scholar
  24. 24.
    Vergara, M., Sancho-Bru, J.L., Gracia-Ibanez, V., Perez-Gonzalez, A.: An introductory study of common grasps used by adults during performance of activities of daily living. J. Hand Ther. 27, 225–234 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Eukene Imatz-Ojanguren
    • 1
    • 2
    Email author
  • Eloy Irigoyen
    • 1
  • Thierry Keller
    • 2
  1. 1.Intelligent Control Research GroupUPV/EHU - University of the Basque CountryBilbaoSpain
  2. 2.TECNALIA Research and InnovationDonostia-San SebastiánSpain

Personalised recommendations