Skip to main content

Learning from the Human Hand: Force Control and Perception Using a Soft-Synergy Prosthetic Hand and Noninvasive Haptic Feedback

  • Chapter
  • First Online:
Advances in Motor Neuroprostheses
  • 650 Accesses

Abstract

Force control and perception plays an important role in activities of daily living when handling objects with different physical properties. These abilities are results of complex sensorimotor pathways that coordinate movements, predict consequences, and process feedback. For prosthetic systems, the ability to exhibit human-like action and perception behavior is critical for the acceptance of the terminal device. In this chapter, we review recent findings obtained from a bioinspired soft-synergy prosthetic hand and a noninvasive mechanotactile feedback device. A series of experiments demonstrated the improvement in force control and perception in closed-loop prosthesis through context-aware myoelectric controllers and contralateral haptic training protocols. By comparing performances between human native hands and prosthetic hands, we provide novel insights on the importance of learning from human sensorimotor mechanisms in the design of upper-limb neuroprosthesis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. S.B. Godfrey, A. Ajoudani, M.G. Catalano, G. Grioli, A. Bicchi, A synergy-driven approach to a myoelectric hand. IEEE Int. Conf. Rehabil. Robot. 2013, 1–6 (2013)

    Google Scholar 

  2. A. Ajoudani et al., Exploring teleimpedance and tactile feedback for intuitive control of the Pisa/IIT SoftHand. IEEE Trans. Haptics 7(2), 203–215 (2014)

    Article  PubMed  Google Scholar 

  3. N. Jiang, H. Rehbaum, I. Vujaklija, B. Graimann, D. Farina, Intuitive, online, simultaneous, and proportional myoelectric control over two degrees-of-freedom in upper limb amputees. IEEE Trans. Neural Syst. Rehabil. Eng. 22(3), 501–510 (2014)

    Article  PubMed  Google Scholar 

  4. F. Clemente, M. D’Alonzo, M. Controzzi, B.B. Edin, C. Cipriani, Non-invasive, temporally discrete feedback of object contact and release improves grasp control of closed-loop myoelectric transradial prostheses. IEEE Trans. Neural Syst. Rehabil. Eng. 24(12), 1314–1322 (2015)

    Article  PubMed  Google Scholar 

  5. M. Santello, M. Flanders, J.F. Soechting, Postural hand synergies for tool use. J. Neurosci. 18(23), 10105–10115 (1998)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. M. Santello, J.F. Soechting, Force synergies for multifingered grasping. Exp. Brain Res. 133(4), 457–467 (2000)

    Article  CAS  PubMed  Google Scholar 

  7. E.J. Weiss, M. Flanders, Muscular and postural synergies of the human hand. J. Neurophysiol. 92(1), 523–535 (2004)

    Article  PubMed  Google Scholar 

  8. M. Santello, G. Baud-Bovy, H. Jörntell, Neural bases of hand synergies. Front. Comput. Neurosci. 7, 23 (2013)

    Article  PubMed  PubMed Central  Google Scholar 

  9. M. Santello et al., Hand synergies: integration of robotics and neuroscience for understanding the control of biological and artificial hands. Phys Life Rev 17, 1–23 (2016)

    Article  PubMed  PubMed Central  Google Scholar 

  10. R.S. Johansson, J.R. Flanagan, Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat. Rev. Neurosci. 10(5), 345–359 (2009)

    Article  CAS  PubMed  Google Scholar 

  11. J.R. Flanagan, S. King, D.M. Wolpert, R.S. Johansson, Sensorimotor prediction and memory in object manipulation. Can. J. Exp. Psychol. 55(2), 87–95 (2001)

    Article  CAS  PubMed  Google Scholar 

  12. C. Antfolk, M. D’Alonzo, B. Rosen, G.N. Lundborg, F. Sebelius, C. Cipriani, Sensory feedback in upper limb prosthetics. Expert Rev. Med. Devices 10(1), 45–54 (2013)

    Article  CAS  PubMed  Google Scholar 

  13. Q. Fu, M. Santello, Improving fine control of grasping force during hand–object interactions for a soft synergy-inspired myoelectric prosthetic hand. Front. Neurorobot. 11, 1–15 (2018)

    Article  Google Scholar 

  14. M.G. Catalano, G. Grioli, E. Farnioli, A. Serio, C. Piazza, A. Bicchi, Adaptive synergies for the design and control of the Pisa/IIT SoftHand. Int. J. Robot. Res. 33(5), 768–782 (2014)

    Article  Google Scholar 

  15. S.B. Godfrey et al., The Softhand Pro: functional evaluation of a novel, flexible, and robust myoelectric prosthesis. PLoS One 13(10), 1–20 (2018)

    Article  CAS  Google Scholar 

  16. S. Casini, M. Morvidoni, M. Bianchi, M.G. Catalano, G. Grioli, A. Bicchi, Design and realization of the CUFF – clenching upper-limb force feedback wearable device for distributed mechano-tactile stimulation of normal and tangential skin forces. IEEE/RSJ Int. Conf. Intell. Robot. Syst., 1186–1193 (2015)

    Google Scholar 

  17. Q. Fu, F. Shao, M. Santello, Inter-limb transfer of grasp force perception with closed-loop hand prosthesis. IEEE Trans. Neural Syst. Rehabil. Eng. 27(5), 927–936 (2019)

    Article  PubMed  Google Scholar 

  18. S. Fani et al., Assessment of myoelectric controller performance and kinematic behavior of a novel soft synergy-inspired robotic hand for prosthetic applications. Front. Neurorobot. 10, 11 (2016)

    Article  PubMed  PubMed Central  Google Scholar 

  19. R.S. Johansson, G. Westling, Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects. Exp. Brain Res. 56(3), 550–564 (1984)

    Article  CAS  PubMed  Google Scholar 

  20. G. Westling, R.S. Johansson, Factors influencing the force control during precision grip. Exp. Brain Res. 53(2), 277–284 (1984)

    Article  CAS  PubMed  Google Scholar 

  21. A.M. Gordon, G. Westling, K.J. Cole, R.S. Johansson, Memory representations underlying motor commands used during manipulation of common and novel objects. J. Neurophysiol. 69(6), 1789–1796 (1993)

    Article  CAS  PubMed  Google Scholar 

  22. R.S. Johansson, G. Westling, Coordinated isometric muscle commands adequately and erroneously programmed for the weight during lifting task with precision grip. Exp. Brain Res. 71, 59–71 (1988)

    CAS  PubMed  Google Scholar 

  23. R.S. Johansson, K.J. Cole, Sensory-motor coordination during grasping and manipulative actions. Curr. Opin. Neurobiol. 2(6), 815–823 (1992)

    Article  CAS  PubMed  Google Scholar 

  24. A.M. Hadjiosif, M.A. Smith, Flexible control of safety margins for action based on environmental variability. J. Neurosci. 35(24), 9106–9121 (2015)

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. S.L. Gorniak, V.M. Zatsiorsky, M.L. Latash, Manipulation of a fragile object. Exp. Brain Res. 202, 413–430 (2010)

    Article  PubMed  Google Scholar 

  26. E.D. Engeberg, S.G. Meek, M.A. Minor, Hybrid force – velocity sliding mode control of a prosthetic hand. IEEE Trans. Biomed. Eng. 55(5), 1572–1581 (2008)

    Article  PubMed  Google Scholar 

  27. E.D. Engeberg, S.G. Meek, Adaptive sliding mode control for prosthetic hands to simultaneously prevent slip and minimize deformation of grasped objects. IEEE Trans. Mechatron. 18(1), 376–385 (2013)

    Article  Google Scholar 

  28. E.D. Engeberg, M. Frankel, S.G. Meek, Biomimetic grip force compensation based on acceleration of a prosthetic wrist under sliding mode control. IEEE Int. Conf. Robot. Biomimetics, 210–215 (2009)

    Google Scholar 

  29. E. Rombokas, C.E. Stepp, C. Chang, M. Malhotra, Y. Matsuoka, Vibrotactile sensory substitution for electromyographic control of object manipulation. IEEE Trans. Biomed. Eng. 60(8), 2226–2232 (2013)

    Article  PubMed  Google Scholar 

  30. S. Raspopovic et al., Restoring natural sensory feedback in real-time bidirectional hand prostheses. Sci. Transl. Med. 6(222), 222ra19 (2014)

    Article  PubMed  Google Scholar 

  31. C. Cipriani, F. Zaccone, S. Micera, M.C. Carrozza, On the shared control of an EMG-controlled prosthetic hand: analysis of user-prosthesis interaction. IEEE Trans. Robot. 24(1), 170–184 (2008)

    Article  Google Scholar 

  32. S. Amsuess et al., Context-dependent upper limb prosthesis control for natural and robust use. IEEE Trans. Neural Syst. Rehabil. Eng. 24(7), 744–753 (2016)

    Article  PubMed  Google Scholar 

  33. N. Jiang, S. Muceli, B. Graimann, D. Farina, Effect of arm position on the prediction of kinematics from EMG in amputees. Med. Biol. Eng. Comput. 51(1–2), 143–151 (2013)

    Article  PubMed  Google Scholar 

  34. G.K. Patel, J.M. Hahne, C. Castellini, D. Farina, S. Dosen, Context-dependent adaptation improves robustness of myoelectric control for upper-limb prostheses. J. Neural Eng. 14(5), 056016 (2017)

    Article  PubMed  Google Scholar 

  35. M. Markovic, S. Dosen, D. Popović, B. Graimann, D. Farina, Sensor fusion and computer vision for context-aware control of a multi degree-of-freedom prosthesis. J. Neural Eng. 12(6), 066022 (2015)

    Article  PubMed  Google Scholar 

  36. M. Markovic, S. Dosen, C. Cipriani, D. Popović, D. Farina, Stereovision and augmented reality for closed-loop control of grasping in hand prostheses. J. Neural Eng. 11(4), 046001 (2014)

    Article  PubMed  Google Scholar 

  37. C. Castellini et al., Proceedings of the first workshop on peripheral machine interfaces: going beyond traditional surface electromyography. Front. Neurorobot. 8, 1–17 (2014)

    Article  Google Scholar 

  38. L.A. Jones, The Control and Perception of Finger Forces, in The Human Hand as an Inspiration for Robot Hand Development, (Springer, Cham, 2014), pp. 99–122

    Chapter  Google Scholar 

  39. S.C. Gandevia, Kinesthesia: Roles for Afferent Signals and Motor Commands, in Handbook of Physiology: Sec.12. Exercise Regulation and Integration of Multiple Systems, ed. by L. B. Rowell, J. T. Shepherd, (Oxford University Press, New York, 1996), pp. 128–172

    Google Scholar 

  40. M. Haruno, D.M. Wolpert, M. Kawato, MOSAIC model for sensorimotor learning and control. Neural Comput. 13, 2201–2220 (2001)

    Article  CAS  PubMed  Google Scholar 

  41. G. Lafargue, J. Paillard, Y. Lamarre, A. Sirigu, Production and perception of grip force without proprioception: is there a sense of effort in deafferented subjects? Eur. J. Neurosci. 17(12), 2741–2749 (2003)

    Article  PubMed  Google Scholar 

  42. L.A. Jones, Perceptual constancy and the perceived magnitude of muscle forces. Exp. Brain Res. 151(2), 197–203 (2003)

    Article  PubMed  Google Scholar 

  43. L.A. Jones, E. Piateski, Contribution of tactile feedback from the hand to the perception of force. Exp. Brain Res. 168(1–2), 298–302 (2006)

    Article  PubMed  Google Scholar 

  44. S.S. Shergill, P.H. Bays, C.D. Frith, D.M. Wotpert, Two eyes for an eye: the neuroscience of force escalation. Science (80-.) 301(5630), 187 (2003)

    Article  CAS  Google Scholar 

  45. R. Ackerley, I. Carlsson, H. Wester, H. Olausson, H. Backlund Wasling, Touch perceptions across skin sites: differences between sensitivity, direction discrimination and pleasantness. Front. Behav. Neurosci. 8, 1–10 (2014)

    Google Scholar 

Download references

Acknowledgments

This work was made possible by grant W911NF-17-1-0049 from the Defense Advanced Research Projects Agency and the US Army Research Office.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Qiushi Fu .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Fu, Q., Santello, M. (2020). Learning from the Human Hand: Force Control and Perception Using a Soft-Synergy Prosthetic Hand and Noninvasive Haptic Feedback. In: Vinjamuri, R. (eds) Advances in Motor Neuroprostheses. Springer, Cham. https://doi.org/10.1007/978-3-030-38740-2_4

Download citation

Publish with us

Policies and ethics