Skip to main content
Log in

Towards the sEMG hand: internet of things sensors and haptic feedback application

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the trend going on in ubiquitous computing, everything is going to be connected to the Internet and its data will be used for various progressive purposes, creating not only information from it, but also, knowledge and even wisdom. Internet of Things (IoT) is becoming important because the amount of data could make it possible to create more usefulness and develop smart applications for the users. Meanwhile, it mainly focuses on how to enable general objects to see, hear, and smell the physical world for themselves, and make them connected to share the observations. In this paper, we focus our attention on the integration of artificial sensory perception and haptic feedback in sEMG hands, which is an intelligent application of the IoT. Artificial sensory perception and haptic feedback are essential elements for amputees with myoelectric hands to restore the grasping function. They can provide information to users, such as forces of interaction and surface properties at points of contact between hands and objects. Recent advancements in robot tactile sensing led to development of many computational techniques that exploit this important sensory channel. At the same time, Surface electromyography (sEMG) is perhaps most useful for providing insight into how the neuromuscular system behaves. Therefore, integration of sEMG technology, artificial sensation and haptic feedback plays an important role in improving the manipulation performance and enhancing perceptual embodiment for users. This paper provides sEMG technologies that involve Multichannel sEMG electrodes array and processing methods, and then reviews current state-of-the-art of artificial sensation and haptic feedback. Drawing from advancements and taking into design considerations of each feedback modality and individual haptic technology, the paper outline challenging issues and future developments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Antfolk C, D'Alonzo M, Controzzi M et al (2013) Artificial redirection of sensation from prosthetic fingers to the phantom hand map on transradial amputees: vibrotactile versus mechanotactile sensory feedback. IEEE Trans Neural Syst Rehab Eng 21(1):112–120

    Google Scholar 

  2. Atieh A, Ahmadi R, Kalantari M et al (2011) A piezoresistive based tactile sensor for use in minimally invasive surgery. Northeast Bioeng Conf 7886(4):1–2

    Google Scholar 

  3. Balasubramanian R, Santos VJ (2014) The human hand as an inspiration for robot hand development. Springer Publishing Company, Incorporated 95

  4. Bandari NM, Ahmadi R, Hooshiar A et al (2017) Hybrid piezoresistive-optical tactile sensor for simultaneous measurement of tissue stiffness and detection of tissue discontinuity in robot-assisted minimally invasive surgery. J Biomed Opt 22(7):77002

    Google Scholar 

  5. Barbadillo G, Dautenhahn K, Wood L (2011) Using FSR sensors to provide tactile skin to the humanoid robot KASPAR. Comput Sci Tech Rep 511:147–153

    Google Scholar 

  6. Bhattacharyya S, Clerc M, Hayashibe M (2016) A study on the effect of electrical stimulation as a user stimuli for motor imagery classification in brain-machine interface. Eur J Translat Myol 26(2):165–168

    Google Scholar 

  7. Casini S, Morvidoni M, Bianchi M, et al. (2015) Design and realization of the CUFF-clenching upper-limb force feedback wearable device for distributed mechano-tactile stimulation of normal and tangential skin forces, IEEE Int Conf Intell Robots Syst: 1186–1193

  8. Cipriani C, Zaccone F, Micera S et al (2008) On the shared control of an EMG-controlled prosthetic hand: analysis of user-prosthesis interaction. IEEE Trans Robot 24(1):170–184

    Google Scholar 

  9. Cipriani C, D'Alonzo M, Carrozza MC (2012) A miniature vibrotactile sensory substitution device for multifingered hand prosthetics. IEEE Trans Biomed Eng 59(2):400–408

    Google Scholar 

  10. Cloutier A, Yang J (2013) Design, control, and sensory feedback of externally powered hand prostheses: a literature review. Crit Rev Biomed Eng 41(2):161

    Google Scholar 

  11. Cotton DPJ, Chappell PH, Cranny A et al (2007) A novel thick-film piezoelectric slip sensor for a prosthetic hand. IEEE Sensors J 7(5):752–761

    Google Scholar 

  12. Cotton DPJ, Graz IM, Lacour SP (2009) A multifunctional capacitive sensor for stretchable electronic skins. IEEE Sensors J 9(12):2008–2009

    Google Scholar 

  13. Dahiya RS, Metta G, Valle M (2010) Tactile sensing:from humans to humanoids. IEEE Trans Robot 26(1):1–20

    Google Scholar 

  14. Dang H, Allen PK (2014) Stable grasping under pose uncertainty using tactile feedback. Auton Robot 36(4):309–330

    Google Scholar 

  15. Englehart K, Hudgins B, Parker PA, Stevenson M (1999) Classification of the myoelectric signal using time-frequency based representations. Med Eng Phys 21(6):431–438

    Google Scholar 

  16. Fang Y, Liu H, Li G et al. (2015) A multichannel surface EMG system for hand motion recognition. Int J Human Robot 12(2). doi:https://doi.org/10.1142/S0219843615500115.

    Google Scholar 

  17. Farina D, Holobar A (2015) Human-machine interfacing by decoding the surface electromyogram [life sciences]. IEEE Signal Process Mag 32(1):115–120

    Google Scholar 

  18. Fishel JA, Loeb GE (2012) Bayesian exploration for intelligent identification of textures. Front Neurorobot 6(4):1–20

    Google Scholar 

  19. Gibson A, Artemiadis P (2015) Neural closed-loop control of a hand prosthesis using cross-modal haptic feedback. IEEE Int Conf Rehab Robot: 37–42

  20. Gonzalez J, Soma H, Sekine M et al (2012) Psycho-physiological assessment of a prosthetic hand sensory feedback system based on an auditory display: a preliminary study. J Neuroeng Rehab 9(1):33

    Google Scholar 

  21. Gonzalez J, Suzuki H, Natsumi N, et al. (2012) Auditory display as a prosthetic hand sensory feedback for reaching and grasping tasks’. Eng Med Biol Soc: 1789

  22. Gonzalez MG, Sorichetti PA, Brazzano LC et al (2014) Electromechanical characterization of piezoelectric polymer thin films in a broad frequency range. Polym Test 37(8):163–169

    Google Scholar 

  23. Hargrove L, Englehart K, Hudgins B. (2006) The effect of electrode displacements on pattern recognition based myoelectric control. Int Conf IEEE. Eng Med Biol Soc: 2203–2206

  24. Hargrove LJ, Englehart K, Hudgins B (2007) A comparison of surface and intramuscular myoelectric signal classification. IEEE Trans Biomed Eng 54(5):847–853

    Google Scholar 

  25. He Y, Li GF, Sun Y et al (2017) Temperature intelligent prediction model of coke oven flue based on CBR and RBFNN. Int J Comput Sci Math. https://doi.org/10.1007/s10586-017-1237-1

    Google Scholar 

  26. Heo JS, Kim JY, Lee JJ (2008) Tactile sensors using the distributed optical fiber sensors. Int Conf Sens Technol 382:486–490

    Google Scholar 

  27. Heyneman B, Cutkosky MR (2012) Biologically inspired tactile classification of object-hand and object-world interactions. 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO): 167–173

  28. Hosoda K, Tada Y, Asada M (2006) Anthropomorphic robotic soft fingertip with randomly distributed receptors. Robot Auto Syst 54(2):104–109

    Google Scholar 

  29. Huang H, Zhou P, Li G, Kuiken TA (2008) An analysis of EMG electrode configuration for targeted muscle reinnervation based neural machine interface. IEEE Trans Neural Syst Rehab Eng 16(1):37–45

    Google Scholar 

  30. Hudgins B, Englehart K, Parker PA, Scott RN (1997) A microprocessor based multifunction myoelectric control system. 23rd Can Med Biol Eng Soc Conf: 138–139

  31. Jara CA, Pomares J, Candelas FA et al (2014) Control framework for dexterous manipulation using dynamic visual servoing and tactile sensors' feedback. Sensors 14(7):1787–1804

    Google Scholar 

  32. Jiang N, Falla D, D'Avella A et al (2010) Myoelectric control in neurorehabilitation. Crit Rev Biomed Eng 38(4):381

    Google Scholar 

  33. Jiang L, Yang B, Huang Q et al (2017) Biomechatronic integration of intelligent prosthetic hand. Robot 39(4):387–394

    Google Scholar 

  34. Kaczmarek KA, Webster JG, Bach-Y-Rita P et al (1991) Electrotactile and vibrotactile displays for sensory substitution systems. IEEE Trans Biomed Eng 38(1):1–16

    Google Scholar 

  35. Kappassov Z, Corrales JA, Perdereau V (2015) Tactile sensing in dexterous robot hands-review. North-Holland Publishing Co 74:195–220

    Google Scholar 

  36. Kimoto A, Sugitani N, Fujisaki S (2010) A multifunctional tactile sensor based on PVDF films for identification of materials. IEEE Sensors J 10(9):1508–1513

    Google Scholar 

  37. Konstantinova J, Stilli A, Althoefer K (2017) Fingertip fiber optical tactile array with two-level spring structure. Sensors 17(10):2337

    Google Scholar 

  38. Kramer RK, Majidi C, Wood RJ (2011) Wearable tactile keypad with stretchable artificial skin. IEEE Int Conf Robot Auto 47:1103–1107

    Google Scholar 

  39. Lang SB, Muensit S (2006) Review of some lesser-known applications of piezoelectric and pyroelectric polymers. Appl Phys A 85(2):125–134

    Google Scholar 

  40. Lee HK, Chung J, Chang SI et al (2011) Real-time measurement of the three-axis contact force distribution using a flexible capacitive polymer tactile sensor. J Micromech Microeng 21(3):035010

    Google Scholar 

  41. Li C, Wu PM, Lee S et al (2008) Flexible dome and bump shape piezoelectric tactile sensors using PVDF-TrFE copolymer. J Microelectromech Syst 17(2):334–341

    Google Scholar 

  42. Li G, Schultz AE, Kuiken TA (2010) Quantifying pattern recognition-based myoelectric control of multifunctional transradial prostheses. IEEE Trans Neural Syst Rehab Eng 18:185–192

    Google Scholar 

  43. Li B, Sun Y, Li G et al (2017) Gesture recognition based on modified adaptive orthogonal matching pursuit algorithm. Clust Comput 3:1–10

    MathSciNet  Google Scholar 

  44. Li K, Fang Y, Zhou Y et al (2017) Non-invasive stimulation-based tactile sensation for upper-extremity prosthesis: a review. IEEE Sensors J 17(9):2625–2635

    Google Scholar 

  45. Liu P, Huang Y, Qiu H et al (2011) Strain and piezoresistive effect in working principle of pressure-sensitive conductive rubber. Polymer Mater Sci Eng 27(5):112–115

    Google Scholar 

  46. Liu H, Greco J, Song X, et al. (2012) Tactile image based contact shape recognition using neural network. IEEE Conf Multisensor Fusion Integra Intell Syst 138–143

  47. Maereg AT, Nagar AK, Reid D et al (2017) Wearable Vibrotactile haptic device for stiffness discrimination during virtual interactions. Front Robot AI 4:42. https://doi.org/10.3389/frobt.2017.00042

    Article  Google Scholar 

  48. Martinez HU, Metta G, Dodd TJ et al. (2013) Active contour following to explore object shape with robot touch. World Haptics Conf IEEE: 341–346

  49. Miao W, Li G, Jiang G et al (2015) Optimal grasp planning of multi-fingered robotic hands: a review. Appl Comput Math 14(3):238–247

    MathSciNet  MATH  Google Scholar 

  50. Ohka M, Morisawa N, Suzuki H et al (2008) A robotic finger equipped with an optical three-axis tactile sensor. IEEE Int Conf Robot Auto 74(740):3425–3430

    Google Scholar 

  51. Peerdeman B, Boere D, Witteveen H (2011) Myoelectric forearm prostheses: state of the art from a user-centered perspective. J Rehab Res Dev 48(6):719–737

    Google Scholar 

  52. Rafiee J, Rafiee MA et al (2011) Feature extraction of forearm EMG signals for prosthetics. Expert Syst Appl 38(4):4058–4067

    MathSciNet  Google Scholar 

  53. Schmitz A, Maiolino P, Maggiali M et al (2011) Methods and technologies for the implementation of large-scale robot tactile sensors. IEEE Trans Robot 27(3):389–400

    Google Scholar 

  54. Schofield JS, Evans KR, Carey JP et al (2014) Applications of sensory feedback in motorized upper extremity prosthesis: a review. Expert Rev Med Dev 11(5):499–511

    Google Scholar 

  55. Seminara L, Pinna L, Valle M et al (2013) Piezoelectric polymer transducer arrays for flexible tactile sensors. IEEE Sensors J 13(10):4022–4029

    Google Scholar 

  56. Shair EF, Zawawi TNST, Abdullah AR et al.(2015) sEMG signals analysis using time-frequency distribution for symmetric and asymmetric lifting. Int Sym Technol Manag Emerg Technol. IEEE: 233–237

  57. Stanley AA, Gwilliam JC, Okamura AM (2013) Haptic jamming: a deformable geometry, variable stiffness tactile display using pneumatics and particle jamming. IEEE World Haptics Conf: 25–30

  58. Stassi S, Cauda V, Canavese G et al (2014) Flexible tactile sensing based on piezoresistive composites: a review. Sensors 14(3):5296–5332

    Google Scholar 

  59. Sun Y, Li C, Li G et al (2018) Gesture recognition based on Kinect and sEMG signal fusion. Mobile Netw Appl. https://doi.org/10.1007/s11036-018-1008-0

    Google Scholar 

  60. Tegin J, Wikander J (2013) Tactile sensing in intelligent robotic manipulation-a review. Ind Robot 32(1):64–70

    Google Scholar 

  61. Tejeiro C, Stepp CE, Malhotra M et al. (2012) Comparison of remote pressure and vibrotactile feedback for prosthetic hand control. IEEE Ras Embs Int Conf Biomed Robot Biomechatron: 521–525

  62. Teshigawara S, Tadakuma K, Ming A et al (2010) High sensitivity initial slip sensor for dexterous grasp. IEEE Int Conf Robot Auto 58(8):4867–4872

    Google Scholar 

  63. Tiwana MI, Redmond SJ, Lovell NH (2012) A review of tactile sensing technologies with applications in biomedical engineering. Sens Actuators A Phys 179(3):17–31

    Google Scholar 

  64. Ulrika W, Ingela C (2015) Forearm amputees' views of prosthesis use and sensory feedback. J Hand Therap Off J Am Soc Hand Therap 28(3):269

    Google Scholar 

  65. Wang H, Boer GD et al (2016) Design methodology for magnetic field-based soft tri-axis tactile sensors. Sensors 16(9):1356

    Google Scholar 

  66. Xu D, Loeb GE, Fishel JA (2013) Tactile identification of objects using Bayesian exploration. IEEE Int Conf Robot Auto: 3056–3061

  67. Yousef H, Boukallel M, Althoefer K (2011) Tactile sensing for dexterous in-hand manipulation in robotics-a review. Sens Actua A Phys 167(2):171–187

    Google Scholar 

  68. Zardoshti KM, Wheeler BC, Badie K et al (1995) EMG feature evaluation for movement control of upper extremity prostheses. IEEE Trans Rehab Eng 3(4):324–333

    Google Scholar 

Download references

Acknowledgements

This work was supported by grants of National Natural Science Foundation of China (Grant No. 51575407, 51575338, 51575412, 61273106) and the Grants of National Defense Pre-Research Foundation of Wuhan University of Science and Technology (GF201705).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Gongfa Li.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, G., Zhang, L., Sun, Y. et al. Towards the sEMG hand: internet of things sensors and haptic feedback application. Multimed Tools Appl 78, 29765–29782 (2019). https://doi.org/10.1007/s11042-018-6293-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6293-x

Keywords

Navigation