Abstract
Current developments being made in upper limb prostheses are focused on replacing lost sensory information to the amputees. Providing sensory stimulation from the prosthesis can directly improve control over the prosthetic and provide a sense of body ownership. The focus of this review article is on recent developments while including foundational knowledge for some of the critical concepts in neural prostheses. Reported concepts follow the flow of information from sensors to signal processing, with emphasis on texture recognition, and then to sensory stimulation strategies that reestablish the lost sensory feedback loop. Prosthetic sensors are used to detect the physical environment, converting pressure, force, and position into electrical signals. The electrical signals can then be processed in an effort to identify the surrounding environment using distinctive characteristics such as stiffness and texture. In order for the amputee to use this information in a natural manner, there must be real-time sensory stimulation, perception, and motor control of the prosthesis. Although truly complete sensory replacement has not yet been realized, some basic percepts can be partially restored, allowing progress towards a more realistic prosthesis with natural sensations.
Similar content being viewed by others
References
Abraira, V. E., and D. D. Ginty. the sensory neurons of touch. Neuron 79:618–639, 2013.
Akhtar, A., M. Nguyen, L. Wan, B. Boyce, P. Slade, and T. Bretl. Passive Mechanical Skin Stretch for Multiple Degree-of-Freedom Proprioception in a Hand Prosthesis, 2014.
Arakeri, T. J., B. A. Hasse, and A. J. Fuglevand. Object discrimination using electrotactile feedback. J. Neural Eng. 15:046007, 2018.
Bensmaia, S. J., and L. E. Miller. Restoring sensorimotor function through intracortical interfaces: progress and looming challenges. Nat. Rev. Neurosci. 15:313, 2014.
Biddiss, E. A., and T. T. Chau. Upper limb prosthesis use and abandonment: a survey of the last 25 years. Prosthet. Orthot. Int. 31:236–257, 2007.
Boretius, T., J. Badia, A. Pascual-Font, M. Schuettler, X. Navarro, K. Yoshida, and T. Stieglitz. A transverse intrafascicular multichannel electrode (TIME) to interface with the peripheral nerve. Biosens. Bioelectron. 26:62–69, 2010.
Brill, N. A., S. N. Naufel, K. Polasek, C. Ethier, J. Cheesborough, S. Agnew, L. E. Miller, and D. J. Tyler. Evaluation of high-density, multi-contact nerve cuffs for activation of grasp muscles in monkeys. J. Neural Eng. 15:036003, 2018.
Cao, Y., T. Li, Y. Gu, H. Luo, S. Wang, and T. Zhang. Fingerprint-inspired flexible tactile sensor for accurately discerning surface texture. Small 14:1703902, 2018.
Casini, S., M. Morvidoni, M. Bianchi, M. Catalano, G. Grioli, and A. Bicchi. Design and realization of the CUFF - clenching upper-limb force feedback wearable device for distributed mechano-tactile stimulation of normal and tangential skin forces. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2015. https://doi.org/10.1109/iros.2015.7353520.
Chaubey, P., T. Rosenbaum-Chou, W. Daly, and D. Boone. Closed-loop vibratory haptic feedback in upper-limb prosthetic users. J. Prosthet. Orthot. 26:120–127, 2014.
Chen, S., A. Z. Weitemier, X. Zeng, L. He, X. Wang, Y. Tao, A. J. Y. Huang, Y. Hashimotodani, M. Kano, H. Iwasaki, L. K. Parajuli, S. Okabe, D. B. L. Teh, A. H. All, I. Tsutsui-Kimura, K. F. Tanaka, X. Liu, and T. J. McHugh. Near-infrared deep brain stimulation via upconversion nanoparticle–mediated optogenetics. Science 359:679–684, 2018.
Chortos, A., J. Liu, and Z. Bao. Pursuing prosthetic electronic skin. Nat. Mater. 15:937–950, 2016.
Christie, B. P., M. Freeberg, W. D. Memberg, G. J. C. Pinault, H. A. Hoyen, D. J. Tyler, and R. J. Triolo. Long-term stability of stimulating spiral nerve cuff electrodes on human peripheral nerves. J. NeuroEng. Rehabil. 14:70, 2017.
Chun, S., A. Hong, Y. Choi, C. Ha, and W. Park. A tactile sensor using a conductive graphene-sponge composite. Nanoscale 8:9185–9192, 2016.
Chun, S., Y. Kim, H.-S. Oh, G. Bae, and W. Park. A highly sensitive pressure sensor using a double-layered graphene structure for tactile sensing. Nanoscale 7:11652–11659, 2015.
Chun, K.-Y., Y. J. Son, E.-S. Jeon, S. Lee, and C.-S. Han. A self-powered sensor mimicking slow- and fast-adapting cutaneous mechanoreceptors. Adv. Mater. 30:1706299, 2018.
Clemente, F., M. D’Alonzo, M. Controzzi, B. B. Edin, and C. Cipriani. Non-invasive, temporally discrete feedback of object contact and release improves grasp control of closed-loop myoelectric transradial prostheses. IEEE Trans. Neural Syst. Rehabil. Eng. 24:1314–1322, 2016.
Clemente, F., S. Dosen, L. Lonini, M. Markovic, D. Farina, and C. Cipriani. Humans can integrate augmented reality feedback in their sensorimotor control of a robotic hand. IEEE Trans. Hum. Mach. Syst. 47:583–589, 2017.
Cula, O. G., and K. J. Dana. Recognition methods for 3D textured surfaces. Proc. SPIE 2001. https://doi.org/10.1117/12.429492.
Dahiya, R. S., G. Metta, M. Valle, and G. Sandini. Tactile sensing-from humans to humanoids. TRO 26:1–20, 2010.
Dalonzo, M., S. Dosen, C. Cipriani, and D. Farina. HyVE: hybrid vibro-electrotactile stimulation for sensory feedback and substitution in rehabilitation. IEEE Trans. Neural Syst. Rehabil. Eng. 22:290–301, 2014.
de Boissieu, F., C. Godin, B. Guilhamat, D. David, C. Serviere, D. Baudois. Tactile texture recognition with a 3-axial force MEMS integrated artificial finger, 2009. https://doi.org/10.15607/rss.2009.v.007
Ehrsson, H. H., B. Rosen, A. Stockselius, C. Ragno, P. Kohler, and G. Lundborg. Upper limb amputees can be induced to experience a rubber hand as their own. Brain 131:3443–3452, 2008.
FitzGerald, J. J., N. Lago, S. Benmerah, J. Serra, C. P. Watling, R. E. Cameron, E. Tarte, S. P. Lacour, S. B. McMahon, and J. W. Fawcett. A regenerative microchannel neural interface for recording from and stimulating peripheral axonsin vivo. J. Neural Eng. 9:016010, 2012.
Han, S., A. Samanta, X. Xie, L. Huang, J. Peng, S. J. Park, D. B. L. Teh, Y. Choi, Y.-T. Chang, A. H. All, Y. Yang, B. Xing, and X. Liu. Gold and hairpin DNA functionalization of upconversion nanocrystals for imaging and in vivo drug delivery. Adv. Mater. 29:1700244, 2017.
Jamali, N., and C. Sammut. Majority voting: material classification by tactile sensing using surface texture. IEEE Trans. Rob. 27:508–521, 2011.
Ji, Z., H. Zhu, H. Liu, N. Liu, T. Chen, Z. Yang, and L. Sun. The design and characterization of a flexible tactile sensing array for robot skin. Sensors (Basel, Switzerland) 16:201, 2016.
Karim, S. A. A., M. H. Kamarudin, B. A. Karim, M. K. Hasan, and J. Sulaiman. Wavelet transform and fast fourier transform for signal compression: a comparative study. J IFET 2011. https://doi.org/10.1109/icedsa.2011.5959031.
Kim, D.-H., J.-H. Ahn, W. M. Choi, H.-S. Kim, T.-H. Kim, J. Song, Y. Y. Huang, Z. Liu, C. Lu, and J. A. Rogers. Stretchable and foldable silicon integrated circuits. Science 320:507–511, 2008.
Kim, S.-H., J. Engel, C. Liu, and D. L. Jones. Texture classification using a polymer-based MEMS tactile sensor. J. Micromech. Microeng. 15:912–920, 2005.
Kim, J., M. Lee, H. J. Shim, R. Ghaffari, H. R. Cho, D. Son, Y. H. Jung, M. Soh, C. Choi, S. Jung, K. Chu, D. Jeon, S.-T. Lee, J. H. Kim, S. H. Choi, T. Hyeon, and D.-H. Kim. Stretchable silicon nanoribbon electronics for skin prosthesis. Nat. Commun. 5:5747, 2014.
Lamport, Z. A., H. F. Haneef, S. Anand, M. Waldrip, and O. D. Jurchescu. Tutorial: organic field-effect transistors: materials, structure and operation. J. Appl. Phys. 124:071101, 2018.
Lee, H. U., A. Blasiak, D. R. Agrawal, D. T. B. Loong, N. V. Thakor, A. H. All, J. S. Ho, and I. H. Yang. Subcellular electrical stimulation of neurons enhances the myelination of axons by oligodendrocytes. PLoS ONE 12:e0179642, 2017.
Lee, S., S. Sheshadri, Z. Xiang, I. Delgado-Martinez, N. Xue, T. Sun, N. V. Thakor, S.-C. Yen, and C. Lee. Selective stimulation and neural recording on peripheral nerves using flexible split ring electrodes. Sens. Actuators B Chem. 242:1165–1170, 2017.
Liao, Z., W. Liu, Y. Wu, C. Zhang, Y. Zhang, X. Wang, and X. Li. A tactile sensor translating texture and sliding motion information into electrical pulses. Nanoscale 7:10801–10806, 2015.
Liu, X., Y. Wang, X. Li, Z. Yi, R. Deng, L. Liang, X. Xie, D. T. B. Loong, S. Song, D. Fan, A. H. All, H. Zhang, L. Huang, and X. Liu. Binary temporal upconversion codes of Mn2+—activated nanoparticles for multilevel anti-counterfeiting. Nat. Commun. 8:1–7, 2017.
Llado, X., M. Petrou, and J. Marti. Surface texture recognition by surface rendering. Optic Eng. 2005. https://doi.org/10.1117/1.1869994.
Lu, N., C. Lu, S. Yang, and J. Rogers. Highly sensitive skin-mountable strain gauges based entirely on elastomers. Adv. Funct. Mater. 22:4044–4050, 2012.
Lucarotti, C., C. M. Oddo, N. Vitiello, and M. C. Carrozza. Synthetic and bio-artificial tactile sensing: a review. Sensors (Basel, Switzerland) 13:1435–1466, 2013.
Maciejasz, P., J. Badia, T. Boretius, D. Andreu, T. Stieglitz, W. Jensen, X. Navarro, and D. Guiraud. Delaying discharge after the stimulus significantly decreases muscle activation thresholds with small impact on the selectivity: an in vivo study using TIME. Med. Biol. Eng. Compu. 53:371–379, 2015.
Mannsfeld, S. C. B., B. C.-K. Tee, R. M. Stoltenberg, C. V. H.-H. Chen, S. Barman, B. V. O. Muir, A. N. Sokolov, C. Reese, and Z. Bao. Highly sensitive flexible pressure sensors with microstructured rubber dielectric layers. Nat. Mater. 9:859–864, 2010.
Markovic, M., H. Karnal, B. Graimann, D. Farina, and S. Dosen. GLIMPSE: Google Glass interface for sensory feedback in myoelectric hand prostheses. J. Neural Eng. 14:036007, 2017.
Muhammad, H. B., C. M. Oddo, L. Beccai, C. Recchiuto, C. J. Anthony, M. J. Adams, M. C. Carrozza, D. W. L. Hukins, and M. C. L. Ward. Development of a bioinspired MEMS based capacitive tactile sensor for a robotic finger. Sens. Actuators A 165:221–229, 2011.
Muhammad, H. B., C. Recchiuto, C. M. Oddo, L. Beccai, C. J. Anthony, M. J. Adams, M. C. Carrozza, and M. C. L. Ward. A capacitive tactile sensor array for surface texture discrimination. Microelectron. Eng. 88:1811–1813, 2011.
Nicolò, B., and B-B Gabriel. Effects of Chai3D Texture Rendering Parameters on Texture Perception, 2018. https://doi.org/10.5281/zenodo.1287011
Oddo, C. M., S. Raspopovic, F. Artoni, A. Mazzoni, G. Spigler, F. Petrini, F. Giambattistelli, F. Vecchio, F. Miraglia, L. Zollo, G. Di Pino, D. Camboni, M. C. Carrozza, E. Guglielmelli, P. M. Rossini, U. Faraguna, and S. Micera. Intraneural stimulation elicits discrimination of textural features by artificial fingertip in intact and amputee humans. eLife 5:e09148, 2016.
Osborn, L. E., A. Dragomir, J. L. Betthauser, C. L. Hunt, H. H. Nguyen, R. R. Kaliki, and N. V. Thakor. Prosthesis with neuromorphic multilayered e-dermis perceives touch and pain. Sci. Rob. 3:eaat3818, 2018.
Pang, C., G.-Y. Lee, T. Kim, S. M. Kim, H. N. Kim, S.-H. Ahn, and K.-Y. Suh. A flexible and highly sensitive strain-gauge sensor using reversible interlocking of nanofibres. Nat. Mater. 11:795–801, 2012.
Peng, J., A. Samanta, X. Zeng, S. Han, L. Wang, D. Su, D. T. B. Loong, N.-Y. Kang, S.-J. Park, A. H. All, W. Jiang, L. Yuan, X. Liu, and Y.-T. Chang. Real-time in vivo hepatotoxicity monitoring through chromophore-conjugated photon-upconverting nanoprobes. Angew. Chem. Int. Ed. 56:4165–4169, 2017.
Prasad, A., D. B. L. Teh, A. Blasiak, C. Chai, Y. Wu, P. M. Gharibani, I. H. Yang, T. T. Phan, K. L. Lim, H. Yang, X. Liu, and A. H. All. Static magnetic field stimulation enhances oligodendrocyte differentiation and secretion of neurotrophic factors. Sci Rep. 7:1–12, 2017.
Qin, L., Z. Yi, and Y. Zhang. Enhanced surface roughness discrimination with optimized features from bio-inspired tactile sensor. Sens. Actuators A 2017. https://doi.org/10.1016/j.sna.2017.07.054.
Qin, L., and Y. Zhang. Roughness discrimination with bio-inspired tactile sensor manually sliding on polished surfaces. Sens. Actuators A 279:433–441, 2018.
Resnik, L., M. R. Meucci, S. Lieberman-Klinger, C. Fantini, D. L. Kelty, R. Disla, and N. Sasson. Advanced upper limb prosthetic devices: implications for upper limb prosthetic rehabilitation. Arch. Phys. Med. Rehabil. 93:710–717, 2012.
Rijnbeek, E. H., N. Eleveld, and W. Olthuis. Update on peripheral nerve electrodes for closed-loop neuroprosthetics. Frontiers Neurosci. 2018. https://doi.org/10.3389/fnins.2018.00350.
Ryu, S., P. Lee, J. B. Chou, R. Xu, R. Zhao, A. J. Hart, and S.-G. Kim. Extremely elastic wearable carbon nanotube fiber strain sensor for monitoring of human motion. ACS Nano 9:5929–5936, 2015.
Schiefer, M. A., M. Freeberg, G. J. C. Pinault, J. Anderson, H. Hoyen, D. J. Tyler, and R. J. Triolo. Selective activation of the human tibial and common peroneal nerves with a flat interface nerve electrode. J. Neural Eng. 10:056006, 2013.
Schoepp, K. R., M. R. Dawson, J. S. Schofield, J. P. Carey, and J. S. Hebert. Design and integration of an inexpensive wearable mechanotactile feedback system for myoelectric prostheses. J. IEEE Transl. Eng. Health Med. 2018. https://doi.org/10.1109/jtehm.2018.2866105.
Shin, H., Z. Watkins, H. Huang, Y. Zhu, and X. Hu. Evoked haptic sensations in the hand via non-invasive proximal nerve stimulation. J. Neural Eng. 15:046005, 2018.
Song, A., Y. Han, H. Hu, and J. Li. A novel texture sensor for fabric texture measurement and classification. IEEE Trans. Instrum. Meas. 63:1739–1747, 2014.
Svensson, P., U. Wijk, A. Björkman, and C. Antfolk. A review of invasive and non-invasive sensory feedback in upper limb prostheses. Expert Rev. Med. Dev. 14:439–447, 2017.
Tan, D. W., M. A. Schiefer, M. W. Keith, J. R. Anderson, and D. J. Tyler. Stability and selectivity of a chronic, multi-contact cuff electrode for sensory stimulation in human amputees. J. Neural Eng. 12:026002, 2015.
Tan, D. W., M. A. Schiefer, M. W. Keith, J. R. Anderson, J. Tyler, and D. J. Tyler. A neural interface provides long-term stable natural touch perception. Sci. Transl. Med. 6:257, 2014.
Tee, B. C.-K., A. Chortos, A. Berndt, A. K. Nguyen, A. Tom, A. McGuire, Z. C. Lin, K. Tien, W.-G. Bae, H. Wang, P. Mei, H.-H. Chou, B. Cui, K. Deisseroth, T. N. Ng, and Z. Bao. A skin-inspired organic digital mechanoreceptor. Science 350:313–316, 2015.
Thota, A. K., S. Kuntaegowdanahalli, A. K. Starosciak, J. J. Abbas, J. Orbay, K. W. Horch, and R. Jung. A system and method to interface with multiple groups of axons in several fascicles of peripheral nerves. J. Neurosci. Methods 244:78–84, 2015.
Tiwana, M. I., S. J. Redmond, and N. H. Lovell. A review of tactile sensing technologies with applications in biomedical engineering. Sens. Actuators A 179:17–31, 2012.
Trbac, M., M. Beli, M. Isakovi, V. Koji, G. Bijeli, I. Popovi, M. Radoti, S. Doen, M. Markovi, D. Farina, and T. Keller. Integrated and flexible multichannel interface for electrotactile stimulation. J. Neural Eng. 13:046014, 2016.
Tyler, D. J. Neural interfaces for somatosensory feedback: bringing life to a prosthesis. Curr. Opin. Neurol. 28:574–581, 2015.
Volkmar, R., S. Dosen, J. Gonzalez-Vargas, M. Baum, and M. Markovic. Improving bimanual interaction with a prosthesis using semi-autonomous control. J. NeuroEng. Rehabil. 16:140, 2019.
Wettels, N., V. J. Santos, R. S. Johansson, and G. E. Loeb. Biomimetic tactile sensor array. Adv. Rob. 22:829–849, 2008.
Wu, Y., Y. Liu, Y. Zhou, Q. Man, C. Hu, W. Asghar, F. Li, Z. Yu, J. Shang, G. Liu, M. Liao, and R.-W. Li. A skin-inspired tactile sensor for smart prosthetics. Sci. Rob. 2018. https://doi.org/10.1126/scirobotics.aat0429.
Xu, H., D. Zhang, J. C. Huegel, W. Xu, and X. Zhu. Effects of different tactile feedback on myoelectric closed-loop control for grasping based on electrotactile stimulation. IEEE Trans. Neural Syst. Rehabil. Eng. 24:827–836, 2016.
Xue, N., T. Sun, W. M. Tsang, I. Delgado-Martinez, S.-H. Lee, S. Sheshadri, Z. Xiang, S. Merugu, Y. Gu, S.-C. Yen, and N. V. Thakor. Polymeric C-shaped cuff electrode for recording of peripheral nerve signal. Sens. Actuators B Chem. 210:640–648, 2015.
Yeo, J. C., Z. Liu, Z. Zhang, P. Zhang, Z. Wang, and C. T. Lim. Wearable mechanotransduced tactile sensor for haptic perception. Adv. Mater. Technol. 2:1700006, 2017.
Yi, Z., and Y. Zhang. Bio-inspired tactile FA-I spiking generation under sinusoidal stimuli. J. Bionic Eng. 13:612–621, 2016.
Yi, Z., Y. Zhang, and J. Peters. Bioinspired tactile sensor for surface roughness discrimination. Sens. Actuators A 2017. https://doi.org/10.1016/j.sna.2016.12.021.
Yousef, H., M. Boukallel, and K. Althoefer. Tactile sensing for dexterous in-hand manipulation in robotics—a review. Sens. Actuators A 167:171–187, 2011.
Zhang, T., L. Jiang, and H. Liu. Design and functional evaluation of a dexterous myoelectric hand prosthesis with biomimetic tactile sensor. IEEE Trans. Neural Syst. Rehabil. Eng. 26:1391–1399, 2018.
Zhao, H., K. O’Brien, S. Li, and R. F. Shepherd. Optoelectronically innervated soft prosthetic hand via stretchable optical waveguides. Sci. Rob. 1:eaai7529, 2016.
Zhu, G., W. Q. Yang, T. Zhang, Q. Jing, J. Chen, Y. S. Zhou, P. Bai, and Z. L. Wang. Self-powered, ultrasensitive, flexible tactile sensors based on contact electrification. Nano Lett. 14:3208, 2014.
Zollo, L., G. Di Pino, A. L. Ciancio, F. Ranieri, F. Cordella, C. Gentile, E. Noce, R. A. Romeo, A. D. Bellingegni, G. Vadalà, S. Miccinilli, A. Mioli, L. Diaz-Balzani, M. Bravi, K.-P. Hoffmann, A. Schneider, L. Denaro, A. Davalli, E. Gruppioni, R. Sacchetti, S. Castellano, V. Di Lazzaro, S. Sterzi, V. Denaro, and E. Guglielmelli. Restoring tactile sensations via neural interfaces for real-time force-and-slippage closed-loop control of bionic hands. Sci. Rob. 4:e9924, 2019.
Zou, L., C. Ge, Z. J. Wang, E. Cretu, and X. Li. Novel tactile sensor technology and smart tactile sensing systems: a review. Sensors 17:2653, 2017.
Zou, Z., C. Zhu, Y. Li, X. Lei, W. Zhang, and J. Xiao. Rehealable, fully recyclable, and malleable electronic skin enabled by dynamic covalent thermoset nanocomposite. Sci. Adv. 4:eaaq0508, 2018.
Acknowledgments
The authors thank Robert Wright, MLS for his guidance and expertise with database searching in order to find relevant publications. We additionally recognize the contributions of Brice Lapin, Jack Wright, and Yiyuan Zhang in the preliminary literature search and data collection. Select illustrations for this article were created with Biorender.com following an academic licensing agreement.
Author contribution
AA, AM, SS, HK, and KD contributed to the conception and design. AM, SS, HK, and KD performed literature search, data analysis and drafted the manuscript. XL provided critical revision of the work. AA supervised and reviewed this work.
Funding
This study did not receive any external funding. This article was in part supported by the Start-Up Tier 1 grant # 21.4531.162640 (PI: A. ALL) and by the Faculty Seed Fund # 31.4531.179234 (PI: A. ALL) from Hong Kong Baptist University (HKBU), Hong Kong.
Conflict of interest
All authors declare no conflict of interest and have nothing to disclose.
Author information
Authors and Affiliations
Corresponding authors
Additional information
Associate Editor Xiaoxiang Zheng oversaw the review of this article.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Masteller, A., Sankar, S., Kim, H.B. et al. Recent Developments in Prosthesis Sensors, Texture Recognition, and Sensory Stimulation for Upper Limb Prostheses. Ann Biomed Eng 49, 57–74 (2021). https://doi.org/10.1007/s10439-020-02678-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10439-020-02678-8