Abstract
Touch receptors in the skin can relay various forms of abstract information, such as words (Braille), haptic feedback (cell phones, game controllers, feedback for prosthetic control), and basic visual information such as edges and shape (sensory substitution devices). The skin can support such applications with ease: They are all low bandwidth and do not require a fine temporal acuity. But what of high-throughput applications? We use sound-to-touch conversion as a motivating example, though others abound (e.g., vision, stock market data). In the past, vibrotactile hearing aids have demonstrated improvement in speech perceptions in the deaf. However, a sound-to-touch sensory substitution device that works with high efficacy and without the aid of lipreading has yet to be developed. Is this because skin simply does not have the capacity to effectively relay high-throughput streams such as sound? Or is this because the spatial and temporal properties of skin have not been leveraged to full advantage? Here, we begin to address these questions with two experiments. First, we seek to determine the best method of relaying information through the skin using an identification task on the lower back. We find that vibrotactile patterns encoding information in both space and time yield the best overall information transfer estimate. Patterns encoded in space and time or “intensity” (the coupled coding of vibration frequency and force) both far exceed performance of only spatially encoded patterns. Next, we determine the vibrotactile two-tacton resolution on the lower back—the distance necessary for resolving two vibrotactile patterns. We find that our vibratory motors conservatively require at least 6 cm of separation to resolve two independent tactile patterns (>80 % correct), regardless of stimulus type (e.g., spatiotemporal “sweeps” versus single vibratory pulses). Six centimeter is a greater distance than the inter-motor distances used in Experiment 1 (2.5 cm), which explains the poor identification performance of spatially encoded patterns. Hence, when using an array of vibrational motors, spatiotemporal sweeps can overcome the limitations of vibrotactile two-tacton resolution. The results provide the first steps toward obtaining a realistic estimate of the skin’s achievable throughput, illustrating the best ways to encode data to the skin (using as many dimensions as possible) and how far such interfaces would need to be separated if using multiple arrays in parallel.
References
Bach-y-Rita P, Collins CC, Saunders FA, White B, Scadeen L (1969) Vision substitution by tactile image projection. Nature 221(5184):963–964. doi:10.1038/221963a0
Bensmaïa SJ, Leung YY, Hsiao SS, Johnson KO (2005) Vibratory adaptation of cutaneous mechanoreceptive afferents. J Neurophysiol 94(5):3023–3036. doi:10.1152/jn.00002.2005
Bikah M, Hallbeck MS, Flowers JH (2008) Supracutaneous vibrotactile perception threshold at various non-glabrous body loci. Ergonomics 51(6):920–934. doi:10.1080/00140130701809341
Brooks PL, Frost BJ, Mason JL, Gibson DM (1986a) Continuing evaluation of the Queen’s University Tactile Vocoder II: identification of open set sentences and tracking narrative. J Rehabil Res Dev 23(1):129–138
Brooks PL, Frost BJ, Mason JL, Gibson DM (1986b) Continuing evaluation of the Queen’s University Tactile Vocoder. I: identification of open set words. J Rehabil Res Dev 23(1):119–128
Cain WS (1973) Spatial discrimination of cutaneous warmth. Am J Psychol 86(1):169–181
Chamberlain MW (2001) A 600 bps MELP vocoder for use on HF channels. In: 2001 MILCOM Proceedings Communications for Network-Centric Operations: Creating the Information Force (Cat. No.01CH37277), vol. 1. IEEE, p. 447–453. doi:10.1109/MILCOM.2001.985836
Chebat D-R, Schneider FC, Kupers R, Ptito M (2011) Navigation with a sensory substitution device in congenitally blind individuals. NeuroReport 22(7):342–347. doi:10.1097/WNR.0b013e3283462def
Cholewiak RW, Collins AA (1995) Vibrotactile pattern discrimination and communality at several body sites. Percept Psychophys, 57(5):724–737. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/7644331
Cholewiak RW, Craig JC (1984) Vibrotactile pattern recognition and discrimination at several body sites. Percept Psychophys, 35(6):503–514. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/6483552
Cohen B, Kirman JH (1986) Vibrotactile frequency discrimination at short durations. J Gen Psychol 113(2):179–186
Craig JC (1982) Temporal integration of vibrotactile patterns. Percept Psychophys, 32(3):219–229. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/7177760
Craig JC (2002) Identification of scanned and static tactile patterns. Percept Psychophys, 64(1):107–20. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/11916294
Ellis EM, Robinson AJ (1993) A phonetic tactile speech listening system. Engineering, p. 1–17
Enriquez M, Maclean KE (2008) Backward and common-onset masking of vibrotactile stimuli. Brain Res Bull 75(6):761–769. doi:10.1016/j.brainresbull.2008.01.018
Evans PM, Craig JC (1991) Tactile attention and the perception of moving tactile stimuli. Percept Psychophys, 49(4):355–64. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/2030933
Galvin K, Mavrias G, Moore A, Cowan R, Blamey P, Clark G (1999) A comparison of tactaid II and tactaid 7 use by adults with a profound hearing impairment. Ear Hear 20(6):471
Galvin KL, Ginis J, Cowan RSC, Blamey PJ, Clark GM (2001) A comparison of a new prototype tickle talker™ with the tactaid 7. Aust N Z J Audiol 23(1):18–36
Gault RH (1924) Progress in experiments on tactual interpretation of oral speech. J Abnorm Psychol Soc Psychol 19(2):155–159. doi:10.1037/h0065752
Geldard FA (1957) Adventures in tactile literacy. Am Psychol 12(3):115–124. doi:10.1037/h0040416
Geldard FA, Sherrick CE (1965) Multiple cutaneous stimulation: the discrimination of vibratory patterns. J Acoust Soc Am 37(5):797–801
Gescheider GA, Bolanowski SJ, Verrillo RT (2004) Some characteristics of tactile channels. Behav Brain Res 148(1–2):35–40. doi:10.1016/S0166-4328(03)00177-3
Gleeson BT, Horschel SK, Provancher WR (2009) Communication of direction through lateral skin stretch at the fingertip. In: World Haptics 2009—Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, p. 172–177. IEEE. doi:10.1109/WHC.2009.4810804
Goble AK, Hollins M (1994) Vibrotactile adaptation enhances frequency discrimination. J Acoust Soc Am 96(2):771. doi:10.1121/1.410314
Grosjean F (1979) A study of timing in a manual and a spoken language: American sign language and English. J Psycholinguist Res, 8(4):379–405. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/490444
Hayward V, Cruz-Hernandez JM (2000) Tactile display device using distributed lateral skin stretch. In: Proceedings of the Haptic Interfaces for Virtual Environment and Teleoperator Systems Symposium, ASME International Mechanical Engineering Congress & Exposition, p. 1309–1314. Orlando
Held R, Hein A (1963) Movement-produced stimulation in the development of visually guided behavior. J Comp Physiol Psychol, 56(5):872–876. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/14050177
ITU (1993) ITU-T Recommendation G.711. In: Telecommunication Policies for the Americas: The Blue Book. Geneva
James W (1890) The principles of psychology, vol 1. Henry Holt and Company, New York
Jones LA (2011) Tactile communication systems optimizing: the display of information. Prog Brain Res 192:113–128. doi:10.1016/B978-0-444-53355-5.00008-7
Jones LA, Berris M (2002) The psychophysics of temperature perception and thermal-interface design. In: Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002. IEEE Comput. Soc, p 137–142. doi:10.1109/HAPTIC.2002.998951
Lamel LF, Kassel RH, Seneff S (1989) Speech database development: design and analysis of the acoustic-phonetic corpus. In: SIOA, vol 2. Noordwijkerhout, p 161–170
Leung YY, Bensmaïa SJ, Hsiao SS, Johnson KO (2005) Time-course of vibratory adaptation and recovery in cutaneous mechanoreceptive afferents. J Neurophysiol 94(5):3037–3045. doi:10.1152/jn.00001.2005
Lim S, Holt LL (2011) Learning foreign sounds in an alien world: videogame training improves non-native speech categorization. Cognit Sci 35(7):1390–1405. doi:10.1111/j.1551-6709.2011.01192.x
Mahns DA, Perkins NM, Sahai V, Robinson L, Rowe MJ (2006) Vibrotactile frequency discrimination in human hairy skin. J Neurophysiol 95(3):1442–1450. doi:10.1152/jn.00483.2005
Miller GA (1953) What is information measurement? Am Psychol 8(1):3–11. doi:10.1037/h0057808
Milnes P, Stevens JC, Brown BH, Summers IR, Cooper PG (1996) Use of micro-controller in a tactile aid for the hearing impaired. In: IEEE Engineering In Medicine And Biology, p 413–414
O’Mara S, Rowe MJ, Tarvin RP (1988) Neural mechanisms in vibrotactile adaptation. J Neurophysiol, 59(2):607–22. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/3351576
Phillips A, Thornton A, Worsfold S, Downie A, Milligan J (1994) Vibrotactile aids with the profoundly deafened. Eur J Disord Commun 29(1):17–26
Pongrac H (2006) Vibrotactile perception: differential effects of frequency, amplitude, and acceleration. In: 2006 IEEE International Workshop on Haptic Audio Visual Environments and their Applications (HAVE 2006). IEEE, p 54–59. doi:10.1109/HAVE.2006.283803
Rabinowitz WM, Houtsma AJ, Durlach NI, Delhorne LA (1987) Multidimensional tactile displays: identification of vibratory intensity, frequency, and contactor area. J Acoust Soc Am, 82(4):1243–52. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/3680781
Ranjbar P, Stranneby D, Borg E (2009) Vibrotactile identification of signal-processed sounds from environmental events. J Rehabil Res Dev 46(8):1021. doi:10.1682/JRRD.2008.11.0150
Reed CM, Delhorne LA (2003) The reception of environmental sounds through wearable tactual aids. Ear Hear 24(6):528–538. doi:10.1097/01.AUD.0000100207.97243.88
Reed CM, Durlach NI (1998) Note on information transfer rates in human communication. Presence Teleoperators Virtual Environ 7(5):509–518. doi:10.1162/105474698565893
Rodman J (2006) The effect of bandwidth on speech intelligibility. White paper, POLYCOM, Inc. Retrieved from http://www.ivci.com/pdf/whitepaper-polycom-bandwith-and-speech-intelligibility.pdf
Ronnberg J, Andersson U, Lyxell B, Spens K-E (1998) Vibrotactile speech tracking support: cognitive prerequisites. J Deaf Stud Deaf Edu, 3(2):143–156. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/15579861
Rothenberg M, Verrillo RT, Zahorian SA, Brachman ML, Bolanowski SJ (1977). Vibrotactile frequency for encoding a speech parameter. J Acoust Soc Am, 62(4):1003–1012. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/908786
Rowe DG (1997) Techniques for harmonic sinusoidal coding. Dissertation, University of South Australia
Rowe D (2011) Codec 2 overview. Retrieved from http://rowetel.com
Scott BL, De Felippo CL (1977) Progress in the development of a tactile aid for the deaf. Acoust Soc Am 62:S76
Summers IR, Chanter CM (2002) A broadband tactile array on the fingertip. J Acoust Soc Am 112(5):2118. doi:10.1121/1.1510140
Summers IR, Gratton DA (1995) Choice of speech features for tactile presentation to the profoundly deaf. IEEE Trans Rehabil Eng 3(1):117–121
Summers IR, Cooper PG, Wright P, Gratton DA, Milnes P, Brown BH (1997) Information from time-varying vibrotactile stimuli. J Acoust Soc Am, 102(6):3686–3696. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/9407660
Summers IR, Whybrow JJ, Gratton DA, Milnes P, Brown BH, Stevens JC (2005) Tactile information transfer: a comparison of two stimulation sites. J Acoust Soc Am 118(4):2527. doi:10.1121/1.2031979
Tan HZ (1996) Information transmission with a multi-finger tactual display. Dissertation, Massachusetts Institute of Technology
Tan HZ, Gray R, Young JJ, Traylor R (2003) A haptic back display for attentional and directional cueing. Haptics E 3(1):1–20
Tikuisis P, Meunier P, Jubenville CE (2001) Human body surface area: measurement and prediction using three dimensional body scans. Eur J Appl Physiol 85(3–4):264–271. doi:10.1007/s004210100484
Traunmuller H (1980) The sentiphone : a Tactual Speech Communication AID. J Commun Disord 13:183–193
Watanabe J, Amemiya T, Nishida S, Johnston A (2010) Tactile duration compression by vibrotactile adaptation. NeuroReport 21(13):856–860. doi:10.1097/WNR.0b013e32833d6bcb
Wedell CH, Cumming SB Jr (1938) Fatigue of the vibratory sense. J Exp Psychol 22(5):429–438
Weinstein S (1968) Intensive and extensive aspects of tactile sensitivity as a function of body part, sex, and laterality. In: Kenshalo DR (ed) The skin senses. Thomas, Springfield, pp 195–222
Weisenberger JM (1989) Evaluation of the siemens minifonator vibrotactile aid. J Speech Hear Res 32(1):24–32
Weisenberger JM, Broadstone S, Kozma-Spytek L (1991a) Relative performance of single-channel and multichannel tactile aids for speech perception. J Rehabil Res Dev 28(2):45. doi:10.1682/JRRD.1991.04.0045
Weisenberger JM, Broadstone SM, Kozma-spytek L (1991b) Relative performance of single-channel and multichannel tactile aids for speech perception. J Rehabil Res. doi:10.1682/JRRD.1991.04.0045
Yuan H, Reed CM, Durlach NI (2005) Tactual display of consonant voicing as a supplement to lipreading. J Acoust Soc Am 118(2):1003. doi:10.1121/1.1945787
Acknowledgments
Funding for this research is supported by a grant from the Renz Neuroscience Initiative (D.M.E.) and by a training fellowship (S.D.N.) from the Keck Center for Interdisciplinary Bioscience Training from the Gulf Coast Consortia (NIBIB Grant No. 5T32EB006350-05).
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Novich, S.D., Eagleman, D.M. Using space and time to encode vibrotactile information: toward an estimate of the skin’s achievable throughput. Exp Brain Res 233, 2777–2788 (2015). https://doi.org/10.1007/s00221-015-4346-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00221-015-4346-1