Using space and time to encode vibrotactile information: toward an estimate of the skin’s achievable throughput

Abstract

Touch receptors in the skin can relay various forms of abstract information, such as words (Braille), haptic feedback (cell phones, game controllers, feedback for prosthetic control), and basic visual information such as edges and shape (sensory substitution devices). The skin can support such applications with ease: They are all low bandwidth and do not require a fine temporal acuity. But what of high-throughput applications? We use sound-to-touch conversion as a motivating example, though others abound (e.g., vision, stock market data). In the past, vibrotactile hearing aids have demonstrated improvement in speech perceptions in the deaf. However, a sound-to-touch sensory substitution device that works with high efficacy and without the aid of lipreading has yet to be developed. Is this because skin simply does not have the capacity to effectively relay high-throughput streams such as sound? Or is this because the spatial and temporal properties of skin have not been leveraged to full advantage? Here, we begin to address these questions with two experiments. First, we seek to determine the best method of relaying information through the skin using an identification task on the lower back. We find that vibrotactile patterns encoding information in both space and time yield the best overall information transfer estimate. Patterns encoded in space and time or “intensity” (the coupled coding of vibration frequency and force) both far exceed performance of only spatially encoded patterns. Next, we determine the vibrotactile two-tacton resolution on the lower back—the distance necessary for resolving two vibrotactile patterns. We find that our vibratory motors conservatively require at least 6 cm of separation to resolve two independent tactile patterns (>80 % correct), regardless of stimulus type (e.g., spatiotemporal “sweeps” versus single vibratory pulses). Six centimeter is a greater distance than the inter-motor distances used in Experiment 1 (2.5 cm), which explains the poor identification performance of spatially encoded patterns. Hence, when using an array of vibrational motors, spatiotemporal sweeps can overcome the limitations of vibrotactile two-tacton resolution. The results provide the first steps toward obtaining a realistic estimate of the skin’s achievable throughput, illustrating the best ways to encode data to the skin (using as many dimensions as possible) and how far such interfaces would need to be separated if using multiple arrays in parallel.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. Bach-y-Rita P, Collins CC, Saunders FA, White B, Scadeen L (1969) Vision substitution by tactile image projection. Nature 221(5184):963–964. doi:10.1038/221963a0

    CAS  Article  PubMed  Google Scholar 

  2. Bensmaïa SJ, Leung YY, Hsiao SS, Johnson KO (2005) Vibratory adaptation of cutaneous mechanoreceptive afferents. J Neurophysiol 94(5):3023–3036. doi:10.1152/jn.00002.2005

    PubMed Central  Article  PubMed  Google Scholar 

  3. Bikah M, Hallbeck MS, Flowers JH (2008) Supracutaneous vibrotactile perception threshold at various non-glabrous body loci. Ergonomics 51(6):920–934. doi:10.1080/00140130701809341

    Article  PubMed  Google Scholar 

  4. Brooks PL, Frost BJ, Mason JL, Gibson DM (1986a) Continuing evaluation of the Queen’s University Tactile Vocoder II: identification of open set sentences and tracking narrative. J Rehabil Res Dev 23(1):129–138

    CAS  PubMed  Google Scholar 

  5. Brooks PL, Frost BJ, Mason JL, Gibson DM (1986b) Continuing evaluation of the Queen’s University Tactile Vocoder. I: identification of open set words. J Rehabil Res Dev 23(1):119–128

    CAS  PubMed  Google Scholar 

  6. Cain WS (1973) Spatial discrimination of cutaneous warmth. Am J Psychol 86(1):169–181

    CAS  Article  PubMed  Google Scholar 

  7. Chamberlain MW (2001) A 600 bps MELP vocoder for use on HF channels. In: 2001 MILCOM Proceedings Communications for Network-Centric Operations: Creating the Information Force (Cat. No.01CH37277), vol. 1. IEEE, p. 447–453. doi:10.1109/MILCOM.2001.985836

  8. Chebat D-R, Schneider FC, Kupers R, Ptito M (2011) Navigation with a sensory substitution device in congenitally blind individuals. NeuroReport 22(7):342–347. doi:10.1097/WNR.0b013e3283462def

    Article  PubMed  Google Scholar 

  9. Cholewiak RW, Collins AA (1995) Vibrotactile pattern discrimination and communality at several body sites. Percept Psychophys, 57(5):724–737. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/7644331

  10. Cholewiak RW, Craig JC (1984) Vibrotactile pattern recognition and discrimination at several body sites. Percept Psychophys, 35(6):503–514. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/6483552

  11. Cohen B, Kirman JH (1986) Vibrotactile frequency discrimination at short durations. J Gen Psychol 113(2):179–186

    CAS  Article  PubMed  Google Scholar 

  12. Craig JC (1982) Temporal integration of vibrotactile patterns. Percept Psychophys, 32(3):219–229. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/7177760

  13. Craig JC (2002) Identification of scanned and static tactile patterns. Percept Psychophys, 64(1):107–20. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/11916294

  14. Ellis EM, Robinson AJ (1993) A phonetic tactile speech listening system. Engineering, p. 1–17

  15. Enriquez M, Maclean KE (2008) Backward and common-onset masking of vibrotactile stimuli. Brain Res Bull 75(6):761–769. doi:10.1016/j.brainresbull.2008.01.018

    Article  PubMed  Google Scholar 

  16. Evans PM, Craig JC (1991) Tactile attention and the perception of moving tactile stimuli. Percept Psychophys, 49(4):355–64. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/2030933

  17. Galvin K, Mavrias G, Moore A, Cowan R, Blamey P, Clark G (1999) A comparison of tactaid II and tactaid 7 use by adults with a profound hearing impairment. Ear Hear 20(6):471

    CAS  Article  PubMed  Google Scholar 

  18. Galvin KL, Ginis J, Cowan RSC, Blamey PJ, Clark GM (2001) A comparison of a new prototype tickle talker™ with the tactaid 7. Aust N Z J Audiol 23(1):18–36

    Article  Google Scholar 

  19. Gault RH (1924) Progress in experiments on tactual interpretation of oral speech. J Abnorm Psychol Soc Psychol 19(2):155–159. doi:10.1037/h0065752

    Article  Google Scholar 

  20. Geldard FA (1957) Adventures in tactile literacy. Am Psychol 12(3):115–124. doi:10.1037/h0040416

    Article  Google Scholar 

  21. Geldard FA, Sherrick CE (1965) Multiple cutaneous stimulation: the discrimination of vibratory patterns. J Acoust Soc Am 37(5):797–801

    CAS  Article  PubMed  Google Scholar 

  22. Gescheider GA, Bolanowski SJ, Verrillo RT (2004) Some characteristics of tactile channels. Behav Brain Res 148(1–2):35–40. doi:10.1016/S0166-4328(03)00177-3

    CAS  Article  PubMed  Google Scholar 

  23. Gleeson BT, Horschel SK, Provancher WR (2009) Communication of direction through lateral skin stretch at the fingertip. In: World Haptics 2009—Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, p. 172–177. IEEE. doi:10.1109/WHC.2009.4810804

  24. Goble AK, Hollins M (1994) Vibrotactile adaptation enhances frequency discrimination. J Acoust Soc Am 96(2):771. doi:10.1121/1.410314

    CAS  Article  PubMed  Google Scholar 

  25. Grosjean F (1979) A study of timing in a manual and a spoken language: American sign language and English. J Psycholinguist Res, 8(4):379–405. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/490444

  26. Hayward V, Cruz-Hernandez JM (2000) Tactile display device using distributed lateral skin stretch. In: Proceedings of the Haptic Interfaces for Virtual Environment and Teleoperator Systems Symposium, ASME International Mechanical Engineering Congress & Exposition, p. 1309–1314. Orlando

  27. Held R, Hein A (1963) Movement-produced stimulation in the development of visually guided behavior. J Comp Physiol Psychol, 56(5):872–876. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/14050177

  28. ITU (1993) ITU-T Recommendation G.711. In: Telecommunication Policies for the Americas: The Blue Book. Geneva

  29. James W (1890) The principles of psychology, vol 1. Henry Holt and Company, New York

    Google Scholar 

  30. Jones LA (2011) Tactile communication systems optimizing: the display of information. Prog Brain Res 192:113–128. doi:10.1016/B978-0-444-53355-5.00008-7

    Article  PubMed  Google Scholar 

  31. Jones LA, Berris M (2002) The psychophysics of temperature perception and thermal-interface design. In: Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002. IEEE Comput. Soc, p 137–142. doi:10.1109/HAPTIC.2002.998951

  32. Lamel LF, Kassel RH, Seneff S (1989) Speech database development: design and analysis of the acoustic-phonetic corpus. In: SIOA, vol 2. Noordwijkerhout, p 161–170

  33. Leung YY, Bensmaïa SJ, Hsiao SS, Johnson KO (2005) Time-course of vibratory adaptation and recovery in cutaneous mechanoreceptive afferents. J Neurophysiol 94(5):3037–3045. doi:10.1152/jn.00001.2005

    PubMed Central  CAS  Article  PubMed  Google Scholar 

  34. Lim S, Holt LL (2011) Learning foreign sounds in an alien world: videogame training improves non-native speech categorization. Cognit Sci 35(7):1390–1405. doi:10.1111/j.1551-6709.2011.01192.x

    Article  Google Scholar 

  35. Mahns DA, Perkins NM, Sahai V, Robinson L, Rowe MJ (2006) Vibrotactile frequency discrimination in human hairy skin. J Neurophysiol 95(3):1442–1450. doi:10.1152/jn.00483.2005

    CAS  Article  PubMed  Google Scholar 

  36. Miller GA (1953) What is information measurement? Am Psychol 8(1):3–11. doi:10.1037/h0057808

    Article  Google Scholar 

  37. Milnes P, Stevens JC, Brown BH, Summers IR, Cooper PG (1996) Use of micro-controller in a tactile aid for the hearing impaired. In: IEEE Engineering In Medicine And Biology, p 413–414

  38. O’Mara S, Rowe MJ, Tarvin RP (1988) Neural mechanisms in vibrotactile adaptation. J Neurophysiol, 59(2):607–22. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/3351576

  39. Phillips A, Thornton A, Worsfold S, Downie A, Milligan J (1994) Vibrotactile aids with the profoundly deafened. Eur J Disord Commun 29(1):17–26

    CAS  Article  PubMed  Google Scholar 

  40. Pongrac H (2006) Vibrotactile perception: differential effects of frequency, amplitude, and acceleration. In: 2006 IEEE International Workshop on Haptic Audio Visual Environments and their Applications (HAVE 2006). IEEE, p 54–59. doi:10.1109/HAVE.2006.283803

  41. Rabinowitz WM, Houtsma AJ, Durlach NI, Delhorne LA (1987) Multidimensional tactile displays: identification of vibratory intensity, frequency, and contactor area. J Acoust Soc Am, 82(4):1243–52. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/3680781

  42. Ranjbar P, Stranneby D, Borg E (2009) Vibrotactile identification of signal-processed sounds from environmental events. J Rehabil Res Dev 46(8):1021. doi:10.1682/JRRD.2008.11.0150

    Article  PubMed  Google Scholar 

  43. Reed CM, Delhorne LA (2003) The reception of environmental sounds through wearable tactual aids. Ear Hear 24(6):528–538. doi:10.1097/01.AUD.0000100207.97243.88

    Article  PubMed  Google Scholar 

  44. Reed CM, Durlach NI (1998) Note on information transfer rates in human communication. Presence Teleoperators Virtual Environ 7(5):509–518. doi:10.1162/105474698565893

    Article  Google Scholar 

  45. Rodman J (2006) The effect of bandwidth on speech intelligibility. White paper, POLYCOM, Inc. Retrieved from http://www.ivci.com/pdf/whitepaper-polycom-bandwith-and-speech-intelligibility.pdf

  46. Ronnberg J, Andersson U, Lyxell B, Spens K-E (1998) Vibrotactile speech tracking support: cognitive prerequisites. J Deaf Stud Deaf Edu, 3(2):143–156. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/15579861

  47. Rothenberg M, Verrillo RT, Zahorian SA, Brachman ML, Bolanowski SJ (1977). Vibrotactile frequency for encoding a speech parameter. J Acoust Soc Am, 62(4):1003–1012. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/908786

  48. Rowe DG (1997) Techniques for harmonic sinusoidal coding. Dissertation, University of South Australia

  49. Rowe D (2011) Codec 2 overview. Retrieved from http://rowetel.com

  50. Scott BL, De Felippo CL (1977) Progress in the development of a tactile aid for the deaf. Acoust Soc Am 62:S76

    Article  Google Scholar 

  51. Summers IR, Chanter CM (2002) A broadband tactile array on the fingertip. J Acoust Soc Am 112(5):2118. doi:10.1121/1.1510140

    Article  PubMed  Google Scholar 

  52. Summers IR, Gratton DA (1995) Choice of speech features for tactile presentation to the profoundly deaf. IEEE Trans Rehabil Eng 3(1):117–121

    Article  Google Scholar 

  53. Summers IR, Cooper PG, Wright P, Gratton DA, Milnes P, Brown BH (1997) Information from time-varying vibrotactile stimuli. J Acoust Soc Am, 102(6):3686–3696. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/9407660

  54. Summers IR, Whybrow JJ, Gratton DA, Milnes P, Brown BH, Stevens JC (2005) Tactile information transfer: a comparison of two stimulation sites. J Acoust Soc Am 118(4):2527. doi:10.1121/1.2031979

    Article  PubMed  Google Scholar 

  55. Tan HZ (1996) Information transmission with a multi-finger tactual display. Dissertation, Massachusetts Institute of Technology

  56. Tan HZ, Gray R, Young JJ, Traylor R (2003) A haptic back display for attentional and directional cueing. Haptics E 3(1):1–20

    Google Scholar 

  57. Tikuisis P, Meunier P, Jubenville CE (2001) Human body surface area: measurement and prediction using three dimensional body scans. Eur J Appl Physiol 85(3–4):264–271. doi:10.1007/s004210100484

    CAS  Article  PubMed  Google Scholar 

  58. Traunmuller H (1980) The sentiphone : a Tactual Speech Communication AID. J Commun Disord 13:183–193

    CAS  Article  PubMed  Google Scholar 

  59. Watanabe J, Amemiya T, Nishida S, Johnston A (2010) Tactile duration compression by vibrotactile adaptation. NeuroReport 21(13):856–860. doi:10.1097/WNR.0b013e32833d6bcb

    Article  PubMed  Google Scholar 

  60. Wedell CH, Cumming SB Jr (1938) Fatigue of the vibratory sense. J Exp Psychol 22(5):429–438

    Article  Google Scholar 

  61. Weinstein S (1968) Intensive and extensive aspects of tactile sensitivity as a function of body part, sex, and laterality. In: Kenshalo DR (ed) The skin senses. Thomas, Springfield, pp 195–222

    Google Scholar 

  62. Weisenberger JM (1989) Evaluation of the siemens minifonator vibrotactile aid. J Speech Hear Res 32(1):24–32

    CAS  Article  PubMed  Google Scholar 

  63. Weisenberger JM, Broadstone S, Kozma-Spytek L (1991a) Relative performance of single-channel and multichannel tactile aids for speech perception. J Rehabil Res Dev 28(2):45. doi:10.1682/JRRD.1991.04.0045

    CAS  Article  PubMed  Google Scholar 

  64. Weisenberger JM, Broadstone SM, Kozma-spytek L (1991b) Relative performance of single-channel and multichannel tactile aids for speech perception. J Rehabil Res. doi:10.1682/JRRD.1991.04.0045

    Google Scholar 

  65. Yuan H, Reed CM, Durlach NI (2005) Tactual display of consonant voicing as a supplement to lipreading. J Acoust Soc Am 118(2):1003. doi:10.1121/1.1945787

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

Funding for this research is supported by a grant from the Renz Neuroscience Initiative (D.M.E.) and by a training fellowship (S.D.N.) from the Keck Center for Interdisciplinary Bioscience Training from the Gulf Coast Consortia (NIBIB Grant No. 5T32EB006350-05).

Author information

Affiliations

Authors

Corresponding author

Correspondence to David M. Eagleman.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 164 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Novich, S.D., Eagleman, D.M. Using space and time to encode vibrotactile information: toward an estimate of the skin’s achievable throughput. Exp Brain Res 233, 2777–2788 (2015). https://doi.org/10.1007/s00221-015-4346-1

Download citation

Keywords

  • Skin
  • Vibrotactile
  • Sound-to-touch
  • Sensory substitution
  • Information transfer