Abstract
Cochlear implants (CIs) allow individuals that can no longer benefit from hearing aids to understand speech with remarkable efficiency. On the other hand, they perform poorly in music perception. Previous research suggest that music experience can be enhanced with the use of other senses such as touch. We present Tickle Tuner, a haptic feedback device suitable for musical training of CI users. The prototype is composed of two high-quality haptic actuators and an external Digital to Analogue Converter (DAC) hosted in a 3D printed enclosure coupled with a smartphone. We describe the design and implementation of the prototype and the analysis of its characteristics. We introduce a test bench for the design of different mappings between sound and vibrations which we assessed with a Melodic Contour Identification (MCI) task. Results from a group of fifteen normal hearing participants using CIs simulation showed significantly higher performance (increase of 26% more correct answers) with haptic feedback than without.
In collaboration with Oticon Medical.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
https://openscad.org, last access July 18, 2022.
- 2.
https://alicevision.org/#meshroom, last access July 18, 2022.
- 3.
https://www.blender.org/, last access July 18, 2022.
- 4.
https://ultimaker.com/3d-printers/ultimaker-3, last access July 18, 2022.
- 5.
https://www.actronika.com/, last access July 18, 2022.
- 6.
https://puredata.info/, last access July 18, 2022.
- 7.
https://danieliglesia.com/mobmuplat/, last access July 18, 2022.
- 8.
https://audiomodeling.com/swam-engine/, last access July 18, 2022.
- 9.
https://www.xlnaudio.com/products/addictive_keys, last access July 18, 2022.
- 10.
http://www.tigerspeech.com/angelsim/angelsim_about, last access July 18, 2022.
- 11.
https://www.reaper.fm/, last access July 18, 2022.
References
Birnbaum, D.M., Wanderley, M.M.: A systematic approach to musical vibrotactile feedback, p. 8 (2011)
Chafe, C.: Tactile audio feedback. In: Proceedings of the International Computer Music Conference, pp. 76–76. International computer music accociation (1993)
Cowan, R.S., Sarant, J.Z., Galvin, K.L., Alcantara, J.I., Blamey, P.J., Clark, G.M.: The tickle talker: a speech perception aid for profoundly hearing impaired children. Sci. Publ. 5(300), 1989–1990 (1990)
Dementyev, A., Getreuer, P., Kanevsky, D., Slaney, M., Lyon, R.: VHP: vibrotactile haptics platform for on-body applications, p. 15 (2021)
Drennan, W.R., et al.: Clinical evaluation of music perception, appraisal and experience in cochlear implant users. Int. J. Audiol. 54(2), 114–123 (2015). https://doi.org/10.3109/14992027.2014.948219
Zeng, F.G., Rebscher, S., Harrison, W., Sun, X., Feng, H.,: Cochlear implants: system design, integration, and evaluation. IEEE Rev. Biomed. Eng. 1, 115–142 (2008). https://doi.org/10.1109/RBME.2008.2008250
Farina, A.: Simultaneous measurement of impulse response and distortion with a swept-sine technique, p. 24 (2000)
Fletcher, M.D.: Can haptic stimulation enhance music perception in hearing-impaired listeners? Front. Neurosci. 15, 723877 (2021). https://doi.org/10.3389/fnins.2021.723877
Fletcher, M.D., Cunningham, R.O., Mills, S.R.: Electro-haptic enhancement of spatial hearing in cochlear implant users. Sci. Rep. 10(1), 1621 (2020). https://doi.org/10.1038/s41598-020-58503-8
Fletcher, M.D., Hadeedi, A., Goehring, T., Mills, S.R.: Electro-haptic enhancement of speech-in-noise performance in cochlear implant users. Sci. Rep. 9(1), 11428 (2019). https://doi.org/10.1038/s41598-019-47718-z
Galvin, J.J., Fu, Q.J., Nogaki, G.: Melodic contour identification by cochlear implant listeners. Ear Hear. 28(3), 302–319 (2007). https://doi.org/10.1097/01.aud.0000261689.35445.20
Galvin, J.J., Fu, Q.J., Shannon, R.V.: Melodic contour identification and music perception by cochlear implant users. Ann. N. Y. Acad. Sci. 1169(1), 518–533 (2009). https://doi.org/10.1111/j.1749-6632.2009.04551.x
Garcia-Valle, G., Ferre, M., Brenosa, J., Vargas, D.: Evaluation of presence in virtual environments: haptic vest and user’s haptic skills. IEEE Access 6, 7224–7233 (2018). https://doi.org/10.1109/ACCESS.2017.2782254
Park, G., Choi, S.: Perceptual space of amplitude-modulated vibrotactile stimuli. In: 2011 IEEE World Haptics Conference, pp. 59–64. IEEE, Istanbul, June 2011. https://doi.org/10.1109/WHC.2011.5945462
Jiam, N.T., Caldwell, M.T., Limb, C.J.: What does music sound like for a cochlear implant user? Otol. Neurotology 38(8), e240–e247 (2017). https://doi.org/10.1097/MAO.0000000000001448
Jones, L.A., Sarter, N.B.: Tactile displays: guidance for their design and application. Hum. Fact. J. Hum. Fact. Ergon. Soc. 50(1), 90–111 (2008). https://doi.org/10.1518/001872008X250638
Karam, M., Russo, F., Fels, D.: Designing the model human cochlea: an ambient crossmodal audio-tactile display. IEEE Trans. Haptics 2(3), 160–169 (2009). https://doi.org/10.1109/TOH.2009.32
Limb, C.J., Roy, A.T.: Technological, biological, and acoustical constraints to music perception in cochlear implant users. Hear. Res. 308, 13–26 (2014). https://doi.org/10.1016/j.heares.2013.04.009
Looi, V., Mcdermott, H., McKay, C.M., Hickson, L.: The effect of cochlear implantation on music perception by adults with usable pre-operative acoustic hearing. Int. J. Audiol. 47, 257–268 (2008)
Luo, X., Hayes, L.: Vibrotactile stimulation based on the fundamental frequency can improve melodic contour identification of normal-hearing listeners with a 4-channel cochlear implant simulation. Front. Neurosci. 13, 1145 (2019). https://doi.org/10.3389/fnins.2019.01145
Merchel, S., Altinsoy, M.E.: Psychophysical comparison of the auditory and tactile perception: a survey. J. Multi. User Interfaces 14(3), 271–283 (2020). https://doi.org/10.1007/s12193-020-00333-z
Omran, S.A., Lai, W., Dillier, N.: Pitch ranking, melody contour and instrument recognition tests using two semitone frequency maps for nucleus cochlear implants. EURASIP J. Audio Speech Music Process. 2010(1), 1–16 (2010). https://doi.org/10.1155/2010/948565
Russo, F.A., Ammirante, P., Fels, D.I.: Vibrotactile discrimination of musical timbre. J. Exp. Psychol. Hum. Percept. Perform. 38(4), 822–826 (2012). https://doi.org/10.1037/a0029046
Singhal, T., Schneider, O.: Juicy haptic design: vibrotactile embellishments can improve player experience in games, p. 11 (2021)
Tan, H.Z., Gray, R., Young, J.J., Traylor, R.: A haptic back display for attentional and directional cueing, p. 20 (2003)
Verrillo, R.T.: Vibration sensation in humans. Music Percept. 9(3), 281–302 (1992). https://doi.org/10.2307/40285553
Weber, M., Saitis, C.: Towards a framework for ubiquitous audio-tactile design, p. 9 (2020)
Wu, J., Zhang, J., Yan, J., Liu, W., Song, G.: Design of a vibrotactile vest for contour perception. Int. J. Adv. Rob. Syst. 9(5), 166 (2012). https://doi.org/10.5772/52373
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ganis, F., Vatti, M., Serafin, S. (2022). Tickle Tuner - Haptic Smartphone Cover for Cochlear Implant Users’ Musical Training. In: Saitis, C., Farkhatdinov, I., Papetti, S. (eds) Haptic and Audio Interaction Design. HAID 2022. Lecture Notes in Computer Science, vol 13417. Springer, Cham. https://doi.org/10.1007/978-3-031-15019-7_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-15019-7_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-15018-0
Online ISBN: 978-3-031-15019-7
eBook Packages: Computer ScienceComputer Science (R0)