Abstract
In this project, we propose an algorithm to convert musical features and structures extracted from monophonic MIDI files to tactile illusions. Mapping music to vibrotactile stimuli is a challenging process since the perceptible frequency range of the skin is lower than that of the auditory system, which may cause the loss of some musical features. Moreover, current proposed models do not warrant the correspondence between the emotional response to music and the vibrotactile version of it. We propose to use tactile illusions as an additional resource to convey more meaningful vibrotactile stimuli. Tactile illusions enable us to add dynamics to vibrotactile stimuli in the form of movement, changes of direction, and localization. The suggested algorithm converts monophonic MIDI files into arrangements of two tactile illusions: “phantom motion” and “funneling”. The validation of the rendered material consisted of presenting the audio rendered from MIDI files to participants and then adding the vibrotactile component to it. The arrangement of tactile illusions was also evaluated alone. Results suggest that the arrangement of tactile illusions evokes more positive emotions than negative ones. This arrangement was also perceived as more agreeable and stimulating than the original audio. Although musical features such as rhythm, tempo, and melody were mostly recognized in the arrangement of tactile illusions, it provoked a different emotional response from that of the original audio.
Supported by the Spanish Government under contract PID2021-125091OB-I00.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bellicha, A., Trujillo-Leon, A., Bachta, W.: Phantom sensation: when the phantom escapes the bounds of the actuators and the end-point is sensed in the air. In: 2019 IEEE World Haptics Conference (WHC), pp. 91–96. IEEE, July 2019. https://doi.org/10.1109/WHC.2019.8816176, https://ieeexplore.ieee.org/document/8816176/
Israr, A., Poupyrev, I.: Tactile brush. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2019–2028. ACM, New York, NY, USA, May 2011. https://doi.org/10.1145/1978942.1979235
Seo, J., Choi, S.: Initial study for creating linearly moving vibrotactile sensation on mobile device. In: 2010 IEEE Haptics Symposium, pp. 67–70. IEEE, March 2010. https://doi.org/10.1109/HAPTIC.2010.5444677, http://ieeexplore.ieee.org/document/5444677/
Karam, M., Russo, F., Branje, C., Fels, D.I.: Towards a model human cochlea: sensory substitution for crossmodal audio-tactile displays. In: Graphics Interface Conference, pp. 267–274 (2008). https://doi.org/10.5555/1375714.1375759
Karam, M., Russo, F.A., Fels, D.I.: Designing the model human cochlea: an ambient crossmodal audio-tactile display. IEEE Trans. Haptics 2(3), 160–169 (2009). https://doi.org/10.1109/TOH.2009.32
Lee, J., Kim, Y., Jounghyun Kim, G.: Rich pinch: perception of object movement with tactile illusion. IEEE Trans. Haptics 9(1), 80–89 (2016). https://doi.org/10.1109/TOH.2015.2475271, https://ieeexplore.ieee.org/document/7234937/
McAngus Todd, N.P.: The kinematics of musical expression. J. Acoust. Soc. Am. 97(3), 1940 (1998). https://doi.org/10.1121/1.412067, https://asa.scitation.org/doi/abs/10.1121/1.412067
Merchel, S., Altinsoy, M.E.: Auditory-tactile experience of music. In: Papetti, S., Saitis, C. (eds.) Musical Haptics. SSTHS, pp. 123–148. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-58316-7_7
Remache-Vinueza, B., Trujillo-León, A., Zapata, M., Sarmiento-Ortiz, F., Vidal-Verdú, F.: Audio-tactile rendering: a review on technology and methods to convey musical information through the sense of touch. Sensors 21(19), 6575 (2021). https://doi.org/10.3390/S21196575, https://www.mdpi.com/1424-8220/21/19/6575/htm www.mdpi.com/1424-8220/21/19/6575
Vieillard, S., Peretz, I., Gosselin, N., Khalfa, S., Gagnon, L., Bouchard, B.: Happy, sad, scary and peaceful musical excerpts for research on emotions 22(4), 720–752 (2008). https://doi.org/10.1080/02699930701503567, https://www.tandfonline.com/doi/abs/10.1080/02699930701503567
Yao, H.Y., Hayward, V.: Design and analysis of a recoil-type vibrotactile transducer. J. Acoust. Soc. Am. 128(2), 619–627 (2010). https://doi.org/10.1121/1.3458852, http://asa.scitation.org/doi/10.1121/1.3458852
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Remache-Vinueza, B. et al. (2022). Mapping Monophonic MIDI Tracks to Vibrotactile Stimuli Using Tactile Illusions. In: Saitis, C., Farkhatdinov, I., Papetti, S. (eds) Haptic and Audio Interaction Design. HAID 2022. Lecture Notes in Computer Science, vol 13417. Springer, Cham. https://doi.org/10.1007/978-3-031-15019-7_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-15019-7_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-15018-0
Online ISBN: 978-3-031-15019-7
eBook Packages: Computer ScienceComputer Science (R0)