3D spider webs exhibit highly intricate fiber architectures and owe their outstanding performance to a hierarchical organization that spans orders of magnitude in length scale from the molecular silk protein, to micrometer-sized fibers, and up to cm-scale web. Similarly, but in a completely different physical manifestation, music has a hierarchical structure composed of elementary sine wave building blocks that can be combined with other waveforms to create complex timbres, which are then arranged within larger-scale musical compositions. Although apparently different, spider webs and music have many similarities, as we point out in this work. Here, we propose an intuitive and interactive way to explore and visualize a 3D Cyrtophora citricola spider web geometry that has been digitally modeled with micron-scale details from full-scale laboratory experiments. We use model-based sonification to translate the web architecture into sound, allowing for aural perception and interpretation of its essential topological features. We implement this sonification using Unity3D and Max/MSP to create an interactive spider web environment in which a user travels through a virtual spider web. Each silk fiber in their field of view is sonified using different sine waves. Together, the sonified fibers create new and more complex timbres that reflects the architecture of 3D spider webs. These concepts are implemented into a spider web-based instrument for live performances, art installations and data exploration. It provides an unprecedented and creative way to immerse the composer, audience and user in an immersive multimedia experience generated by the complexity of a 3D spider web.
This is a preview of subscription content,to check access.
Access this article
Availability of data and material
The spider web images and model are available from the corresponding author on reasonable request.
The sonification model (Unity3D game and Max/MSP patch) are available from the corresponding author on reasonable request.
Su I, Qin Z, Bisshop A et al (2021) Sonification of a 3-D spider web and reconstitution for musical composition using granular synthesis. Comput Music J (in Press)
Hermann T, Hunt A, Neuhoff JG (2011) The sonification handbook. Logos Verlag, Berlin
Bovermann T, Hermann T, Ritter H (2006) Tangible data scanning sonification model. Georgia Institute of Technology, Atlanta
Kramer G, Walker BN, Bonebright T et al (1999) The sonification report: status of the field and research agenda. Report prepared for the national science foundation by members of the international community for auditory display. Int Community Audit Disp (ICAD), St Fe, NM
Pauletto S, Hunt A (2009) Interactive sonification of complex data. Int J Hum Comput Stud 67:923–933. https://doi.org/10.1016/j.ijhcs.2009.05.006
Idreos S, Papaemmanouil O, Chaudhuri S (2015) Overview of data exploration techniques. In: Proceedings of the 2015 ACM SIGMOD international conference on management of data, pp 277–281
Papachristodoulou P, Betella A, Verschure P (2014) Sonification of large datasets in a 3D immersive environment: a neuroscience case study. In: ACHI2014: the seventh international conference on advances in computer-human interactions, pp 35–40
Yu CH, Qin Z, Martin-Martinez F, Buehler MJ (2019) A self-consistent sonification method to translate amino acid sequences into musical compositions and application in protein design using AI. ACS Nano 13:7471–7482
Jeon M, Smith MT, Walker JW, Kuhl SA (2014) Constructing the immersive interactive sonification platform (iISoP). In: Streitz N, Markopoulos P (eds) Distributed, ambient, and pervasive interactions. Springer, Cham, pp 337–348
Barrett N (2016) Interactive spatial sonification of multidimensional data for composition and auditory display. Comput Music J 40:47–69. https://doi.org/10.1162/COMJ_a_00358
Gu GX, Su I, Sharma S et al (2016) Three-dimensional-printing of bio-inspired composites. J Biomech Eng 138:21006
Su I, Buehler MJ (2016) Spider silk: dynamic mechanics. Nat Mater 15:1054–1055. https://doi.org/10.1038/nmat4721
Su I, Buehler MJ (2016) Nanomechanics of silk: the fundamentals of a strong, tough and versatile material. Nanotechnology 27:302001. https://doi.org/10.1088/0957-4484/27/30/302001
Su I, Jung GS, Narayanan N, Buehler MJ (2020) Perspectives on three-dimensional printing of self-assembling materials and structures. Curr Opin Biomed Eng 15:59–67. https://doi.org/10.1016/j.cobme.2020.01.003
Giesa T, Spivak D, Buehler M (2011) Reoccurring patterns in hierarchical protein materials and music: the power of analogies. Bionanoscience 1:153
Wong JY, McDonald J, Taylor-Pinney M et al (2012) Materials by design: merging proteins and music. Nano Today 7:488–495. https://doi.org/10.1016/j.nantod.2012.09.001
Mortimer B (2017) Biotremology: do physical constraints limit the propagation of vibrational information? Anim Behav 130:165–174. https://doi.org/10.1016/j.anbehav.2017.06.015
Hill PSM, Wessel A (2016) Biotremology. Curr Biol 26:R187–R191
Mortimer B, Soler A, Siviour CR et al (2016) Tuning the instrument: sonic properties in the spider’s web. J R Soc Interface. https://doi.org/10.1098/rsif.2016.0341
Mortimer B, Gordon S, Drodge D et al (2013) Sonic properties of silks. XIV Int Conf Invertebr Sound Vib ISV 2013:43
Saraceno T, Bisshop A, Krell A, Mühlethaler R (2019) Arachnid orchestras: artistic research in vibrational interspecies communication. In: Biotremology: studying vibrational behavior. Springer, pp 485–509
Su I, Qin Z, Saraceno T et al (2018) Imaging and analysis of a threedimensional spider web architecture. J R Soc Interface. https://doi.org/10.1098/rsif.2018.0193
Wulff C (2010) Zu Hause bei der Schwarzen Witwe oder das letzte Projekt mit Rolf-Dieter Düppe. Festschrift anlässlich der Pensionierung von Dr-Ing Rolf-Dieter Düppe nach 36 Jahren am Inst für Photogramm und Kartographie; Schriftenr/Fachrichtung Geodäsie 30:8
Luhmann T (2017) Nahbereichsphotogrammetrie, Errata. Wichmann Verlag, Heidelberg
Luhmann T (2010) 3D-Rekonstruktion von Spinnennetzen. Wichmann Verlag, Heidelberg
Papachristodoulou P, Betella A, Manzolli J, Verschure PFMJ (2015) Augmenting the navigation of complex data sets using sonification: a case study with BrainX3 Panagiota. In: 2015 IEEE 2nd VR Workshop on sonic interactions for virtual environments, SIVE 2015—proceedings. Institute of Electrical and Electronics Engineers Inc., pp 3–8
Rossing TD, SpringerLink (2010) The science of string instruments. [electronic resource]. Springer, New York, 2010
Bennett WR, Holland CK (2018) The science of musical sound. Springer, Berlin
Robinson DW, Dadson RS (1956) A re-determination of the equal-loudness relations for pure tones. Br J Appl Phys 7:166–181. https://doi.org/10.1088/0508-3443/7/5/302
Chadabe J (1984) Interactive composing: an overview. Comput Music J. https://doi.org/10.2307/3679894
Engelmann S (2016) Social spiders and hybrid webs at Studio Tomás Saraceno. Cult Geogr 24:161–169. https://doi.org/10.1177/1474474016647371
Vets T, Nijs L, Lesaffre M et al (2017) Gamified music improvisation with BilliArT: a multimodal installation with balls. J Multimodal User Interfaces 11:25–38. https://doi.org/10.1007/s12193-016-0224-9
MATLAB (2019) 188.8.131.529231 (R2019a). The MathWorks Inc., Natick, Massachusetts
Sporns O (2013) The human connectome: origins and challenges. Neuroimage 80:53–61. https://doi.org/10.1016/j.neuroimage.2013.03.023
Lindborg P (2018) Interactive sonification of weather data for the locust wrath, a multimedia dance performance. Leonardo 51:466–474
Haixia Z, Smith BK, Norman K et al (2005) Interactive sonification of choropleth maps. IEEE Multimed 12:26–35. https://doi.org/10.1109/MMUL.2005.28
Zotkin DYN, Hwang J, Duraiswaini R, Davis LS (2003) HRTF personalization using anthropometric measurements. In: 2003 IEEE workshop on applications of signal processing to audio and acoustics (IEEE Cat. No.03TH8684). pp 157–160
This research was supported by the MIT Center for Art, Science and Technology (CAST) program, The Mellon Foundation, and Studio Tomás Saraceno GmbH. We are grateful for the inspiration from Studio Saraceno to develop this instrument for a public performance in the context of his exhibition ON AIR at the Palais de Tokyo, Paris in November 2018, supported by Festival d'Automne à Paris and curated by Rebecca Lamarche-Vadel. Additional support provided by the Office of Naval Research (grant # N00014-16-1-2333), the National Institutes of Health (NIH) U01EB014976, and The Mellon Foundation.
Conflict of interest
We declare we have no conflicts of interest and competing interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Below is the link to the electronic supplementary material.
About this article
Cite this article
Su, I., Hattwick, I., Southworth, C. et al. Interactive exploration of a hierarchical spider web structure with sound. J Multimodal User Interfaces 16, 71–85 (2022). https://doi.org/10.1007/s12193-021-00375-x