Abstract
Artificial Intelligence (AI) and Extended Realities (XR) have the potential to create new worlds, new narrative spaces. From our standpoint, to be believable, those worlds need not only to convince our senses, but also our experience reality: a complex, non-linear net of relationships among objects, beings, actions and concepts. In other words, we wanted to create a narrative space rich and complex, in terms of content and behaviour. At times, non-linear and unpredictable.
In Oracle, we explored some boundaries of this idea. Oracle is an installation based on a variable chain of spatialized sound processing algorithms directly controlled by a multilayer perceptron neural network (the “oracle”). The sound input of the system is the voice of any participant who wants to play as a “visitor” (the one who poses a question to the oracle). The output will be a processed version of that input, a slight modification, a completely different sound texture or anything in between these extremes. By wearing a VR headset, the visitor can interact with the oracle and influence the output of the neural network, therefore the sound output. However, it is not possible for any of the users to exactly predict the outcome of such interaction. The oracle’s answer is then a net of sound relationships the user needs to decipher, not to get the answer, but to better understand the question.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Those type of addresses are typically used for OSC control data. However, they are actually just strings (followed by values) that can be “routed”. The typical format is “/effect/controlParameter [values]”. By dividing the message through its “/” character, it is possible to deliver data only where needed.
- 2.
From Benjamin Smith, https://www.benjamindaysmith.com/ml-machine-learning-toolkit-in-max.
References
Jourdan, T., Caramiaux, B.: Machine learning for musical expression. In: New Interfaces for Musical Expression (NIME). Mexico City (2023)
Donahue, C., McAuley, J., Puckette, M.: Adversarial audio synthesis. In: 7th International Conference on Learning Representations, ICLR 2019 (2019)
Engel, J., Agrawal, K.K., Chen, S., Gulrajani, I., Donahue, C., Roberts, A.: Gansynth: Adversarial neural audio synthesis. In: 7th International Conference on Learning Representations, ICLR 2019 (2019)
Tahiroğlu, K., Kastemaa, M., Koli, O.: Al-terity: Non-rigid musical instrument with artificial intelligence applied to real-time audio synthesis. In: Proceedings of the International Conference on New Interfaces for Musical Expression (2020)
Fiebrink, R.: Wekinator homepage, http://www.wekinator.org
Ronchi, G., Benghi, C.: Interactive light and sound installation using artificial intelligence. Int. J. Arts Technol. 7, (2014). https://doi.org/10.1504/IJART.2014.066456
Schacher, J.C., Miyama, C., Bisig, D.: Gestural electronic music using machine learning as generative device. In: Proceedings of the International Conference on New Interfaces for Musical Expression (2015)
Nyström, E.: Intra-actions: experiments with velocity and position in continuous controllers. In: Proceedings of the International Conference on New Interfaces for Musical Expression (2020)
Serafin, S., Erkut, C., Kojs, J., Nilsson, N.C., Nordahl, R.: Virtual reality musical Instruments: state of the art, esign principles, and future directions. Comput. Music. J. (2016). https://doi.org/10.1162/COMJ_a_00372
Lee, M.: Entangled: A Multi-modal, Multi-user interactive instrument in virtual 3D space using the smartphone for gesture control. In: Proceedings of the International Conference on New Interfaces for Musical Expression (2021). https://doi.org/10.21428/92fbeb44.eae7c23f
Santini, G.: Synesthesizer: Physical modelling and machine learning for a Color-based synthesizer in virtual reality. In: lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (2019). https://doi.org/10.1007/978-3-030-21392-3_18
Zwicker, E.: Subdivision of the audible frequency range into critical bands (Frequenzgruppen). J. Acoust. Soc. Am. (1961). https://doi.org/10.1121/1.1908630
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Santini, G., Chen, Z. (2024). An AI-Mediated VR Sound Installation. In: Di Marco, G., Lombardi, D., Tedjosaputro, M. (eds) Creativity in the Age of Digital Reproduction. xArch 2023. Lecture Notes in Civil Engineering, vol 343. Springer, Singapore. https://doi.org/10.1007/978-981-97-0621-1_27
Download citation
DOI: https://doi.org/10.1007/978-981-97-0621-1_27
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-0620-4
Online ISBN: 978-981-97-0621-1
eBook Packages: EngineeringEngineering (R0)