FuXi: A Fish-Driven Instrument for Real-Time Music Performance

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9027)

Abstract

In this paper we present a system for real-time computer music performance (live electronics) and live visuals based on the behavior of a fish in an aquarium. The system is comprised of (1) an aquarium with a fish; (2) a computer vision module; (3) a visual display of the fish overlaid by graphical elements controlled by the user, (4) a sound synthesis module and (5) a standard MIDI controller. The musical expression and graphic generation is a combination of the fish movements and decisions made by the performer in real-time. By making use of a live animal, the system provides indeterminacy and natural gestures to the sound being generated. The match between sound and image shows some semantic redundancy, aiming at a more narrative compositional approach where the fish is the main character. The system is targeted to soundscape composition and electroacoustic music featuring a high degree of improvisation.

Keywords

Fish-driven instrument Electroacoustic music NIME HCI Improvisation Indeterminism 

Notes

Acknowledgments

We would like to acknowledge the valuable artistic inputs coming from Prof. Álvaro Barbosa, Dr. José Alberto Gomes and Ana Cristina Dias; the expertise and encouragement from Professor David Flores Gonalves in the field of biology and fish behavior; and the financial support from GAES, which made possible the presentation of this work abroad.

References

  1. 1.
    Taruskin, R.: Music in the Late Twentieth Century: The Oxford History of Western Music. Oxford University Press, New York (2009)Google Scholar
  2. 2.
    Chadabe, J.: Electric Sound: The Past and Promise of Electronic Music. Pearson Education, Upper Saddle River (1996)Google Scholar
  3. 3.
    Cage, J.: Silence, 1st edn. Wesleyan University Press, Middletown (1961)Google Scholar
  4. 4.
    de Leeuw, T.: Music in Twenty Century: A Study of Its Elements and Structure. Amsterdam University Press, Amsterdam (2005)CrossRefGoogle Scholar
  5. 5.
    Collopy, F.: Color, form, and motion: Dimensions of a musical art of light. Leonardo 33(5), 355–360 (2000)CrossRefGoogle Scholar
  6. 6.
    DeWitt, T.: Visual music: searching for an aesthetic. Leonardo 20(2), 15–122 (1987)CrossRefGoogle Scholar
  7. 7.
    Paradiso, J., O’modhrain, S.: Current trends in electronic music interfaces. J. New Music Res. 32(4), 345–349 (2003). (Guest Editors’ Introduction)CrossRefGoogle Scholar
  8. 8.
    Peacock, K.: Instruments to perform color-music: two centuries of technological experimentation. Leonardo 21(4), 397–406 (1988)CrossRefGoogle Scholar
  9. 9.
    Franco, E., Griffith, N., Fernström, M.: Issues for Designing a flexible expressive audiovisual system for real-time performance & composition. In: Proceedings of the 2004 Conference on New Interfaces for Musical Expression (2004)Google Scholar
  10. 10.
    Levin, G., Lieberman, Z.: Sounds from shapes: audiovisual performance with hand silhouette contours in the manual input sessions. In: Proceedings of the 2005 Conference on New Interfaces for Musical Expression, pp. 115–120 (2005)Google Scholar
  11. 11.
    Levin, G.: Painterly interfaces for audiovisual performance. Ph.D. thesis, Massachusetts Institute of Technology (2000)Google Scholar
  12. 12.
    Lew, M.: Live Cinema: designing an instrument for cinema editing as a live performance. In: Proceedings of the 2004 Conference on New Interfaces for Musical Expression, pp. 144–149 (2004)Google Scholar
  13. 13.
    Schott, G.: Magia universalis natura et artis. Number, vol. 1 in Magia universalis natura et artis (1657)Google Scholar
  14. 14.
    Caccuri, V.: Canções Submersas (2008)Google Scholar
  15. 15.
    Quiet Enseble: Quintetto (2009)Google Scholar
  16. 16.
    Walker, B.N., Godfrey, M.T., Orlosky, J.E., Bruce, C., Sanford, J.: Aquarium sonification: Soundscapes for accessible dynamic informal learning environments. In: Proceedings of the 12th International Conference on Auditory Display, pp. 238–241. Citeseer (2006)Google Scholar
  17. 17.
    Walker, B.N., Kim, J., Pendse, A., Others: Musical soundscapes for an accessible aquarium: Bringing dynamic exhibits to the visually impaired. In: Proceedings of the International Computer Music Conference (ICMC 2007), Copenhagen, Denmark (2007)Google Scholar
  18. 18.
    Pendse, A., Pate, M., Walker, B.N.: The accessible aquarium: identifying and evaluating salient creature features for sonification. In: Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility, Assets 2008, pp. 297–298. ACM, New York (2008)Google Scholar
  19. 19.
    Bruce, C., Walker, B.: Designing effective sound-based aquarium exhibit interpretation for visitors with vision impairments. In: Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 251–252. ACM (2010)Google Scholar
  20. 20.
    Jeon, M., Winton, R., Yim, J., Bruce, C., Walker, B.: Aquarium fugue: interactive sonification for children and visually impaired audience in informal learning environments. In: Proceedings of the 18th International Conference on Auditory Display, Atlanta (2012)Google Scholar
  21. 21.
    Jeon, M., Winton, R.J., Henry, A.G., Oh, S., Bruce, C.M., Walker, B.N.: Designing interactive sonification for live aquarium exhibits. In: Stephanidis, C. (ed.) HCII 2013, Part I. CCIS, vol. 373, pp. 332–336. Springer, Heidelberg (2013) CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Faculty of Creative IndustriesUniversity of Saint JosephMacauChina
  2. 2.CITAR-Research Center in Science and Technology of the ArtsPortuguese Catholic University of PortugalOportoPortugal

Personalised recommendations