Skip to main content
SpringerLink
Log in

ARLooper: A Mobile AR Application for Collaborative Sound Recording and Performance

  • Conference paper
  • First Online:
Perception, Representations, Image, Sound, Music (CMMR 2019)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12631))

Included in the following conference series:

  • 1030 Accesses

Abstract

This paper introduces ARLooper, an AR-based iOS application for multi-user sound recording and performance, that aims to explore the possibility of using mobile AR technology in creating novel musical interfaces and collaborative audiovisual experience. ARLooper allows the user to record sound through microphones in mobile devices and, at the same time, visualizes and places recorded sounds as 3D waveforms in an AR space. The user can play, modify, and loop the recorded sounds with several audio filters attached to them. Since ARLooper generates the world map information through iOS ARKit’s tracking technique called visual-inertial odometry which tracks the real world and a correspondence between real and AR spaces, it enables multiple users to connect to the same AR space by sharing and synchronize the world map data. In this shared AR space, the user can see each other’s 3D waveforms and activities, such as selection and manipulation, as a result, having a potential of collaborative AR performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://developer.apple.com/arkit/.

  2. 2.

    https://audiokit.io/.

  3. 3.

    http://www.create.ucsb.edu/.

References

  1. Arango, J.J., Giraldo, D.M.: The smartphone ensemble. Exploring mobile computer mediation in collaborative musical performance. In: The New Interfaces for Musical Expression Conference, Brisbane, Australia, pp. 61–64 (2016)

    Google Scholar 

  2. Arth, C., Grasset, R., Gruber, L., Langlotz, T., Mulloni, A., Wagner, D.: The history of mobile augmented reality. arXiv preprint arXiv:1505.01319 (2015)

  3. d’Alessandro, N., Pon, A., Wang, J., Eagle, D., Sharlin, E., Fels, S.: A digital mobile choir: joining two interfaces towards composing and performing collaborative mobile music. In: the International Conference on New Interfaces for Musical Expression. University of Michigan, Ann Arbor (2012)

    Google Scholar 

  4. Berry, R., Makino, M., Hikawa, N., Suzuki, M.: The augmented composer project: the music table. In: the 2nd IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 338–339. IEEE (2003)

    Google Scholar 

  5. Billinghurst, M., Weghorst, S., Furness III, T.: Shared space: an augmented reality interface for computer supported collaborative work. In: Proceedings of CVE, vol. 96 (1996)

    Google Scholar 

  6. Broll, W., et al.: ARTHUR: a collaborative augmented environment for architectural design and urban planning. J. Virtual Reality Broadcast. 1(1), (2004)

    Google Scholar 

  7. Butz, A., Hollerer, T., Feiner, S., MacIntyre, B., Beshers, C.: Enveloping users and computers in a collaborative 3D augmented reality. In: The 2nd IEEE and ACM International Workshop on Augmented Reality, pp. 35–44. IEEE (1999)

    Google Scholar 

  8. Clouth, R.: Mobile augmented reality as a control mode for real-time music systems. Universitat Pompeu Fabra, Barcelona (2013)

    Google Scholar 

  9. Davison, A.J.: Real-time simultaneous localisation and mapping with a single camera. In: ICCV, vol. 3, pp. 1403–1410 (2003)

    Google Scholar 

  10. Essl, G., Rohs, M.: Interactivity for mobile music-making. Organ. Sound 14(2), 197–207 (2009)

    Article  Google Scholar 

  11. Essl, G., Lee, S.W.: Mobile devices as musical instruments - state of the art and future prospects. In: Aramaki, M., Davies, M.E.P., Kronland-Martinet, R., Ystad, S. (eds.) CMMR 2017. LNCS, vol. 11265, pp. 525–539. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01692-0_35

    Chapter  Google Scholar 

  12. Gaye, L., Holmquist, L.E., Behrendt, F., Tanaka, A.: Mobile music technology: report on an emerging community. In: the International Conference on New Interfaces for Musical Expression, Paris, France, pp. 22–25 (2006)

    Google Scholar 

  13. Kaltenbrunner, M., Jorda, S., Geiger, G., Alonso, M.: The reacTable*: a collaborative musical instrument. In: The 15th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises, pp. 406–411. IEEE (2006)

    Google Scholar 

  14. Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: The 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 1–10. IEEE Computer Society (2007)

    Google Scholar 

  15. Laurenzo, T., Rodríguez, E., Castro, J.F.: YARMI: an augmented reality musical instrument. In: the New Interfaces for Musical Expression Conference, Pittsburgh, PA, USA, pp. 268–269 (2009)

    Google Scholar 

  16. Lee, S.W., Srinivasamurthy, A., Tronel, G., Shen, W., Freeman, J.: Tok!: a collaborative acoustic instrument using mobile phones. In: the International Conference on New Interfaces for Musical Expression. University of Michigan, Ann Arbor (2012)

    Google Scholar 

  17. Levin, G.: Dialtones (A Telesymphony) (2001). http://www.flong.com/projects/telesymphony/. Accessed 05 Mar 2019

  18. Oh, J., Herrera, J., Bryan, N.J., Dahl, L., Wang, G.: Evolving the mobile phone orchestra. In: The International Conference on New Interfaces for Musical Expression, Sydney, Australia, pp. 82–87 (2010)

    Google Scholar 

  19. Poupyrev, I., et al.: Augmented groove: collaborative jamming in augmented reality. In: ACM SIGGRAPH 2000 Conference Abstracts and Applications, vol. 17, no. 7, p. 77 (2000)

    Google Scholar 

  20. Rekimoto, J., Saitoh, M.: Augmented surfaces: a spatially continuous work space for hybrid computing environments. In: the SIGCHI Conference on Human Factors in Computing Systems, pp. 378–385. ACM (1999)

    Google Scholar 

  21. Rohs, M., Essl, G.: CaMus 2: collaborative music performance with mobile camera phones. In: The International Conference on New Interfaces for Musical Expression, pp. 190–195 (2007)

    Google Scholar 

  22. Salazar, S., Pipepenbrink, A., Reid, S.: Developing a performance practice for mobile music technology. In: The International Conference on New Interfaces for Musical Expression, pp. 59–64. Virginia Tech, Blacksburg (2018)

    Google Scholar 

  23. Szalavári, Z., Schmalstieg, D., Fuhrmann, A., Gervautz, M.: “Studierstube”: an environment for collaboration in augmented reality. Virtual Reality 3(1), 37–48 (1998)

    Article  Google Scholar 

  24. Tanaka, A.: Mobile music making. In: The International Conference on New Interfaces for Musical Expression, Hamamatsu, Japan, pp. 154–156 (2004)

    Google Scholar 

  25. Zhou, F., Duh, H.B.L., Billinghurst, M.: Trends in augmented reality tracking, interaction and display: a review of ten years of ISMAR. In: The 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 193–202. IEEE Computer Society (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Media Arts and Technology, University of California, Santa Barbara, Santa Barbara, CA, 93106, USA

    Sihwa Park

Authors
  1. Sihwa Park
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sihwa Park .

Editor information

Editors and Affiliations

  1. Laboratoire PRISM, CNRS-AMU, Marseille, France

    Richard Kronland-Martinet

  2. Laboratoire PRISM, CNRS-AMU, Marseille, France

    Sølvi Ystad

  3. Laboratoire PRISM, CNRS-AMU, Marseille, France

    Mitsuko Aramaki

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Park, S. (2021). ARLooper: A Mobile AR Application for Collaborative Sound Recording and Performance. In: Kronland-Martinet, R., Ystad, S., Aramaki, M. (eds) Perception, Representations, Image, Sound, Music. CMMR 2019. Lecture Notes in Computer Science(), vol 12631. Springer, Cham. https://doi.org/10.1007/978-3-030-70210-6_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-70210-6_35

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-70209-0

  • Online ISBN: 978-3-030-70210-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation