Skip to main content

An Audience Data-Driven Alternate Reality Storytelling Design

  • 300 Accesses

Part of the Lecture Notes in Computer Science book series (LNCS,volume 13324)


This case study, @LAs, made audiences the authors of exhibits as a way to unite audiences and artworks through a method of collecting their data and using artificial intelligence to process it. Three types of audience data were shared with this prototype exhibit system—identity data, real-time interaction data, and real-time location data. Tagged by folksonomy, audience data became a novel element for the AI system to process to yield media content. Thus, the system selected and generated real-time multimedia content with audiences’ personal features and allowed real-time interaction between audiences and the system, which realized dynamic control in the alternate reality. Developed and built during the 2019 University of California, Los Angeles, School of Theater, Film and Television Future Storytelling Summer Institute program, @LAs contained three prototype exhibits for a multimedia alternate reality storytelling pavilion, created as a prototype for a hypothetical mixed reality pop-up installation run throughout Los Angeles during the 2028 Summer Olympic Games. The three exhibits—i.e., three different virtual interaction spaces—were thematically based on murals, foods, and public transportation development in Los Angeles. This paper 1) introduces the theoretical and practical bases of the case study for new forms of alternate reality, 2) describes the data processing method used and the audiences’ experiences, and 3) shares the prototype’s design lessons based on the questionnaire responses from designers.


  • Audience data
  • Alternate reality
  • Artificial intelligence
  • Dynamic control
  • Personalized experience
  • Immersion

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-031-05434-1_10
  • Chapter length: 18 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   69.99
Price excludes VAT (USA)
  • ISBN: 978-3-031-05434-1
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   89.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.


  1. Rostami, A., Rossitto, C., Waern, A.: Frictional realities: enabling immersion in mixed-reality performances. In: Proceedings of the 2018 ACM International Conference on Interactive Experiences for TV and Online Video, pp. 15–27, June 2018

    Google Scholar 

  2. Lens. Accessed 01 May 2020

  3. Olafur Eliasson: The unspeakable openness of things ( Accessed 01 May 2020

  4. IGNANT. Accessed 01 May 2020

  5. MacGregor, B.: Cybernetic serendipity revisited. In: Proceedings of the 4th Conference on Creativity & Cognition, pp. 11–13, October 2002

    Google Scholar 

  6. Agent Ruby. Accessed 01 May 2020

  7. Röggla, T., Wang, C., Perez Romero, L., Jansen, J., Cesar, P.: Tangible air: an interactive installation for visualising audience engagement. In: Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition, pp. 263–265, June 2017

    Google Scholar 

  8. Borderless. Accessed 01 May 2020

  9. Bowers, J., SHAPE Consortium: TONETABLE: a multi-user, mixed-media, interactive installation. In Proceedings of COST G-6 Conference on Digital Audio Effects (DAFX-01), December 2001

    Google Scholar 

  10. Kim, J.Y., Allen, J.P., Lee, E.: Alternate reality gaming. Commun. ACM 51(2), 36–42 (2008)

    CrossRef  Google Scholar 

  11. Accessed 01 Jan 2022

  12. Dannenberg, R.B., Bates, J.: A model for interactive art (1995)

    Google Scholar 

  13. Elgammal, A.: AI Is blurring the definition of artist: advanced algorithms are using machine learning to create art autonomously. Am. Sci. 107(1), 18–22 (2019)

    CrossRef  Google Scholar 

  14. Shanken, E.A.: Art in the information age: Technology and conceptual art. Leonardo 35(4), 433–438 (2002)

    CrossRef  Google Scholar 

  15. Berry, D.M., Fagerjord, A.: Digital Humanities: Knowledge and Critique in a Digital Age. Polity Press, Cambridge (2017)

    Google Scholar 

  16. Burke, J., Stein, J.: Live performance and post-cinematic filmmaking. Perform. Matt. 6(1), 28–47 (2020)

    Google Scholar 

  17. Daer, A.R., Hoffman, R., Goodman, S.: Rhetorical functions of hashtag forms across social media applications. In: Proceedings of the 32nd ACM International Conference on the Design of Communication CD-ROM, pp. 1–3, September 2014

    Google Scholar 

  18. Zhang, C., Hoel, A.S., Perkis, A.: Quality of alternate reality experience and its QoE influencing factors. In: Proceedings of the 2nd International Workshop on Multimedia Alternate Realities, pp. 3–8, October 2017

    Google Scholar 

  19. Morrison, M., Krugman, D.M.: A look at mass and computer mediated technologies: understanding the roles of television and computers in the home. J. Broadcast. Electron. Media 45(1), 135–161 (2001)

    CrossRef  Google Scholar 

  20. Teng, S.Y.: The Description of Aesthetic Psychology. China Social Science Press (1985)

    Google Scholar 

  21. Koleva, B., et al.: Orchestrating a mixed reality performance. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 38–45, March 2001

    Google Scholar 

Download references


The Institute was supported by the UCLA TFT Skoll Center for Social Impact Entertainment.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Xiaowen Sun .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Sun, X., Calderón, D.G., Subbaraman, B., Burke, J.A. (2022). An Audience Data-Driven Alternate Reality Storytelling Design. In: Rauterberg, M. (eds) Culture and Computing. HCII 2022. Lecture Notes in Computer Science, vol 13324. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05433-4

  • Online ISBN: 978-3-031-05434-1

  • eBook Packages: Computer ScienceComputer Science (R0)