Skip to main content
SpringerLink
Log in

Evaluation in scenarios of ubiquity of technology: a systematic literature review on interactive installations

  • Original Paper
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

Literature in the field of human–computer interaction (HCI) has shown a long tradition of evaluation methods for and along with interactive systems design. We have experienced in the last years an impressive development in ubiquitous and pervasive systems, motivated by technological development, low cost of sensors and actuators, and a rise in the maker culture for the construction of computational systems. While such systems naturally inherit methods for evaluating the user interaction from previous interaction paradigms, it is not clear whether they reach specificities of the interaction of people within ubiquitous and pervasive systems scenarios. This work aimed at shedding light on this subject by conducting a systematic literature review on ubiquitous and pervasive technology scenarios of interactive installations. Results have shown that most of the selected contributions use classical methods of data collection and analysis, and combinations of these methods. Analysis of results also points out some new aspects to be considered in evaluation methods, regarding the human (social) actions promoted or afforded by ubiquitous and pervasive systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

Not applicable.

Code availability

Not applicable.

Notes

  1. https://dl.acm.org/

  2. https://ieeexplore.ieee.org/

  3. https://link.springer.com/

  4. https://www.scopus.com/

  5. https://scielo.org/

Systematic literature review references

  1. Adhitya S, Scott D (2018) The London Soundmap: integrating sonic interaction design in the urban realm. In Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion (New York, NY, USA, 2018), AM’18, Association for Computing Machinery.

  2. Afonso AG, Ergin E, Fatah G, Schieck A. (2019) Flowing bodies: exploring the micro and macro scales of bodily interactions with urban media installations. In Proceedings of the 2019 on Designing Interactive Systems Conference (New York, NY, USA, 2019), DIS’19, Association for Computing Machinery, 1183–1193.

  3. Batras D, Guez J, Jégo J-F, and Tramus M-H. (2016) A virtual reality agent-based platform for improvisation between real and virtual actors using gestures. In Proceedings of the 2016 Virtual Reality International Conference (New York, NY, USA, 2016), VRIC ’16, Association for Computing Machinery.

  4. Biloria N, Dritsa D (2020) Real-time interactive multimodal systems for physiological and emotional wellbeing. Springer International Publishing, Cham, pp 181–203

    Google Scholar 

  5. Brown C, Paine G (2019) A case study in collaborative learning via participatory music interactive systems: Interactive Tango Milonga. Springer International Publishing, Cham, pp 285–306

    Google Scholar 

  6. Ceconello M, and Spallazzo D (2016) Vis-à-vis with Leonardo. Designing Digital Encoun-ters. In Eurographics Workshop on Graphics and Cultural Heritage (2016), C. E. Catalanoand L. D. Luca, Eds., The Eurographics Association.

  7. Cuan C, Berl E, LaViers A (2019) Time to compile: a performance installationas human-robot interaction study examining self-evaluation and perceived control. Paladyn, J Behav Robotics 10(1):267–285

    Article  Google Scholar 

  8. Dalsgaard P, Dindler C, Halskov K (2011) Understanding the dynamics of engaging interaction in public spaces. In: Campos P, Graham N, Jorge J, Nunes N, Palanque P, Winckler M (eds) Human-Computer Interaction - INTERACT 2011 (Berlin, Heidelberg, 2011). Springer, Berlin, Heidelberg, pp 212–229

    Chapter  Google Scholar 

  9. Erkut C, Fehr J (2017) Structuring design and evaluation of an interactive installation through swarms of light rays with human-artifact model. In: Brooks AL, Brooks E (eds) Interactivity, Game Creation, Design, Learning, and Innovation (Cham, 2017). Springer International Publishing, pp 39–46

    Chapter  Google Scholar 

  10. Françoise J, Candau Y, Fdili Alaoui S, and Schiphorst T. (2017) Designing for kinesthetic awareness: revealing user experiences through second-person inquiry. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (New York, NY, USA,2017), CHI ’17, Association for Computing Machinery, 5171–5183.

  11. Froese T, Suzuki K, Ogai Y, Ikegami T (2012) Using human–computer interfaces to investigate ‘mind-as-it-could-be’ from the first-person perspective. Cogn Comput 4(3):365–382

    Article  Google Scholar 

  12. Grace K, Grace S, Maher ML, Mahzoon MJ, Lee L, LoCurto L, and Outcault B. (2017) The willful marionette: exploring responses to embodied interaction. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition (New York, NY, USA, 2017), C&C ’17, Association for Computing Machinery, 15–27.

  13. Hsueh S, Alaoui SF, and Mackay WE (2019) Understanding kinaesthetic creativity in dance. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2019), CHI ’19, Association for Computing Machinery, 1–12.

  14. Ingebritsen R, Knowlton C, Sato H, Mott E. (2020) Social movements: A case study in dramaturgically-driven sound design for contemporary dance performance to mediate human-human interaction. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction (New York, NY, USA, 2020), TEI ’20, Association for Computing Machinery, 227–237.

  15. Janauskait L, and Palamas G. (2019) Establishing dialogues between movement and atmospheric ambiances. In Proceedings of the 6th International Conference on Movement and Computing (New York, NY, USA, 2019), MOCO ’19, Association for Computing Machinery.

  16. Kelton ML, Ma JY (2018) Reconfiguring mathematical settings and activity through multi-party, whole-body collaboration. Educ Stud Math 98(2):177–196

    Article  Google Scholar 

  17. Levinsky S, and Russell A. (2019) Agency in dialogue how choreographic thought emerges through dancing with tools that propel. In 2019 AISB Convention (2019), The Society for the Study of Artificial Intelligence and Simulation of Behaviour, 7–14.

  18. Liang R-H, Chung W-M, Kao H-L, and Lin T-Y (2013) Intouch: crossing social interaction with perception. In: Marcus A (ed) Design, User Experience, and Usability. User Experience in Novel Technological Environments (Berlin, Heidelberg, 2013). Springer, Berlin, Heidelberg, pp 306–315

  19. Loke L, Khut G P, Kocaballi AB. (2012) Bodily experience and imagination: designing ritual interactions for participatory live-art contexts. In Proceedings of the Designing Interactive Systems Conference (New York, NY, USA, 2012), DIS ’12, Association for Computing Machinery, 779–788.

  20. Lopes MM (2016) Body storytelling and the performance of memory: arts-based-research and human enhancement. In Distributed, Ambient and Pervasive Interactions (Cham, 2016), N. Streitz and P. Markopoulos, Eds., Springer International Publishing, 257–269.

  21. Malinverni L, Burguès NP. (2015) The medium matters: the impact of full-body interaction on the socio-affective aspects of collaboration. In Proceedings of the 14th International Conference on Interaction Design and Children (New York, NY, USA, 2015), IDC ’15, Association for Computing Machinery, 89–98.

  22. Manzolli J, Moroni A, Valarini GA. (2018) Selfhood: an evolutionary and interactive experience synthesizing images and sounds. In Music Technology with Swing (Cham,2018), M. Aramaki, M. E. P. Davies, R. Kronland-Martinet, and S. Ystad, Eds., Springer International Publishing. 625–636.

  23. Mendoza YLM, Baranauskas MCC. (2019) Tangitime: designing a (socio)enactive experience for deep time in an educational exhibit. In Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems (New York, NY, USA, 2019), IHC ’19, Association for Computing Machinery.

  24. Millar GC, Tabrizian P, Petrasova A, Petras V, Harmon B, Mitasova H, Meetenmeyer RK (2018) Tangible landscape: a hands-on method for teaching terrain analysis. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2018), CHI ’18, Association for Computing Machinery, 1–12.

  25. Mora-Guiard J, Crowell C, Pares N (2018) A play therapy based full-body interaction intervention tool for children with autism. In Pervasive Computing Paradigms for Mental Health (Cham, 2018), N. Oliver, S. Serino, A. Matic, P. Cipresso, N. Filipovic, and L. Gavrilovska, Eds., Springer International Publishing, 28–40.

  26. Mora-Guiard J, Pares N. (2014) “Child as the measure of all things”: the body as a referent in designing a museum exhibit to understand the nanoscale. In Proceedings of the 2014 Conference on Interaction Design and Children (New York, NY, USA, 2014), IDC ’14,Association for Computing Machinery, 27–36.

  27. Morgan E, Gunes H (2013) Human nonverbal behaviour understanding in the wild for new media art. In: Salah AA, Hung H, Aran O, Gunes H (eds) Human Behavior Understanding (Cham, 2013). Springer International Publishing, pp 27–39

    Chapter  Google Scholar 

  28. Núñez-Pacheco C, Loke L (2014) Aesthetic resources for technology-mediated bodily self-reflection: the case of eloquent robes. In Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: The Future of Design (New York, NY, USA, 2014), OzCHI ’14, Association for Computing Machinery, 1–10

  29. Prpa M, Tatar K, Riecke BE, Pasquier P (2017) The pulse breath water system: exploring breathing as an embodied interaction for enhancing the affective potential of virtual reality. In: Lackey S, Chen J (eds) Virtual, Augmented and Mixed Reality (Cham, 2017). Springer International Publishing, pp 153–172

    Chapter  Google Scholar 

  30. Roberts J, Lyons L (2017) The value of learning talk: applying a novel dialogue scoring method to inform interaction design in an open-ended, embodied museum exhibit. Int J Comput-Support Collab Learn 12(4):343–376

    Article  Google Scholar 

  31. Salmon R, Paine G (2013) Embodiment: auditory visual enhancement of interactive environments. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (New York, NY, USA, 2013), TEI ’13, Association for Computing Machinery, 129–136.

  32. Schacher JC, Bisig D (2017) Haunting space, social interaction in a large-scale media environment. In: Bern-haupt R, Dalvi G, Joshi A, Balkrishan DK, O’Neill J, Winckler M (eds) Human-Computer Interaction - INTERACT 2017 (Cham, 2017). Springer International Publishing, pp 242–262

    Chapter  Google Scholar 

  33. Schiphorst T. (2011) Self-evidence: applying somatic connoisseurship to experience design. In CHI ’11 Extended Abstracts on Human Factors in Computing Systems (New York, NY, USA, 2011), CHI EA ’11, Association for Computing Machinery, 145–160.

  34. Shaer O, Valdes C, Liu S, Lu K, Chang K, Xu W, Haddock TL, Bhatia S, Densmore D, Kincaid R (2014) Designing reality-based interfaces for experiential bio-design. Pers Ubiquit Comput 18(6):1515–1532

    Article  Google Scholar 

  35. Simon LU, van der Vlugt M, Calvi L. (2016) Triggers to entice an audience to ‘perform as interface’ in an interactive installation. In Proceedings of the 20th International Academic Mindtrek Conference (New York, NY, USA, 2016), Academic Mindtrek ’16, Association for Computing Machinery, 322–330.

  36. Stepanova ER, Desnoyers-Stewart J, Pasquier P, Riecke BE (2020) Jel: Breathing together to connect with others and nature. In Proceedings of the 2020 ACM Designing Interactive Systems Conference (New York, NY, USA, 2020), DIS ’20, Association for Computing Machinery, 641–654.

  37. Tan L, and Chow KKN (2017) Facilitating meaningful experience with ambient media: an embodied engagement model. In Proceedings of the Fifth International Symposium of Chinese CHI (New York, NY, USA, 2017), Chinese CHI 2017, Association for Computing Machinery, 36–46.

  38. Tanenbaum K, Hatala M, Tanenbaum J, Wakkary R, Antle AA (2014) case study of intended versus actual experience of adaptivity in a tangible storytelling system. User Modeling User-Adapted Interact 24(3):175–217

    Article  Google Scholar 

  39. Tikka H, Na SV, Jacucci G, Korpilahti T (2011) Provoking the city–touch installations for urban space. Digital Creativity 22(3):200–214

    Article  Google Scholar 

  40. Torpus J. (2018) Extending museum exhibits by embedded media content for an embodied interaction experience. In Proceedings of the 10th Nordic Conference on Human-Computer Interaction (New York, NY, USA, 2018), NordiCHI ’18, Association for Computing Machinery, 236–246.

  41. Vets T, Nijs L, Lesaffre M, Moens B, Bressan F, Colpaert P, Lambert P, Van de Walle R, Leman M (2017) Gamified music improvisation with billiart: a multimodal installation with balls. J Multimodal User Interfaces 11(1):25–38

    Article  Google Scholar 

  42. Vidyarthi J, Riecke BE, and Gromala D. (2012) Sonic Cradle: designing for an immersive experience of meditation by connecting respiration to music. In Proceedings of the Designing Interactive Systems Conference (New York, NY, USA, 2012), DIS ’12, Association for Computing Machinery, 408–417.

  43. Volpe G, Camurri A (2011) A system for embodied social active listening to sound and music content. J Comput Cult Herit 4:1

    Article  Google Scholar 

  44. Wiethoff A, Gerstberger J, Gehring S. (2015) Starlight: exploring embodied interactions with media architecture and public audiences. In Proceedings of the 4th International Symposium on Pervasive Displays (New York, NY, USA, 2015), PerDis ’15, Association for Computing Machinery, 83–89.

  45. Xambó A, Hornecker E, Marshall P, Jordà S, Dobbyn C, Laney R (2016) Exploring social interaction with a tangible music interface. Interacting with Comp 29(2):248–270

    Google Scholar 

References

  1. Bradley MM, Lang PJ (1994) Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Ther Exp Psychiatry 25(1):49–59. https://doi.org/10.1016/0005-7916(94)90063-92

    Article  Google Scholar 

  2. Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res Psychol 3(2):77–101. https://doi.org/10.1191/1478088706qp063oa

    Article  Google Scholar 

  3. Brockmyer JH, Fox CM, Curtiss KA, McBroom E, Burkhart KM, Pidruzny JN (2009) The development of the game engagement questionnaire: A measure of engagement in video game-playing. J Exp Soc Psychol 45(4):624–634. https://doi.org/10.1016/j.jesp.2009.02.016

    Article  Google Scholar 

  4. Costello B, Edmonds E. (2007) A study in play, pleasure and interaction design. In: Proceedings of the 2007 Conference on Designing Pleasurable Products and Interfaces. 76–91. DPPI ’07, ACM, New York, NY, USA. https://doi.org/10.1145/1314161.13141685.

  5. Díaz-Oreiro I, López G, Quesada L, Guerrero LA (2019) Standardized questionnaires for user experience evaluation: a systematic literature review. Proceedings 31(1). https://doi.org/10.3390/proceedings2019031014

  6. Gallagher S, Lindgren R (2015) Enactive metaphors: learning through full-body engagement. Educ Psychol Rev 27:391–404. https://doi.org/10.1007/s10648-015-9327-1

    Article  Google Scholar 

  7. Gough D, Oliver S, Thomas J. (2012) An introduction to systematic reviews. SAGE Publications

  8. Harms C, Biocca F. (2004) Internal consistency and reliability of the networked minds measure of social presence. In: Seventh Annual International Workshop: Presence, http://cogprints.org/7026/

  9. Hassenzahl M, Burmester M, Koller F (2003) Attrakdiff: Ein fragebogen zur messung wahrgenommener hedonischer und pragmatischer qualität. In: Szwillus G, Ziegler J (eds) Mensch & Computer 2003: Interaktion in Bewegung. Vieweg Teubner Verlag, Wiesbaden, pp 187–196. https://doi.org/10.1007/978-3-322-80058-919

    Chapter  Google Scholar 

  10. Höök K, Sengers P, Andersson G (2003). Sense and sensibility: evaluation and interactive art. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, 241–248. https://doi.org/10.1145/642611.642654

  11. Kaipainen M, Ravaja N, Tikka P, Vuori R, Pugliese R, Rapino M, Takala T (2011) Enactive systems and enactive media: embodied human-machine coupling beyond interfaces. Leonardo 44(5):433–438. https://doi.org/10.1162/LEONa00244

    Article  Google Scholar 

  12. Korhonen H, Montola M, Arrasvuori J (2009) Understanding playful user experience through digital games. Int Conference Designing Pleasur Prod Interfaces 09:274–285

    Google Scholar 

  13. Lazar J, Feng JH, Hochheiser H (2010) Research methods in human-computer interaction. Wiley Publishing

    Google Scholar 

  14. Lucero A, Arrasvuori J. (2010) Plex cards: a source of inspiration when designing for playfulness. In: Proceedings of the 3rd International Conference on Fun and Games. 28–37. Fun and Games ’10, Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/1823818.1823821

  15. McLennan J, Omodei M, Wearing A. (2001) Cognitive processes of first-on-scene fire officers in command at emergency incidents as an analogue of small-unit command in peace support operations. In: The human in command peace support operations, 312–329

  16. Moher D, Liberati A, Tetzlaff J, Altman DG, Group TP (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLOS Medicine 6(7):1–6. https://doi.org/10.1371/journal.pmed.1000097

    Article  Google Scholar 

  17. O’Brien HL, Toms EG (2008) What is user engagement? A conceptual framework for defining user engagement with technology. J Am Soc Inform Sci Technol 59(6):938–955. https://doi.org/10.1002/asi.20801

    Article  Google Scholar 

  18. Petitmengin C (2006) Describing one’s subjective experience in the second person: an Interview method for the science of consciousness. Phenomenol Cogn Sci 5(3):229–269. https://doi.org/10.1007/s11097-006-9022-2

    Article  Google Scholar 

  19. Smith J, Flowers P, Larkin M (2009) Interpretative phenomenological analysis: theory, method and research. SAGE Publications

    Google Scholar 

  20. Torpus J (2018) Extending museum exhibits by embedded media content for an embodied interaction experience. In Proceedings of the 10th Nordic Conference on Human-Computer Interaction (New York, NY, USA, 2018), NordiCHI ’18, Association for Computing Machinery, 236–246.

  21. Varela F, Thompson E, Rosch E (1993) The embodied mind: cognitive science and human experience. Philosophy, psychology, MIT Press, Cognitive science

    Google Scholar 

  22. Vermersch P (1996) L’explicitation de l’action. Cahiers de linguistique sociale 3:113–120

    Google Scholar 

  23. Watson D, Clark LA, Tellegen A (1988) Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol 54(6):1063–1070

    Article  Google Scholar 

  24. Weiser M (1991) The computer for the 21st century. Sci Am 265(3):94–104

    Article  Google Scholar 

Download references

Funding

This study was financially supported by the National Council for Scientific and Technological Development (CNPq) through grant #306272/2017–2, by the São Paulo Research Foundation (FAPESP) through grants #2015/16528–0, #2015/24300–9, #2020/03503–7, and #2020/04242–2, and the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES) – Finance Code 001.

Author information

Authors and Affiliations

  1. Institute of Computing, University of Campinas (UNICAMP), Campinas, SP, Brazil

    Yusseli Lizeth Méndez Mendoza, Emanuel Felipe Duarte, Maria Jêsca Nobre de Queiroz & M. Cecília C. Baranauskas

Authors
  1. Yusseli Lizeth Méndez Mendoza
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emanuel Felipe Duarte
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maria Jêsca Nobre de Queiroz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. M. Cecília C. Baranauskas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yusseli Lizeth Méndez Mendoza.

Ethics declarations

Conflicts of interest

Not applicable.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Highlights

• Scenarios constructed with ubiquitous technology involve dimensions that might not be covered by current evaluation methods.

• A systematic literature review on the evaluation of interactive installations is conducted.

• Results raised the mainstream methods for data collection and analysis in interactive installations.

• The work reveals a complex relationship between categories of data collection and objects of evaluation.

• Missing aspects are related to the experience concept, the reachness of methods, and ethical issues.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mendoza, Y.L.M., Duarte, E.F., de Queiroz, M.J.N. et al. Evaluation in scenarios of ubiquity of technology: a systematic literature review on interactive installations. Pers Ubiquit Comput 27, 343–361 (2023). https://doi.org/10.1007/s00779-022-01696-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-022-01696-8

Keywords

Search

Navigation