Skip to main content
SpringerLink
Log in

Evaluating Augmented Reality Based Remote Collaboration: A Contextualized Approach

  • Chapter
  • First Online:
Human-Automation Interaction

Part of the book series: Automation, Collaboration, & E-Services ((ACES,volume 12))

Abstract

Augmented Reality (AR) has tremendous capacity to assist scenarios of remote collaboration in the context of Industry 4.0, where distributed team-members with different expertise need to achieve shared targets. Solutions using AR enable integration of virtual and real-world content, empowering individuals with a common ground that improves understanding of the problem and its context, as well as knowledge retention. Given the field novelty, most efforts have been dedicated to proposing novel technologies to support the basic needs of collaboration. However, as the field evolves, evaluation must also be addressed, moving beyond the technological aspects to better understand its purpose to the collaborative process. Nevertheless, evaluating such contexts is a challenging goal given the number of characteristics that may impact collaboration. Furthermore, existing literature suggests evaluation methods still take advantage on single-user methods, not suitable to collaborative scenarios. As such, evaluation of such activities needs to advance into more contextualized perspectives, considering different members actions and how they relate with the technology being used. Also, the environment and how distinct factors affect the process of decision-making. In this vein, a conceptual framework for AR-based remote scenarios is proposed for supporting more contextualized evaluations. It encompasses identification of important dimensions, fostering an additional overview of the added value of AR to the nuances of the collaborative effort.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    - https://sdgs.un.org/goals (accessed on July, 2022).

References

  1. Kim S, Billinghurst M, Lee G (2018) The effect of collaboration styles and view independence on video-mediated remote collaboration. Comput Support Cooperat Work, CSCW 27(3–6):569–607

    Article  Google Scholar 

  2. Kim K, Billinghurst M, Bruder G, Duh HB, Member S (2018) Revisiting trends in augmented reality research: a review of the 2nd decade of ISMAR (2008–2017). IEEE Trans Visual Comput Graph TVCG 24(11):2947–2962

    Article  Google Scholar 

  3. Ens B et al (2019) Revisiting collaboration through mixed reality: the evolution of groupware. Int J Hum Comput Stud 131:81–98

    Article  Google Scholar 

  4. Brkic VS, Putnik G, Veljković ZA, Shah V, Essdai A (2017) Interfaces for distributed remote user controlled manufacturing as collaborative environment. In: Proceedings of the AHFE 2016 international conference on human factors and system interactions, pp 335–347

    Google Scholar 

  5. Marques B, Silva S, Teixeira A, Dias P, Santos BS (2022) A vision for contextualized evaluation of remote collaboration supported by AR. Comput Graph 102:413–425

    Article  Google Scholar 

  6. Schneider M, Rambach J, Stricker D (2017) Augmented reality based on edge computing using the example of remote live support. In: IEEE international conference on industrial technology, pp 1277–1282

    Google Scholar 

  7. Lukosch S, Billinghurst M, Alem L, Kiyokawa K (2015) Collaboration in augmented reality. Comput Support Cooperat Work, CSCW 24(6):515–525

    Article  Google Scholar 

  8. Arias E, Eden H, Fischer G, Gorman A, Scharff E (2000) Transcending the individual human mind-creating shared understanding through collaborative design. ACM Trans Comput-Human Interact 7(1):84–113

    Article  Google Scholar 

  9. Hoppe AH, Westerkamp K, Maier S, van de Camp F, Stiefelhagen R (2018) Multi-user collaboration on complex data in virtual and augmented reality. In: International conference on human-computer interaction, HCI international 2018: HCII 2018—posters’ extended abstracts, pp 258–265

    Google Scholar 

  10. Marques B, Teixeira A, Silva SS, Alves J, Dias P, Santos BS (2021) A critical analysis on remote collaboration mediated by augmented reality: making a case for improved characterization and evaluation of the collaborative process. Comput Graph 102:619–633

    Article  Google Scholar 

  11. Johson S, Gibson M, Mutlu B (2015) Handheld or handsfree? Remote collaboration via lightweight head-mounted displays and handheld devices. In: ACM conference on computer supported cooperative work and social computing, pp 1825–1836

    Google Scholar 

  12. Lapointe JF, Molyneaux H, Allili MS (2020) A literature review of AR-based remote guidance tasks with user studies. In: International conference on human-computer interaction, HCII 2020: virtual, augmented and mixed reality. industrial and everyday life applications, pp 111–120

    Google Scholar 

  13. Ong SK, Yuan ML, Nee AYC (2008) Augmented reality applications in manufacturing: a survey. Int J Prod Res 46(10):2707–2742

    Article  MATH  Google Scholar 

  14. Wang X, Ong SK, Nee AYC (2016) A comprehensive survey of augmented reality assembly research. Adv Manuf 4(1):1–22

    Article  Google Scholar 

  15. Hall M, McMahon CA, Bermell-Garcia P, Johansson A, Ravindranath R (2018) Capturing synchronous collaborative design activities: a state-of-the-art technology review. In: International design conference, DESIGN, pp 347–358

    Google Scholar 

  16. Röltgen D, Dumitrescu R (2020) Classification of industrial augmented reality use cases. Procedia CIRP 91:93–100

    Article  Google Scholar 

  17. Marques B, Silva SS, Alves J, Araujo T, Dias PM, Sousa Santos B (2021) A conceptual model and taxonomy for collaborative augmented reality. In: IEEE transactions on visualization and computer graphics, pp 1–21

    Google Scholar 

  18. Bottani E, Vignali G (2019) Augmented reality technology in the manufacturing industry: a review of the last decade. IISE Trans 51(3):284–310

    Article  Google Scholar 

  19. Altug Y, Mahdy AM (2016) A perspective on distributed and collaborative augmented reality. Int J Recent Trends Human Comput Interact (IJHCI) 7(2):23–41

    Google Scholar 

  20. Piumsomboon T, Dey A, Ens B, Lee G, Billinghurst M (2019) The effects of sharing awareness cues in collaborative mixed reality. Front Robot AI 6:1–18

    Article  Google Scholar 

  21. Aromaa S, Väätänen A, Hakkarainen M, Kaasinen E (2017) User experience and user acceptance of an augmented reality based knowledge-sharing solution in industrial maintenance work. In: Proceedings of the AHFE 2017 international conference on usability and user experience, vol 607, pp 145–156

    Google Scholar 

  22. Alexander T, Ripkens A, Westhoven M, Kleiber M, Pfendler C (2017) Virtual tele-cooperation: applying AR and VR for cooperative tele-maintenance and advanced distance learning. In: Proceedings of the AHFE 2017 international conference on human factors in training, education, and learning sciences, pp 234–244

    Google Scholar 

  23. Neale DC, Carroll JM, Rosson MB (2004) Evaluating computer-supported cooperative work: models and frameworks. In: ACM conference on computer supported cooperative work, CSCW, pp 112–121

    Google Scholar 

  24. Kim S, Billinghurst M, Lee C, Lee G (2018) Using freeze frame and visual notifications in an annotation drawing interface for remote collaboration. KSII Trans Internet Inf Syst 12(12):6034–6056

    Google Scholar 

  25. Choi SH, Kim M, Lee JY (2018) Situation-dependent remote AR collaborations: image-based collaboration using a 3D perspective map and live video-based collaboration with a synchronized VR mode. Comput Ind 101:51–66

    Article  Google Scholar 

  26. Barroso J, Fonseca L, Marques B, Dias P, Sousa Santos B (2020) Remote collaboration using mixed reality: exploring a shared model approach through different interaction methods. In: European conference on computer-supported cooperative work, ECSCW, pp 1–6

    Google Scholar 

  27. Elvezio C, Sukan M, Oda O, Feiner S, Tversky B (2017) Remote collaboration in AR and VR using virtual replicas. In: ACM SIGGRAPH 2017 VR Village (SIGGRAPH ’17), pp 1–2

    Google Scholar 

  28. Bai H, Sasikumar P, Yang J, Billinghurst M (2020) A user study on mixed reality remote collaboration with eye gaze and hand gesture sharing. In: CHI ’20: Proceedings of the 2020 CHI conference on human factors in computing systems, pp 1–13

    Google Scholar 

  29. Zillner J, Mendez E, Wagner D (2018) Augmented reality remote collaboration with dense reconstruction. In: Adjunct proceedings—2018 IEEE international symposium on mixed and augmented reality, ISMAR-Adjunct 2018, pp 38–39

    Google Scholar 

  30. Marques B (2021) Concepts and methods to support the development and evaluation of remote collaboration using augmented reality, Ph.D. Thesis—University of Aveiro

    Google Scholar 

  31. Antunes P, Herskovic V, Ochoa SF, Pino JA (2014) Reviewing the quality of awareness support in collaborative applications. J Syst Softw 89(1):146–169

    Article  Google Scholar 

  32. Hamadache K, Lancieri L (2009) Strategies and taxonomy, tailoring your CSCW evaluation. Lect Notes Comput Sci 5784:206–221

    Article  Google Scholar 

  33. Marques B, Teixeira A, Silva S, Alves J, Dias P, Santos BS (2020) A conceptual model for data collection and analysis for AR-based remote collaboration evaluation. In: IEEE international symposium on mixed and augmented reality adjunct (ISMAR-Adjunct), 2020, pp 1–2

    Google Scholar 

  34. Dey A, Billinghurst M, Lindeman RW, Ii JES (2018) A systematic review of 10 years of augmented reality usability studies: 2005 to 2014. Front Robot AI 5

    Google Scholar 

  35. Marques B, Silva S, Dias P, Santos BS (2021) An ontology for evaluation of remote collaboration using augmented reality. In: European conference on computer-supported cooperative work, ECSCW, pp 1–8

    Google Scholar 

  36. Belen R, Nguyen H, Filonik D, Del Favero D, Bednarz T (2019) A systematic review of the current state of collaborative mixed reality technologies: 2013–2018. AIMS Electron Electr Eng 3(2):181–223

    Article  Google Scholar 

  37. Merino L, Schwarzl M, Kraus M, Sedlmair M, Schmalstieg D, Weiskopf D (2020) Evaluating mixed and augmented reality: a systematic literature review (2009–2019). In: IEEE international symposium on mixed and augmented reality, ISMAR, pp 1–14

    Google Scholar 

  38. Grudin J, Poltrock S (2011) Computer supported cooperative work. In: The encyclopedia of human-computer interaction, 2nd ed. Interaction Design Foundation

    Google Scholar 

  39. Cidota M, Lukosch S, Datcu D, Lukosch H (2016) Comparing the effect of audio and visual notifications on workspace awareness using head-mounted displays for remote collaboration in augmented reality. Augment Human Res 1(1):1–15

    Article  Google Scholar 

  40. Billinghurst M, Clark A, Lee G (2015) A survey of augmented reality. Foundat Trends Human-Comput Interact 8(2–3):73–272

    Article  Google Scholar 

  41. Gurevich P, Lanir J, Cohen B (2015) Design and implementation of teleadvisor: a projection-based augmented reality system for remote collaboration. Comput Support Cooperat Work, CSCW 24(6):527–562

    Article  Google Scholar 

  42. Teo T, Lawrence L, Lee GA, Billinghurst M, Adcock M (2019) Mixed reality remote collaboration combining 360 video and 3D reconstruction. In: Conference on human factors in computing systems, CHI, pp 1–14

    Google Scholar 

  43. Lamberti F, Manuri F, Sanna A, Paravati G, Pezzolla P, Montuschi P (2014) Challenges, opportunities, and future trends of emerging techniques for augmented reality-based maintenance. IEEE Trans Emerg Top Comput 2(4):411–421

    Article  Google Scholar 

  44. Palmarini R, Erkoyuncu JA, Roy R, Torabmostaedi H (2018) A systematic review of augmented reality applications in maintenance. Robot Comput-Integr Manuf 49:215–228

    Article  Google Scholar 

  45. Zigart T, Schlund S (2020) Evaluation of augmented reality technologies in manufacturing—a literature review. In: Proceedings of the AHFE 2020 international conference on human factors and ergonomics, pp 75–82

    Google Scholar 

  46. Marques B, Silva S, Alves J, Rocha A, Dias P, Santos BS (2022) Remote collaboration in maintenance contexts using augmented reality: insights from a participatory process. Int J Inter Design Manuf, IJIDeM, pp 1–20

    Google Scholar 

  47. Madeira T, Marques B, Alves J, Dias P, Santos BS (2021) Exploring annotations and hand tracking in augmented reality for remote collaboration. In: Advances in intelligent systems and computing, pp 83–89

    Google Scholar 

  48. Fakourfar O, Ta K, Tang R, Bateman S, Tang A (2016) Stabilized annotations for mobile remote assistance. In: Conference on human factors in computing systems, CHI, pp 1548–1560

    Google Scholar 

  49. Anton D, Kurillo G, Bajcsy R (2018) User experience and interaction performance in 2D/3D telecollaboration. Futur Gener Comput Syst 82:77–88

    Article  Google Scholar 

  50. Bai Z, Blackwell AF (2012) Analytic review of usability evaluation in ISMAR. Interact Comput 24(6):450–460

    Article  Google Scholar 

  51. Ratcliffe J, Soave F, Bryan-Kinns N, Tokarchuk L, Farkhatdinov I (2021) Extended reality (XR) remote research: a survey of drawbacks and opportunities. In: CHI conference on human factors in computing systems, pp 1–13

    Google Scholar 

  52. Ratclife J et al. (2021) Remote XR studies: exploring three key challenges of remote XR experimentation jack. In: CHI conference on human factors in computing systems extended abstracts (CHI ’21 extended abstracts), pp 1–4

    Google Scholar 

  53. Pereira C, Teixeira A, Silva MO (2014) Live evaluation within ambient assisted living scenarios. In: International conference on pervasive technologies related to assistive environments, PETRA, pp 1–6

    Google Scholar 

  54. Glomann L (2017) Introducing ‘human-centered agile workflow’ (HCAW)—an agile conception and development process model. In: Proceedings of the AHFE 2017 international conference on usability and user experience, pp 646–655

    Google Scholar 

  55. Jerald J (2015) The VR book: human-centered design for virtual reality. ACM Books

    Google Scholar 

  56. Edwards C (2016) Iterative design. In: The bloomsbury encyclopedia of design. Bloomsbury Academic

    Google Scholar 

  57. Marques B et al (2020) Interaction with virtual content using augmented reality: a user study in assembly procedures. Proc ACM Human-Comput Interact 4(196):1–17

    Article  Google Scholar 

  58. Marques B, Silva S, Rocha A, Dias P, Santos BS (2021) Remote asynchronous collaboration in maintenance scenarios using augmented reality and annotations. In: IEEE conference on virtual reality and 3D user interfaces abstracts and workshops (VRW), IEEE VR 2021, pp 567–568

    Google Scholar 

  59. Pereira C et al (2015) Evaluation of complex distributed multimodal applications: evaluating a Tele rehabilitation system when it really matters. Lect Notes Comput Sci 9194:146–157

    Article  Google Scholar 

  60. Marques B, Dias P, Rocha A, Alves J, Santos BS (2019) An augmented reality framework for supporting technicians during maintenance procedures. Int Conf Graph Interact ICGI 2019:1–4

    Google Scholar 

  61. Alves J et al (2019) Augmented reality platform to foster collaborative manipulation of 3D virtual objects: a prototype. In: International conference on graphics and interaction, ICGI 2019

    Google Scholar 

  62. Alves J et al (2018) Remote interaction in virtual and augmented reality: a case study using multiple devices. In: International conference on graphics and interaction, ICGI 2018

    Google Scholar 

  63. Paelke V, Büttner S, Mucha H, Röcker C (2017) A checklist based approach for evaluating augmented reality displays in industrial applications. In: Proceedings of the AHFE 2017 international conference on human factors and ergonomics, pp 225–234

    Google Scholar 

  64. Marques B, Silva S, Dias P, Santos BS (2021) A toolkit to evaluate and characterize the collaborative process in scenarios of remote collaboration supported by AR. In: IEEE international symposium on mixed and augmented reality adjunct (ISMAR-Adjunct), pp 336–337

    Google Scholar 

Download references

Acknowledgements

We would like to thank the reviewers for their comments and efforts towards improving our manuscript. Also, António Teixeira for his thoughtful and constructive comments and suggestions, helping us improving our on-going research. This research was developed in the scope of the PhD grant [SFRH/BD/143276/2019], funded by FCT - Foundation for Science and Technology. It was also supported by IEETA - Institute of Electronics and Informatics Engineering of Aveiro, funded by FCT, in the context of the project [UID/CEC/00127/2020].

Author information

Authors and Affiliations

  1. IEETA, DETI, University of Aveiro, 3800-193, Aveiro, Portugal

    Bernardo Marques, Samuel Silva, Paulo Dias & Beatriz Sousa Santos

Authors
  1. Bernardo Marques
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Samuel Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paulo Dias
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Beatriz Sousa Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Beatriz Sousa Santos .

Editor information

Editors and Affiliations

  1. School of Industrial Engineering, Purdue University, West Lafayette, IN, USA

    Vincent G. Duffy

  2. Human-Computer Interaction Center, Aachen University, Aachen, Nordrhein-Westfalen, Germany

    Martina Ziefle

  3. Department of Industrial Engineering, Tsinghua University, Beijing, China

    Pei-Luen Patrick Rau

  4. International School of Technology and Management, Feng Chia University, Taiwan, China

    Mitchell M. Tseng

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Marques, B., Silva, S., Dias, P., Santos, B.S. (2023). Evaluating Augmented Reality Based Remote Collaboration: A Contextualized Approach. In: Duffy, V.G., Ziefle, M., Rau, PL.P., Tseng, M.M. (eds) Human-Automation Interaction. Automation, Collaboration, & E-Services, vol 12. Springer, Cham. https://doi.org/10.1007/978-3-031-10788-7_16

Download citation

Publish with us

Policies and ethics

Search

Navigation