Skip to main content
SpringerLink
Log in

Spatial Augmented Reality in the Factory: Can In-Situ Projections Be Used to Communicate Dangers and Health Risks?

  • Conference paper
  • First Online:
Human-Computer Interaction – INTERACT 2023 (INTERACT 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14143))

Included in the following conference series:

  • 689 Accesses

Abstract

In the context of industrial settings, extensive research of in-situ projections has proven their benefits for task performance. However, to date, these projections have not explicitly addressed policies designed to mitigate the dangers and health risks that are just as important, if not more than task performance considerations in such settings. We developed in-situ projections for three different use cases: (1) assembly support at a workbench, (2) ergonomic lifting, (3) restricted areas, which we studied with 15 representative target users. We found the expected benefits of the task-supporting projection (use case 1), increasing task performance and causing minimal cognitive load. However, our data also suggest that the other projections (use case 2 and 3) did not improve policy compliance. Our findings indicate that in-situ projections are not the most suitable solution to nudge workers to policy compliance in an industrial assembly setting, as most participants ignored the policy after evaluating the dangers themselves. Furthermore, based on our limitations and findings, we reflect on how current study practices can be improved for ubiquitous systems, especially when aiding policy compliance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://research-and-innovation.ec.europa.eu/research-area/industry/industry-50.

  2. 2.

    For a demo version of the software please contact the authors.

  3. 3.

    https://www.iso.org/standard/62996.html.

  4. 4.

    Since there is currently no agreed-upon way to interpret objective measures used to estimate cognitive load meaningfully [11], we opted to use the presented mix.

References

  1. Bezerra, C., et al.: Challenges for usability testing in ubiquitous systems. In: Proceedings of the 26th Conference on l’Interaction Homme-Machine, pp. 183–188. IHM 2014, Association for Computing Machinery, New York (2014). https://doi.org/10.1145/2670444.2670468

  2. Bimber, O., Iwai, D., Wetzstein, G., Grundhöfer, A.: The visual computing of projector-camera systems. In: ACM SIGGRAPH 2008 Classes, pp. 84:1–84:25. SIGGRAPH 2008, ACM, New York (2008). https://doi.org/10.1145/1401132.1401239

  3. Bimber, O., Raskar, R.: Spatial Augmented Reality: Merging Real and Virtual Worlds. A. K. Peters Ltd, USA (2005)

    Book  Google Scholar 

  4. Borro, D., Suescun, A., Brazalez, A., Gonzalez, J.M., Ortega, E., Gonzalez, E.: Warm: wearable AR and tablet-based assistant systems for bus maintenance. Appl. Sci. 11(4), 1–20 (2021). https://doi.org/10.3390/app11041443

    Article  Google Scholar 

  5. Brizzi, F., Peppoloni, L., Graziano, A., Stefano, E.D., Avizzano, C.A., Ruffaldi, E.: Effects of augmented reality on the performance of teleoperated industrial assembly tasks in a robotic embodiment. IEEE Trans. Hum. Mach. Syst. 48(2), 197–206 (2018). https://doi.org/10.1109/THMS.2017.2782490

    Article  Google Scholar 

  6. Büttner, S., Prilla, M., Röcker, C.: Augmented reality training for industrial assembly work - are projection-based AR assistive systems an appropriate tool for assembly training? In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–12. CHI 2020, Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3313831.3376720

  7. Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., Sheikh, Y.: OpenPose: realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans. Pattern Anal. Mach. Intell. 43(1), 172–186 (2021). https://doi.org/10.1109/TPAMI.2019.2929257

    Article  Google Scholar 

  8. Carvalho, R.M., Andrade, RMd.C., de Oliveira, K.M.: Aquarium - a suite of software measures for HCI quality evaluation of ubiquitous mobile applications. J. Syst. Softw. 136, 101–136 (2018). https://doi.org/10.1016/j.jss.2017.11.022

    Article  Google Scholar 

  9. Choi, W., Park, S., Kim, D., Lim, Y.-k, Lee, U.: Multi-stage receptivity model for mobile just-in-time health intervention. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3(2), 1–26 (2019). https://doi.org/10.1145/3328910

    Article  Google Scholar 

  10. Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., Tian, Q.: CenterNet: Keypoint triplets for object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6569–6578. ICCV 2019, Seoul (2019)

    Google Scholar 

  11. Duran, R., Zavgorodniaia, A., Sorva, J.: Cognitive load theory in computing education research: a review. ACM Trans. Comput. Educ. 22(4), 1–27 (2022). https://doi.org/10.1145/3483843

    Article  Google Scholar 

  12. Ericsson, K.A., Simon, H.A.: Protocol Analysis: Verbal Reports as Data. The MIT Press, Cambridge, MA (1993)

    Book  Google Scholar 

  13. Estrada-Lugo, H.D., et al.: Video analysis for ergonomics assessment in the manufacturing industry: initial feedback on a case study. In: Proceedings of the 32nd European Safety and Reliability Conference, TBP. ESREL 2022, ESRA, Dublin (2022)

    Google Scholar 

  14. Fite-Georgel, P.: Is there a reality in industrial augmented reality? In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality, pp. 201–210. IEEE, Basel (2011). https://doi.org/10.1109/ISMAR.2011.6092387

  15. Fraga-Lamas, P., Fernádez-Caramés, T.M., Blanco-Novoa, O., Vilar-Montesinos, M.: A review on industrial augmented reality systems for the industry 4.0 shipyard. IEEE Access 6, 13358–13375 (2018). https://doi.org/10.1109/ACCESS.2018.2808326

  16. Funk, M., Bächler, A., Bächler, L., Kosch, T., Heidenreich, T., Schmidt, A.: Working with augmented reality?: a long-term analysis of in-situ instructions at the assembly workplace. In: Proceedings of the 10th International Conference on Pervasive Technologies Related to Assistive Environments, pp. 222–229. Association for Computing Machinery, New York (2017)

    Google Scholar 

  17. Funk, M., Mayer, S., Schmidt, A.: Using in-situ projection to support cognitively impaired workers at the workplace. In: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility, pp. 185–192. ASSETS 2015, Association for Computing Machinery, New York (2015). https://doi.org/10.1145/2700648.2809853

  18. González-Franco, M., et al.: Immersive mixed reality for manufacturing training. Front. Robot. AI 4, 3 (2017)

    Article  Google Scholar 

  19. Guinet, A.L., Bouyer, G., Otmane, S., Desailly, E.: Reliability of the head tracking measured by microsoft hololens during different walking conditions. Comput. Methods Biomech. Biomed. Eng. 22(sup1), S169–S171 (2019). https://doi.org/10.1080/10255842.2020.1714228

    Article  Google Scholar 

  20. Hart, S.G., Staveland, L.E.: Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Human Mental Workload, Advances in Psychology, vol. 52, pp. 139–183. Amsterdam (1988). https://doi.org/10.1016/S0166-4115(08)62386-9, https://www.sciencedirect.com/science/article/pii/S0166411508623869

  21. Heindl, C., Stübl, G., Pönitz, T., Pichler, A., Scharinger, J.: Visual large-scale industrial interaction processing. In: Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers, pp. 280–283. UbiComp/ISWC 2019 Adjunct, Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3341162.3343769

  22. Hitachi: Putting fun into maintaining physical distance by system to link walking people and spatial distance : Research & Development : Hitachi (2020), https://www.hitachi.com/rd/news/topics/2020/1130.html

  23. Korn, O., Funk, M., Schmidt, A.: Assistive Systems for the Workplace: Towards Context-Aware Assistance, pp. 121–135. IGI Global, Hershey (2015). https://doi.org/10.4018/978-1-4666-7373-1.ch006

  24. Korn, O., Schmidt, A., Hörz, T.: The potentials of in-situ-projection for augmented workplaces in production: a study with impaired persons. In: CHI 2013 Extended Abstracts on Human Factors in Computing Systems, pp. 979–984. CHI EA 2013, Association for Computing Machinery, New York (2013). https://doi.org/10.1145/2468356.2468531

  25. Lewis, J., Sauro, J.: The factor structure of the system usability scale. In: Proceedings of the 1st International Conference on Human Centered Design: Held as Part of HCI International, vol. 5619, pp. 94–103. Springer, Berlin Heidelberg, Berlin (2009). https://doi.org/10.1007/978-3-642-02806-9_12

  26. Lin, G., Haynes, M., Srinivas, S., Kotipalli, P., Starner, T.: Towards finding the optimum position in the visual field for a head worn display used for task guidance with non-registered graphics. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5(1), 1–26 (2021). https://doi.org/10.1145/3448091

    Article  Google Scholar 

  27. Lin, P.C., Chen, Y.J., Chen, W.S., Lee, Y.J.: Automatic real-time occupational posture evaluation and select corresponding ergonomic assessments. Sci. Rep. 12(1), 2139 (2022). https://doi.org/10.1038/s41598-022-05812-9

    Article  Google Scholar 

  28. Masoni, R., et al.: Supporting remote maintenance in industry 4.0 through augmented reality. Procedia Manufacturing 11, 1296–1302 (2017). https://doi.org/10.1016/j.promfg.2017.07.257. https://www.sciencedirect.com/science/article/pii/S2351978917304651. In: 27th International Conference on Flexible Automation and Intelligent Manufacturing, FAIM2017, 27-30 June 2017, Modena, Italy

  29. Niu, S.: Ergonomics and occupational safety and health: an ILO perspective. Appl. Ergon. 41(6), 744–753 (2010)

    Article  Google Scholar 

  30. Pönitz, T., Ebenhofer, G., Stübl, G., Heindl, C., Scharinger, J.: On the potential of large-scale extended reality interaction for industrial environments. In: UbiComp 2021: The 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 61–63. Association for Computing Machinery, New York (2021). https://doi.org/10.1145/3460418.3479304

  31. Xia, Q., Korpela, J., Namioka, Y., Maekawa, T.: robust unsupervised factory activity recognition with body-worn accelerometer using temporal structure of multiple sensor data motifs. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4(3), 1–30 (2020). https://doi.org/10.1145/3411836

    Article  Google Scholar 

  32. Qingxin, X., Wada, A., Korpela, J., Maekawa, T., Namioka, Y.: Unsupervised factory activity recognition with wearable sensors using process instruction information. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3(2), 1–23 (2019). https://doi.org/10.1145/3328931

  33. QueueSight: Social Distancing Tool (2020). https://www.queuesight.com

  34. Rocha, L.C., Andrade, R.M.C., Sampaio, A.L., Lelli, V.: Heuristics to evaluate the usability of ubiquitous systems. In: Distributed, Ambient and Pervasive Interactions: 5th International Conference, DAPI 2017, Held as Part of HCI International 2017, Vancouver, BC, Canada, July 9–14, 2017, Proceedings, pp. 120–141. Springer-Verlag, Berlin (2017). https://doi.org/10.1007/978-3-319-58697-7_9

  35. de Souza Filho, J.C., Brito, M.R.F., Sampaio, A.L.: Comparing heuristic evaluation and MALTU model in interaction evaluation of ubiquitous systems. In: Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems. IHC 2020, Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3424953.3426639

  36. Tomitsch, M., et al.: Design. think. make. break. repeat. A Handbook of Methods. Bis Publishers, The Netherlands (2018)

    Google Scholar 

  37. Uva, A.E., Gattullo, M., Manghisi, V.M., Spagnulo, D., Cascella, G.L., Fiorentino, M.: Evaluating the effectiveness of spatial augmented reality in smart manufacturing: a solution for manual working stations. Int. J. Adv. Manuf. Technol. 94(1), 509–521 (2017). https://doi.org/10.1007/s00170-017-0846-4

    Article  Google Scholar 

  38. Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M.: YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors (2022). https://doi.org/10.48550/ARXIV.2207.02696

  39. Weiss, A., Wortmeier, A.-K., Kubicek, B.: Cobots in industry 4.0: a roadmap for future practice studies on human–robot collaboration. IEEE Trans. Hum. Mach. Syst. 51(4), 335–345 (2021). https://doi.org/10.1109/THMS.2021.3092684

    Article  Google Scholar 

  40. Zhou, J., Lee, I., Thomas, B., Menassa, R., Farrant, A., Sansome, A.: Applying spatial augmented reality to facilitate in-situ support for automotive spot welding inspection. In: Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry, pp. 195–200. VRCAI 2011, Association for Computing Machinery, New York (2011). https://doi.org/10.1145/2087756.2087784

Download references

Acknowledgements

This work was supported by the European Union’s Horizon 2020 research and innovation programme within the project TEAMING.AI (grant number 957402) as well as by the country of Upper Austria as part of the FTI strategy, project “Zer0P”.

Author information

Authors and Affiliations

  1. Technische Universität Wien, Vienna, Austria

    Aaron Wedral, Rafael Vrecar & Astrid Weiss

  2. PROFACTOR GmbH, Steyr-Gleink, 4407, Upper Austria, Austria

    Gerhard Ebenhofer, Thomas Pönitz, Paul H. Wührer & Gernot Stübl

Authors
  1. Aaron Wedral
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rafael Vrecar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gerhard Ebenhofer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomas Pönitz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Paul H. Wührer
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Astrid Weiss
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Gernot Stübl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aaron Wedral .

Editor information

Editors and Affiliations

  1. University of West London, London, UK

    José Abdelnour Nocera

  2. Reykjavik University, Reykjavik, Iceland

    Marta Kristín Lárusdóttir

  3. University of York, York, UK

    Helen Petrie

  4. University of Bari Aldo Moro, Bari, Italy

    Antonio Piccinno

  5. Université Côte d’Azur, Sophia Antipolis Cedex, France

    Marco Winckler

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wedral, A. et al. (2023). Spatial Augmented Reality in the Factory: Can In-Situ Projections Be Used to Communicate Dangers and Health Risks?. In: Abdelnour Nocera, J., Kristín Lárusdóttir, M., Petrie, H., Piccinno, A., Winckler, M. (eds) Human-Computer Interaction – INTERACT 2023. INTERACT 2023. Lecture Notes in Computer Science, vol 14143. Springer, Cham. https://doi.org/10.1007/978-3-031-42283-6_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-42283-6_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-42282-9

  • Online ISBN: 978-3-031-42283-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation