Abstract
In the context of industrial settings, extensive research of in-situ projections has proven their benefits for task performance. However, to date, these projections have not explicitly addressed policies designed to mitigate the dangers and health risks that are just as important, if not more than task performance considerations in such settings. We developed in-situ projections for three different use cases: (1) assembly support at a workbench, (2) ergonomic lifting, (3) restricted areas, which we studied with 15 representative target users. We found the expected benefits of the task-supporting projection (use case 1), increasing task performance and causing minimal cognitive load. However, our data also suggest that the other projections (use case 2 and 3) did not improve policy compliance. Our findings indicate that in-situ projections are not the most suitable solution to nudge workers to policy compliance in an industrial assembly setting, as most participants ignored the policy after evaluating the dangers themselves. Furthermore, based on our limitations and findings, we reflect on how current study practices can be improved for ubiquitous systems, especially when aiding policy compliance.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
For a demo version of the software please contact the authors.
- 3.
- 4.
Since there is currently no agreed-upon way to interpret objective measures used to estimate cognitive load meaningfully [11], we opted to use the presented mix.
References
Bezerra, C., et al.: Challenges for usability testing in ubiquitous systems. In: Proceedings of the 26th Conference on l’Interaction Homme-Machine, pp. 183–188. IHM 2014, Association for Computing Machinery, New York (2014). https://doi.org/10.1145/2670444.2670468
Bimber, O., Iwai, D., Wetzstein, G., Grundhöfer, A.: The visual computing of projector-camera systems. In: ACM SIGGRAPH 2008 Classes, pp. 84:1–84:25. SIGGRAPH 2008, ACM, New York (2008). https://doi.org/10.1145/1401132.1401239
Bimber, O., Raskar, R.: Spatial Augmented Reality: Merging Real and Virtual Worlds. A. K. Peters Ltd, USA (2005)
Borro, D., Suescun, A., Brazalez, A., Gonzalez, J.M., Ortega, E., Gonzalez, E.: Warm: wearable AR and tablet-based assistant systems for bus maintenance. Appl. Sci. 11(4), 1–20 (2021). https://doi.org/10.3390/app11041443
Brizzi, F., Peppoloni, L., Graziano, A., Stefano, E.D., Avizzano, C.A., Ruffaldi, E.: Effects of augmented reality on the performance of teleoperated industrial assembly tasks in a robotic embodiment. IEEE Trans. Hum. Mach. Syst. 48(2), 197–206 (2018). https://doi.org/10.1109/THMS.2017.2782490
Büttner, S., Prilla, M., Röcker, C.: Augmented reality training for industrial assembly work - are projection-based AR assistive systems an appropriate tool for assembly training? In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–12. CHI 2020, Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3313831.3376720
Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., Sheikh, Y.: OpenPose: realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans. Pattern Anal. Mach. Intell. 43(1), 172–186 (2021). https://doi.org/10.1109/TPAMI.2019.2929257
Carvalho, R.M., Andrade, RMd.C., de Oliveira, K.M.: Aquarium - a suite of software measures for HCI quality evaluation of ubiquitous mobile applications. J. Syst. Softw. 136, 101–136 (2018). https://doi.org/10.1016/j.jss.2017.11.022
Choi, W., Park, S., Kim, D., Lim, Y.-k, Lee, U.: Multi-stage receptivity model for mobile just-in-time health intervention. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3(2), 1–26 (2019). https://doi.org/10.1145/3328910
Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., Tian, Q.: CenterNet: Keypoint triplets for object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6569–6578. ICCV 2019, Seoul (2019)
Duran, R., Zavgorodniaia, A., Sorva, J.: Cognitive load theory in computing education research: a review. ACM Trans. Comput. Educ. 22(4), 1–27 (2022). https://doi.org/10.1145/3483843
Ericsson, K.A., Simon, H.A.: Protocol Analysis: Verbal Reports as Data. The MIT Press, Cambridge, MA (1993)
Estrada-Lugo, H.D., et al.: Video analysis for ergonomics assessment in the manufacturing industry: initial feedback on a case study. In: Proceedings of the 32nd European Safety and Reliability Conference, TBP. ESREL 2022, ESRA, Dublin (2022)
Fite-Georgel, P.: Is there a reality in industrial augmented reality? In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality, pp. 201–210. IEEE, Basel (2011). https://doi.org/10.1109/ISMAR.2011.6092387
Fraga-Lamas, P., Fernádez-Caramés, T.M., Blanco-Novoa, O., Vilar-Montesinos, M.: A review on industrial augmented reality systems for the industry 4.0 shipyard. IEEE Access 6, 13358–13375 (2018). https://doi.org/10.1109/ACCESS.2018.2808326
Funk, M., Bächler, A., Bächler, L., Kosch, T., Heidenreich, T., Schmidt, A.: Working with augmented reality?: a long-term analysis of in-situ instructions at the assembly workplace. In: Proceedings of the 10th International Conference on Pervasive Technologies Related to Assistive Environments, pp. 222–229. Association for Computing Machinery, New York (2017)
Funk, M., Mayer, S., Schmidt, A.: Using in-situ projection to support cognitively impaired workers at the workplace. In: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility, pp. 185–192. ASSETS 2015, Association for Computing Machinery, New York (2015). https://doi.org/10.1145/2700648.2809853
González-Franco, M., et al.: Immersive mixed reality for manufacturing training. Front. Robot. AI 4, 3 (2017)
Guinet, A.L., Bouyer, G., Otmane, S., Desailly, E.: Reliability of the head tracking measured by microsoft hololens during different walking conditions. Comput. Methods Biomech. Biomed. Eng. 22(sup1), S169–S171 (2019). https://doi.org/10.1080/10255842.2020.1714228
Hart, S.G., Staveland, L.E.: Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Human Mental Workload, Advances in Psychology, vol. 52, pp. 139–183. Amsterdam (1988). https://doi.org/10.1016/S0166-4115(08)62386-9, https://www.sciencedirect.com/science/article/pii/S0166411508623869
Heindl, C., Stübl, G., Pönitz, T., Pichler, A., Scharinger, J.: Visual large-scale industrial interaction processing. In: Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers, pp. 280–283. UbiComp/ISWC 2019 Adjunct, Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3341162.3343769
Hitachi: Putting fun into maintaining physical distance by system to link walking people and spatial distance : Research & Development : Hitachi (2020), https://www.hitachi.com/rd/news/topics/2020/1130.html
Korn, O., Funk, M., Schmidt, A.: Assistive Systems for the Workplace: Towards Context-Aware Assistance, pp. 121–135. IGI Global, Hershey (2015). https://doi.org/10.4018/978-1-4666-7373-1.ch006
Korn, O., Schmidt, A., Hörz, T.: The potentials of in-situ-projection for augmented workplaces in production: a study with impaired persons. In: CHI 2013 Extended Abstracts on Human Factors in Computing Systems, pp. 979–984. CHI EA 2013, Association for Computing Machinery, New York (2013). https://doi.org/10.1145/2468356.2468531
Lewis, J., Sauro, J.: The factor structure of the system usability scale. In: Proceedings of the 1st International Conference on Human Centered Design: Held as Part of HCI International, vol. 5619, pp. 94–103. Springer, Berlin Heidelberg, Berlin (2009). https://doi.org/10.1007/978-3-642-02806-9_12
Lin, G., Haynes, M., Srinivas, S., Kotipalli, P., Starner, T.: Towards finding the optimum position in the visual field for a head worn display used for task guidance with non-registered graphics. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5(1), 1–26 (2021). https://doi.org/10.1145/3448091
Lin, P.C., Chen, Y.J., Chen, W.S., Lee, Y.J.: Automatic real-time occupational posture evaluation and select corresponding ergonomic assessments. Sci. Rep. 12(1), 2139 (2022). https://doi.org/10.1038/s41598-022-05812-9
Masoni, R., et al.: Supporting remote maintenance in industry 4.0 through augmented reality. Procedia Manufacturing 11, 1296–1302 (2017). https://doi.org/10.1016/j.promfg.2017.07.257. https://www.sciencedirect.com/science/article/pii/S2351978917304651. In: 27th International Conference on Flexible Automation and Intelligent Manufacturing, FAIM2017, 27-30 June 2017, Modena, Italy
Niu, S.: Ergonomics and occupational safety and health: an ILO perspective. Appl. Ergon. 41(6), 744–753 (2010)
Pönitz, T., Ebenhofer, G., Stübl, G., Heindl, C., Scharinger, J.: On the potential of large-scale extended reality interaction for industrial environments. In: UbiComp 2021: The 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 61–63. Association for Computing Machinery, New York (2021). https://doi.org/10.1145/3460418.3479304
Xia, Q., Korpela, J., Namioka, Y., Maekawa, T.: robust unsupervised factory activity recognition with body-worn accelerometer using temporal structure of multiple sensor data motifs. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4(3), 1–30 (2020). https://doi.org/10.1145/3411836
Qingxin, X., Wada, A., Korpela, J., Maekawa, T., Namioka, Y.: Unsupervised factory activity recognition with wearable sensors using process instruction information. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3(2), 1–23 (2019). https://doi.org/10.1145/3328931
QueueSight: Social Distancing Tool (2020). https://www.queuesight.com
Rocha, L.C., Andrade, R.M.C., Sampaio, A.L., Lelli, V.: Heuristics to evaluate the usability of ubiquitous systems. In: Distributed, Ambient and Pervasive Interactions: 5th International Conference, DAPI 2017, Held as Part of HCI International 2017, Vancouver, BC, Canada, July 9–14, 2017, Proceedings, pp. 120–141. Springer-Verlag, Berlin (2017). https://doi.org/10.1007/978-3-319-58697-7_9
de Souza Filho, J.C., Brito, M.R.F., Sampaio, A.L.: Comparing heuristic evaluation and MALTU model in interaction evaluation of ubiquitous systems. In: Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems. IHC 2020, Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3424953.3426639
Tomitsch, M., et al.: Design. think. make. break. repeat. A Handbook of Methods. Bis Publishers, The Netherlands (2018)
Uva, A.E., Gattullo, M., Manghisi, V.M., Spagnulo, D., Cascella, G.L., Fiorentino, M.: Evaluating the effectiveness of spatial augmented reality in smart manufacturing: a solution for manual working stations. Int. J. Adv. Manuf. Technol. 94(1), 509–521 (2017). https://doi.org/10.1007/s00170-017-0846-4
Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M.: YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors (2022). https://doi.org/10.48550/ARXIV.2207.02696
Weiss, A., Wortmeier, A.-K., Kubicek, B.: Cobots in industry 4.0: a roadmap for future practice studies on human–robot collaboration. IEEE Trans. Hum. Mach. Syst. 51(4), 335–345 (2021). https://doi.org/10.1109/THMS.2021.3092684
Zhou, J., Lee, I., Thomas, B., Menassa, R., Farrant, A., Sansome, A.: Applying spatial augmented reality to facilitate in-situ support for automotive spot welding inspection. In: Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry, pp. 195–200. VRCAI 2011, Association for Computing Machinery, New York (2011). https://doi.org/10.1145/2087756.2087784
Acknowledgements
This work was supported by the European Union’s Horizon 2020 research and innovation programme within the project TEAMING.AI (grant number 957402) as well as by the country of Upper Austria as part of the FTI strategy, project “Zer0P”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wedral, A. et al. (2023). Spatial Augmented Reality in the Factory: Can In-Situ Projections Be Used to Communicate Dangers and Health Risks?. In: Abdelnour Nocera, J., Kristín Lárusdóttir, M., Petrie, H., Piccinno, A., Winckler, M. (eds) Human-Computer Interaction – INTERACT 2023. INTERACT 2023. Lecture Notes in Computer Science, vol 14143. Springer, Cham. https://doi.org/10.1007/978-3-031-42283-6_31
Download citation
DOI: https://doi.org/10.1007/978-3-031-42283-6_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-42282-9
Online ISBN: 978-3-031-42283-6
eBook Packages: Computer ScienceComputer Science (R0)