Skip to main content
SpringerLink
Log in

The effects of complex assembly task type and assembly experience on users’ demands for augmented reality instructions

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

Compared to traditional techniques, augmented reality (AR) confers notable benefits in facilitating complex product assembly processes. The efficacy of AR systems in assembly contexts is notably influenced by the pivotal role of AR instructions. As such, meeting users’ demands for AR instructions is crucial during AR-guided assembly processes. In the present study, an investigation was conducted into the influence of complex assembly task types and user assembly experience on their demands for AR instructions. Firstly, complex assembly tasks were categorized into repetitive complex assembly tasks (RAT) and non-repetitive complex assembly tasks (NRAT) based on their complex characteristics. A user study was conducted using HMD-HoloLens 2 as the experimental device. User performances were recorded during iterative execution of AR experimental tasks under the aforementioned task conditions. The specific measures included users’ attention process, interface interaction behaviors, assembly errors, and users’ subjective experience. The results indicate significant differences in users’ demands for AR instructions across different task types. Moreover, users’ demands for AR instructions also changed with increasing assembly experience. Through comprehensive analysis, the rules of users’ demands for AR instructions were summarized. The present findings enhance the current comprehension of users’ demands regarding AR instructions and offer valuable insights for designing and developing efficient AR-guided assembly systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

All the data are obtained by experiments and are authentic.

References

  1. Chu C-H, Ko C-H (2021) An experimental study on augmented reality assisted manual assembly with occluded components. J Manuf Syst 61:685–695

    Article  Google Scholar 

  2. Carmigniani J, Furht B (2011) Augmented reality: An overview. In: Furht B (ed) Handbook of augmented reality: Springer New York, pp 3–46

  3. Nee AYC, Ong SK, Chryssolouris G, Mourtzis D (2012) Augmented reality applications in design and manufacturing. CIRP Ann 61(2):657–679

    Article  Google Scholar 

  4. Malik AA, Masood T, Bilberg A (2019) Virtual reality in manufacturing: immersive and collaborative artificial-reality in design of human-robot workspace. Int J Comput Integr Manuf 33(1):22–37

    Article  Google Scholar 

  5. Gupta S, Chaudhary R, Gupta S (2019) A survey on tracking techniques in augmented reality based application. International Conference on Image Information Processing (ICIIP), pp 215–220

  6. Li W, Wang J, Lan S, Li S (2019) Augmented reality assembly process information content editing technology. Comput Integr Manuf Syst 25(7):1676–1684

    Google Scholar 

  7. Wu D, Zhou S, Xu H (2021) Assembly operation process assistance based on augmented reality and artificial intelligence. Aeronaut Manuf Technol 64(13):26–32

    Google Scholar 

  8. Schwerdtfeger Bo, Reif R, G¨unthner WA, Klinker G, Hamacher D, Schega L, Bockelmann I, Doil F, T¨umler J (2009) Pick-by-vision: a first stress test. In: IEEE International Symposium on Mixed and Augmented Reality, pp 115–124

  9. Choong Y-Y, Goad K, Mangold KC (2022) Augmented reality (AR) usability evaluation framework: the case of public safety communications research. NIST Interagency/Internal Report (NISTIR), pp 1–21

  10. Bottani E, Vignali G (2019) Augmented reality technology in the manufacturing industry: a review of the last decade. IISE Transactions 51(3):284–310

    Article  Google Scholar 

  11. Akçayır M, Akçayır G, Pektaş HM, Ocak MA (2016) Augmented reality in science laboratories: the effects of augmented reality on university students’ laboratory skills and attitudes toward science laboratories. Comput Hum Behav 57:334–342

    Article  Google Scholar 

  12. Poushneh A (2018) Augmented reality in retail: a trade-off between user’s control of access to personal information and augmentation quality. J Retail Consum Serv 41:169–176

    Article  Google Scholar 

  13. Yuan ML, Ong SK, Nee AYC (2008) Augmented reality for assembly guidance using a virtual interactive tool. Int J Prod Res 46(7):1745–1767

    Article  Google Scholar 

  14. Hou L, Wang X, Bernold L, Love PED (2013) Using animated augmented reality to cognitively guide assembly. J Comput Civ Eng 27(5):439–451

    Article  Google Scholar 

  15. Hou L, Wang X, Truijens M (2015) Using augmented reality to facilitate piping assembly: an experiment-based evaluation. J Comput Civil Eng 29:1

    Article  Google Scholar 

  16. Brizzi F, Peppoloni L, Graziano A, Stefano ED, Avizzano CA, Ruffaldi E (2018) Effects of augmented reality on the performance of teleoperated industrial assembly tasks in a robotic embodiment. IEEE Trans Human-Mach Syst 48(2):197–206

    Article  Google Scholar 

  17. Yang Z, Shi J, Jiang W, Sui Y, Wu Y, Ma S, Kang C, Li H (2019) Influences of augmented reality assistance on performance and cognitive loads in different stages of assembly task. Front Psychol 10:1703

    Article  PubMed  PubMed Central  Google Scholar 

  18. Vanneste P, Huang Y, Park JY, Cornillie F, Decloedt B, Van den Noortgate W (2020) Cognitive support for assembly operations by means of augmented reality: an exploratory study. Int J Human-Comput Stud 143:102480

  19. Werrlich S, Lorber C, Nguyen P-A, Yanez CEF, Notni G (2018) Assembly training: comparing the effects of head-mounted displays and face-to-face training. In: Chen J, Fragomeni G (eds) Virtual, augmented and mixed reality: interaction, navigation, visualization, embodiment, and simulation. Springer International Publishing, pp 462–476

  20. Barfield W, Caudell TP (2001) Boeing’s wire bundle assembly project

  21. Fiorentino M, Monno G, Uva AE (2009) Tangible digital master for product lifecycle management in augmented reality. Int J Interact Des Manuf (IJIDeM) 3(2):121–129

    Article  Google Scholar 

  22. Wiedenmaier S, Oehme O, Schmidt L, Luczak H (2003) Augmented reality (AR) for assembly processes design and experimental evaluation. Int J Human-Comput Interact 16(3):497–514

    Article  Google Scholar 

  23. Feiner SHS (2011) Exploring the benefits of augmented reality documentation for maintenance and repair. IEEE Trans Vis Comput Graph 17(10):1355–1368

    Article  PubMed  Google Scholar 

  24. Huang JM, Ong SK, Nee AYC (2017) Visualization and interaction of finite element analysis in augmented reality. Comput Aided Des 84:1–14

    Article  Google Scholar 

  25. Billinghurst M, Hakkarainen M, Woodward C (2008) Augmented assembly using a mobile phone. In: 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, pp 167–168

  26. Jeffri NFS, Rambli DRA (2020) Guidelines for the interface design of AR systems for manual assembly. In: Proceedings of the 2020 4th International Conference on Virtual and Augmented Reality Simulations, pp 70–77

  27. Funk M, Bächler A, Bächler L, Korn O, Krieger C, Heidenreich T, Schmidt A (2015) Comparing projected in-situ feedback at the manual assembly workplace with impaired workers. In: Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments, pp 1–8

  28. Funk M, Kosch T, Greenwald SW, Schmidt A (2015) A benchmark for interactive augmented reality instructions for assembly tasks. In: Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia, pp 253–257

  29. Wang Z, Bai X, Zhang S, Billinghurst M, He W, Wang Y, Han D, Chen G, Li J (2021) The role of user-centered AR instruction in improving novice spatial cognition in a high-precision procedural task. Adv Eng Inform 47:101250

  30. Funk M, Bächler A, Bächler L, Kosch T, Heidenreich T, Schmidt A (2017) Working with augmented reality? In: Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments, pp 222–229

  31. Wang C-H, Tsai N-H, Lu J-M, Wang M-JJ (2019) Usability evaluation of an instructional application based on Google Glass for mobile phone disassembly tasks. Appl Ergon 77:58–69

    Article  PubMed  Google Scholar 

  32. Rodriguez FS, Saleem K, Spilski J, Lachmann T (2021) Performance differences between instructions on paper vs digital glasses for a simple assembly task. Appl Ergon 94:103423

  33. Drouot M, Le Bigot N, Bricard E, Bougrenet JL, Nourrit V (2022) Augmented reality on industrial assembly line: impact on effectiveness and mental workload. Appl Ergon 103:103793

    Article  PubMed  Google Scholar 

  34. Wang Z, Bai X, Zhang S, Billinghurst M, He W, Wang P, Lan W, Min H, Chen Y (2022) A comprehensive review of augmented reality-based instruction in manual assembly, training and repair. Robot Comput -Integr Manuf 78:102407

  35. Gattullo M, Dalena V, Evangelista A, Uva AE (2019) A context-aware technical information manager for presentation in augmented reality. Virtual Reality and 3D User Interfaces (VR) 939–940

  36. Wang Z, Bai X, Zhang S, He W, Zhang X, Zhang L, Wang P, Han D, Yan Y (2019) Information-level AR instruction: a novel assembly guidance information representation assisting user cognition. Int J Adv Manuf Technol 106(1–2):603–626

    Google Scholar 

  37. Wang X, Ong SK, Nee AYC (2016) Multi-modal augmented-reality assembly guidance based on bare-hand interface. Adv Eng Inform 30(3):406–421

    Article  Google Scholar 

  38. Endsley TC, Sprehn KA, Brill RM, Ryan KJ, Vincent EC, Martin JM (2017) Augmented reality design heuristics: designing for dynamic interactions. Proc Human Factors Ergon Soc Ann Meet 61(1):2100–2104

    Article  Google Scholar 

  39. Ariansyah D, Erkoyuncu JA, Eimontaite I, Johnson T, Oostveen A-M, Fletcher S, Sharples S (2022) A head mounted augmented reality design practice for maintenance assembly: toward meeting perceptual and cognitive needs of AR users. Appl Ergon 98(2):103597

    Article  PubMed  Google Scholar 

  40. Jones AT, Romero D, Wuest T (2018) Modeling agents as joint cognitive systems in smart manufacturing systems. Manuf Lett 17:6–8

    Article  Google Scholar 

  41. Hollnagel E (2005) Designing for joint cognitive systems. People and Systems - Who are we Designing for 47–51

  42. Rumelhart DE (1977) Cognitive psychology: cognition and reality. Science 198(4319):816–817

  43. Haritos T, Macchiarella ND (2005) A mobile application of augmented reality for aerospace maintenance training. In: 24th Digital Avionics Systems Conference 1:5.B.3–5.1

  44. Wang Z, Bai X, Zhang S, He W, Wang P, Zhang X, Yan Y (2020) SHARIdeas: a visual representation of intention sharing between designer and executor supporting AR assembly. In: Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference, pp 1–2

  45. Stork S, Schubö A (2010) Human cognition in manual assembly: theories and applications. Adv Eng Inform 24(3):320–328

    Article  Google Scholar 

  46. Radkowski R, Herrema J, Oliver J (2015) Augmented reality-based manual assembly support with visual features for different degrees of difficulty. Int J Human-Comput Interact 31(5):337–349

    Article  Google Scholar 

  47. Blattgerste J, Strenge B, Renner P, Pfeiffer T, Essig K (2017) Comparing conventional and augmented reality instructions for manual assembly tasks. In: Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments, pp 75–82

  48. Funk M, Kosch T, Schmidt A (2016) Interactive worker assistance: comparing the effects of in-situ projection, head-mounted displays, tablet, and paper instructions. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp 934–939

  49. Smith E, Semple G, Evans D, McRae K, Blackwell P (2020) Augmented instructions: analysis of performance and efficiency of assembly tasks. In: Chen JYC, Fragomeni G (eds) Virtual, Augmented and Mixed Reality. Industrial and Everyday Life Applications, 12191:166–177

  50. Takemura BMKKKAMJJLVTMH (2014) The effectiveness of an AR-based context-aware assembly support system in object assembly. In: 2014 IEEE Virtual Reality (VR), pp 57–62

  51. Livingston MA (2005) Evaluating human factors in augmented reality systems. IEEE Comput Graphics Appl 25(6):6–9

    Article  Google Scholar 

  52. Porter ME, Heppelmann JE (2017) Why every organization needs an augmented reality strategy. Harv Bus Rev 95(6):46–57

    Google Scholar 

  53. Holm M, Danielsson O, Syberfeldt A, Moore P, Wang L (2017) Adaptive instructions to novice shop-floor operators using augmented reality. J Ind Prod Eng 34(5):362–374

    Google Scholar 

  54. Li D, Babcock J, and Parkhurst DJ (2006) openEyes: a low-cost head-mounted eye-tracking solution. In: Proceedings of the 2006 symposium on Eye tracking research & applications, pp 95–100

  55. Chen F, Ruiz N, Choi E, Epps J, Khawaja MA, Taib R, Yin B, Wang Y (2012) Multimodal behavior and interaction as indicators of cognitive load. ACM Trans Interact Intell Syst 2(4):1–36

    Article  Google Scholar 

  56. Liang C, Lin C-T, Yao S-N, Chang W-S, Liu Y-C, Chen S-A (2017) Visual attention and association: an electroencephalography study in expert designers. Des Stud 48:76–95

    Article  Google Scholar 

  57. Thomas LE, Lleras A (2007) Moving eyes and moving thought: on the spatial compatibility between eye movements and cognition. Psychon Bull Rev 14(4):663–668

    Article  PubMed  Google Scholar 

  58. Nuthmann A, Smith TJ, Engbert R, Henderson JM (2010) CRISP: a computational model of fixation durations in scene viewing. Psychol Rev 117(2):382–405

    Article  PubMed  Google Scholar 

  59. Kalyani R and Gadiraju U (2019) Understanding user search behavior across varying cognitive levels. In: Proceedings of the 30th ACM Conference on Hypertext and Social Media, pp 123–132

  60. Duncan J, Humphreys GW (1989) Visual search and stimulus similarity. Psychol Rev 96(3):433–458

    Article  CAS  PubMed  Google Scholar 

  61. Desimone R, Duncan J (1995) Neural mechanisms of selective visual attention. Annu Rev Neurosci 18:193–222

    Article  CAS  PubMed  Google Scholar 

  62. Richardson M, Jones G, Torrance M (2004) Identifying the task variables that influence perceived object assembly complexity. Ergonomics 47(9):945–964

    Article  PubMed  Google Scholar 

  63. Wickens CD (1992) Engineering psychology and human performance (2nd ed.): HarperCollins Publishers

  64. Clark R, Harrelson GL (2002) Designing instruction that supports cognitive learning processes. J Athl Train 37(4):152–159

    Google Scholar 

  65. Basil MD (2012) Multiple resource theory. In: Seel NM (eds) Encyclopedia of the Sciences of Learning, pp 2384–2385

  66. Lee BC, Duffy VG (2015) The effects of task interruption on human performance: a study of the systematic classification of human behavior and interruption frequency. Human Factors Ergon Manuf Serv Ind 25(2):137–152

    Article  Google Scholar 

  67. Lai Z-H, Tao W, Leu MC, Yin Z (2020) Smart augmented reality instructional system for mechanical assembly towards worker-centered intelligent manufacturing. J Manuf Syst 55:69–81

    Article  Google Scholar 

  68. Sable-Meyer M, Fagot J, Caparos S, van Kerkoerle T, Amalric M, Dehaene S (2021) Sensitivity to geometric shape regularity in humans and baboons: a putative signature of human singularity. Proc Natl Acad. Sci U.S.A 118:16

    Article  Google Scholar 

Download references

Funding

This work is partly supported by the National Key R&D Program of China (Grant No. 2019YFB1703804, 2021YFB1716200, 2021YFB1714900), the National Natural Science Foundation of China (Grant No.52375515, 52275513), and the Programme of Introducing Talents of Discipline to Universities (111 Project), China, (Grant No. B13044).

Author information

Authors and Affiliations

  1. Laboratory of Cyber-Physical Interaction, School of Mechanical Engineering, Northwestern Polytechnical University, Xi’an, 710072, China

    Jianghong Li, Shuxia Wang, Guanfeng Wang, Jie Zhang, Shuo Feng, Yifan Xiao & Shixin Wu

Authors
  1. Jianghong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuxia Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guanfeng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shuo Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yifan Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Shixin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All the authors contributed to the study’s conception and design. Material preparation, data collection, and analysis were performed by Jianghong Li, Jie Zhang, Shuo Feng, Yifan Xiao, and Shixin Wu. The first draft of the manuscript was written by Jianghong Li, and all the authors commented on previous versions of the manuscript. All the authors read and approved the final manuscript.

Corresponding author

Correspondence to Shuxia Wang.

Ethics declarations

Ethics approval

Not applicable.

Consent to participate

All authors read and approved the final manuscript.

Consent for publication

All authors agree to publish in The International Journal of Advanced Manufacturing Technology.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, J., Wang, S., Wang, G. et al. The effects of complex assembly task type and assembly experience on users’ demands for augmented reality instructions. Int J Adv Manuf Technol 131, 1479–1496 (2024). https://doi.org/10.1007/s00170-024-13091-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-024-13091-z

Keywords

Search

Navigation