Skip to main content
SpringerLink
Log in

User-oriented AR assembly guideline: a new classification method of assembly instruction for user cognition

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

In augmented reality (AR) assembly, due to the unintuitive expression of the information content of the assembly instructions and the irregular design of the information form, the user’s assembly efficiency is low and the operation error rate is high. This paper elaborates and defines a user-oriented classification method of assembly instructions. This method reclassifies traditional assembly instructions based on the user’s cognitive needs for assembly tasks to improve users’ cognitive efficiency of physical tasks. In this article, the definition of traditional assembly instructions is first reiterated, and the geometrical relationship between it and AR assembly instructions is explained. Then, the definition of AR assembly instructions at the information level is given, the traditional assembly instructions are classified according to this definition, and the explanation of each classification is made. We compared the role of the old and new instructions in typical assembly use cases. The data shows that there are significant differences in performance between the new and old instructions. The new instructions significantly improve the user’s performance in terms of assembly time and operating experience (including enjoyment, concentration, confidence, natural intuition, feasibility, effectiveness, usability, and understandability), through the use of AR instructions on the information level and auxiliary reclassification, so as to achieve efficient and concise operation purposes. In addition, we also discussed the significance of this research and future research directions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Radkowski R, Herrema JS, Oliver JH (2015) Augmented reality-based manual assembly support with visual features for different degrees of difficulty. Int J Human-Comput Interac 31:337–349

    Google Scholar 

  2. Reiners D, Stricker D, Klinker G, Muller S (1999) Augmented reality for construction tasks: doorlock assembly. In: Proceedings of the international workshop on Augmented reality: placing artificial objects in real scenes: placing artificial objects in real scenes, pp 31–46

    Google Scholar 

  3. Mizell D (2001) Boeing’s wire bundle assembly project. In: Fundamentals of wearable computers and augmented reality, vol 5

    Google Scholar 

  4. Henderson S, Feiner S (2011) Exploring the benefits of augmented reality documentation for maintenance and repair. IEEE Trans Vis Comput Graph 17:1355–1368

    Google Scholar 

  5. Stork S, Schubö A (2010) Human cognition in manual assembly: theories and applications. Adv Eng Inform 24:320–328

    Google Scholar 

  6. Hou L, Wang X, Bernold L, Love PE (2013) Using animated augmented reality to cognitively guide assembly. J Comput Civ Eng 27:439–451

    Google Scholar 

  7. Funk M, Bächler A, Bächler L, Korn O, Krieger C, Heidenreich T et al (2015) Comparing projected in-situ feedback at the manual assembly workplace with impaired workers. In: Acm International Conference on Pervasive Technologies Related to Assistive Environments

    Google Scholar 

  8. Wang Z, Bai X, Zhang S, He W, Zhang X, Zhang L, Wang P, Han D, Yan Y (2020) Information-level AR instruction: a novel assembly guidance information representation assisting user cognition. Int J Adv Manuf Technol 106:603–626

    Google Scholar 

  9. Wang Z, Bai X, Zhang S, He W, Zhang X, Yan Y et al (2020) Information-level real-time AR instruction: a novel dynamic assembly guidance information representation assisting human cognition. Int J Adv Manuf Technol:1–19

  10. Lee K, Andrews G (1985) Inference of the positions of components in an assembly: part 2. Comput Aided Des 17:20–24

    Google Scholar 

  11. Sodhi R, Joshua UT (1991) Representing tolerance and assembly information in a feature-based design environment. In: Proceedings of the 1991 ASME design automation conference, vol 32

    Google Scholar 

  12. J. J. Shah and T. Ravi 1992, "Feature based assembly modeling," Proceedings of the 1992 ASME International Computers in Engineering Conference and Exposition,

  13. Deneux D (1999) Introduction to assembly features: an illustrated synthesis methodology. J Intell Manuf 10:29–39

    Google Scholar 

  14. De Fazio TL, Edsall AC, Gustavson RE, Hernandez JA, Hutchins PM, Leung HW et al (1993) A prototype of feature-based design for assembly. J Mech Des 115:723–734

    Google Scholar 

  15. Van Holland W, Bronsvoort WF (2000) Assembly features in modeling and planning. Robot Comput Integr Manuf 16:277–294

    Google Scholar 

  16. Sung R, Corney J, Clark DER (2001) Automatic assembly feature recognition and disassembly sequence generation. J Comput Inf Sci Eng 1:291–299

    Google Scholar 

  17. Shyamsundar N, Gadh R (2001) Internet-based collaborative product design with assembly features and virtual design spaces. Comput Aided Des 33:637–651

    Google Scholar 

  18. Zha XF, Du H (2002) A PDES/STEP-based model and system for concurrent integrated design and assembly planning. Comput Aided Des 34:1087–1110

    Google Scholar 

  19. Chan C, Tan ST (2003) Generating assembly features onto split solid models. Comput Aided Des 35:1315–1336

    Google Scholar 

  20. Hamidullah E, Irfan MA (2006) Assembly features: definition, classification, and instantiation. In: International conference on emerging technologies, pp 617–623

    Google Scholar 

  21. Pang Y, Nee AYC, Ong SK, Yuan ML, Youceftoumi K (2006) Assembly feature design in an augmented reality environment. Assem Autom 26:34–43

    Google Scholar 

  22. T. P. Caudell and D. W. Mizell 1992, "Augmented reality: an application of heads-up display technology to manual manufacturing processes," in System Sciences. Proceedings of the Twenty-Fifth Hawaii International Conference on, 1992, pp. 659–669

  23. Webster A (1996) Augmented reality in architectural construction, inspection and renovation. In: Asce Third Congress on Computing Civil Engineering

    Google Scholar 

  24. Neumann U, Majoros A (1998) Cognitive, performance, and systems issues for augmented reality applications in manufacturing and maintenance. In: Virtual Reality Annual International Symposium. Proceedings., IEEE, pp 4–11

    Google Scholar 

  25. Stork S, Schuboe A (2010) Human cognition in manual assembly: theories and applications. Adv Eng Inform 24:320–328

    Google Scholar 

  26. Henderson SJ, Feiner SK (2011) Augmented reality in the psychomotor phase of a procedural task. In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality, pp 191–200

    Google Scholar 

  27. Gorecky D, Worgan SF, Meixner G (2011) COGNITO: a cognitive assistance and training system for manual tasks in industry. In: ECCE 2011 - European Conference on Cognitive Ergonomics, Rostock, Germany, August 24–26, 2011, Proceedings of the 29th annual conference of the European Association of Cognitive Ergonomics

    Google Scholar 

  28. Gauglitz S, Lee C, Turk M, Höllerer T (2012) Integrating the physical environment into mobile remote collaboration. In: International Conference on Human-computer Interaction with Mobile Devices & Services

    Google Scholar 

  29. Petersen N, Stricker D (2015) Cognitive augmented reality. Comput Graph 53:82–91

    Google Scholar 

  30. Cunha P, Brandão J, Vasconcelos J, Soares F, Carvalho V (2016) Augmented reality for cognitive and social skills improvement in children with ASD. In: 2016 13th International Conference on Remote Engineering and Virtual Instrumentation (REV), pp 334–335

    Google Scholar 

  31. Blattgerste J, Strenge B, Renner P, Pfeiffer T, Essig K (2017) Comparing conventional and augmented reality instructions for manual assembly tasks. In: Pervasive technologies related to assistive environments, pp 75–82

    Google Scholar 

  32. Blattgerste J, Renner P, Strenge B, Pfeiffer T (2018) In-situ instructions exceed side-by-side instructions in augmented reality assisted assembly. In: Pervasive technologies related to assistive environments, pp 133–140

    Google Scholar 

  33. Deshpande A, Kim I (2018) The effects of augmented reality on improving spatial problem solving for object assembly. Adv Eng Inform 38:760–775

    Google Scholar 

  34. Ramachandran A, Palanivelu K, Mudgal A et al (2019) Mini-Me: an adaptive avatar for mixed reality remote collaboration. In: Scientific Reports

    Google Scholar 

  35. Wang P, Zhang S, Bai X, Billinghurst M, He W, Sun M et al (2019) 2.5DHANDS: a gesture-based MR remote collaborative platform. Int J Adv Manuf Technol 105:3031–3043

    Google Scholar 

  36. Wang P, Zhang S, Bai X, Billinghurst M, He W, Sun M, Chen Y, Lv H, Ji H (2019) 2.5 DHANDS: a gesture-based MR remote collaborative platform. Int J Adv Manuf Technol 102:1339–1353

    Google Scholar 

  37. Doshi A, Smith RT, Thomas BH, Bouras C (2017) Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing. Int J Adv Manuf Technol 89:1279–1293

    Google Scholar 

  38. Alemanni M, Destefanis F, Vezzetti E (2011) Model-based definition design in the product lifecycle management scenario. Int J Adv Manuf Technol 52:1–14

    Google Scholar 

  39. Nguyen D, Gerrit M (2019) Comparison user engagement of gamified and non-gamified augmented reality assembly training. In: Advances in agile and user-centred software engineering, pp 142–152

    Google Scholar 

  40. Fiorentino M, Monno G, Uva A (2009) Tangible digital master for product lifecycle management in augmented reality. Int J Interact. Des. Manuf. (IJIDeM) 3:121–129

    Google Scholar 

  41. Wiedenmaier S, Oehme O, Schmidt L, Luczak H (2003) Augmented reality (AR) for assembly processes design and experimental evaluation. Int J Human-Comput Interac 16:497–514

    Google Scholar 

  42. Zhang J, Ong S, Nee A (2011) RFID-assisted assembly guidance system in an augmented reality environment. Int J Prod Res 49:3919–3938

    Google Scholar 

  43. Feiner S, Macintyre B, Seligmann D (1993) Knowledge-based augmented reality. Commun ACM 36:52–63

    Google Scholar 

  44. T.-C. Optronique, F. Guyancourt, and T. SA, "STARMATE: using augmented reality technology for computer guided maintenance of complex mechanical elements," 2003

    Google Scholar 

  45. Billinghurst M, Hakkarainen M, Woodward C (2008) Augmented assembly using a mobile phone. In: Proceedings of the 7th International Conference on Mobile and Ubiquitous Multimedia, pp 84–87

    Google Scholar 

  46. Funk M, Kosch T, Greenwald SW, Schmidt A (2015) A benchmark for interactive augmented reality instructions for assembly tasks. In: Mobile and ubiquitous multimedia, pp 253–257

    Google Scholar 

  47. Huang JM, Ong SK, Nee AYC (2016) Visualization and interaction of finite element analysis in augmented reality. In: Computer Aided Design, vol 84

    Google Scholar 

  48. Uva AE, Cristiano S, Fiorentino M, Monno G (2010) Distributed design review using tangible augmented technical drawings. Comput Aid Des 42:364–372

    Google Scholar 

  49. Webel S, Bockholt U, Engelke T, Gavish N, Tecchia F (2011) Design recommendations for augmented reality based training of maintenance skills. Springer, New York

    Google Scholar 

  50. Gavish N, Gutierrez T, Webel S, Rodriguez J, Tecchia F (2011) Design guidelines for the development of virtual reality and augmented reality training systems for maintenance and assembly tasks. In: Bio Web of Conferences: the International Conference Skills

    Google Scholar 

  51. Wang X, Ong S, Nee AY-C (2016) Multi-modal augmented-reality assembly guidance based on bare-hand interface. Adv Eng Inform 30:406–421

    Google Scholar 

  52. Baumeister J, Ssin SY, Elsayed NAM, Dorrian J, Webb DP, Walsh JA et al (2017) Cognitive cost of using augmented reality displays. IEEE Transac Visual Comput Graph 23:2378–2388

    Google Scholar 

  53. Palmarini R, Erkoyuncu JA, Roy R, Torabmostaedi H (2018) A systematic review of augmented reality applications in maintenance. Robot Comput Integr Manuf 49:215–228

    Google Scholar 

  54. Wang P, Zhang S, Bai X, Billinghurst M, Zhang L, Wang S, Han D, Lv H, Yan Y (2019) A gesture- and head-based multimodal interaction platform for MR remote collaboration. Int J Adv Manuf Technol 105:3031–3043

    Google Scholar 

Download references

Acknowledgments

We would like to appreciate the anonymous reviewers for their constructive suggestions for enhancing this paper. In addition, I would like to thank Zhishuo Xiong of the London School of Economics and Political Science for checking the early versions of the English manuscript and helping the author to correct the grammatical errors in the paper. We thank the CPILab VR/AR research team for their contributions to this research. Shu Han provided some valuable design solutions for this UX experiment. Yuxiang Yan established the basic hardware environment for our research. Peng Wang broke through the technical difficulty of this research, and Hao Lv did a lot of work for the collection of experimental data. In addition, we would like to thank Prof. Shusheng Zhang and Associate Prof. Xiaoliang Bai for their constructive comments on the improvement of the experiment. We would also like to thank the volunteers of Northwestern Polytechnical University for participating in the experiment.

Funding

This work is partly supported by the National Key R&D Program of China (Grant No. 2019YFB1703800), Fundamental Research Funds for the Central Universities (Grant No. 3102020gxb003), Natural Science Basic Research Plan in Shaanxi Province of China (Grant No. 2016JM6054), Programme of Introducing Talents of Discipline to Universities (111 Project), China (Grant No. B13044), Civil Aircraft Special Project (MJZ-2017-G73), and Seed Foundation of Innovation and Creation for Graduate Students in the Northwestern Polytechnical University (ZZ2018084).

Author information

Authors and Affiliations

  1. Laboratory of Cyber-Physical Interaction, College of Mechanical and Electrical Engineering, Northwestern Polytechnical University, Xi’an, 710072, China

    Zhuo Wang, Xiaoliang Bai, Shusheng Zhang, Shu Han, Xiangyu Zhang & Yuxiang Yan

  2. Cyber-Reality Innovation Centre (China), Nanjing, 211200, China

    Zhuo Wang, Xiaoliang Bai, Shu Han, Xiangyu Zhang & Yuxiang Yan

  3. AVIC Xinxiang Aviation Industry (Group) Co., Ltd., Xinxiang, 453049, China

    Yang Wang

  4. The London School of Economics and Political Science, London, WC2A 2AE, UK

    Zhishuo Xiong

Authors
  1. Zhuo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoliang Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shusheng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shu Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiangyu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yuxiang Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Zhishuo Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Xiaoliang Bai or Shusheng Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

(MP4 81,799 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Bai, X., Zhang, S. et al. User-oriented AR assembly guideline: a new classification method of assembly instruction for user cognition. Int J Adv Manuf Technol 112, 41–59 (2021). https://doi.org/10.1007/s00170-020-06291-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-020-06291-w

Keywords

Search

Navigation