Advertisement

A comprehensive survey of AR/MR-based co-design in manufacturing

  • Peng Wang
  • Shusheng ZhangEmail author
  • Mark Billinghurst
  • Xiaoliang BaiEmail author
  • Weiping He
  • Shuxia Wang
  • Mengmeng Sun
  • Xu Zhang
Original Article

Abstract

For more than 2 decades, Augmented Reality (AR)/Mixed Reality (MR) has received an increasing amount of attention by researchers and practitioners in the manufacturing community, because it has applications in many fields, such as product design, training, maintenance, assembly, and other manufacturing operations. However, to the best of our knowledge, there has been no comprehensive review of AR-based co-design in manufacturing. This paper presents a comprehensive survey of existing research, projects, and technical characteristics between 1990 and 2017 in the domain of co-design based on AR technology. Among these papers, more than 90% of them were published between 2000 and 2017, and these recent relevant works are discussed at length. The paper provides a comprehensive academic roadmap and useful insight into the state-of-the-art of AR-based co-design systems and developments in manufacturing for future researchers all over the world. This work will be useful to researchers who plan to utilize AR as a tool for design research.

Keywords

Augmented reality (AR) AR-based co-design Collaborative work Virtual reality (VR) CAD 

Notes

Acknowledgements

This research was sponsored by the civil aircraft special project (MJZ-2017-G73). Besides, we would like to appreciate the anonymous reviewers for their constructive suggestions for enhancing this paper. Specifically, Mark Billinghurst has checked the English of an early version, and his constructive comments are gratefully acknowledged which have helped the author to improve the paper. Moreover, we thank Dechuan Han and Mengmeng Sun for filtering and downloading papers. We also thank other schoolfellows, including Jinping Chen, Qian Cheng, Li Zhang, Yongxing Chen, Zhuang Chang, Yue Wang, and Xiaokun Zhang for their invaluable discussion, criticism, and advice for further improving this paper.

References

  1. 1.
    Chandrasegaran SK, Ramani K, Sriram RD, Horváth I, Bernard A, Harik RF et al (2013) The evolution, challenges, and future of knowledge representation in product design systems. Comput Aided Des 45(2):204–228CrossRefGoogle Scholar
  2. 2.
    Wang X, Dunston PS (2011) Comparative effectiveness of mixed reality-based virtual environments in collaborative design. IEEE Trans Syst Man Cybern Part C (Appl Rev) 41(3):284–296CrossRefGoogle Scholar
  3. 3.
    Gu N, Mi JK, Maher ML (2011) Technological advancements in synchronous collaboration: the effect of 3d virtual worlds and tangible user interfaces on architectural design. Autom Constr 20(3):270–278CrossRefGoogle Scholar
  4. 4.
    Karakaya AF, Demirkan H (2015) Collaborative digital environments to enhance the creativity of designers. Comput Hum Behav 42:176–186CrossRefGoogle Scholar
  5. 5.
    Mitchell V, Ross T, May A, Sims R, Parker C (2016) Empirical investigation of the impact of using co-design methods when generating proposals for sustainable travel solutions. Codesign 12(4):205–220CrossRefGoogle Scholar
  6. 6.
    Tseng MM, Du X (1998) Design by customers for mass customization products. CIRP Ann Manuf Technol 47(1):103–106CrossRefGoogle Scholar
  7. 7.
    Porter ME, Heppelmann JE (2017) Why every organization needs an augmented reality strategy. Harv Bus Rev 95(6):46–57Google Scholar
  8. 8.
    Mohr P, Mandl D, Tatzgern M, Veas E, Schmalstieg D, Kalkofen D. (2017) Retargeting video tutorials showing tools with surface contact to augmented reality. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, pp 6547–6558Google Scholar
  9. 9.
    Lukosch S, Billinghurst M, Alem L, Kiyokawa K (2015) Collaboration in Augmented Reality. Comput Support Cooperative Work 24:515CrossRefGoogle Scholar
  10. 10.
    See ZS, Billinghurst M, Rengganaten V, Soo S (2016) Medical learning murmurs simulation with mobile audible augmented reality. In: SIGGRAPH ASIA 2016 mobile graphics and interactive applications, ACM, p. 4Google Scholar
  11. 11.
    Wang X, Ong SK, Nee AYC (2016) A comprehensive survey of augmented reality assembly research. Adv Manuf 4(1):1–22CrossRefGoogle Scholar
  12. 12.
    Henderson S, Feiner S (2011) Exploring the benefits of augmented reality documentation for maintenance and repair. IEEE Trans Visual Comput Graph 17(10):1355CrossRefGoogle Scholar
  13. 13.
    Shen Y, Ong SK, Nee AYC (2010) Augmented reality for collaborative product design and development. Des Stud 31(2):118–145CrossRefGoogle Scholar
  14. 14.
    Ong SK, Shen Y (2009) A mixed reality environment for collaborative product design and development. CIRP Ann 58(1):139–142CrossRefGoogle Scholar
  15. 15.
    Nee AY, Ong SK, Chryssolouris G, Mourtzis D (2012) Augmented reality applications in design and manufacturing. CIRP Ann Manuf Technol 61(2):657–679CrossRefGoogle Scholar
  16. 16.
    Ong SK, Yuan ML, Nee AYC (2008) Augmented reality applications in manufacturing: a survey. Int J Prod Res 46(10):2707–2742zbMATHCrossRefGoogle Scholar
  17. 17.
    Huang TC, Chen CC, Chou YW (2016) Animating eco-education: to see, feel, and discover in an augmented reality-based experiential learning environment. Comput Educ 96:72–82CrossRefGoogle Scholar
  18. 18.
    Wu HK, Lee WY, Chang HY, Liang JC (2013) Current status, opportunities and challenges of augmented reality in education. Comput Educ 62:41–49CrossRefGoogle Scholar
  19. 19.
    Arroyave-Tobón S, Osorio-Gómez G, Cardona-McCormick JF (2015) Air-modelling: a tool for gesture-based solid modelling in context during early design stages in AR environments. Comput Ind 66:73–81CrossRefGoogle Scholar
  20. 20.
    Millette A, McGuffin MJ (2016) DualCAD: integrating augmented reality with a desktop GUI and smartphone interaction. In: Mixed and augmented reality (ISMAR-Adjunct), 2016 IEEE International Symposium, IEEE, pp 21–26Google Scholar
  21. 21.
    Uva AE, Cristiano S, Fiorentino M, Monno G (2010) Distributed design review using tangible augmented technical drawings. Comput Aided Des 42(5):364–372CrossRefGoogle Scholar
  22. 22.
    Januszka M, Moczulski W (2011) Augmented reality system for aiding engineering design process of machinery systems. J Syst Sci Syst Eng 20(3):294CrossRefGoogle Scholar
  23. 23.
    Wang X, Love PE, Kim MJ, Wang W (2014) Mutual awareness in collaborative design: an augmented reality integrated telepresence system. Comput Ind 65(2):314–324CrossRefGoogle Scholar
  24. 24.
    Chen Y, Feng P, Lin Z (2005) A genetics-based approach for the principle conceptual design of mechanical products. Int J Adv Manuf Technol 27(3–4):225–233CrossRefGoogle Scholar
  25. 25.
    Wang L, Shen W, Xie H, Neelamkavil J, Pardasani A (2002) Collaborative conceptual design—state of the art and future trends. Comput Aided Des 34(13):981–996CrossRefGoogle Scholar
  26. 26.
    Li WD, Ong SK, Fuh JYH, Wong YS, Lu YQ, Nee AYC (2004) Feature-based design in a distributed and collaborative environment. Comput Aided Des 36(9):775–797CrossRefGoogle Scholar
  27. 27.
    Chu CH, Cheng CY, Wu CW (2006) Applications of the web-based collaborative visualization in distributed product development. Comput Ind 57(3):272–282CrossRefGoogle Scholar
  28. 28.
    Yin X, Wu Y, Liu Q, Wang S (2015) A web-based, collaborative modeling, simulation, and parallel computing environment for electromechanical systems. Adv Mech Eng 7(3):1–6CrossRefGoogle Scholar
  29. 29.
    Qiang L, Zhang YF, Nee AYC (2001) A distributive and collaborative concurrent product design system through the www/internet. Int J Adv Manuf Technol 17(5):315–322CrossRefGoogle Scholar
  30. 30.
    Fan H, Liu Y, Hu B, Ye X (2016) Multidomain model integration for online collaborative system design and detailed design of complex mechatronic systems. IEEE Trans Autom Sci Eng 13(2):709–728CrossRefGoogle Scholar
  31. 31.
    Wang K, Takahashi A (2012) Semantic web based innovative design knowledge modeling for collaborative design. Expert Syst Appl 39(5):5616–5624CrossRefGoogle Scholar
  32. 32.
    Monostori L, Váncza J, Kumara SRT (2006) Agent-based systems for manufacturing. CIRP Ann Manuf Technol 55(2):697–720CrossRefGoogle Scholar
  33. 33.
    Wang JX, Tang MX, Song LN, Jiang SQ (2009) Design and implementation of an agent-based collaborative product design system. Comput Ind 60(7):520–535CrossRefGoogle Scholar
  34. 34.
    Shen W, Maturana F, Norrie DH (2000) Metamorph ii: an agent-based architecture for distributed intelligent design and manufacturing. J Intell Manuf 11(3):237–251zbMATHCrossRefGoogle Scholar
  35. 35.
    Chen Y, Liu ZL, Xie YB (2014) A multi-agent-based approach for conceptual design synthesis of multi-disciplinary systems. Int J Prod Res 52(6):1681–1694CrossRefGoogle Scholar
  36. 36.
    Shen W, Norrie DH, Barthés JPA (2000) Multi-agent systems for concurrent intelligent design and manufacturing, vol 47, no 11. CRC Press, Boca Raton, pp 1253–1254CrossRefGoogle Scholar
  37. 37.
    Wu D, Rosen DW, Wang L, Schaefer D (2015) Cloud-based design and manufacturing: a new paradigm in digital manufacturing and design innovation. Comput Aided Des 59:1CrossRefGoogle Scholar
  38. 38.
    Schaefer D (2014) Cloud-based design and manufacturing (CBDM). Springer, BerlinCrossRefGoogle Scholar
  39. 39.
    Li WD, Lu WF, Fuh JYH, Wong YS (2005) Collaborative computer-aided design-research and development status. Comput Aided Des 37(9):931–940CrossRefGoogle Scholar
  40. 40.
    Fuh JYH, Li WD (2005) Advances in collaborative cad: the-state-of-the-art. Comput Aided Des 37(5):571–581CrossRefGoogle Scholar
  41. 41.
    Shen W, Wang L (2003) Web-based and agent-based approaches for collaborative product design: an overview. Int J Comput Appl Technol 16(2–3):103–112CrossRefGoogle Scholar
  42. 42.
    French D, Jensen G, Walker SS, Madsen P, Red E (2013) Emerging design methods and tools in collaborative product development. J Comput Inf Sci Eng 13(3):031001CrossRefGoogle Scholar
  43. 43.
    Huang GQ (2002) Web-based support for collaborative product design review. Comput Ind 48(1):71–88CrossRefGoogle Scholar
  44. 44.
    Ren L, Zhang L, Tao F, Zhang XL, Zhang Y, Zhang Y (2012) A methodology towards virtualisation-based high performance simulation platform supporting multidisciplinary design of complex products. Enterp Inf Syst 6(3):267–290CrossRefGoogle Scholar
  45. 45.
    Gao S, Wan H, Peng Q (2000) An approach to solid modeling in a semi-immersive virtual environment. Comput Graph 24(2):191–202CrossRefGoogle Scholar
  46. 46.
    Kiyokawa K, Billinghurst M, Hayes SE, Gupta A, Sannohe Y, Kato H (2002) Communication behaviors of co-located users in collaborative AR interfaces. In: mixed and augmented reality, 2002. ISMAR 2002. Proceedings. International symposium on IEEE, pp 139–148Google Scholar
  47. 47.
    Rekimoto J (1996) Transvision: a hand-held augmented reality system for collaborative design. Proc Virtual Syst Multimed 96:18–20Google Scholar
  48. 48.
    Masai K, Kunze K, Billinghurst M (2016) Empathy glasses. In: Proceedings of the 2016 CHI Conference Extended abstracts on human factors in computing systems. ACM, pp 1257–1263Google Scholar
  49. 49.
    Kouprie M, Visser FS (2009) A framework for empathy in design: stepping into and out of the user’s life. J Eng Des 20(5):437–448CrossRefGoogle Scholar
  50. 50.
    Yuan S, Dong H (2014) Empathy building through co-design. In: International conference on universal access in human-computer interaction. Springer, Cham, pp 85–91Google Scholar
  51. 51.
    Schmalstieg D, Fuhrmann A, Gervautz M (1998) “studierstube”: an environment for collaboration in augmented reality. Virtual Real 3(1):37–48CrossRefGoogle Scholar
  52. 52.
    Blanding R, Turkiyyah GECAD (2007) A prototype screen-based VR solid modeling environment incorporating tangible deformable models. Comput Aided Des Appl 4(5):595–605CrossRefGoogle Scholar
  53. 53.
    Jezernik A, Hren G (2003) A solution to integrate computer-aided design (CAD) and virtual reality (VR) databases in design and manufacturing processes. Int J Adv Manuf Technol 22(11–12):768–774CrossRefGoogle Scholar
  54. 54.
    Ma W, Zhong Y, Tso SK, Zhou T (2004) A hierarchically structured and constraint-based data model for intuitive and precise solid modeling in a virtual reality environment. Comput Aided Des 36(10):903–928CrossRefGoogle Scholar
  55. 55.
    Zhong Y, Ma W, Shirinzadeh B (2005) A methodology for solid modelling in a virtual reality environment. Robot Comput Integr Manuf 21(6):528–549CrossRefGoogle Scholar
  56. 56.
    Wang QH, Li JR, Wu BL, Zhang XM (2010) Live parametric design modifications in cad-linked virtual environment. Int J Adv Manuf Technol 50(9–12):859–869CrossRefGoogle Scholar
  57. 57.
    Kan HY, Duffy VG, Su CJ (2001) An internet virtual reality collaborative environment for effective product design. Comput Ind 45(2):197–213CrossRefGoogle Scholar
  58. 58.
    Mahdjoub M, Monticolo D, Gomes S, Sagot JC (2010) A collaborative design for usability approach supported by virtual reality and a multi-agent system embedded in a plm environment. Comput Aided Des 42(5):402–413CrossRefGoogle Scholar
  59. 59.
    Ishii H (1997) Tangible bits: towards seamless interfaces between people, bits and atoms. In: ACM Sigchi conference on human factors in computing systems. ACM, pp 234–241Google Scholar
  60. 60.
    Dunston P, Wang X, Billinghurst M, Hampson B (2003) Mixed reality benefits for design perception. In: 19th International Symposium on automation and robotics construction (ISARC 2002), pp 191–196Google Scholar
  61. 61.
    Zhou F, Duh BL, Billinghurst M (2008) Trends in augmented reality tracking, interaction and display: a review of ten years of ISMAR. In: IEEE/ACM International Symposium on mixed and augmented reality. IEEE, pp 193–202Google Scholar
  62. 62.
    Piya C, Vinayak, Chandrasegaran S, Elmqvist N, Ramani K (2017) Co-3Deator: ateam-first collaborative 3D design ideation tool. In: Chi Conference on human factors in computing systems. ACM, pp 6581–6592Google Scholar
  63. 63.
    Kiyokawa K, Iwasa H, Takemura H, Yokoya N (1998) Collaborative immersive workspace through. In: Proceedings of SPIE International Symposium on intelligent systems and advanced manufacturing, Vol. 3517, pp 2–13Google Scholar
  64. 64.
    Billinghurst M, Kato H (2002) Collaborative augmented reality. Commun ACM 45(7):64–70CrossRefGoogle Scholar
  65. 65.
    Tait M, Billinghurst M (2015) The effect of view independence in a collaborative AR system. Comput Support Coop Work 24(6):563–589CrossRefGoogle Scholar
  66. 66.
    Tan CSS, Luyten K, Jan VDB, Schöning J, Coninx K (2014) The role of physiological cues during remote collaboration. Presence 23(1):90–107CrossRefGoogle Scholar
  67. 67.
    Xolocotzin Eligio U, Ainsworth SE, Crook CK (2012) Emotion understanding and performance during computer-supported collaboration. Comput Hum Behav 28:2046CrossRefGoogle Scholar
  68. 68.
    Lu CY, Shpitalni M, Gadh R (1999) Virtual and augmented reality technologies for product realization. CIRP Ann Manuf Technol 48(2):471–495CrossRefGoogle Scholar
  69. 69.
    Wang X, Gu N, Marchant D (2008) An empirical study on designers’ perceptions of augmented reality within an architectural firm. Electron J Inf Technol Constr 13:536–552Google Scholar
  70. 70.
    Li JR, Tang C, Wang QH (2015) A proxy approach to integrate heterogeneous CAD resources for distributed collaborative design. Int J Comput Integr Manuf 28(6):593–606CrossRefGoogle Scholar
  71. 71.
    Shen Y, Ong SK, Nee AYC (2006) A Framework for Multiple-View Product Representation Using Augmented Reality. In: International Conference on cyberworlds. IEEE Computer Society, pp 157–164Google Scholar
  72. 72.
    Shen Y, Ong SK, Nee AYC (2008) Product information visualization and augmentation in collaborative design. Comput Aided Des 40(9):963–974CrossRefGoogle Scholar
  73. 73.
    Xia H, Herscher S, Perlin K, Wigdor D (2018) Spacetime: enabling fluid individual and collaborative editing in virtual reality. In: The 31st Annual ACM Symposium on user interface software and technology. ACM, pp 853–866Google Scholar
  74. 74.
    Vinayak, Ramanujan D, Piya C, Ramani K (2016) MobiSweep: exploring spatial design ideation using a smartphone as a hand-held reference plane. Proceedings of the TEI'16: tenth international conference on tangible, embedded, and embodied interaction. ACM, pp 12–20Google Scholar
  75. 75.
    Huo K, Vinayak, Ramani K (2017) Window-Shaping: 3D design ideation by creating on, borrowing from, and looking at the physical world. In: Eleventh International Conference on Tangible, 2017Google Scholar
  76. 76.
    Sang HY, Ke H, Ramani K (2016) TMotion: embedded 3D mobile input using magnetic sensing technique. In: Tei 16: Tenth International Conference on Tangible, 2016Google Scholar
  77. 77.
    Zhao Z, Badam S K, Chandrasegaran S, Park D G, Elmqvist N, Kisselburgh L, Ramani K (2014) skWiki:a multimedia sketching system for collaborative creativity. In: Acm Conference on Human Factors in Computing Systems, 2014Google Scholar
  78. 78.
    Dey A, Billinghurst M, Lindeman RW, Ii JES (2017) A systematic review of usability studies in augmented reality between 2005 and 2014. In: IEEE International Symposium on Mixed and Augmented Reality, IEEE, pp 49–50Google Scholar
  79. 79.
    Ahlers KH, Kramer A, Breen DE, Chevalier PY, Crampton C, Rose E, Tuceryan M, Whitaker RT, Greer D (1995) Distributed augmented reality for collaborative design applications. Comput Graph Forum 14(3):3–14CrossRefGoogle Scholar
  80. 80.
    Kiyokawa K, Takemura H, Katayama Y, Iwasa H, Yokoya N (1996) Vlego: a simple two-handed modeling environment based on toy blocks. Proceedings of the ACM symposium on virtual reality software and technology. ACM, pp 27–34Google Scholar
  81. 81.
    Kiyokawa K, Takemura H, Yokoya N (1999) Seamlessdesign: a face-to-face collaborative virtual/augmented environment for rapid prototyping of geometrically constrained 3-d objects. In: Multimedia computing and systems, 1999. IEEE International Conference on (Vol. 2, pp 447–453), IEEEGoogle Scholar
  82. 82.
    Schumann H, Burtescu S, Siering F (1998) Applying augmented reality techniques in the field of interactive collaborative design. 3D structure from multiple images of large-scale environments. Springer, Berlin, pp 290–303CrossRefGoogle Scholar
  83. 83.
    Schmalstieg D, Fuhrmann A, Hesina G, Gervautz M, Purgathofer W (2002) The studierstube augmented reality project. Presence Teleoperators Virtual Environ 11(1):33–54CrossRefGoogle Scholar
  84. 84.
    Grønbæk K, Mogensen P, Rbk P (2001) Interactive room support for complex and distributed design projects. INTERACT, pp 407–414Google Scholar
  85. 85.
    Gupta K, Lee GA, Billinghurst M (2016) Do you see what i see? the effect of gaze tracking on task space remote collaboration. IEEE Trans Visual Comput Graphics 22(11):2413–2422CrossRefGoogle Scholar
  86. 86.
    Piumsomboon T, Lee Y, Lee GA, Dey A, Billinghurst M (2017) Empathic mixed reality: sharing what you feel and interacting with what you see. In: International Symposium on Ubiquitous Virtual Reality, IEEE, pp 38–41Google Scholar
  87. 87.
    Lee Y, Masai K, Kai K, Sugimoto M, Billinghurst M (2017) A remote collaboration system with empathy glasses. In: IEEE International Symposium on mixed and augmented reality, IEEE, pp 342–343Google Scholar
  88. 88.
    Gao L, Bai H, Lee G, Billinghurst M (2016) An oriented point-cloud view for MR remote collaboration. In: SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications. ACM, p 8Google Scholar
  89. 89.
    Regenbrecht HT, Wagner M, Baratoff G (2002) Magicmeeting: a collaborative tangible augmented reality system. Virtual Real 6(3):151–166CrossRefGoogle Scholar
  90. 90.
    Wagner MT, Wagner MT (2002) Interaction in a collaborative augmented reality environment. In: CHI ‘02 Extended Abstracts on human factors in computing systems, ACM, pp 504–505Google Scholar
  91. 91.
    Klinker G, Dutoit AH, Bauer M, Bayer J, Novak V, Matzke D (2002) Fata Morgana “A Presentation System for Product Design”. In: International Symposium on mixed and augmented reality, IEEE Computer Society, pp 76Google Scholar
  92. 92.
    Liu P, Georganas ND, Boulanger P (2002) Designing real-time vision based augmented reality environments for 3D collaborative applications. Des Real, IEEEGoogle Scholar
  93. 93.
    Verlinden JC (2003) Development of a flexible augmented prototyping system. UNION Agency–Science Press, pp 1–8Google Scholar
  94. 94.
    Hirakawa M, Koike S (2004) A collaborative augmented reality system using transparent display. In: IEEE Sixth International Symposium on multimedia software engineering, IEEE Computer Society, pp 410–416Google Scholar
  95. 95.
    Penn A, Mottram C, Schieck AFG, Wittkämper M, Störring M, Romell O, Strothmann A, Aish F (2004) Recent advances in design and decision support systems in architecture and urban planning. Springer, Dordrecht, pp 213–231CrossRefGoogle Scholar
  96. 96.
    Broll W, Lindt I, Ohlenburg J, Wittkamper M, Yuan C, Novotny T, Schieck AFG, Mottram C, Strothmann A (2004) Arthur: a collaborative augmented environment for architectural design and urban planning. J Virtual Real Broadcast 1(1):1–10Google Scholar
  97. 97.
    Yuan ML, Ong SK, Nee AYC (2004) The virtual interaction panel: an easy control tool in augmented reality systems. Comput Anim Virtual Worlds 15(3–4):425–432CrossRefGoogle Scholar
  98. 98.
    Shen Y, Ong SK, Nee AYC (2008) Collaborative design in 3D space. In: ACM SIGGRAPH International Conference on virtual-reality continuum and its applications in industry, ACM, p 29Google Scholar
  99. 99.
    Maher ML, Mi JK (2005) Do tangible user interfaces impact spatial cognition in collaborative design?. In: International Conference on cooperative design, visualization, and engineering (vol 3675), Springer, pp 30–41Google Scholar
  100. 100.
    Kim MJ, Maher ML (2005) Comparison of designers using a tangible user interface and a graphical user interface and the impact on spatial cognition. In: International Workshop on human behaviour in designing, pp 81–94Google Scholar
  101. 101.
    Lee W, Park J (2006) Augmented foam: touchable and graspable augmented reality for product design simulation. Bull Jpn Soc Sci Des 52(6):17–26Google Scholar
  102. 102.
    Sidharta R, Oliver J, Sannier A (2006) Augmented reality tangible interface for distributed design review. In: computer graphics, imaging and visualisation, 2006 International Conference on IEEE, pp 464–470Google Scholar
  103. 103.
    Sakong K, Nam TJ (2006) Supporting telepresence by visual and physical cues in distributed 3D collaborative design environments. In: CHI’06 extended abstracts on human factors in computing systems, ACM, pp 1283–1288Google Scholar
  104. 104.
    Wang X, Chen R (2009) An experimental study on collaborative effectiveness of augmented reality potentials in urban design. CoDesign 5(4):229–244CrossRefGoogle Scholar
  105. 105.
    Wang X, Dunston PS (2006) Potential of augmented reality as an assistant viewer for computer-aided drawing. J Comput Civil Eng 20(6):437–441CrossRefGoogle Scholar
  106. 106.
    Wang X, Dunston PS (2008) User perspectives on mixed reality tabletop visualization for face-to-face collaborative design review. Autom Constr 17(4):399–412CrossRefGoogle Scholar
  107. 107.
    Wang X (2007) Exploring an innovative collaborative design space through mixed reality boundaries. In: Computer supported cooperative work in design, 2007. CSCWD 2007. 11th International Conference on IEEE, pp 264–269Google Scholar
  108. 108.
    Seichter H (2007) Augmented reality and tangible interfaces in collaborative urban design. In: computer-aided architectural design futures (CAADFutures) 2007, Springer, Dordrecht, pp 3–16Google Scholar
  109. 109.
    Santos P, Stork A, Gierlinger T, Pagani A, Paloc C, Barandarian I, Jiménez JM (2007) Improve: an innovative application for collaborative mobile mixed reality design review. Int J Interactive Des Manuf 1(2):115–126CrossRefGoogle Scholar
  110. 110.
    Santos P, Stork A, Gierlinger T, Pagani A, Araújo B, Jota R, Conti G (2007) Improve: collaborative design review in mobile mixed reality. In: International Conference on Virtual Reality. Springer, Berlin, pp 543–553Google Scholar
  111. 111.
    Chastine J, Nagel K, Zhu Y, Hudachek-Buswell M (2008) Studies on the effectiveness of virtual pointers in collaborative augmented reality. In: 3D User Interfaces, 2008. 3DUI 2008. IEEE Symposium on IEEE, pp 117–124Google Scholar
  112. 112.
    Lee JY, Rhee GW, Park H (2009) AR/RP-based tangible interactions for collaborative design evaluation of digital products. Int J Adv Manuf Technol 45(7–8):649–665CrossRefGoogle Scholar
  113. 113.
    Hammad A, Wang H, Mudur SP (2009) Distributed augmented reality for visualizing collaborative construction tasks. J Comput Civil Eng 23(6):418–427CrossRefGoogle Scholar
  114. 114.
    Dolinsky M (2010) Interactive augmented reality system for product design review. IS&T/SPIE Electron Imaging 7525:75250H–75250HGoogle Scholar
  115. 115.
    Jota R, Araújo BRD, Bruno LC, Pereira JM, Jorge JA (2010) Immiview: a multi-user solution for design review in real-time. J Real Time Image Process 5(2):91–107CrossRefGoogle Scholar
  116. 116.
    Januszka M, Moczulski W (2010) Augmented reality for machinery systems design and development. New world situation: new directions in concurrent engineering. Springer, LondonGoogle Scholar
  117. 117.
    Januszka M, Moczulski W (2007) Machinery designing aided by augmented reality technology. Comput Assist Mech Eng Sci 14(14):621–630Google Scholar
  118. 118.
    Januszka M, Moczulski W (2006) Collaborative augmented reality in CAD design. Mach Dyn Probl 30(3):124–131Google Scholar
  119. 119.
    Moczulski W, Panfil W, Januszka M, Mikulski G (2007) Applications of augmented reality in machinery design, maintenance and diagnostics. Recent Advances in Mechatronics. Springer, Berlin, Heidelberg, pp 52–56Google Scholar
  120. 120.
    Thomas BH, Itzstein GSV, Vernik R, Porter S, Marner MR, Smith RT et al. (2011) Spatial augmented reality support for design of complex physical environments. In: IEEE International Conference on pervasive computing and communications workshops, IEEE, pp 588–593Google Scholar
  121. 121.
    Poppe E, Brown R, Johnson D, Recker J (2012) Preliminary evaluation of an augmented reality collaborative process modelling system. In: International Conference on cyberworlds. IEEE Computer Society, pp 77–84Google Scholar
  122. 122.
    Poppe E, Brown R, Johnson D, Recker J (2011) A prototype augmented reality collaborative process modelling tool. In: Demo Track of the Nineth Conference on Business Process Management 2011, Clermont-Ferrand, France, August. DBLPGoogle Scholar
  123. 123.
    Ko CH, Chang TC (2012) Evaluation and student perception of augmented reality-based design collaboration[J]. Management 6(6):6Google Scholar
  124. 124.
    Gauglitz S, Nuernberger B, Turk M (2014) World-stabilized annotations and virtual scene navigation for remote collaboration. In: Proceedings of the 27th annual ACM symposium on User interface software and technology. ACM, pp 449–459Google Scholar
  125. 125.
    Gauglitz S, Lee C, Turk M (2012) Integrating the physical environment into mobile remote collaboration. In: International Conference on human-computer interaction with mobile devices and services. ACM, pp 241–250Google Scholar
  126. 126.
    Sodhi RS, Jones BR, Forsyth D, Maciocci G, Maciocci G (2013) BeThere: 3D mobile collaboration with spatial input. In: Sigchi Conference on human factors in computing systems. ACM, pp 179–188Google Scholar
  127. 127.
    Gurevich P, Lanir J, Cohen B (2015) Design and implementation of teleadvisor: a projection-based augmented reality system for remote collaboration. Comput Support Coop Work 24(6):527–562CrossRefGoogle Scholar
  128. 128.
    Gurevich P, Lanir J, Cohen B, Ran S (2012) TeleAdvisor: a versatile augmented reality tool for remote assistance. In: Sigchi Conference on human factors in computing systems. ACM, pp 619–622Google Scholar
  129. 129.
    Gül LF, Halıcı M, Uzun C, Esengün M (2016) Understanding the impact of mobile augmented reality on co-design cognition and co-modelling. In: International Conference on cooperative design, visualization and engineering. Springer International Publishing, pp 362–370Google Scholar
  130. 130.
    Gül LF (2017) Studying gesture-based interaction on a mobile augmented reality application for co-design activity. J Multimodal User Interfaces 12(2):109–124CrossRefGoogle Scholar
  131. 131.
    Grandi JG (2017) Design of collaborative 3D user interfaces for virtual and augmented reality. In: virtual reality (VR), 2017 IEEE. IEEE, pp 419–420Google Scholar
  132. 132.
    Grasset R, Lamb P, Billinghurst M (2005) Evaluation of mixed-space collaboration. In: Proceedings of the 4th IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE Computer Society, pp 90–99Google Scholar
  133. 133.
    Ohshima T, Satoh K, Yamamoto H, Tamura H (1998) AR2 Hockey: a case study of collaborative augmented reality. In: Virtual reality annual international symposium, 1998. Proceedings., IEEE 1998. IEEE, pp 268–275Google Scholar
  134. 134.
    Wang Y, Zhang S, Yang S, He W, Bai X, Zeng Y (2017) A LINE-MOD-based markerless tracking approachfor AR applications. Int J Adv Manuf Technol 89(5–8):1699–1707CrossRefGoogle Scholar
  135. 135.
    Agrawala M, Beers AC, McDowall I, Fröhlich B, Bolas M, Hanrahan P (1997) The two-user Responsive Workbench: support for collaboration through individual views of a shared space. In: Proceedings of the 24th annual conference on Computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., pp 327–332Google Scholar
  136. 136.
    Yu D, Jin JS, Luo S, Lai W, Huang Q (2009) A useful visualization technique: a literature review for augmented reality and its application, limitation and future direction. Visual information communication. Springer, Boston, MA, pp 311–337Google Scholar
  137. 137.
    Papagiannakis G, Singh G, Magnenat-Thalmann N (2008) A survey of mobile and wireless technologies for augmented reality systems. Comput Anim Virtual Worlds 19(1):3–22CrossRefGoogle Scholar
  138. 138.
    Gutwin C, Greenberg S (1998) Effects of awareness support on groupware usability. In: Sigchi Conference on Human Factors in Computing Systems (vol 98, pp 511–518). ACM Press/Addison-Wesley Publishing CoGoogle Scholar
  139. 139.
    Szalavári Z, Gervautz M (2010) The personal interaction panel—a two-handed interface for augmented reality. Comput Graph Forum 16(3):C335–C346CrossRefGoogle Scholar
  140. 140.
    Budhiraja R, Lee GA, Billinghurst M (2013) Interaction techniques for HMD-HHD hybrid AR systems. IEEE International Symposium on mixed and augmented reality (vol 8783, pp 243–244). IEEEGoogle Scholar
  141. 141.
    Billinghurst M (2017) The coming age of empathic computing. http://www.slideshare.net/marknb00/the-coming-age-of-empathic-computing
  142. 142.
    Hancock JT, Gee K, Ciaccio K, Lin MH (2008) I’m sad you’re sad: emotional contagion in CMC. In: ACM Conference on computer supported cooperative work. ACM, pp 295–298Google Scholar
  143. 143.
    Barsade SG (2002) The ripple effect: emotional contagion and its influence on group behavior. Adm Sci Q 47(4):644–675CrossRefGoogle Scholar
  144. 144.
    Cheshin A, Rafaeli A, Bos N (2011) Anger and happiness in virtual teams: emotional influences of text and behavior on others’ affect in the absence of non-verbal cues. Organ Behav Hum Decis Process 116(1):2–16CrossRefGoogle Scholar
  145. 145.
    Höllerer T, Feiner S, Terauchi T, Rashid G, Hallaway D (1999) Exploring mars: developing indoor and outdoor user interfaces to a mobile augmented reality system. Comput Graph 23(6):779–785CrossRefGoogle Scholar
  146. 146.
    Billinghurst M, Thomas BH (2011) Mobile collaborative augmented reality. In: recent trends of mobile collaborative augmented reality systems. Springer, New York, pp 1–19Google Scholar
  147. 147.
    Jeon S, Hwang J, Kim GJ, Billinghurst M (2010) Interaction with large ubiquitous displays using camera-equipped mobile phones. Pers Ubiquitous Comput 14(2):83–94CrossRefGoogle Scholar
  148. 148.
    Gül LF, Halici SM (2016) Collaborative design with mobile augmented reality. In: 34th eCAADe Conference Proceedings. Oulu, Finland, pp 493–500Google Scholar
  149. 149.
    Billinghurst M, Grasset R, Looser J (2005) Designing augmented reality interfaces. ACM Siggraph Comput Graph 39(1):17–22CrossRefGoogle Scholar
  150. 150.
    Ishii H, Lakatos D, Bonanni L, Labrune JB (2012) Radical atoms: beyond tangible bits, toward transformable materials. Interactions 19(1):38–51CrossRefGoogle Scholar
  151. 151.
    Purdy TG, Choi YM (2014) Enhancing augmented reality for use in product design. In: CHI’14 Extended Abstracts on human factors in computing systems. ACM, pp 1303–1308Google Scholar
  152. 152.
    Wang X (2008) Exploring augmented reality benefits in collaborative conceptualization. In: Computer supported cooperative work in design, 2008. CSCWD 2008. 12th International Conference on IEEE, pp 699–704Google Scholar
  153. 153.
    Xing NL (2013) Augmented reality 3D design space. Doctoral dissertationGoogle Scholar
  154. 154.
    Ng LX, Ong SK, Nee AYC (2015) Conceptual design using functional 3D models in augmented reality. Int J Interact Des Manuf 9(2):115–133CrossRefGoogle Scholar
  155. 155.
    Safin S, Delfosse V, Leclercq P (2010) Mixed-reality prototypes to support early creative design. The Engineering of Mixed Reality Systems. Springer, London, pp 419–445Google Scholar
  156. 156.
    Fuge M, Yumer ME, Orbay G, Kara LB (2012) Conceptual design and modification of freeform surfaces using dual shape representations in augmented reality environments. Comput Aided Des 44(10):1020–1032CrossRefGoogle Scholar
  157. 157.
    Turk M (2014) Multimodal interaction: a review. Pattern Recogn Lett 36:189–195CrossRefGoogle Scholar
  158. 158.
    Deng S, Jiang N, Chang J, Guo S, Zhang JJ (2017) Understanding the impact of multimodal interaction using gaze informed mid-air gesture control in 3D virtual objects manipulation. Int J Hum Comput Stud 105:68–80CrossRefGoogle Scholar
  159. 159.
    Quek F, McNeill D, Bryll R, Duncan S, Ma XF, Kirbas C, Ansari R (2002) Multimodal human discourse: gesture and speech. ACM Trans Comput Hum Interact 9(3):171–193CrossRefGoogle Scholar
  160. 160.
    Shim J, Yang Y, Kang N, Seo J, Han TD (2016) Gesture-based interactive augmented reality content authoring system using HMD. Virtual Real 20(1):57–69CrossRefGoogle Scholar
  161. 161.
    Henry P, Krainin M, Herbst E, Ren X, Fox D (2010) RGB-D mapping: using depth cameras for dense 3D modeling of indoor environments. In: In the 12th International Symposium on Experimental Robotics (ISER)Google Scholar
  162. 162.
    Ismail AW, Billinghurst M, Sunar MS (2015) Vision-based technique and issues for multimodal interaction in augmented reality. In: Proceedings of the 8th International Symposium on Visual Information Communication and Interaction. ACM, pp 75–82Google Scholar
  163. 163.
    Lee JY, Park HM, Lee SH, Shin SH, Kim TE, Choi JS (2014) Design and implementation of an augmented reality system using gaze interaction. Multimed Tools Appl 68(2):265–280CrossRefGoogle Scholar
  164. 164.
    Krafka K, Khosla A, Kellnhofer P, Kannan H, Bhandarkar S, Matusik W, Torralba A (2016) Eye tracking for everyone. Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2176–2184Google Scholar
  165. 165.
    Chatterjee I, Xiao R, Harrison C (2015) Gaze + gesture: expressive, precise and targeted free-space interactions. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. ACM, pp 131–138Google Scholar
  166. 166.
    Song J, Cho S, Baek SY, Lee K, Bang H (2014) GaFinC: gaze and Finger Control interface for 3D model manipulation in CAD application. Comput Aided Des 46:239–245CrossRefGoogle Scholar
  167. 167.
    Wu D, Rosen DW, Schaefer D (2014) Cloud-based design and manufacturing: status and promise. Cloud-Based Design and Manufacturing (CBDM). Springer, Cham, pp 1–24Google Scholar
  168. 168.
    Wang X, Dunston PS (2013) Tangible mixed reality for remote design review: a study understanding user perception and acceptance. Vis Eng 1(1):1–15CrossRefGoogle Scholar
  169. 169.
    Huang JM, Ong SK, Nee AYC (2016) Visualization and interaction of finite element analysis in augmented reality. Comput Aided Des 84:1–14CrossRefGoogle Scholar
  170. 170.
    Huang JM, Ong SK, Nee AYC (2015) Real-time finite element structural analysis in augmented reality. Adv Eng Softw 87(2):43–56CrossRefGoogle Scholar
  171. 171.
    Burdea GC (1999) Invited review: the synergy between virtual reality and robotics. IEEE Trans Robot Autom 15(3):400–410CrossRefGoogle Scholar
  172. 172.
    Seth A, Vance JM, Oliver JH (2011) Virtual reality for assembly methods prototyping: a review. Virtual Real 15(1):5–20CrossRefGoogle Scholar
  173. 173.
    Bordegoni M, Ferrise F (2013) Designing interaction with consumer products in a multisensory virtual reality environment. Virtual Phys Prototyp 8(1):51–64CrossRefGoogle Scholar
  174. 174.
    Wang P, Zhang S, Bai X, Billinghurst M, He W, Sun M, Chen Y, Lv H, Ji H (2019) 2.5DHANDS: a gesture-based MR remote collaborative platform. Int J Adv Manuf Technol 2019:1–15Google Scholar
  175. 175.
    Wang P, Zhang S, Bai X, Billinghurst M, He W, Zhang L, Du J, Wang S (2018) Do you know what i mean? an mr-based collaborative platform. In: IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2018Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  • Peng Wang
    • 1
  • Shusheng Zhang
    • 1
    Email author
  • Mark Billinghurst
    • 1
    • 2
  • Xiaoliang Bai
    • 1
    Email author
  • Weiping He
    • 1
  • Shuxia Wang
    • 1
  • Mengmeng Sun
    • 1
  • Xu Zhang
    • 1
  1. 1.Key Laboratory of Contemporary Designing and Integrated Manufacturing Technology, Ministry of Education, Cyber-Physical Interaction LabNorthwestern Polytechnical UniversityXi’anChina
  2. 2.Empathic Computing LabUniversity of South AustraliaMawson LakesAustralia

Personalised recommendations