Skip to main content
SpringerLink
Log in

A context-aware augmented reality system to assist the maintenance operators

  • Original Paper
  • Published:
International Journal on Interactive Design and Manufacturing (IJIDeM) Aims and scope Submit manuscript

Abstract

The increasing complexity and the technology advancement of equipment pose challenges to the maintenance operators nowadays. In this paper, a context-aware augmented reality (AR) system is proposed to assist the operators in routine and ad hoc maintenance tasks by providing them with context relevant information. This system integrates AR technology and context-awareness. It is able to analyze the contexts of the maintenance tasks and provide relevant and useful information to the operators through rendering the information on the real equipment and environment. In addition, the authoring of context-aware AR contents, which is essential for developing such applications, is explored. The properties of AR contents and their context-adaptation schemes are discussed, and a low-level authoring approach is proposed for constructing context-aware AR information. Finally, a case study is conducted to evaluate the system performance, and the results demonstrated that the system can adapt to dynamic maintenance situations to provide context relevant information to assist the maintenance operators.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Azuma, R.: A survey of augmented reality. Presence Teleoperators Virtual Environ. 6(4), 355–385 (1997)

    Google Scholar 

  2. Bottecchia, S., Cieutat, J.M., Merlo, C., Jessel, J.P.: A new AR interaction paradigm for collaborative teleassistance system: the POA. Int. J. Interact. Design Manuf. 3(1), 35–40 (2009)

    Article  Google Scholar 

  3. Morkos, B., Taiber, J., Summers, J., Mears, L., Fadel, G., Rilka, T.: Mobile devices within manufacturing environments: a BMW applicability study. Int. J. Interact. Design Manuf. 6(2), 101–111 (2012)

    Article  Google Scholar 

  4. Dey, A.K.: Understanding and using context. Pers. Ubiquitous Comput. 5(1), 4–7 (2001)

    Article  Google Scholar 

  5. Byun, H.E., Cheverst, K.: Utilizing context history to provide dynamic adaptations. Appl. Artif. Intell. 18(6), 533–548 (2004)

    Article  Google Scholar 

  6. Zhang, J.X., Sheng, Y.H., Hao, W., Wang, P.P., Tian, P., Miao, K., Pickering, C.K.: A context-aware framework supporting complex ubiquitous scenarios with augmented reality enabled. In: Proceedings of 5th International Conference on Pervasive Computing and Applications, Maribor, pp. 69–74 (2010)

  7. Lewandowski, J., Arochena, H.E., Naguib, R.N.G., Chao, K.M.: A portable framework design to support user context aware augmented reality applications. In: Proceedings of 3rd International Conference on Games and Virtual Worlds for Serious Applications, Athens, pp. 144–147 (2011)

  8. Shin, C., Kim, H., Kang, C., Jang, Y., Choi, A., Woo, W.: Unified context-aware augmented reality application framework for user-driven tour guides. In: Proceedings of 2010 International Symposium on Ubiquitous Virtual Reality, Gwangju, pp. 52–55 (2010)

  9. Feiner, S., Macintyre, B., Seligmann, D.: Knowledge-based augmented reality. Commun. ACM 36, 53–62 (1993)

    Article  Google Scholar 

  10. Friedrich, W.: ARVIKA-augmented reality for development. Production and Service, International Symposium on Mixed and Augmented Reality, Darmstadt, , pp. 3–4 (2002)

  11. Schwald, B., et al.: STARMATE: using augmented reality technology for computer guided maintenance of complex mechanical elements. In: Proceedings of eBusiness and eWork Conference (e2001), Venice, pp 17–19 (2001)

  12. Savioja, P., Järvinen, P., Karhela, T., Siltanen, P., Woodward, C.: Developing a mobile, service-based augmented reality tool for modern maintenance work. In: Proceedings of the 2nd International Conference on Virtual Reality, Beijing, pp. 554–563 (2007)

  13. ARTESAS: Advanced augmented reality technologies for industrial service applications. http://www.artesas.de (2012)

  14. Harmo, P., Halme, A., Virekoski, P., Halinen, M., Pitkänen, H.: Etälä—virtual reality assisted telepresence system for remote maintenance. In: Proceedings of the 1st IFAC Conference on Mechatronic Systems, Darmstadt, pp. 1075–1080 (2000)

  15. Didier, J.Y., Roussel, D.: AMRA: augmented reality assistance in train maintenance tasks. In: ISMAR’05: Workshop on Industrial Augmented Reality, pp. xvii–xviii (2005)

  16. Neubert, J., Pretlove, J., Drummond, T.: Rapidly constructed appearance models for tracking in augmented reality applications. Mach. Vis. Appl. 23(5), 843–856 (2012)

    Article  Google Scholar 

  17. Alvarez, H., Aguinaga, I., Borro, D.: Providing guidance for maintenance operations using automatic markerless augmented reality system. IEEE International Symposium on Mixed and Augmented Reality, Basel, pp. 181–90 (2011)

  18. Caponio, A., Hincapie, M., Gonzalez Mendivil, E.: lMAR: highly parallel architecture for markerless augmented reality in aircraft maintenance. International Conference on Virtual and Mixed Reality, Orlando, pp. 20–29 (2011)

  19. Yan, W., Ishii, H., Shimoda, H., Izumi, M.: A feasible tracking method of augmented reality for supporting fieldwork of nuclear power plant, Lecture Notes in Computer Science, vol. 5622. LNCS, pp. 639–646 (2009)

  20. Klein, G., Murray, D.: Parallel tracking and mapping for small ar workspaces. International Symposium on Mixed and Augmented Reality, Nara (2007)

  21. Goose, S., Sudarsky, S., Zhang, X., Navab, N.: Speech-enabled augmented reality supporting mobile industrial maintenance. IEEE Pervasive Comput. 2(1), 65–70 (2003)

    Article  Google Scholar 

  22. Park, H.M., Lee, S.H., Choi, J.S.: Wearable augmented reality system using gaze interaction. 7th IEEE International Symposium on Mixed and Augmented Reality, Cambridge, pp. 175–176 (2008)

  23. Yuan, M.L., Ong, S.K., Nee, A.Y.C.: The Virtual Interaction Panel: an easy control tool in augmented reality systems. Comput. Anim. Virtual Worlds J. (Special Issue: The Very Best Papers from CASA 2004 15(3–4):425–432 (2004)

  24. Henderson, S.J., Feiner, S.: Opportunistic tangible user interfaces for augmented reality. IEEE Trans. Vis. Comput. Graphics 16(1), 4–16 (2010)

    Article  Google Scholar 

  25. Jacob, R.J.K.: Eye-movement-based human-computer interaction techniques: toward non-command interface, chap. 6. Advances in Human–Computer Interaction, vol. 4. Ablex Publishing Corporation, Norwood, pp. 151–190 (1993)

  26. Sakata, N., Kurata, T., Kuzuoka, H.: Visual assist with a laser pointer and wearable display for remote collaboration. Trans. Virtual Real. Soc. Japan 11(4), 561–568 (2006)

    Google Scholar 

  27. Ong, S.K., Yuan, M.L., Nee, A.Y.C.: Augmented reality applications in manufacturing: a survey. Int. J. Prod. Res. 46(10), 2707–2742 (2008)

    Article  MATH  Google Scholar 

  28. HITLabNZ ARToolkit: http://www.artoolkit.sourceforge.net/ (2005)

  29. Wang, X.H., Zhang, D., Gu, T., Pung, H.H.: Ontology-based context modeling and reasoning using OWL. 2nd IEEE Annual conference on Pervasive Computing and Communications Workshops, Singapore, pp. 18–22 (2004)

  30. SWRL: A semantic web rule language combining OWL and RuleML. http://www.w3.org/Submission/SWRL/ (retrieved 28 April 2012)

  31. Pellet: OWL 2 Reasoner for Java. http://clarkparsia.com/pellet (retrieved 28 April 2012)

  32. Shen, Y., Ong, S.K., Nee, A.Y.C.: Vision-based hand interaction in augmented reality environment. Int. J. Hum. Comput. Interact. 27(6), 523–544 (2011)

    Article  Google Scholar 

  33. The OWL API. http://owlapi.sourceforge.net/ (retrieved 28 April 2012)

Download references

Author information

Authors and Affiliations

  1. NUS Graduate School for Integrative Sciences and Engineering, National University of Singapore, 28 Medical Drive, Singapore , 117456, Singapore

    J. Zhu, S. K. Ong & A. Y. C. Nee

  2. Mechanical Engineering Department, National University of Singapore, 9 Engineering Drive 1, Singapore , 117576, Singapore

    S. K. Ong & A. Y. C. Nee

Authors
  1. J. Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. K. Ong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. Y. C. Nee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. K. Ong.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhu, J., Ong, S.K. & Nee, A.Y.C. A context-aware augmented reality system to assist the maintenance operators. Int J Interact Des Manuf 8, 293–304 (2014). https://doi.org/10.1007/s12008-013-0199-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12008-013-0199-7

Keywords

Search

Navigation