Skip to main content
SpringerLink
Log in

Machine Vision for Device Tracking in a Smart Manufacturing Environment Based on Augmented Reality

  • Original Paper
  • Published:
Augmented Human Research Aims and scope Submit manuscript

Abstract

In a controlled network environment, such as the smart indoor manufacturing environment, the device identification and detection of components is challenging without prior knowledge of the design and implementation process. Thus, the concept of device identification for diagnosis and equipment maintenance by means of markerless augmented reality (AR) merits investigation. AR, when coupled with machine vision, caters for obtaining real-time device information regarding the position and features of the robotic elements within indoor manufacturing plants. Thus, this article proposes an efficient machine vision model to detect and identify devices based within a manufacturing plant, with the aid of AR for extending the device operational details. This offers an alternative solution in the absence of user built-in maps for the calculation of device positions based on uncertainties of the exact locations. To achieve this, a two-part validation is conducted involving (1) device recognition based on position and (2) Data integration to A Supervisory Control and Data Acquisition (SCADA) model developed in National Instruments Labview. The findings demonstrate that the AR application can detect devices within the manufacturing plant without the need for alteration. The results also indicate that the application can be integrated into a SCADA model without the need to alter the application, provided that the array index is the same. Only when the array index differs are alterations necessary for utilising the AR application.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Antonijevic M, Sucic S, Keserica H (2018) Augmented reality applications for substation management by utilizing standards-compliant SCADA communication. 599(11):2

  2. Kukuni TG, Kotze B (2019) Industrial augmented reality (IAR) as an approach for device identification within a manufacturing plant for property alteration purpose. In: Visual 2019: the fourth international conference on applications and systems of visual paradigm, p 13

  3. Brunner KE, Perey C, Rauschnabel PA, Sage M (2020) Augmented reality and virual reality: changing realities in a dynamic world. In: Implementation of augmented reality in manufacturing: a case study exercise, pp 85–86

  4. Vidal-Balea A, Blanco-Novoa O, Fraga-Lamas P, Vilar-Montesinos M, Fernandez-Carames TM (2020) A collaborative augmented reality application for training and assistance during shipbuilding assembly process. 54(4):1–3

  5. Cognex, Introduction to machine vision. In: A guide to automating process and quality improvements, p 3

  6. Kukuni TG, Kotze B (2019) Industrial augmented reality (IAR) as an approach for device identification within a manufacturing plant for property alteration purpose, pp 13–18

  7. Van Krevelen DW, Poelman R, A survey of augmented reality technologies, applications and limitations. Int J Virtual Real, pp 1–20

  8. Nandini V, Vishal DR, Prakash AC, Aishwarya S (2016) A review on applications of machine vision systems in industries. Indian J Sci Technol 9(48):1

    Article  Google Scholar 

  9. Georgel PF (2011) Is there a reality in industrial augmented reality? In: Proceedings of the 10th IEEE international symposium on mixed reality and augmented reality, pp 201–210

  10. Navab N, Heining SM, Traub J (2009) Camera augmented mobile c-arm (camc): calibration accuracy study and clinical applications. IEEE Trans Med Imaging, 1412–1423

  11. Cheng Y, Jafari MA (2005) Vision-based Online process control in manufacturing applications. p 6

  12. Lee Y, Rhee S (2015) Efficient photo image retrieval system based on combination of smart sensing and virtual descriptor. Intell Auto Soft Comput, pp 39–50

  13. Etonam AK, Di Gravio G, Kuloba PW, Njiri JG (2019) Augmented reality (AR) application in manufacturing encompassing quality control and maintenance. Int J Eng Adv Technol 9(1):2019

  14. Geiger P, Pryss R, Schickler M, Reichert M (2013) Engineering an advanced location-based augmented reality engine for smart mobile devices. University of Ulm

  15. Pryss R, Geiger P, Schickler M, Schobel J, Reitchert M (2017) The AREA framework for location-based smart mobile augmented reality applications. Int J Ubiquious Syst Pervasive Netw 9(1):13–21

    Article  Google Scholar 

  16. Silva RL, Paulo RS, Mazala D, Giraldi G (2014) Applying object recognition and tracking to augmented reality for informatioin visualization. 5

  17. Qi AN, Voon KL, Ismail M, Mustaffa N, Ismail MH (2015) Design and development of a mechanism of robotic arm for lifting part1. In: 2nd Integrated design project conference, Pahang

  18. Jin Y-H, Hwang I-T, Lee W-H (2020) A mobile augmented reality system for the real-time visualization of pipes in point cloud data with a depth sensor 836(9):1–20

  19. Kukuni TG, Kotze B, Hurst W, Lenkoe L (2022) Augmented reality in smart manufacturing: a user experience evaluation. Webology 19(3):2405–2423

    Google Scholar 

  20. Blaga A, Tamas L (2018) Augmented reality for digital manufacturing. In: 26th Mediterranean conference on control and automation

  21. Blaga A, Militaru C, Mezei A, Tamas L (2012) Augmented reality integration into MES for connected workers. Robot Comput-Integr Manuf, pp 14–16

  22. Plakas G, Ponis S, Agalianos K, Aretoulaki E, Gayialis P (2021) Augmented reality in manufacturing and logistics. Lessons Learnt from a Real-Life application, 1629–1635

  23. Ziaee O, Hamedi M (2018) Augmented reality applications in manufacturing and its future scope in industry 4.0

  24. Gattulo M, Scurati W, Fiorentino M, Uva E, Ferrise F, Bordegoni M (2019) Towards augmented reality manuals for industry 4.0. A methodology. Robotics and Computer Integrated Manufacturing, pp 276–286

  25. Urbas U, Vrabic R, Vukasinovic N (2019) Display product manufacturing information in augmented reality for inspection. In: 52nd CIRP conference on manufacuring systems, Ljubljana, Slovenia

  26. ANDREASJAKL https://www.andreasjakl.com/basics-of-ar-slam-simultaneous-localization-and-mapping/. [Online]. Available: https://www.andreasjakl.com/basics-of-ar-slam-simultaneous-localization-and-mapping/. Accessed 10 May 2020

  27. Sagitov A, Shabalina K, Lavrenov R, Magid E. Comparing fiducial marker systems in the presence of occlusion

  28. Kumar S, Das N, Islam S (2014) Performance analysis of substation automation systems architecture based on IEC 61850. In: Proceedings of the 2014 Australasian university power engineering conference (AUPEC), pp 1–6

  29. Wang TJ, Shyi CN, Hou TW, Fong CP (2010) Design, and implementation of augmented reality system collaborating with QR code. In: International computer symposium, pp 414–418

  30. Mur-Artal R, Montiel JJ (2015) ORB-SLAM: a versatile and accurate monocular SLAM system 31(5)

Download references

Author information

Authors and Affiliations

  1. Department of Electrical, Electronic and Computer Engineering, Central University of Technology, 20 President Brandt Street, Bloemfontein, Free-State, South Africa

    Tshepo Godfrey Kukuni & Ben Kotze

  2. Information Technology Group, Wageningen University and Research, Leeuwenborch, Hollandseweg 1, 6706 KN, Wageningen, The Netherlands

    William Hurst

Authors
  1. Tshepo Godfrey Kukuni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ben Kotze
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. William Hurst
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to William Hurst.

Ethics declarations

Conflicts of interest

The authors declare no conflict of interest.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kukuni, T.G., Kotze, B. & Hurst, W. Machine Vision for Device Tracking in a Smart Manufacturing Environment Based on Augmented Reality. Augment Hum Res 8, 3 (2023). https://doi.org/10.1007/s41133-023-00060-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s41133-023-00060-6

Keywords

Search

Navigation