Skip to main content
SpringerLink
Log in

Fault detection and classification in automated assembly machines using machine vision

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

Automated assembly machines operate continuously to achieve high production rates. Continuous operation increases the potential for faults such as jams, missing parts, and electromechanical failures of subsystems. The goal of this research project was to develop and validate a machine vision inspection (MVI) system to detect and classify multiple faults using a single camera as a sensor. An industrial automated O-ring assembly machine that places O-rings on to continuously moving plastic carriers at a rate of over 100 assemblies per minute was modified to serve as the test apparatus. An industrial camera with LED panel lights for illumination was used to acquire videos of the machine’s operation. A programmable logic controller (PLC) with a human-machine interface (HMI) allowed for the generation of faults in a controlled fashion. Three MVI methods, based on computer vision techniques available in the literature, were developed for this application. The methods used features extracted from the videos to classify the machine’s condition. The first method was based on Gaussian mixture models (GMMs); the second method used an optical flow approach; and the third method was based on running average and morphological image processing operations. In order to provide a single metric to quantify relative performance, a machine vision performance index (MVPI) was developed with five measures of performance: accuracy, processing time, speed of response, robustness against noise, and ease of tuning. The MVPI for the three MVI methods is reported along with the significance of the results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Holloway L, Krogh B (1990) Fault detection and diagnosis in manufacturing systems: a behavioral model approach. Proc. of Rensselaer’s Second International Conference on Computer Integrated Manufacturing 252–259

  2. Boothroyd G (2005) Assembly automation and product design, 2nd edn. Florence: Marcel Dekker, New York

    Book  Google Scholar 

  3. Viswanadham N, Johnson T (1988) Fault detection and diagnosis of automated manufacturing systems. Proc of the 27th IEEE Conference on Decision and Control 3:2301–2306

    Article  Google Scholar 

  4. Konrad H (1996) Fault detection in milling, using parameter estimation and classification methods. Control Eng Pract 4(11):1573–1578. doi:10.1016/0967-0661(96)00172-4

    Article  Google Scholar 

  5. Wang W, Mcfadden P (1996) Application of wavelets to gearbox vibration signals for fault detection. J Sound Vib 192(5):927–939

    Article  Google Scholar 

  6. Dalpiaz G, Rivola A (1997) Condition monitoring and diagnostics in automatic machines: comparison of vibration analysis techniques. Mech Syst Signal Process 11(1):53–73

    Article  Google Scholar 

  7. Patel S, Kamrani A, Orady E (1995) A knowledge-based system for fault diagnosis and maintenance of advanced automated systems. Comput Ind Eng 29(1–4):147–151

    Article  Google Scholar 

  8. Demetgul M, Tansel I, Taskin S (2009) Fault diagnosis of pneumatic systems with artificial neural network algorithms. Expert Syst Appl 36(7):10512–10519

    Article  Google Scholar 

  9. Demetgul M, Unal M, Tansel I, Yazicioğlu O (2011a) Fault diagnosis on bottle filling plant using genetic-based neural network. Adv Eng Softw 42(12):1051–1058

    Article  Google Scholar 

  10. Fernando H, Surgenor B (2015) An unsupervised artificial neural network versus a rule-based approach for fault detection and identification in an automated assembly machine. Robotics Comp Integrated Manuf. doi:10.1016/j.rcim.2015.11.006

    Google Scholar 

  11. Sekar R, Hsieh S, Wu Z (2011) Remote diagnosis design for a plc-based automated system: 1-implementation of three levels of architectures. Int J Adv Manuf Technol 57(5–8):683–700

    Article  Google Scholar 

  12. Sekar R, Hsieh S, Wu Z (2013) Remote diagnosis design for a plc-based automated system: 2-evaluation of factors affecting remote diagnosis performance. Int J Adv Manuf Technol 65(5–8):1091–1109

    Article  Google Scholar 

  13. Malamas E, Petrakis E, Zervakis M, Petit L, Legat J (2003) A survey on industrial vision systems, applications and tools. Image Vis Comput 21(2):171–188

    Article  Google Scholar 

  14. Yao A (2005) Design and implementation of web-based diagnosis and management system for an FMS. Int J Adv Manuf Technol 26(11–12):1379–1387

    Article  MathSciNet  Google Scholar 

  15. Killing J, Surgenor B, Mechefske C (2009) A machine vision system for the detection of missing fasteners on steel stampings. Int J Adv Manuf Technol 41(7–8):808–819

    Article  Google Scholar 

  16. Ke Y, Sukthankar R, Hebert M (2007) Event detection in crowded videos. IEEE 11th International Conference on Computer Vision (ICCV) 1–8

  17. Benezeth Y, Jodoin P, Saligrama V, Rosenberger C (2009) Abnormal events detection based on spatio-temporal co-occurrences. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2458–2465

  18. Hughes K, Fernando H, Szkilnyk G, Surgenor B, Greenspan M (2014) Video event detection for fault monitoring in assembly automation. Int J Intell Syst Technol Appl 13(1/2):103–116

    Google Scholar 

  19. Bhuvanesh A, Ratnam M (2007) Automatic detection of stamping defects in lead frames using machine vision: overcoming translational and rotational misalignment. Int J Adv Manuf Technol 32(11):1201–1210

    Article  Google Scholar 

  20. Shahabi H, Ratnam M (2009) In-cycle monitoring of tool nose wear and surface roughness of turned parts using machine vision. Int J Adv Manuf Technol 40(11):1148–1157

    Article  Google Scholar 

  21. Xiaokun L, Porikli F (2004) A hidden Markov model framework for traffic event detection using video features. Proc of the Int Conference Image Process (ICIP) 5:2901–2904

    Google Scholar 

  22. Zezhi C, Pears N, Freeman M, Austin J (2014) A Gaussian Mixturemodel and support vector machine approach to vehicle type and colour classification. Intelligent Trans Syst (IET) 8(2):135–144

    Article  Google Scholar 

  23. Horn B, Schunck B (1981) Determining optical flow. Proc. International Society for Optics and Photonics, Technical Symposium East 319–331

  24. Patel E, Shukla D (2013) Comparison of optical flow algorithms for speed determination of moving objects. Int J Comp Appl 63(5):32–37

    Google Scholar 

  25. Usamentiaga R, Molleda J, Garcia D, Bulnes F, Perez J (2013) Jam detector for steel pickling lines using machine vision. IEEE Trans Ind Appl 49(5):1954–1961

    Article  Google Scholar 

  26. Chauhan V, Surgenor B (2015) A comparative study of machine vision based methods for fault detection in an automated assembly machine. Procedia Manufacturing 1:416–428

    Article  Google Scholar 

  27. Miles BC, Surgenor BW, Killing J (2008) Effect of lighting on the performance of a machine vision system. Proc. of 18th Conference on Flexible Automation and Intelligent Inspection (FAIM), Skövde, Sweden

  28. Kopparapu SK (2006) Lighting design for machine vision application. Image Vis Comput 24(7):720–726

    Article  Google Scholar 

  29. Chauhan V, Fernando H, Surgenor B (2014) Effect of illumination techniques on machine vision inspection for automated assembly machines. Proc. of The Canadian Society for Mechanical Engineering (CSME) International Congress, Toronto 1–6

  30. Stauffer C, Grimson WE (1999) Adaptive background mixture models for real-time tracking. IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2:1–7

    Google Scholar 

  31. MATLAB (2014) Image acquisition, processing and computer vision toolboxes. The Mathworks, Inc., U.S.

    Google Scholar 

  32. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Proc. of 25th Annual Conference on Advances in Neural Information Processing Systems (NIPS) 1097–1105

Download references

Author information

Authors and Affiliations

  1. Mechanical and Materials Engineering Department, Queen’s University, K7L3N6, Kingston, ON, Canada

    Vedang Chauhan & Brian Surgenor

Authors
  1. Vedang Chauhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Brian Surgenor
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vedang Chauhan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chauhan, V., Surgenor, B. Fault detection and classification in automated assembly machines using machine vision. Int J Adv Manuf Technol 90, 2491–2512 (2017). https://doi.org/10.1007/s00170-016-9581-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-016-9581-5

Keywords

Search

Navigation