Distributed defect recognition on steel surfaces using an improved random forest algorithm with optimal multi-feature-set fusion

  • Yalin Wang
  • Haibing Xia
  • Xiaofeng Yuan
  • Ling Li
  • Bei Sun


Inspecting steel surfaces is important to ensure steel quality. Numerous defect-detection methods have been developed for steel surfaces. However, they are primarily used for local defects, and their accuracy in detecting distributed defects is unsatisfactory because such defects are difficult to locate and have complex texture characteristics. To solve these issues, an improved random forest algorithm with optimal multi-feature-set fusion (OMFF-RF algorithm) is proposed for distributed defect recognition in this paper. The OMFF-RF algorithm includes the following three aspects. First, a histogram of oriented gradient (HOG) feature-set and a gray-level co-occurrence matrix (GLCM) feature-set are extracted and fused to describe local and global texture characteristics, respectively. Second, given the small number of samples of distributed defect images and the high dimensionality of the extracted feature-sets, a random forest algorithm is introduced to perform defect classification. Third, the feature-sets vary greatly in performance and dimensionality. To improve the fusion efficiency, OMFF-RF merges the HOG feature-set and the GLCM feature-set through a multi-feature-set fusion factor, which changes the number of decision trees that correspond to each feature-set in the RF algorithm. The OMFF factor is found by optimizing the fitting curve of the classification accuracy of the test set using a stepping multi-feature-set fusion factor. In experiments, the effectiveness of the proposed OMFF-RF was verified using 5 types of distributed defects collected from an actual steel production line. OMFF-RF achieved a recognition accuracy of 91%, a result superior to support vector machine (SVM) and conventional RF algorithms.


Steel surface Distributed defect recognition Histogram of oriented gradient (HOG) Gray-level co-occurrence matrix (GLCM) Random forest (RF) Optimal multi-feature-set fusion (OMFF) 



This work is supported by the Major Program of the National Natural Science Foundation of China (Grant No. 61590921), the National Natural Science Foundation of China (Grant No. 61273187), the Foundation for Innovative Research Groups of the National Natural Science Foundation of China (Grant No. 61321003) and the Fundamental Research Funds for the Central Universities of Central South University (Grant No. 2017zzts488).


  1. 1.
    Agarwal K, Shivpuri R, Zhu Y, Chang T-S, Huang H (2011) Process knowledge based multi-class support vector classification (PK-MSVM) approach for surface defects in hot rolling. Expert Syst Appl 38:7251–7262. CrossRefGoogle Scholar
  2. 2.
    Ameer A, Kumar KS (2015) Efficient automatic image annotation using optimized weighted complementary feature fusion using genetic algorithm. Procedia Comput Sci 58:731–739. CrossRefGoogle Scholar
  3. 3.
    Azhar R, Tuwohingide D, Kamudi D, Sarimuddin, Suciati N (2015) Batik image classification using SIFT feature extraction, bag of features and support vector machine. Procedia Comput Sci 72:24–30. CrossRefGoogle Scholar
  4. 4.
  5. 5.
    Blackledge JM, Dubovitskiy DA (2008) A surface inspection machine vision system that includes fractal texture analysis. ISAST Trans Electron Sig Process 3:76–89Google Scholar
  6. 6.
    Borselli A, Colla V, Vannucci M, Veroli M (2010) A fuzzy inference system applied to defect detection in flat steel production. In: International conference on fuzzy systems. IEEE, Piscataway, NJ, pp 21–26Google Scholar
  7. 7.
    Breiman L (2001) Random forest. Mach Learn 45:5–32. CrossRefMATHGoogle Scholar
  8. 8.
    Caleb P, Steuer M (2000) Classification of surface defects on hot rolled steel using adaptive learning methods. In: Proceedings KBIESAT fourth international conference on knowledge-based intelligent engineering systems and allied technologies. IEEE, Piscataway, NJ, pp 103–108Google Scholar
  9. 9.
    Cateni S, Colla V, Vannucci M, Borselli A (2012) Fuzzy inference systems applied to image classification in the industrial field. In: Azeem MF (ed) Fuzzy inference system - theory and applications. InTech, Rijeka, pp 243–270Google Scholar
  10. 10.
    Chen Y, Chen L, Liu X, Ding S, Zhang H (2011) Real-time steel inspection system based on support vector machine and multiple kernel learning. In: Advances in intelligent and soft computing, 2011. Springer, Berlin, pp 185–190Google Scholar
  11. 11.
    Choi K, Koo K, Lee J (2006) Development of defect classification algorithm for POSCO rolling strip surface inspection system. In: 2006 SICE-ICASE international joint conference. IEEE, Piscataway, NJ, pp 2499–2502Google Scholar
  12. 12.
    Choi D-C, Jeon Y-J, Lee SJ, Yun JP, Kim SW (2014) Algorithm for detecting seam cracks in steel plates using a gabor filter combination method. Appl Opt 53:4865. CrossRefGoogle Scholar
  13. 13.
    Cord A, Bach F, Jeulin D (2010) Texture classification by statistical learning from morphological image processing: application to metallic surfaces. J Microsc 239:159–166. MathSciNetGoogle Scholar
  14. 14.
    Dutta S, Das A, Barat K, Roy H (2012) Automatic characterization of fracture surfaces of AISI 304LN stainless steel using image texture analysis. Measurement 45:1140–1150. CrossRefGoogle Scholar
  15. 15.
    Gao LL, Song JK, Nie FP, Yan Y, Sebe N, Shen HT (2015) Optimal graph learning with partial tags and multiple features for image and video annotation. In: 2015 I.E. Conference on Computer Vision and Pattern Recognition. IEEE, Piscataway, NJ, pp 4371–4379Google Scholar
  16. 16.
    Ghorai S, Mukherjee A, Gangadaran M, Dutta PK (2013) Automatic defect detection on hot-rolled flat steel products. IEEE Trans Instrum Meas 62:612–621. CrossRefGoogle Scholar
  17. 17.
    Gray KR, Aljabar P, Heckemann RA, Hammers A, Rueckert D (2015) Random forest-based similarity measures for multi-modal classification of Alzheimer's disease. NeuroImage 65:167–175. CrossRefGoogle Scholar
  18. 18.
    Haralick RM (1979) Statistical and structural approaches to texture. Proc IEEE 67:786–804. CrossRefGoogle Scholar
  19. 19.
    Hemalatha G, Sumathi CP (2016) Preprocessing techniques of facial image with Median and Gabor filters. In: 2016 international conference on information communication and embedded systems (ICICES). IEEE, Piscataway, NJ, pp 1–6Google Scholar
  20. 20.
    Hu H, Liu Y, Liu M, Nie L (2016) Surface defect classification in large-scale strip steel image collection via hybrid chromosome genetic algorithm. Neurocomputing 181:86–95. CrossRefGoogle Scholar
  21. 21.
    Jeon YJ, Choi DC, Yun JP, Park C, Kim SW (2011) Detection of scratch defects on slab surface. In: 2011 11th international conference on control, automation and systems (ICCAS). IEEE, Piscataway, NJ, pp 1274–1278Google Scholar
  22. 22.
    Jia H, Murphey YL, Shi J, Chang T-S (2004) An intelligent real-time vision system for surface defect detection. In: Proceedings of the 17th international conference on pattern recognition, 2004. ICPR 2004. IEEE, Piscataway, NJ, pp 239–242Google Scholar
  23. 23.
    Kang G-W, Liu H-B (2005) Surface defects inspection of cold rolled strips based on neural network. In: 2005 international conference on machine learning and cybernetics. IEEE, Piscataway, NJ, pp 5034–5037Google Scholar
  24. 24.
    Kumudham R, Dhanalakshmi, Swaminathan A, Geetha, Rajendran V (2016) Comparison of the performance metrics of median filter and wavelet filter when applied on SONAR images for denoising. In: 2016 international conference on computation of power, energy information and commuincation (ICCPEIC). IEEE, Piscataway, NJ, pp 288–290Google Scholar
  25. 25.
    Lenc L, Král P (2015) Automatic face recognition system based on the SIFT features. Comput Electr Eng 46:256–272. CrossRefGoogle Scholar
  26. 26.
    Liang C-W, Juang C-F (2015) Moving object classification using local shape and HOG features in wavelet-transformed space with hierarchical SVM classifiers. Appl Soft Comput 28:483–497. CrossRefGoogle Scholar
  27. 27.
    Liu W, Yan Y, Li J, Zhang Y, Sun H (2008) Automated on-line fast detection for surface defect of steel strip based on multivariate discriminant function. In: 2008 second international symposium on intelligent information technology application. IEEE, Piscataway, NJ, pp 493–497Google Scholar
  28. 28.
    Liu YC, Hsu YL, Sun YN, Tsai SJ, Ho CY, Chen CM (2010) A computer vision system for automatic steel surface inspection. In: 2010 5th IEEE conference on industrial electronics and applications. IEEE, Piscataway, NJ, pp 1667–1670Google Scholar
  29. 29.
    Liu M, Liu Y, Hu H, Nie L (2016) Genetic algorithm and mathematical morphology based binarization method for strip steel defect image with non-uniform illumination. J Vis Commun Image Represent 37:70–77. CrossRefGoogle Scholar
  30. 30.
    Martins LAO, Padua FLC, Almeida PEM (2010) Automatic detection of surface defects on rolled steel using computer vision and artificial neural networks. In: IECON 2010 - 36th annual conference on IEEE industrial electronics society. IEEE, Piscataway, NJ, pp 1081–1086Google Scholar
  31. 31.
    Luo Q, He Y (2016) A cost-effective and automatic surface defect inspection system for hot-rolled flat steel. Robot Comput Integr Manuf 38:16–30. CrossRefGoogle Scholar
  32. 32.
    Mäenpää T (2006) Surface quality assessment with advanced texture analysis techniques. In: Proceedings of the international surface inspection summit (ISIS 2006). International Surface Inspection Summit, p 7Google Scholar
  33. 33.
    Masci J, Meier U, Ciresan D, Schmidhuber J, Fricout G (2012) Steel defect classification with max-pooling convolutional neural networks. In: The 2012 international joint conference on neural networks (IJCNN). IEEE, Piscataway, NJ, pp 1–6Google Scholar
  34. 34.
    Neogi N, Mohanta DK, Dutta PK (2014) Review of vision-based steel surface inspection systems. EURASIP J Image Video Process 2014:1–19. CrossRefGoogle Scholar
  35. 35.
    Pang Y, Yuan Y, Li X, Pan J (2011) Efficient HOG human detection. Signal Process 91:773–781. CrossRefMATHGoogle Scholar
  36. 36.
    Park C, Won S (2009) An automated web surface inspection for hot wire rod using undecimated wavelet transform and support vector machine. In: 2009 35th annual conference of IEEE industrial electronics. IEEE, Piscataway, NJ, pp 2411–2415Google Scholar
  37. 37.
    Peng K, Zhang X (2009) Classification technology for automatic surface defects detection of steel strip based on improved BP algorithm. In: 2009 fifth international conference on natural computation. IEEE, Piscataway, NJ, pp 110–114Google Scholar
  38. 38.
    Sasaki T, Takada H, Tomura Y (2007) Automatic surface inspection system for tin mill black plate (TMBP). JPE Technical Report 9:60–63Google Scholar
  39. 39.
    Song JK, Yang Y, Huang Z, Shen HT, Luo JB (2013) Effective multiple feature hashing for large-scale near-duplicate video retrieval. IEEE Trans Multimedia 15:1997–2008. CrossRefGoogle Scholar
  40. 40.
    Suvdaa B, Ahn J, Ko J (2012) Steel surface defects detection and classification using SIFT and voting strategy. Int J Softw Eng Appl 2:161–165Google Scholar
  41. 41.
    Tang B, Kong J-Y, Wang X-D, Chen L (2009) Surface inspection system of steel strip based on machine vision. In: 2009 first international workshop on database technology and applications. IEEE, Piscataway, NJ, pp 359–362Google Scholar
  42. 42.
    Wang XH, Gao LL, Song JK, Shen HT (2017) Beyond frame-level CNN: saliency-aware 3-D CNN with LSTM for video action recognition. IEEE Signal Proc Let 24(4):510–514. CrossRefGoogle Scholar
  43. 43.
    Wu G, Zhang H, Sun X, Xu J, Xu K (2007) A bran-new feature extraction method and its application to surface defect recognition of hot rolled strips. In: 2007 I.E. international conference on automation and logistics. IEEE, Piscataway, NJ, pp 2069–2074Google Scholar
  44. 44.
    Wu G, Kwak H, Jang S, Xu K, Xu J (2008) Design of online surface inspection system of hot rolled strips. In: 2008 I.E. international conference on automation and logistics. IEEE, Piscataway, NJ, pp 2291–2295Google Scholar
  45. 45.
    Yan Y, Song K, Xing Z, Feng X (2011) The strip steel surface defects classification method based on weak classifier adaptive enhancement. In: 2011 third international conference on measuring technology and mechatronics automation. IEEE, Piscataway, NJ, pp 958–961Google Scholar
  46. 46.
    Yang Y, Song JK, Huang Z, Ma ZG, Sebe N, Hauptmann AG (2013) Multi-feature fusion via hierarchical regression for multimedia analysis. IEEE Trans Multimedia 15:572–581. CrossRefGoogle Scholar
  47. 47.
    Yao S, Pan S, Wang T, Zheng C, Shen W, Chong Y (2015) A new pedestrian detection method based on combined HOG and LSS features. Neurocomputing 151:1006–1014. CrossRefGoogle Scholar
  48. 48.
    Yazdchi MR, Mahyari AG, Nazeri A (2008) Detection and classification of surface defects of cold rolling mill steel using morphology and neural network. In: 2008 international conference on computational intelligence for modelling control & automation. IEEE, Piscataway, NJ, pp 1071–1076Google Scholar
  49. 49.
    Yazdchi M, Yazdi M, Mahyari AG (2009) Steel surface defect detection using texture segmentation based on multifractal dimension. In: 2009 international conference on digital image processing. IEEE, Piscataway, NJ, pp 346–350Google Scholar
  50. 50.
    Yun JP, Park C, Bae H, Hwang H, Choi S (2010) Vertical scratch detection algorithm for high-speed scale-covered steel BIC(Bar in Coil). In: 2010 international conference on control automation and systems (ICCAS). IEEE, Piscataway, NJ, pp 342–345Google Scholar
  51. 51.
    Yun JP, Choi D-C, Jeon Y-J, Park C, Kim SW (2014) Defect inspection system for steel wire rods produced by hot rolling process. Int J Adv Manuf Technol 70:1625–1634. CrossRefGoogle Scholar
  52. 52.
    Zhao J, Yang Y, Li G (2010) The cold rolling strip surface defect on-line inspection system based on machine vision. In: 2010 second Pacific-Asia conference on circuits, communications and system. IEEE, Piscataway, NJ, pp 402–405Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Yalin Wang
    • 1
  • Haibing Xia
    • 1
  • Xiaofeng Yuan
    • 1
  • Ling Li
    • 1
  • Bei Sun
    • 1
  1. 1.School of Information Science and EngineeringCentral South UniversityChangshaChina

Personalised recommendations