Abstract
Crack image segmentation has recently become a major research topic in nondestructive inspection. However, the image segmentation methods are not robust to variations such as illumination, weather, noise and the segmentation accuracy which cannot meet the requirements of practical applications. Therefore, a neural network ensemble method is proposed for effective crack segmentation in this paper, which consists of fully convolution networks (FCN) and multi-scale structured forests for edge detection (SFD). In order to improve the accuracy of crack segmentation and reduce the error mark under complex background, a new network model based on FCN model is proposed to address the problems that lose local information and the capacity of partial refinement, which are frequently encountered in FCN model in the crack segmentation. In addition, SFD is combined with the half-reconstruction method of anti-symmetrical bi-orthogonal wavelet to overcome the limitation of crack edge detection. Finally, the result of the two maps is merged after resizing to the original image dimensions. Qualitative and quantitative evaluations of the proposed methods are performed, showing that they can obtain better results than certain existing methods for crack segmentation.
Similar content being viewed by others
References
Luo, Q.W., He, Y.G.: A cost-effective and automatic surface defect inspection system for hot-rolled flat steel. Robot. Comput.-Integr. Manuf. 38(3), 16–30 (2016)
Kehoe, A., Parker, G.A.: Image processing for industrial radiographic inspection: image enhancement. Br. J. NDT 32(4), 183–190 (1990)
Amet, L.L., Ertuzun, A., Ercil, A.: An efficient method for texture defect detection: subband domain co-occurrence matrices. Image Vis. Comput. 18(6–7), 543–553 (2000)
Bodnarova, A., Bennamoun, M., Latham, S.: Optimal gabor filters for textile flaw detection. Pattern Recognit. 35(12), 2973–2991 (2002)
Wiltschi, K., Pinz, A., Lindeberg, T.: An automatic assessment scheme for steel quality inspection. Mach. Vis. Appl. 12(3), 113–128 (2000)
Clelia, M., Ettore, S., Massimiliano, N., Nicola, A., Arcangelo, D.: Rail corrugation detection by Gabor filtering. In: IEEE International Conference on Image Processing, pp. 626–628 (2001)
Kasban, H., Zahran, O., Arafa, H., El-Kordy, M., ELARABY, S.M.S., Abd EI-Samie, F.E.: Welding defect detection from radiography images with a cepstral approach. NDT & E Int. 44, 226–231 (2011)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 640–651 (2017)
Piotr, D., Lawrence, Z.C.: Fast edge detection using structured forests. IEEE Trans. Pattern Anal. Mach. Intell. 37(8), 1558–1570 (2015)
Chi, J.N., Zhang, C., Zhang, Z.H.: Image enhancement based on antisymmetrical bioryhogonal wavelet recontruction. Acta Autom. Sin. 36(4), 475–487 (2010)
Martnez, S.S., Ortega, J.G., Garcia, J.G., Garcia, A.S.: A machine vision system for defect characterization on transparent parts with non-plane surfaces. Mach. Vis. Appl. 23(1), 1–13 (2012)
He, Y.Q., Qiu, H.X.: A method of cracks image segmentation based on the means of multiple thresholds. J. Commun. Comput. 9, 1147–1151 (2012)
Kima, H., Ahna, E., Chob, S.: Comparative analysis of image binarization methods for crack identification in concrete structures. Cem. Concr. Res. 99, 53–61 (2017)
Li, Y.Y., Sagar, D., Peng, Y.K.: A machine vision system for identification of micro-crack in egg shell. J. Food Eng. 109, 127–134 (2012)
Aswini, E., Divya, S., Kardheepan, S., Manikandan, T.: Mathematical morphology and bottom-hat filtering approach for crack detection on relay surfaces. In: IEEE International Conference on Smart Structures & Systems, pp. 108–113 (2013)
Jetsadaporn, P., Chawalit, K., Somying, T.: Crack detection on unwashed eggs using image processing. J. Food Eng. 209, 76–82 (2017)
Bharathi, S.S., Radhakrishnan, N., Priya, L.: Surface defect detection of rubber oil seals based on texture analysis. In: IEEE 4th International Conference on Signal and Image Processing, pp. 207–216 (2012)
Withawat, W., Pichate, K., Channarong, A., Pitikhate, S.: Rapid detection of hairline cracks on the surface of piezoelectric ceramics. Int. J. Adv. Manuf. Technol. 64, 1275–1283 (2013)
Yusuke, F., Yoshihiko, H.: A robust automatic crack detection method from noisy concrete surfaces. Mach. Vis. Appl. 22, 245–254 (2011)
Yang, C.L., Liu, P.Y., Yin, G.F., Wang, L.: Crack detection in magnetic tile images using nonsubsampled shearlet transform and envelope gray level gradient. Opt. Laser Technol. 90, 7–17 (2017)
Yamaguchi, T., Hashimoto, S.: Fast crack detection method for large-size concrete surface images using percolation-based image processing. Mach. Vis. Appl. 21, 797–809 (2010)
Fadel, M.M., Jaime, A.C.: Real-time fault detection in manufacturing environments using face recognition techniques. J. Intell. Manuf. 23, 393–408 (2012)
Cinara, A.F., Barhlib, S.M., Hollis, D., Flansbjer, M., Tomlinson, R.A., Marrow, T.J., Mostafavi, M.: An autonomous surface discontinuity detection and quantification method by digital image correlation and phase congruency. Opt. Lasers Eng. 96, 94–106 (2017)
Zou, Q., Cao, Y., Li, Q.Q., Mao, Q.Z., Wang, S.: CrackTree: automatic crack detection from pavement images. Pattern Recognit. Lett. 3, 227–238 (2012)
Mohamad, M.H., Satya, M.M., Desire, S., Fabrice, M.: Multiple features extraction for timber defects detection and classification using SVM. In: IEEE International Conference on Image Processing, pp. 427–431 (2015)
Suvdaa, B., Ahn, J., Ko, J.: Steel Surface defects detection and classification using SIFT and voting strategy. Int. J. Softw. Eng. Appl. 6(2), 161–165 (2012)
Hu, H.J., Liu, Y., Liu, M.F., Nie, L.Q.: Surface defect classification in large-scale strip steel image collection via hybrid chromosome genetic algorithm. Neurocomputing 181, 86–95 (2016)
Sukalpa, C., Bu, G.P., Guan, H., Jun, J., Umapada, P., Yew-Chaye, L., Michael, B.: Automatic bridge crack detection—a texture analysis-based approach. In: IAPR Workshop on Artificial Neural Networks in Pattern Recognition, pp. 193–203 (2014)
Li, G., Zhao, X.X., Du, K., Ru, F., Zhang, Y.B.: Recognition and evaluation of bridge cracks with modified active contour model and greedy search-based support vector machine. Autom. Constr. 78, 51–61 (2017)
Hoang, N., Nguyen, Q., Bui, D.T.: Image processing-based classification of asphalt pavement cracks using support vector machine optimized by artificial bee colony. J. Comput. Civ. Eng. 32(5), 04018037 (2018)
Banharnsakun, A.: Hybrid ABC-ANN for pavement surface distress detection and classification. Int. J. Mach. Learn. Cybern. 8, 699–710 (2017)
Cubero-Fernandez, A., Rodriguez-Lozano, F.J., Rafael, V., Joaquin, O., Jose, M.P.: Efficient pavement crack detection and classification. EURASIP J. Image Video Process. 2017, 39–49 (2017)
Chen, S.G., Lin, B., Han, X.S., Liang, X.H.: Automated inspection of engineering ceramic grinding surface damage based on image recognition. Int. J. Adv. Manuf. Technol. 66, 431–443 (2013)
Shi, P.F., Fan, X.N., Wang, G.R.: A novel underwater dam crack detection algorithm based on sonar images. In: Proceedings of the 5th International Conference on Computer Sciences and Automation Engineering, pp. 452–456 (2015)
Meng, F., Li, A.: Pavement crack detection using sketch token. Procedia Comput. Sci. 139, 151–157 (2018)
Luo, Q.J., Ge, B.Z., Tian, Q.G.: A fast adaptive crack detection algorithm based on a double-edge extraction operator of FSM. Constr. Build. Mater. 24, 244–254 (2019)
Cha, Y.J., Choi, W.: Deep learning-based crack damage detection using convolutional neural networks. Comput.-Aided Civ. Infrastruct. Eng. 32, 361–378 (2017)
Zhang, L., Yang, F., Zhang, Y.D., Zhu, Y.J.: Road crack detection using deep convolutional neural network. In: IEEE International Conference on Image Processing, pp. 3708–3712 (2016)
Gopalakrishnan, K., Khaitan, S.K., Choudhary, A., Agrawal, A.: Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection. Constr. Build. Mater. 157, 322–330 (2017)
Zhang, A., Wang, K.C.P.: Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network. Comput.-Aided Civ. Infrastruct. Eng. 32, 805–819 (2017)
Zhang, A., Wang, K.C.P., Fei, Y., Liu, Y., Tao, S.Y., Chen, C., Li, J.Q., Li, B.X.: Deep learning-based fully automated pavement crack detection on 3D asphalt surfaces with an improved CrackNet. J. Comput. Civ. Eng. 32(5), 04018041 (2018)
Chen, F.C., Jahanshahi, M.R.: NB-CNN: deep learning-based crack detection using convolutional neural network and Naive Bayes data fusion. IEEE Trans. Ind. Electron. 65(5), 4392–4400 (2018)
Liu, H., Guo, R.Y.: Detection and identification of SAWH pipe weld defects based on X-ray image and CNN. Chin. J. Sci. Instrum. 39(4), 247–256 (2014)
Chang, H.T., Gou, J.N., Li, X.M.: Application of faster R-CNN in image defect detection of industrial CT. J. Image Graph. 23(7), 1061–1071 (2018)
Huang, H.W., Li, Q.T., Zhang, D.M.: Deep learning based image recognition for crack and leakage defects of metro shield tunnel. Tunn. Undergr. Space Technol. 77, 166–176 (2018)
Dung, C.V., Anh, L.D.: Autonomous concrete crack detection using deep fully convolutional neural network. Autom. Construct. 99, 52–58 (2019)
Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFS. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018)
He, K., Zhang, X.Y., Ren, S.Q., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder–decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)
Wang, S., Wu, X., Zhang, Y.H., Chen, Q.: Image crack detection with multi-scale down-sample normalized cut. Chin. J. Sci. Instrum. 38(11), 2788–2796 (2017)
Boykov, Y., Kolmogorov, V., Zhang, D.M.: A experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137 (2004)
Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 3rd edn. Publishing House of Electronics Industry, Beijing (2011)
Perazzi, F., Krahenbuhl, P., Pritch, Y., Hornung, A.: Saliency filters: contrast based filtering for salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 733–740 (2012)
Acknowledgements
This work was supported by the National Natural Science Foundation of China under (51875272, 61761024), the Talent Development Project of Yunnan Province under (KKSY201801018), Yunnan Provincial Department of Education Science Research Fund Project under (2019J0045) and the Introduction Talents Project of Guizhou University under (2018(18)).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wang, S., Wu, X., Zhang, Y. et al. A neural network ensemble method for effective crack segmentation using fully convolutional networks and multi-scale structured forests. Machine Vision and Applications 31, 60 (2020). https://doi.org/10.1007/s00138-020-01114-0
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00138-020-01114-0