Abstract
Computer vision builds a connection between image processing and industrials, bringing modern perception to the automated manufacture of magnetic tiles. In this article, we propose a real-time model called MCuePush U-Net, specifically designed for saliency detection of surface defect. Our model consists of three main components: MCue, U-Net and Push network. MCue generates three-channel resized inputs, including one MCue saliency image and two raw images; U-Net learns the most informative regions, and essentially it is a deep hierarchical structured convolutional network; Push network defines the specific location of predicted surface defects with bounding boxes, constructed by two fully connected layers and one output layer. We show that the model exceeds the state of the art in saliency detection of magnetic tiles, in which it both effectively and explicitly maps multiple surface defects from low-contrast images. The proposed model significantly reduces time cost of machinery from 0.5 s per image to 0.07 s and enhances detection accuracy for image-based defect examinations.
Similar content being viewed by others
References
Yang, C., Liu, P., Yin, G., Jiang, H., Li, X.: Defect detection in magnetic tile images based on stationary wavelet transform. NDT & E Int. 83, 78–87 (2016)
Li, X., Jiang, H., Yin, G.: Detection of surface crack defects on ferrite magnetic tile. NDT & E Int. 62, 6–13 (2014)
Xie, L., Lin, L., Yin, M., Meng, L., Yin, G.: A novel surface defect inspection algorithm for magnetic tile. Appl. Surf. Sci. 375, 118–126 (2016)
Yang, C., Liu, P., Yin, G., Wang, L.: Crack detection in magnetic tile images using nonsubsampled shearlet transform and envelope gray level gradient. Opt. Laser Technol. 90, 7–17 (2017)
Cha, Y.-J., Choi, W., Büyüköztürk, O.: Deep learning-based crack damage detection using convolutional neural networks. Comput. Aided Civ. Infrastruct. Eng. 32(5), 361–378 (2017)
Masci, J., Meier, U., Ciresan, D., Schmidhuber, J., Fricout, G.: Steel defect classification with max-pooling convolutional neural networks. In: The 2012 International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE (2012)
Soukup, D., Huber-Mörk, R.: Convolutional neural networks for steel surface defect detection from photometric stereo images. In: International Symposium on Visual Computing, pp. 668–677. Springer (2014)
Ren, R., Hung, T., Tan, K.C.: A generic deep-learning-based approach for automated surface inspection. IEEE Trans. Cybern. 48(3), 929–940 (2018)
Yang, C., Pu, J., Dong, Y., Xie, G.S., Si, Y., Liu, Z.: Scene classification-oriented saliency detection via the modularized prescription. Vis. Comput. 1, 1–16 (2018)
Yang, Z., Xiong, H.: Computing object-based saliency via locality-constrained linear coding and conditional random fields. Vis. Comput. 33(11), 1403–1413 (2017)
Zhou, X., Wang, Y., Zhu, Q., Xiao, C., Xiao, L.: Ssg: superpixel segmentation and grabcut-based salient object segmentation. Vis. Comput. 11, 1–14 (2018)
Gorji, S., Clark, J.J.: Attentional push: a deep convolutional network for augmenting image salience with shared attention modeling in social scenes. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3472–3481, (2017)
Ramanishka, V., Das, A., Zhang, J., Saenko, K.: Top-down visual saliency guided by captions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, p. 7 (2017)
Zhang, J., Sclaroff, S.: Saliency detection: a boolean map approach. In: IEEE International Conference on Computer Vision, pp. 153–160 (2013)
Donoser, M., Urschler, M., Hirzer, M., Bischof, H.: Saliency driven total variation segmentation. In: IEEE International Conference on Computer Vision, pp. 817–824 (2009)
Cheng, M.M., Warrell, J., Lin, W.Y., Zheng, S., Vineet, V., Crook, N.: Efficient salient region detection with soft image abstraction. In: IEEE International Conference on Computer Vision, pp. 1529–1536 (2013)
Ran, M., Zelnik-Manor, L., Tal, A.: Saliency for image manipulation. Vis. Comput. 29(5), 381–392 (2013)
Gao, Y., Wang, M., Tao, D., Ji, R., Dai, Q.: 3-d object retrieval and recognition with hypergraph analysis. IEEE Trans. Image Process. 21(9), 4290–4303 (2012)
Yang, B., Li, F.W., Wang, X., Mingliang, X., Liang, X., Jiang, Z., Jiang, Y.: Visual saliency guided textured model simplification. Vis. Comput. 32(11), 1415–1432 (2016)
Tang, Y., Tong, R., Tang, M., Zhang, Y.: Depth incorporating with color improves salient object detection. Vis. Comput. 32(1), 111–121 (2016)
Kapoor, A., Biswas, K.K., Hanmandlu, M.: An evolutionary learning based fuzzy theoretic approach for salient object detection. Vis. Comput. 33(5), 1–21 (2016)
Borji, A., Cheng, M.M., Jiang, H., Li, J.: Salient object detection: a benchmark. IEEE Trans. Image Process. 24(12), 5706–5722 (2015)
Thur15000. http://mmcheng.net/gsal/. Accessed 26 Mar 2018
Yan, Q., Xu, L., Shi, J., Jia, J.: Hierarchical saliency detection. In: Computer Vision and Pattern Recognition, pp. 1155–1162 (2013)
Borji, A.: What is a salient object? a dataset and a baseline model for salient object detection. IEEE Trans. Image Process. A Publ. IEEE Signal Process. Soc. 24(2), 742 (2015)
Yang, C., Zhang, L., Lu, H., Xiang, R., Yang, M.H.: Saliency detection via graph-based manifold ranking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3166–3173 (2013)
Alpert, S., Galun, M., Brandt, A., Basri, R.: Image segmentation by probabilistic bottom-up aggregation and cue integration. IEEE Trans. Pattern Anal. Mach. Intell. 34(2), 315–26 (2012)
Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2009. pp. 1597–1604 (2009)
Achanta, R., Ssstrunk, S.: Saliency detection using maximum symmetric surround. In: IEEE International Conference on Image Processing, pp. 2653–2656 (2010)
Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. 20(11), 1254–1259 (1998)
Song, K.C., Hu, S.P., Yan, Y.H., Li, J.: Surface defect detection method using saliency linear scanning morphology for silicon steel strip under oil pollution interference. ISIJ Int. 54(11), 2598–2607 (2014)
Song, K., Yan, Y.: Micro surface defect detection method for silicon steel strip based on saliency convex active contour model. Math. Probl. Eng. 2013(8), 1–13 (2013)
Guan, S.: Strip steel defect detection based on saliency map construction using gaussian pyramid decomposition. Trans. Iron Steel Inst. Jpn. 55(9), 1950–1955 (2015)
Achanta, R., Estrada, F., Wils, P., Ssstrunk, S.: Salient region detection and segmentation. In: International Conference on Computer Vision Systems, pp. 66–75 (2008)
Gharsallah, M.B., Braiek, E.B.: Weld inspection based on radiography image segmentation with level set active contour guided off-center saliency map. Adv. Mater. Sci. Eng. 2015(11), 1–10 (2015)
Aiger, D., Talbot, H.: The phase only transform for unsupervised surface defect detection. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 295–302. IEEE (2010)
Bai, X., Fang, Y., Lin, W., Wang, L.: Saliency-based defect detection in industrial images by using phase spectrum. IEEE Trans. Ind. Inform. 10(4), 2135–2145 (2014)
He, S., Jiao, J., Zhang, X., Han, G., Lau, R.W.H.: Delving into salient object subitizing and detection. In: IEEE International Conference on Computer Vision, pp. 1059–1067 (2017)
Zhang, P., Wang, D., Lu, H., Wang, H., Xiang, R.: Amulet: Aggregating multi-level convolutional features for salient object detection. In: IEEE International Conference on Computer Vision, pp. 202–211 (2017)
Zhang, P., Wang, D., Lu, H., Wang, H., Yin, B.: Learning uncertain convolutional features for accurate saliency detection. In: IEEE International Conference on Computer Vision, pp. 212–221 (2017)
Wang, T., Borji, A., Zhang, L., Zhang, P., Lu, H.: A stagewise refinement model for detecting salient objects in images. In: IEEE International Conference on Computer Vision, pp. 4039–4048 (2017)
Oh, S.J., Benenson, R., Khoreva, A., Akata, Z., Fritz, M., Schiele, B.: Exploiting saliency for object segmentation from image level labels. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4410–4419 (2017)
Simon, M., Yang, G., Darrell, T., Denzler, J., Rodner, E.: Generalized orderless pooling performs implicit salient matching. In: IEEE International Conference on Computer Vision, pp. 4970–4979 (2017)
Jonathan, L., Evan, S., Trevor, D.: Fully convolutional networks for semantic segmentation. IEEE Conference on Computer Vision and Pattern Recognition. CVPR, pp. 3431–3440. IEEE, Boston, MA, USA (2015)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomedical Image Segmentation. Springer International Publishing, Berlin (2015)
Zhai, Y., Shah, M.: Visual attention detection in video sequences using spatiotemporal cues. In: ACM International Conference on Multimedia, pp. 815–824 (2006)
Cheng, M.M., Zhang, G.X., Mitra, N.J., Huang, X., Hu, S.M.: Global contrast based salient region detection. In: Computer Vision and Pattern Recognition, pp. 409–416 (2011)
Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: IEEE Conference on Computer Vision and Pattern Recognition. CVPR ’07, pp. 1–8 (2007)
Rudinac, M., Jonker, P.P.: Saliency detection and object localization in indoor environments. In: International Conference on Pattern Recognition, pp. 404–407 (2010)
Krahenbuhl, P.: Saliency filters: contrast based filtering for salient region detection. In: Computer Vision and Pattern Recognition, pp. 733–740 (2012)
Zhang, J., Sclaroff, S., Lin, Z., Shen, X., Price, B., Mech, R.: Minimum barrier salient object detection at 80 fps. In: IEEE International Conference on Computer Vision, pp. 1404–1412 (2015)
Harris, C.: A combined corner and edge detector. Proc. Alvey Vis. Conf. 1988(3), 147–151 (1988)
Schaefer, S., Mcphail, T., Warren, J.: Image deformation using moving least squares. In: ACM SIGGRAPH, pp. 533–540 (2006)
Zhang, M., Li, X., Xu, M., Li, Q.: Image segmentation and classification for sickle cell disease using deformable u-net (2017). arXiv preprint arXiv:1710.08149
Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., Wei, Y.: Deformable convolutional networks. CoRR, abs/1703.06211 1(2), 3 (2017)
Zhu, W., Liang, S., Wei, Y., Sun, J.: Saliency optimization from robust background detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2814–2821 (2014)
Tavakoli, H.R., Rahtu, E.: Fast and efficient saliency detection using sparse sampling and kernel density estimation. In: Scandinavian Conference on Image Analysis, pp. 666–675 (2011)
Jiang, B., Zhang, L., Lu, H., Yang, C., Yang, M.H.: Saliency detection via absorbing Markov chain. In: IEEE International Conference on Computer Vision, pp. 1665–1672 (2013)
Wang, J., Jiang, H., Yuan, Z., Cheng, M.M., Hu, X., Zheng, N.: Salient object detection: a discriminative regional feature integration approach. Int. J. Comput. Vis. 123(2), 251–268 (2017)
Acknowledgements
This study is supported by National Natural Science Foundation of China (Grant No. 61421004).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Authors Yibin Huang, Congying Qiu, Maria Feng and Kui Yuan declare that they have no conflict of interest.
Rights and permissions
About this article
Cite this article
Huang, Y., Qiu, C. & Yuan, K. Surface defect saliency of magnetic tile. Vis Comput 36, 85–96 (2020). https://doi.org/10.1007/s00371-018-1588-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-018-1588-5