Surface defect saliency of magnetic tile

Abstract

Computer vision builds a connection between image processing and industrials, bringing modern perception to the automated manufacture of magnetic tiles. In this article, we propose a real-time model called MCuePush U-Net, specifically designed for saliency detection of surface defect. Our model consists of three main components: MCue, U-Net and Push network. MCue generates three-channel resized inputs, including one MCue saliency image and two raw images; U-Net learns the most informative regions, and essentially it is a deep hierarchical structured convolutional network; Push network defines the specific location of predicted surface defects with bounding boxes, constructed by two fully connected layers and one output layer. We show that the model exceeds the state of the art in saliency detection of magnetic tiles, in which it both effectively and explicitly maps multiple surface defects from low-contrast images. The proposed model significantly reduces time cost of machinery from 0.5 s per image to 0.07 s and enhances detection accuracy for image-based defect examinations.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

References

  1. 1.

    Yang, C., Liu, P., Yin, G., Jiang, H., Li, X.: Defect detection in magnetic tile images based on stationary wavelet transform. NDT & E Int. 83, 78–87 (2016)

    Article  Google Scholar 

  2. 2.

    Li, X., Jiang, H., Yin, G.: Detection of surface crack defects on ferrite magnetic tile. NDT & E Int. 62, 6–13 (2014)

    Article  Google Scholar 

  3. 3.

    Xie, L., Lin, L., Yin, M., Meng, L., Yin, G.: A novel surface defect inspection algorithm for magnetic tile. Appl. Surf. Sci. 375, 118–126 (2016)

    Article  Google Scholar 

  4. 4.

    Yang, C., Liu, P., Yin, G., Wang, L.: Crack detection in magnetic tile images using nonsubsampled shearlet transform and envelope gray level gradient. Opt. Laser Technol. 90, 7–17 (2017)

    Article  Google Scholar 

  5. 5.

    Cha, Y.-J., Choi, W., Büyüköztürk, O.: Deep learning-based crack damage detection using convolutional neural networks. Comput. Aided Civ. Infrastruct. Eng. 32(5), 361–378 (2017)

    Article  Google Scholar 

  6. 6.

    Masci, J., Meier, U., Ciresan, D., Schmidhuber, J., Fricout, G.: Steel defect classification with max-pooling convolutional neural networks. In: The 2012 International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE (2012)

  7. 7.

    Soukup, D., Huber-Mörk, R.: Convolutional neural networks for steel surface defect detection from photometric stereo images. In: International Symposium on Visual Computing, pp. 668–677. Springer (2014)

  8. 8.

    Ren, R., Hung, T., Tan, K.C.: A generic deep-learning-based approach for automated surface inspection. IEEE Trans. Cybern. 48(3), 929–940 (2018)

    Article  Google Scholar 

  9. 9.

    Yang, C., Pu, J., Dong, Y., Xie, G.S., Si, Y., Liu, Z.: Scene classification-oriented saliency detection via the modularized prescription. Vis. Comput. 1, 1–16 (2018)

    Google Scholar 

  10. 10.

    Yang, Z., Xiong, H.: Computing object-based saliency via locality-constrained linear coding and conditional random fields. Vis. Comput. 33(11), 1403–1413 (2017)

    Article  Google Scholar 

  11. 11.

    Zhou, X., Wang, Y., Zhu, Q., Xiao, C., Xiao, L.: Ssg: superpixel segmentation and grabcut-based salient object segmentation. Vis. Comput. 11, 1–14 (2018)

    Google Scholar 

  12. 12.

    Gorji, S., Clark, J.J.: Attentional push: a deep convolutional network for augmenting image salience with shared attention modeling in social scenes. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3472–3481, (2017)

  13. 13.

    Ramanishka, V., Das, A., Zhang, J., Saenko, K.: Top-down visual saliency guided by captions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, p. 7 (2017)

  14. 14.

    Zhang, J., Sclaroff, S.: Saliency detection: a boolean map approach. In: IEEE International Conference on Computer Vision, pp. 153–160 (2013)

  15. 15.

    Donoser, M., Urschler, M., Hirzer, M., Bischof, H.: Saliency driven total variation segmentation. In: IEEE International Conference on Computer Vision, pp. 817–824 (2009)

  16. 16.

    Cheng, M.M., Warrell, J., Lin, W.Y., Zheng, S., Vineet, V., Crook, N.: Efficient salient region detection with soft image abstraction. In: IEEE International Conference on Computer Vision, pp. 1529–1536 (2013)

  17. 17.

    Ran, M., Zelnik-Manor, L., Tal, A.: Saliency for image manipulation. Vis. Comput. 29(5), 381–392 (2013)

    Article  Google Scholar 

  18. 18.

    Gao, Y., Wang, M., Tao, D., Ji, R., Dai, Q.: 3-d object retrieval and recognition with hypergraph analysis. IEEE Trans. Image Process. 21(9), 4290–4303 (2012)

    MathSciNet  Article  Google Scholar 

  19. 19.

    Yang, B., Li, F.W., Wang, X., Mingliang, X., Liang, X., Jiang, Z., Jiang, Y.: Visual saliency guided textured model simplification. Vis. Comput. 32(11), 1415–1432 (2016)

    Article  Google Scholar 

  20. 20.

    Tang, Y., Tong, R., Tang, M., Zhang, Y.: Depth incorporating with color improves salient object detection. Vis. Comput. 32(1), 111–121 (2016)

    Article  Google Scholar 

  21. 21.

    Kapoor, A., Biswas, K.K., Hanmandlu, M.: An evolutionary learning based fuzzy theoretic approach for salient object detection. Vis. Comput. 33(5), 1–21 (2016)

    Google Scholar 

  22. 22.

    Borji, A., Cheng, M.M., Jiang, H., Li, J.: Salient object detection: a benchmark. IEEE Trans. Image Process. 24(12), 5706–5722 (2015)

    MathSciNet  Article  Google Scholar 

  23. 23.

    Thur15000. http://mmcheng.net/gsal/. Accessed 26 Mar 2018

  24. 24.

    Yan, Q., Xu, L., Shi, J., Jia, J.: Hierarchical saliency detection. In: Computer Vision and Pattern Recognition, pp. 1155–1162 (2013)

  25. 25.

    Borji, A.: What is a salient object? a dataset and a baseline model for salient object detection. IEEE Trans. Image Process. A Publ. IEEE Signal Process. Soc. 24(2), 742 (2015)

    MathSciNet  Article  Google Scholar 

  26. 26.

    Yang, C., Zhang, L., Lu, H., Xiang, R., Yang, M.H.: Saliency detection via graph-based manifold ranking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3166–3173 (2013)

  27. 27.

    Alpert, S., Galun, M., Brandt, A., Basri, R.: Image segmentation by probabilistic bottom-up aggregation and cue integration. IEEE Trans. Pattern Anal. Mach. Intell. 34(2), 315–26 (2012)

    Article  Google Scholar 

  28. 28.

    Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2009. pp. 1597–1604 (2009)

  29. 29.

    Achanta, R., Ssstrunk, S.: Saliency detection using maximum symmetric surround. In: IEEE International Conference on Image Processing, pp. 2653–2656 (2010)

  30. 30.

    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. 20(11), 1254–1259 (1998)

    Google Scholar 

  31. 31.

    Song, K.C., Hu, S.P., Yan, Y.H., Li, J.: Surface defect detection method using saliency linear scanning morphology for silicon steel strip under oil pollution interference. ISIJ Int. 54(11), 2598–2607 (2014)

    Article  Google Scholar 

  32. 32.

    Song, K., Yan, Y.: Micro surface defect detection method for silicon steel strip based on saliency convex active contour model. Math. Probl. Eng. 2013(8), 1–13 (2013)

    Google Scholar 

  33. 33.

    Guan, S.: Strip steel defect detection based on saliency map construction using gaussian pyramid decomposition. Trans. Iron Steel Inst. Jpn. 55(9), 1950–1955 (2015)

    Article  Google Scholar 

  34. 34.

    Achanta, R., Estrada, F., Wils, P., Ssstrunk, S.: Salient region detection and segmentation. In: International Conference on Computer Vision Systems, pp. 66–75 (2008)

  35. 35.

    Gharsallah, M.B., Braiek, E.B.: Weld inspection based on radiography image segmentation with level set active contour guided off-center saliency map. Adv. Mater. Sci. Eng. 2015(11), 1–10 (2015)

    Article  Google Scholar 

  36. 36.

    Aiger, D., Talbot, H.: The phase only transform for unsupervised surface defect detection. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 295–302. IEEE (2010)

  37. 37.

    Bai, X., Fang, Y., Lin, W., Wang, L.: Saliency-based defect detection in industrial images by using phase spectrum. IEEE Trans. Ind. Inform. 10(4), 2135–2145 (2014)

    Article  Google Scholar 

  38. 38.

    He, S., Jiao, J., Zhang, X., Han, G., Lau, R.W.H.: Delving into salient object subitizing and detection. In: IEEE International Conference on Computer Vision, pp. 1059–1067 (2017)

  39. 39.

    Zhang, P., Wang, D., Lu, H., Wang, H., Xiang, R.: Amulet: Aggregating multi-level convolutional features for salient object detection. In: IEEE International Conference on Computer Vision, pp. 202–211 (2017)

  40. 40.

    Zhang, P., Wang, D., Lu, H., Wang, H., Yin, B.: Learning uncertain convolutional features for accurate saliency detection. In: IEEE International Conference on Computer Vision, pp. 212–221 (2017)

  41. 41.

    Wang, T., Borji, A., Zhang, L., Zhang, P., Lu, H.: A stagewise refinement model for detecting salient objects in images. In: IEEE International Conference on Computer Vision, pp. 4039–4048 (2017)

  42. 42.

    Oh, S.J., Benenson, R., Khoreva, A., Akata, Z., Fritz, M., Schiele, B.: Exploiting saliency for object segmentation from image level labels. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4410–4419 (2017)

  43. 43.

    Simon, M., Yang, G., Darrell, T., Denzler, J., Rodner, E.: Generalized orderless pooling performs implicit salient matching. In: IEEE International Conference on Computer Vision, pp. 4970–4979 (2017)

  44. 44.

    Jonathan, L., Evan, S., Trevor, D.: Fully convolutional networks for semantic segmentation. IEEE Conference on Computer Vision and Pattern Recognition. CVPR, pp. 3431–3440. IEEE, Boston, MA, USA (2015)

  45. 45.

    Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomedical Image Segmentation. Springer International Publishing, Berlin (2015)

    Google Scholar 

  46. 46.

    Zhai, Y., Shah, M.: Visual attention detection in video sequences using spatiotemporal cues. In: ACM International Conference on Multimedia, pp. 815–824 (2006)

  47. 47.

    Cheng, M.M., Zhang, G.X., Mitra, N.J., Huang, X., Hu, S.M.: Global contrast based salient region detection. In: Computer Vision and Pattern Recognition, pp. 409–416 (2011)

  48. 48.

    Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: IEEE Conference on Computer Vision and Pattern Recognition. CVPR ’07, pp. 1–8 (2007)

  49. 49.

    Rudinac, M., Jonker, P.P.: Saliency detection and object localization in indoor environments. In: International Conference on Pattern Recognition, pp. 404–407 (2010)

  50. 50.

    Krahenbuhl, P.: Saliency filters: contrast based filtering for salient region detection. In: Computer Vision and Pattern Recognition, pp. 733–740 (2012)

  51. 51.

    Zhang, J., Sclaroff, S., Lin, Z., Shen, X., Price, B., Mech, R.: Minimum barrier salient object detection at 80 fps. In: IEEE International Conference on Computer Vision, pp. 1404–1412 (2015)

  52. 52.

    Harris, C.: A combined corner and edge detector. Proc. Alvey Vis. Conf. 1988(3), 147–151 (1988)

    Google Scholar 

  53. 53.

    Schaefer, S., Mcphail, T., Warren, J.: Image deformation using moving least squares. In: ACM SIGGRAPH, pp. 533–540 (2006)

  54. 54.

    Zhang, M., Li, X., Xu, M., Li, Q.: Image segmentation and classification for sickle cell disease using deformable u-net (2017). arXiv preprint arXiv:1710.08149

  55. 55.

    Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., Wei, Y.: Deformable convolutional networks. CoRR, abs/1703.06211 1(2), 3 (2017)

  56. 56.

    Zhu, W., Liang, S., Wei, Y., Sun, J.: Saliency optimization from robust background detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2814–2821 (2014)

  57. 57.

    Tavakoli, H.R., Rahtu, E.: Fast and efficient saliency detection using sparse sampling and kernel density estimation. In: Scandinavian Conference on Image Analysis, pp. 666–675 (2011)

  58. 58.

    Jiang, B., Zhang, L., Lu, H., Yang, C., Yang, M.H.: Saliency detection via absorbing Markov chain. In: IEEE International Conference on Computer Vision, pp. 1665–1672 (2013)

  59. 59.

    Wang, J., Jiang, H., Yuan, Z., Cheng, M.M., Hu, X., Zheng, N.: Salient object detection: a discriminative regional feature integration approach. Int. J. Comput. Vis. 123(2), 251–268 (2017)

    Article  Google Scholar 

Download references

Acknowledgements

This study is supported by National Natural Science Foundation of China (Grant No. 61421004).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Yibin Huang.

Ethics declarations

Conflict of interest

Authors Yibin Huang, Congying Qiu, Maria Feng and Kui Yuan declare that they have no conflict of interest.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Huang, Y., Qiu, C. & Yuan, K. Surface defect saliency of magnetic tile. Vis Comput 36, 85–96 (2020). https://doi.org/10.1007/s00371-018-1588-5

Download citation

Keywords

  • Saliency detection
  • Surface defect
  • Convolutional network