Skip to main content

Advertisement

SpringerLink
Log in

SEHSNet: Stage Enhancement and Hierarchical Supervision Network for edge detection

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Edge detection is a challenging low-level vision task. After a long period of development, there are already edge detectors whose edge recognition performance exceeds that of humans. However, there are still three problems in the existing modern deep CNN edge detection methods, which can be summarized as low-level under chaos, high-level over suppression, and inter-layer without interaction. For the first problem, we propose a cascade structure of dilated branches with truncated gradient flow, called the low-level proofreading module. A high-level cognition module is used to excavate the global edge cues to solve the over-suppression problem of high level. Considering the limitations of deep supervision, we create a deep hierarchical supervision strategy to facilitate inter-layer communication. We combine all the components to form our network, Stage Enhancement and Hierarchical Supervision Network. It is from the perspective of the deep edge detector, synthesizing and optimizing the advantages of previous works, and creating a new architecture to accommodate edge detection tasks. Our network outperforms the SOTA algorithms on several metrics. For example, when only using BSDS500 dataset for training, we achieved the F-measure ODS of 0.820, breaking the previous best record of 0.808.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability

Data will be made available on request.

References

  1. Ma, J., Wang, T., Li, G., Zhan, Q., Wu, D., Chang, Y., Xue, Y., Zhang, Y., Zuo, J.: Concrete surface roughness measurement method based on edge detection. The Visual Computer, 1–12 (2023)

  2. Zhao, L., Zhao, Q., Liu, H., Lv, P., Gu, D.: Structural sparse representation-based semi-supervised learning and edge detection proposal for visual tracking. Vis. Comput. 33, 1169–1184 (2017)

    Article  Google Scholar 

  3. Andersson, M., Johnsson, B., Munkberg, J., Clarberg, P., Hasselgren, J., Akenine-Möller, T.: Efficient multi-view ray tracing using edge detection and shader reuse. Vis. Comput. 27, 665–676 (2011)

    Article  Google Scholar 

  4. Gaur, D., Mehrotra, D., Singh, K.: A new approach to simulate the dynamic behavior of particulate matter using a canny edge detector embedded piv algorithm. Vis. Comput. 39(9), 4293–4304 (2023)

    Article  Google Scholar 

  5. Canny, J.: A Computational Approach to Edge Detection. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-8(6), 679–698 (1986) https://doi.org/10.1109/TPAMI.1986.4767851

  6. Konishi, S., Yuille, A.L., Coughlan, J.M., Zhu, S.C.: Statistical edge detection: learning and evaluating edge cues. IEEE Trans. Pattern Anal. Mach. Intell. 25(1), 57–74 (2003). https://doi.org/10.1109/TPAMI.2003.1159946

    Article  Google Scholar 

  7. Arbeláez, P., Maire, M., Fowlkes, C., Malik, J.: Contour Detection and Hierarchical Image Segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011). https://doi.org/10.1109/TPAMI.2010.161

    Article  PubMed  Google Scholar 

  8. Mély, D.A., Kim, J., McGill, M., Guo, Y., Serre, T.: A systematic comparison between visual cues for boundary detection. Vis. Res. 120, 93–107 (2016)

    Article  PubMed  Google Scholar 

  9. Xie, S., Tu, Z.: Holistically-Nested Edge Detection. In: ICCV, pp. 1395–1403 (2016). https://doi.org/10.1109/ICCV.2015.164

  10. Liu, Y., Cheng, M.-M., Hu, X., Wang, K., Bai, X.: Richer Convolutional Features for Edge Detection. In: CVPR, pp. 5872–5881 (2017). https://doi.org/10.1109/CVPR.2017.622

  11. He, J., Zhang, S., Yang, M., Shan, Y., Huang, T.: Bi-Directional Cascade Network for Perceptual Edge Detection. In: CVPR, pp. 3823–3832 (2019). https://doi.org/10.1109/CVPR.2019.00395

  12. Su, Z., Liu, W., Yu, Z., Hu, D., Liao, Q., Tian, Q., Pietikäinen, M., Liu, L.: Pixel Difference Networks for Efficient Edge Detection. In: ICCV, pp. 5097–5107 (2021). https://doi.org/10.1109/ICCV48922.2021.00507

  13. Zhou, C., Huang, Y., Pu, M., Guan, Q., Huang, L., Ling, H.: The treasure beneath multiple annotations: An uncertainty-aware edge detector. arXiv preprint arXiv:2303.11828 (2023)

  14. Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large-Scale Image Recognition (2014) arXiv:1409.1556

  15. Kittler, J.: On the accuracy of the Sobel edge detector. Image Vis. Comput. 1(1), 37–42 (1983). https://doi.org/10.1016/0262-8856(83)90006-9

    Article  Google Scholar 

  16. Dollár, P., Zitnick, C.L.: Fast edge detection using structured forests. IEEE Trans. Pattern Anal. Mach. Intell. 37(8), 1558–1570 (2014)

    Article  Google Scholar 

  17. Lim, J.J., Zitnick, C.L., Dollár, P.: Sketch Tokens: A Learned Mid-level Representation for Contour and Object Detection. In: CVPR, pp. 3158–3165 (2013). https://doi.org/10.1109/CVPR.2013.406

  18. Hwang, J., Liu, T.: Pixel-wise Deep Learning for Contour Detection. In: ICLR (2015)

  19. Bertasius, G., Shi, J., Torresani, L.: Deepedge: A multi-scale bifurcated deep network for top-down contour detection. In: CVPR, pp. 4380–4389 (2015)

  20. Shen, W., Wang, X., Wang, Y., Bai, X., Zhang, Z.: Deepcontour: A deep convolutional feature learned by positive-sharing loss for contour detection. In: CVPR, pp. 3982–3991 (2015)

  21. Kumawat, A., Panda, S.: A robust edge detection algorithm based on feature-based image registration (fbir) using improved canny with fuzzy logic (icwfl). Vis. Comput. 38(11), 3681–3702 (2022)

    Article  Google Scholar 

  22. Yang, W., Chen, X.-D., Wang, H., Mao, X.: Edge detection using multi-scale closest neighbor operator and grid partition. The Visual Computer, 1–18 (2023)

  23. Aboutabit, N.: A new construction of an image edge detection mask based on caputo-fabrizio fractional derivative. Vis. Comput. 37(6), 1545–1557 (2021)

    Article  Google Scholar 

  24. Wang, Y., Zhao, X., Huang, K.: Deep crisp boundaries. In: CVPR, pp. 3892–3900 (2017)

  25. Soria, X., Riba, E., Sappa, A.: Dense Extreme Inception Network: Towards a Robust CNN Model for Edge Detection. In: WACV, pp. 1912–1921 (2020). https://doi.org/10.1109/WACV45572.2020.9093290

  26. Wibisono, J., Hang, H.-M.: Fined: Fast Inference Network for Edge Detection. In: ICME, pp. 1–6 (2021). https://doi.org/10.1109/ICME51207.2021.9428230

  27. Lin, C., Zhang, Z., Hu, Y.: Bio-inspired feature enhancement network for edge detection. Appl. Intell. 52(10), 11027–11042 (2022)

    Article  Google Scholar 

  28. Pan, S., Wang, R., Lin, C.: Bio-inspired feature cascade network for edge detection. Vis. Comput. 39(9), 4149–4164 (2023)

    Article  Google Scholar 

  29. Lee, C.-Y., Xie, S., Gallagher, P., Zhang, Z., Tu, Z.: Deeply-supervised nets. In: Artificial Intelligence and Statistics, pp. 562–570 (2015)

  30. Yu, Z., Feng, C., Liu, M.-Y., Ramalingam, S.: Casenet: Deep Category-Aware Semantic Edge Detection. In: CVPR, pp. 1761–1770 (2017). https://doi.org/10.1109/CVPR.2017.191

  31. Hu, Y., Chen, Y., Li, X., Feng, J.: Dynamic Feature Fusion for Semantic Edge Detection. In: IJCAI, pp. 782–788 (2019). https://doi.org/10.24963/ijcai.2019/110

  32. Wang, G., Wang, X., Li, F., Liang, X.: DOOBNet: Deep Object Occlusion Boundary Detection from an Image. In: ACCV, pp. 686–702 (2019). https://doi.org/10.1007/978-3-030-20876-9_43

  33. Lu, R., Xue, F., Zhou, M., Ming, A., Zhou, Y.: Occlusion-Shared and Feature-Separated Network for Occlusion Relationship Reasoning. In: ICCV, pp. 10342–10351 (2019). https://doi.org/10.1109/ICCV.2019.01044

  34. Pu, M., Huang, Y., Guan, Q., Ling, H.: RINDNet: Edge Detection for Discontinuity in Reflectance, Illumination, Normal and Depth. In: ICCV, pp. 6859–6868 (2021). https://doi.org/10.1109/ICCV48922.2021.00680

  35. Pu, M., Huang, Y., Liu, Y., Guan, Q., Ling, H.: EDTER: Edge Detection with Transformer. In: CVPR, pp. 1402–1412 (2022)

  36. Wang, C.-Y., Liao, H.-Y.M., Wu, Y.-H., Chen, P.-Y., Hsieh, J.-W., Yeh, I.-H.: CSPNet: A new backbone that can enhance learning capability of CNN. In: CVPR, pp. 1571–1580 (2020)

  37. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017)

    Article  PubMed  Google Scholar 

  38. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. CoRR (2020)

  39. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin transformer: Hierarchical vision transformer using shifted windows. In: ICCV, pp. 10012–10022 (2021)

  40. Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., Shao, L.: Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In: ICCV, pp. 568–578 (2021)

  41. Li, Z., Peng, C., Yu, G., Zhang, X., Deng, Y., Sun, J.: Detnet: Design backbone for object detection. In: ECCV, pp. 334–350 (2018)

  42. Chen, Q., Wang, Y., Yang, T., Zhang, X., Cheng, J., Sun, J.: You only look one-level feature. In: CVPR, pp. 13039–13048 (2021)

  43. Huan, L., Xue, N., Zheng, X., He, W., Gong, J., Xia, G.-S.: Unmixing convolutional features for crisp edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 44(10), 6602–6609 (2021)

    Article  Google Scholar 

  44. Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal Loss for Dense Object Detection. IEEE Trans. Pattern Anal. Mach. Intell. 42(2), 318–327 (2020). https://doi.org/10.1109/TPAMI.2018.2858826

    Article  PubMed  Google Scholar 

  45. Yang, F., Zhang, L., Yu, S., Prokhorov, D., Mei, X., Ling, H.: Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection. IEEE Trans. Intell. Transp. Syst. 21(4), 1525–1535 (2020). https://doi.org/10.1109/TITS.2019.2910595

    Article  Google Scholar 

  46. Wang, X., Yu, K., Dong, C., Loy, C.C.: Recovering realistic texture in image super-resolution by deep spatial feature transform. In: CVPR, pp. 606–615 (2018)

  47. Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from rgbd images. In: ECCV, pp. 746–760 (2012)

  48. Deng, R., Liu, S.: Deep structural contour detection. In: ACMMM, pp. 304–312 (2020)

  49. Adam, P., Sam, G., Soumith, C., Gregory, C., Edward, Y., Zachary, D., Zeming, L., Alban, D., Luca, A., Adam, L.: Automatic differentiation in pytorch. In: NIPS (2017)

  50. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: A large-scale hierarchical image database. In: CVPR, pp. 248–255 (2009). https://doi.org/10.1109/CVPR.2009.5206848

  51. Ren, X., Bo, L.: Discriminatively trained sparse code gradients for contour detection. In: NIPS, pp. 584–592 (2012)

  52. Isola, P., Zoran, D., Krishnan, D., Adelson, E.H.: Crisp boundary detection using pointwise mutual information. In: ECCV, pp. 799–814 (2014)

  53. Hallman, S., Fowlkes, C.C.: Oriented edge forests for boundary detection. In: CVPR, pp. 1732–1740 (2015)

  54. Kokkinos, I.: Pushing the boundaries of boundary detection using deep learning. In: ICLR (2016)

  55. Maninis, K.-K., Pont-Tuset, J., Arbeláez, P., Gool, L.V.: Convolutional oriented boundaries. In: ECCV, pp. 580–596 (2016)

  56. Liao, Y., Fu, S., Lu, X., Zhang, C., Tang, Z.: Deep-learning-based object-level contour detection with CCG and CRF optimization. In: ICME, pp. 859–864 (2017)

  57. Xu, D., Ouyang, W., Alameda-Pineda, X., Ricci, E., Wang, X., Sebe, N.: Learning deep structured multi-scale features using attention-gated crfs for contour prediction. In: NIPS, pp. 3961–3970 (2017)

  58. Yang, J., Price, B., Cohen, S., Lee, H., Yang, M.-H.: Object contour detection with a fully convolutional encoder-decoder network. In: CVPR, pp. 193–202 (2016)

  59. Bertasius, G., Shi, J., Torresani, L.: High-for-low and low-for-high: Efficient boundary detection from deep object features and its applications to high-level vision. In: ICCV, pp. 504–512 (2015)

  60. Deng, R., Shen, C., Liu, S., Wang, H., Liu, X.: Learning to predict crisp boundaries. In: ECCV, pp. 562–578 (2018)

  61. Cao, Y.-J., Lin, C., Li, Y.-J.: Learning crisp boundaries using deep refinement network and adaptive weighting loss. IEEE Trans. Multimedia 23, 761–771 (2020)

    Article  Google Scholar 

  62. Gupta, S., Arbelaez, P., Malik, J.: Perceptual organization and recognition of indoor scenes from RGB-D images. In: CVPR, pp. 564–571 (2013)

  63. Gupta, S., Girshick, R., Arbeláez, P., Malik, J.: Learning rich features from RGB-D images for object detection and segmentation. In: ECCV, pp. 345–360 (2014)

  64. Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., Xie, S.: A convnet for the 2020s. In: CVPR, pp. 11976–11986 (2022)

Download references

Acknowledgements

The authors are grateful to the anonymous reviewers for their insightful comments which have certainly improved this paper. This research was funded by the Provincial Science and Technology Innovation Special Fund Project of Jilin Province, Grant Number 20190302026GX, Natural Science Foundation of Jilin Province, Grant Number 20200201037JC, and the Fundamental Research Funds for the Central Universities, JLU.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Jilin University, Changchun, 130012, Jilin, China

    Jianhang Zhou, Hongwei Zhao & Mingsi Sun

  2. Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012, Jilin, China

    Jianhang Zhou & Hongwei Zhao

  3. College of Electrical and Information Engineering, Jilin Agricultural Science and Technology College, Jilin, 132101, Jilin, China

    Mingsi Sun

Authors
  1. Jianhang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongwei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mingsi Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongwei Zhao.

Ethics declarations

conflict of Interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, J., Zhao, H. & Sun, M. SEHSNet: Stage Enhancement and Hierarchical Supervision Network for edge detection. Vis Comput (2024). https://doi.org/10.1007/s00371-024-03280-y

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00371-024-03280-y

Keywords

Search

Navigation