Advertisement

Learning to Predict Crisp Boundaries

  • Ruoxi Deng
  • Chunhua Shen
  • Shengjun LiuEmail author
  • Huibing Wang
  • Xinru Liu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11210)

Abstract

Recent methods for boundary or edge detection built on Deep Convolutional Neural Networks (CNNs) typically suffer from the issue of predicted edges being thick and need post-processing to obtain crisp boundaries. Highly imbalanced categories of boundary versus background in training data is one of main reasons for the above problem. In this work, the aim is to make CNNs produce sharp boundaries without post-processing. We introduce a novel loss for boundary detection, which is very effective for classifying imbalanced data and allows CNNs to produce crisp boundaries. Moreover, we propose an end-to-end network which adopts the bottom-up/top-down architecture to tackle the task. The proposed network effectively leverages hierarchical features and produces pixel-accurate boundary mask, which is critical to reconstruct the edge map. Our experiments illustrate that directly making crisp prediction not only promotes the visual results of CNNs, but also achieves better results against the state-of-the-art on the BSDS500 dataset (ODS F-score of .815) and the NYU Depth dataset (ODS F-score of .762).

Keywords

Edge detection Contour detection Convolutional neural networks 

Notes

Acknowledgement

This work is funded by the China Scholarship Council (Grant No. 201506370087), the National Natural Science Foundation of China (Grant No. 61572527, Grant No. 61628211, Grant No. 61602524).

References

  1. 1.
    Marr, D., Hildreth, E.: Theory of edge detection. Proc. Roy. Soc. Lond. B: Biol. Sci. 207(1167), 187–217 (1980)CrossRefGoogle Scholar
  2. 2.
    Gonzalez, R.C., Wood, R.E.: Digital Image Processing, 2nd edn. Prentice Hall, Upper Saddle. River (2014)Google Scholar
  3. 3.
    Torre, V., Poggio, T.A.: On edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 2, 147–163 (1986)CrossRefGoogle Scholar
  4. 4.
    Senthilkumaran, N., Rajesh, R.: Edge detection techniques for image segmentation-a survey of soft computing approaches. Int. J. Recent Trends Eng. 1(2), 250–254 (2009)Google Scholar
  5. 5.
    Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011)CrossRefGoogle Scholar
  6. 6.
    Chen, L.C., Barron, J.T., Papandreou, G., Murphy, K., Yuille, A.L.: Semantic image segmentation with task-specific edge detection using CNNs and a discriminatively trained domain transform. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4545–4554 (2016)Google Scholar
  7. 7.
    Bertasius, G., Shi, J., Torresani, L.: Semantic segmentation with boundary neural fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3602–3610 (2016)Google Scholar
  8. 8.
    Ren, X.: Local grouping for optical flow. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8. IEEE (2008)Google Scholar
  9. 9.
    Revaud, J., Weinzaepfel, P., Harchaoui, Z., Schmid, C.: EpicFlow: edge-preserving interpolation of correspondences for optical flow. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1164–1172 (2015)Google Scholar
  10. 10.
    Bertasius, G., Shi, J., Torresani, L.: Deepedge: A multi-scale bifurcated deep network for top-down contour detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4380–4389 (2015)Google Scholar
  11. 11.
    Shen, W., Wang, X., Wang, Y., Bai, X., Zhang, Z.: Deepcontour: a deep convolutional feature learned by positive-sharing loss for contour detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3982–3991 (2015)Google Scholar
  12. 12.
    Xie, S., Tu, Z.: Holistically-nested edge detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1395–1403 (2015)Google Scholar
  13. 13.
    Liu, Y., Cheng, M.M., Hu, X., Wang, K., Bai, X.: Richer convolutional features for edge detection. arXiv preprint arXiv:1612.02103 (2016)
  14. 14.
    Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 746–760. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33715-4_54CrossRefGoogle Scholar
  15. 15.
    Sobel, I.: Camera models and machine perception. Technical report, Department of Computer Science, Stanford University, California (1970)Google Scholar
  16. 16.
    Yu, Z., Feng, C., Liu, M.Y., Ramalingam, S.: Casenet: Deep category-aware semantic edge detection. arXiv e-Prints (2017)Google Scholar
  17. 17.
    Yang, J., Price, B., Cohen, S., Lee, H., Yang, M.H.: Object contour detection with a fully convolutional encoder-decoder network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 193–202 (2016)Google Scholar
  18. 18.
    Kittler, J.: On the accuracy of the sobel edge detector. Image Vis. Comput. 1(1), 37–42 (1983)CrossRefGoogle Scholar
  19. 19.
    Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 6, 679–698 (1986)CrossRefGoogle Scholar
  20. 20.
    Fram, J.R., Deutsch, E.S.: On the quantitative evaluation of edge detection schemes and their comparison with human performance. IEEE Trans. Comput. C-24(6), 616–628 (1975)CrossRefGoogle Scholar
  21. 21.
    Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12(7), 629–639 (1990)CrossRefGoogle Scholar
  22. 22.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Siddiqui, M., Medioni, G.: Human pose estimation from a single view point, real-time range sensor. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1–8. IEEE (2010)Google Scholar
  24. 24.
    Martin, D.R., Fowlkes, C.C., Malik, J.: Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans. Pattern Anal. Mach. Intell. 26(5), 530–549 (2004)CrossRefGoogle Scholar
  25. 25.
    Dollar, P., Tu, Z., Belongie, S.: Supervised learning of edges and object boundaries. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 1964–1971. IEEE (2006)Google Scholar
  26. 26.
    Dollár, P., Zitnick, C.L.: Fast edge detection using structured forests. IEEE Trans. Pattern Anal. Mach. Intell. 37(8), 1558–1570 (2015)CrossRefGoogle Scholar
  27. 27.
    Kokkinos, I.: Pushing the boundaries of boundary detection using deep learning. arXiv preprint arXiv:1511.07386 (2015)
  28. 28.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  29. 29.
    LeCun, Y., et al.: Handwritten digit recognition with a back-propagation network. In: Advances in Neural Information Processing Systems, pp. 396–404 (1990)Google Scholar
  30. 30.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  31. 31.
    Xie, S., Tu, Z.: Holistically-nested edge detection. Int. J. Comput. Vis. 125(1–3), 3–18 (2017)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Gu, J., Zhou, Y., Zuo, X.: Making class bias useful: a strategy of learning from imbalanced data. In: Yin, H., Tino, P., Corchado, E., Byrne, W., Yao, X. (eds.) IDEAL 2007. LNCS, vol. 4881, pp. 287–295. Springer, Heidelberg (2007).  https://doi.org/10.1007/978-3-540-77226-2_30CrossRefGoogle Scholar
  33. 33.
    Tang, L., Liu, H.: Bias analysis in text classification for highly skewed data. In: Fifth IEEE International Conference on Data Mining, 4-p. IEEE (2005)Google Scholar
  34. 34.
    Lusa, L., et al.: Class prediction for high-dimensional class-imbalanced data. BMC Bioinform. 11(1), 523 (2010)CrossRefGoogle Scholar
  35. 35.
    Haider, A.H., et al.: Unconscious race and class bias: its association with decision making by trauma and acute care surgeons. J. Trauma Acute Care Surg. 77(3), 409–416 (2014)CrossRefGoogle Scholar
  36. 36.
    Phillips, S.J., Dudík, M.: Generative and discriminative learning with unknown labeling bias. In: Advances in Neural information Processing Systems, pp. 401–408 (2009)Google Scholar
  37. 37.
    Milletari, F., Navab, N., Ahmadi, S.A.: V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571. IEEE (2016)Google Scholar
  38. 38.
    Dice, L.R.: Measures of the amount of ecologic association between species. Ecology 26(3), 297–302 (1945)CrossRefGoogle Scholar
  39. 39.
    Pinheiro, P.O., Lin, T.-Y., Collobert, R., Dollár, P.: Learning to refine object segments. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 75–91. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46448-0_5CrossRefGoogle Scholar
  40. 40.
    Xie, S., Girshick, R., Dollr, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks (2016)Google Scholar
  41. 41.
    Lin, G., Milan, A., Shen, C., Reid, I.: RefineNet: Multi-path refinement networks for high-resolution semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)Google Scholar
  42. 42.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  43. 43.
    Adam Paszke, Sam Gross, S.C., Chanan, G.: PyTorch (2017)Google Scholar
  44. 44.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014)Google Scholar
  45. 45.
    Wang, Y., Zhao, X., Huang, K.: Deep crisp boundaries. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3892–3900 (2017)Google Scholar
  46. 46.
    Isola, P., Zoran, D., Krishnan, D., Adelson, E.H.: Crisp boundary detection using pointwise mutual information. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 799–814. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10578-9_52CrossRefGoogle Scholar
  47. 47.
    Bertasius, G., Shi, J., Torresani, L.: High-for-low and low-for-high: efficient boundary detection from deep object features and its applications to high-level vision. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 504–512 (2015)Google Scholar
  48. 48.
    Mottaghi, R., et al.: The role of context for object detection and semantic segmentation in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 891–898 (2014)Google Scholar
  49. 49.
    Hallman, S., Fowlkes, C.C.: Oriented edge forests for boundary detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1732–1740 (2015)Google Scholar
  50. 50.
    Gupta, S., Arbelaez, P., Malik, J.: Perceptual organization and recognition of indoor scenes from RGB-D images. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 564–571. IEEE (2013)Google Scholar
  51. 51.
    Gupta, S., Girshick, R., Arbeláez, P., Malik, J.: Learning rich features from RGB-D images for object detection and segmentation. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 345–360. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10584-0_23CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Ruoxi Deng
    • 1
  • Chunhua Shen
    • 2
  • Shengjun Liu
    • 1
    Email author
  • Huibing Wang
    • 3
  • Xinru Liu
    • 1
  1. 1.Central South UniversityChangshaChina
  2. 2.The University of AdelaideAdelaideAustralia
  3. 3.Dalian University of TechnologyDalianChina

Personalised recommendations