Advertisement

Deep Hough Transform for Semantic Line Detection

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12354)

Abstract

In this paper, we put forward a simple yet effective method to detect meaningful straight lines, a.k.a. semantic lines, in given scenes. Prior methods take line detection as a special case of object detection, while neglect the inherent characteristics of lines, leading to less efficient and suboptimal results. We propose a one-shot end-to-end framework by incorporating the classical Hough transform into deeply learned representations. By parameterizing lines with slopes and biases, we perform Hough transform to translate deep representations to the parametric space and then directly detect lines in the parametric space. More concretely, we aggregate features along candidate lines on the feature map plane and then assign the aggregated features to corresponding locations in the parametric domain. Consequently, the problem of detecting semantic lines in the spatial domain is transformed to spotting individual points in the parametric domain, making the post-processing steps, i.e. non-maximal suppression, more efficient. Furthermore, our method makes it easy to extract contextual line features, that are critical to accurate line detection. Experimental results on a public dataset demonstrate the advantages of our method over state-of-the-arts. Codes are available at https://mmcheng.net/dhtline/.

Keywords

Straight line detection Hough transform CNN 

Notes

Acknowledgements

This research was supported by Major Project for New Generation of AI under Grant No. 2018AAA0100400, NSFC (61922046), Tianjin Natural Science Foundation (18ZXZNGX00110), and the Fundamental Research Funds for the Central Universities (Nankai University: 63201169).

Supplementary material

504446_1_En_15_MOESM1_ESM.pdf (2.2 mb)
Supplementary material 1 (pdf 2233 KB)

References

  1. 1.
    Aggarwal, N., Karl, W.C.: Line detection in images through regularized hough transform. IEEE Trans. Image Process. 15(3), 582–591 (2006)CrossRefGoogle Scholar
  2. 2.
    Akinlar, C., Topal, C.: Edlines: a real-time line segment detector with a false detection control. Pattern Recogn. Lett. 32(13), 1633–1642 (2011)CrossRefGoogle Scholar
  3. 3.
    Ballard, D.: Generating the hough transform to detect arbitary shapes. Pattern Recogn. 13(2) (1981)Google Scholar
  4. 4.
    Borji, A., Cheng, M.M., Hou, Q., Jiang, H., Li, J.: Salient object detection: a survey. Comput. Vis. Media 5(2), 117–150 (2019).  https://doi.org/10.1007/s41095-019-0149-9CrossRefGoogle Scholar
  5. 5.
    Burns, J.B., Hanson, A.R., Riseman, E.M.: Extracting straight lines. IEEE Trans. Pattern Anal. Mach. Intell. PAMI–8(4), 425–455 (1986)CrossRefGoogle Scholar
  6. 6.
    Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. PAMI–8(6), 679–698 (1986)CrossRefGoogle Scholar
  7. 7.
    Caplin, S.: Art and Design in Photoshop. Elsevier/Focal (2008)Google Scholar
  8. 8.
    Chan, T., Yip, R.K.: Line detection algorithm. In: Proceedings of 13th International Conference on Pattern Recognition, vol. 2, pp. 126–130. IEEE (1996)Google Scholar
  9. 9.
    Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 801–818 (2018)Google Scholar
  10. 10.
    Cheng, Z.Q., Li, J.X., Dai, Q., Wu, X., Hauptmann, A.G.: Learning spatial awareness to improve crowd counting. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 6152–6161 (2019)Google Scholar
  11. 11.
    Duda, R.O., Hart, P.E.: Use of the hough transformation to detect lines and curves in pictures. Technical report, Sri International Menlo Park Ca Artificial Intelligence Center (1971)Google Scholar
  12. 12.
    Etemadi, A.: Robust segmentation of edge data. In: 1992 International Conference on Image Processing and its Applications, pp. 311–314. IET (1992)Google Scholar
  13. 13.
    Fan, D.P., Lin, Z., Zhang, Z., Zhu, M., Cheng, M.M.: Rethinking RGB-D salient object detection: models, datasets, and large-scale benchmarks. IEEE TNNLS (2020)Google Scholar
  14. 14.
    Fan, D.-P., Zhai, Y., Borji, A., Yang, J., Shao, L.: BBS-Net: RGB-D salient object detection with a bifurcated backbone strategy network. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12357, pp. 275–292. Springer, Cham (2020).  https://doi.org/10.1007/978-3-030-58610-2_17CrossRefGoogle Scholar
  15. 15.
    Fan, R., Cheng, M.M., Hou, Q., Mu, T.J., Wang, J., Hu, S.M.: S4Net: single stage salient-instance segmentation. Comput. Vis. Media 6(2), 191–204 (2020).  https://doi.org/10.1007/s41095-020-0173-9CrossRefGoogle Scholar
  16. 16.
    Fernandes, L.A., Oliveira, M.M.: Real-time line detection through an improved hough transform voting scheme. Pattern Recogn. 41(1), 299–314 (2008)CrossRefGoogle Scholar
  17. 17.
    Gao, S.H., Cheng, M.M., Zhao, K., Zhang, X.Y., Yang, M.H., Torr, P.: Res2Net: a new multi-scale backbone architecture. IEEE Trans. Pattern Anal. Mach. Intell. 1 (2020)Google Scholar
  18. 18.
    Gao, S.H., Tan, Y.Q., Cheng, M.M., Lu, C., Chen, Y., Yan, S.: Highly efficient salient object detection with 100k parameters. In: European Conference on Computer Vision (ECCV) (2020)Google Scholar
  19. 19.
    Girshick, R.: Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)Google Scholar
  20. 20.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  21. 21.
    Hou, Q., Cheng, M.M., Hu, X., Borji, A., Tu, Z., Torr, P.: Deeply supervised salient object detection with short connections. IEEE TPAMI 41(4), 815–828 (2019).  https://doi.org/10.1109/TPAMI.2018.2815688CrossRefGoogle Scholar
  22. 22.
    Hough, P.V.: Method and means for recognizing complex patterns. US Patent 3,069,654 (1962)Google Scholar
  23. 23.
    Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., Liu, W.: CCNet: criss-cross attention for semantic segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 603–612 (2019)Google Scholar
  24. 24.
    Illingworth, J., Kittler, J.: The adaptive hough transform. IEEE Trans. Pattern Anal. Mach. Intell. PAMI–9(5), 690–698 (1987)CrossRefGoogle Scholar
  25. 25.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  26. 26.
    Kiryati, N., Eldar, Y., Bruckstein, A.M.: A probabilistic hough transform. Pattern Recogn. 24(4), 303–316 (1991)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Krages, B.: Photography: The Art of Composition. Simon and Schuster, New York (2012)Google Scholar
  28. 28.
    Law, H., Deng, J.: CornerNet: detecting objects as paired keypoints. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 734–750 (2018)Google Scholar
  29. 29.
    Lee, J.T., Kim, H.U., Lee, C., Kim, C.S.: Semantic line detection and its applications. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3229–3237 (2017)Google Scholar
  30. 30.
    Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125 (2017)Google Scholar
  31. 31.
    Liu, L., Chen, R., Wolf, L., Cohen-Or, D.: Optimizing photo composition. Comput. Graph. Forum 29(2), 469–478 (2010)CrossRefGoogle Scholar
  32. 32.
    Liu, W., Salzmann, M., Fua, P.: Context-aware crowd counting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5099–5108 (2019)Google Scholar
  33. 33.
    Liu, Y., et al.: Richer convolutional features for edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 41(8), 1939–1946 (2019).  https://doi.org/10.1109/TPAMI.2018.2878849CrossRefGoogle Scholar
  34. 34.
    Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, pp. 8024–8035 (2019)Google Scholar
  35. 35.
    Princen, J., Illingworth, J., Kittler, J.: A hierarchical approach to line extraction based on the hough transform. Comput. Vis. Graph. Image Process. 52(1), 57–77 (1990)CrossRefGoogle Scholar
  36. 36.
    Qi, C.R., Chen, X., Litany, O., Guibas, L.J.: ImvoteNet: boosting 3D object detection in point clouds with image votes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4404–4413 (2020)Google Scholar
  37. 37.
    Qi, C.R., Litany, O., He, K., Guibas, L.J.: Deep hough voting for 3D object detection in point clouds. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 9277–9286 (2019)Google Scholar
  38. 38.
    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)Google Scholar
  39. 39.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)Google Scholar
  40. 40.
    Sobel, I.: An isotropic 3 \(\times \) 3 image gradient operator. Presentation at Stanford A.I. Project 1968, February 2014Google Scholar
  41. 41.
    Tan, Y.Q., Gao, S., Li, X.Y., Cheng, M.M., Ren, B.: Vecroad: Point-based iterative graph exploration for road graphs extraction. In: IEEE CVPR (2020)Google Scholar
  42. 42.
    Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794–7803 (2018)Google Scholar
  43. 43.
    Xie, S., Tu, Z.: Holistically-nested edge detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1395–1403 (2015)Google Scholar
  44. 44.
    Yacoub, S.B., Jolion, J.M.: Hierarchical line extraction. IEE Proc.-Vis. Image Signal Process. 142(1), 7–14 (1995)CrossRefGoogle Scholar
  45. 45.
    Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122 (2015)
  46. 46.
    Zhang, Z., et al.: PPGnet: learning point-pair graph for line segment detection. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, 16–20 June 2019 (2019)Google Scholar
  47. 47.
    Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2881–2890 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.TKLNDST, CSNankai UniversityTianjinChina

Personalised recommendations