Multi-scale Improves Boundary Detection in Natural Images

  • Xiaofeng Ren
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5304)


In this work we empirically study the multi-scale boundary detection problem in natural images. We utilize local boundary cues including contrast, localization and relative contrast, and train a classifier to integrate them across scales. Our approach successfully combines strengths from both large-scale detection (robust but poor localization) and small-scale detection (detail-preserving but sensitive to clutter). We carry out quantitative evaluations on a variety of boundary and object datasets with human-marked groundtruth. We show that multi-scale boundary detection offers large improvements, ranging from 20% to 50%, over single-scale approaches. This is the first time that multi-scale is demonstrated to improve boundary detection on large datasets of natural images.


Average Precision Natural Image Boundary Detection Relative Contrast Berkeley Segmentation Dataset 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Canny, J.: A computational approach to edge detection. IEEE Trans. PAMI 8, 679–698 (1986)CrossRefGoogle Scholar
  2. 2.
    Witkin, A.: Scale-space filtering. Int’l. J. Conf. on Artificial Intell. 2, 1019–1022 (1983)Google Scholar
  3. 3.
    Koenderink, J.: The structure of images. Biological Cybernetics 50, 363–370 (1984)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Lindeberg, T.: Scale-Space Theory in Computer Vision. Kluwer Academic Publishers, Dordrecht (1994)CrossRefzbMATHGoogle Scholar
  5. 5.
    Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. PAMI 12, 629–639 (1990)CrossRefGoogle Scholar
  6. 6.
    Lindeberg, T.: Edge detection and ridge detection with automatic scale selection. Int’l. J. Comp. Vision 30, 117–156 (1998)CrossRefGoogle Scholar
  7. 7.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV, vol. II, pp. 416–423 (2001)Google Scholar
  8. 8.
    Konishi, S., Yuille, A., Coughlan, J., Zhu, S.: Fundamental bounds on edge detection: an information theoretic evaluation of different edge cues. In: CVPR, pp. 573–579 (1999)Google Scholar
  9. 9.
    Martin, D., Fowlkes, C., Malik, J.: Learning to detect natural image boundaries using local brightness, color and texture cues. IEEE Trans. PAMI 26(5), 530–549 (2004)CrossRefGoogle Scholar
  10. 10.
    Konishi, S., Yuille, A., Coughlan, J.: A statistical approach to multi-scale edge detection. In: ECCV Workshop on Generative-Model-Based Vision (2002)Google Scholar
  11. 11.
    Dollar, P., Tu, Z., Belongie, S.: Supervised learning of edges and object boundaries. In: CVPR, vol. 2, pp. 1964–1971 (2006)Google Scholar
  12. 12.
    Galun, M., Basri, R., Brandt, A.: Multiscale edge detection and fiber enhacement using differences of oriented means. In: ICCV (2007)Google Scholar
  13. 13.
    Bouman, C., Shapiro, M.: A multiscale random field model for bayesian image segmentation. IEEE Trans. Im. Proc. 3(2), 162–177 (1994)CrossRefGoogle Scholar
  14. 14.
    Koepfler, G., Lopez, C., Morel, J.: A multiscale algorithm for image segmentation by variational method. SIAM J. Numer. Anal. 31(1), 282–299 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Sharon, E., Brandt, A., Basri, R.: Segmentation and boundary detection using multiscale intensity measurements. In: CVPR (2001)Google Scholar
  16. 16.
    Ruderman, D.L., Bialek, W.: Statistics of natural images: Scaling in the woods. Physics Review Letters 73(6), 814–817 (1994)CrossRefGoogle Scholar
  17. 17.
    Elder, J., Goldberg, R.: Ecological statistics of gestalt laws for the perceptual organization of contours. Journal of Vision 2(4), 324–353 (2002)CrossRefGoogle Scholar
  18. 18.
    Ren, X., Malik, J.: A probabilistic multi-scale model for contour completion based on image statistics. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 312–327. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  19. 19.
    Bergholm, F.: Edge focusing. IEEE Trans. PAMI 9, 726–741 (1987)CrossRefGoogle Scholar
  20. 20.
    Basu, M.: Gradient-based edge detection methods - a survey. IEEE. Trans. System, Man and Cybernatics 32(3), 252–260 (2002)CrossRefGoogle Scholar
  21. 21.
    Rijsbergen, C.V.: Information Retrieval, 2nd edn. Univ. of Glasgow (1979)Google Scholar
  22. 22.
    Ren, X., Fowlkes, C., Malik, J.: Scale-invariant contour completion using conditional random fields. In: ICCV, vol. 2, pp. 1214–1221 (2005)Google Scholar
  23. 23.
    Arbelaez, P.: Boundary extraction in natural images using ultrametric contour maps. In: Workshop on Perceptual Organization in Computer Vision (POCV) (2006)Google Scholar
  24. 24.
    Felzenszwalb, P., McAllester, D.: A min-cover approach for finding salient curves. In: Workshop on Perceptual Organization in Computer Vision (POCV) (2006)Google Scholar
  25. 25.
    Zhu, Q., Song, G., Shi, J.: Untangling cycles for contour grouping. In: ICCV (2007)Google Scholar
  26. 26.
    Stein, A., Hoiem, D., Hebert, M.: Learning to find object boundaries using motion cues. In: ICCV (2007)Google Scholar
  27. 27.
    Shotton, J., Winn, J., Rother, C., Criminisi, A.: Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951. Springer, Heidelberg (2006)Google Scholar
  28. 28.
    Malisiewicz, T., Efros, A.: Improving spatial support for objects via multiple segmentations. In: BMVC (2007)Google Scholar
  29. 29.
    Everingham, M., Van Gool, L., Williams, C., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge (VOC 2007). (2007),
  30. 30.
    Russell, B., Torralba, A., Murphy, K., Freeman, W.: LabelMe: a database and web-based tool for image annotation. Technical Report AI Memo AIM-2005-025, MIT (2005)Google Scholar
  31. 31.
    Martin, D., Fowlkes, C., Walker, L., Malik, J.: Local boundary detection in natural images: Matching human and machine performance. In: ECVP (2003)Google Scholar
  32. 32.
    Yu, S.: Segmentation induced by scale invariance. In: CVPR, vol. I, pp. 445–451 (2005)Google Scholar
  33. 33.
    Estrada, F., Elder, J.: Multi-scale contour extraction based on natural image statistics. In: Workshop on Perceptual Organization in Computer Vision (POCV) (2006)Google Scholar
  34. 34.
    Hoiem, D., Efros, A., Hebert, M.: Recovering occlusion boundaries from a single image. In: ICCV (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Xiaofeng Ren
    • 1
  1. 1.Toyota Technological Institute at ChicagoChicagoUSA

Personalised recommendations