Enhancing Interactive Image Segmentation with Automatic Label Set Augmentation

  • Lei Ding
  • Alper Yilmaz
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6316)


We address the problem of having insufficient labels in an interactive image segmentation framework, for which most current methods would fail without further user interaction. To minimize user interaction, we use the appearance and boundary information synergistically. Specifically, we perform distribution propagation on the image graph constructed with color features to derive an initial estimate of the segment labels. Following that, we include automatically estimated segment distributions at “critical pixels” with uncertain labels to improve the segmentation performance. Such estimation is realized by incorporating boundary information using a non-parametric Dirichlet process for modeling diffusion signatures derived from the salient boundaries. Our main contribution is fusion of image appearance with probabilistic modeling of boundary information to segment the whole-object with a limited number of labeled pixels. Our proposed framework is extensively tested on a standard dataset, and is shown to achieve promising results both quantitatively and qualitatively.


Dirichlet Process Diffusion Signature Average Error Rate Label Vertex Adaptive Thresholding 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Akaho, S.: The e-PCA and m-PCA: Dimension reduction of parameters by information geometry. In: IJCNN (2004)Google Scholar
  2. 2.
    Arbelaez, P., Cohen, L.: Constrained image segmentation from hierarchical boundaries. In: CVPR (2008)Google Scholar
  3. 3.
    Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: From contours to regions: An empirical evaluation. In: CVPR (2009)Google Scholar
  4. 4.
    Blackwell, D., MacQueen, J.B.: Ferguson distributions via polya urn scheme. Annals of Statistics 1, 353–355 (1973)zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Blake, A., Rother, C., Brown, M., Perez, P., Torr, P.: Interactive image segmentation using an adaptive GMMRF model. In: ECCV (2006)Google Scholar
  6. 6.
    Boykov, Y.Y., Jolly, M.P.: Interactive graph cuts for optimal boundary & region segmentation of objects in n-d images. In: ICCV (2001)Google Scholar
  7. 7.
    Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001) (software available online)Google Scholar
  8. 8.
    Corduneanu, A., Jaakkola, T.: Distributed information regularization on graphs. In: NIPS (2005)Google Scholar
  9. 9.
    Das, P., Veksler, O., Zavadsky, V., Boykov, Y.: Semiautomatic segmentation with compact shape prior. Image and Vision Computing 27(1-2), 206–219 (2009)CrossRefGoogle Scholar
  10. 10.
    Dempster, A., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B 39(1), 1–38 (1977)zbMATHMathSciNetGoogle Scholar
  11. 11.
    Ding, L., Yilmaz, A.: Interactive image segmentation using probabilistic hypergraphs. Pattern Recognition 43(5) (2010)Google Scholar
  12. 12.
    Duchenne, O., Audibert, J.Y., Keriven, R., Ponce, J., Segonne, F.: Segmentation by transduction. In: CVPR (2008)Google Scholar
  13. 13.
    Grady, L.: Random walks for image segmentation. IEEE PAMI 28(11), 1768–1783 (2006)Google Scholar
  14. 14.
    Guan, J., Qiu, G.: Interactive image segmentation using optimization with statistical priors. In: ECCV Workshops (2006)Google Scholar
  15. 15.
    Kass, M., Witkin, A., Terzopoulos, D.: Snakes: Active contour models. IJCV 1(4), 321–331 (1987)CrossRefGoogle Scholar
  16. 16.
    Lempitsky, V., Kohli, P., Rother, C., Sharp, T.: Image segmentation with a bounding box prior. In: ICCV (2009)Google Scholar
  17. 17.
    Maire, M., Arbelaez, P., Fowlkes, C., Malik, J.: Using contours to detect and localize junctions in natural images. In: CVPR (2008)Google Scholar
  18. 18.
    Martin, D., Fowlkes, C., Malik, J.: Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE PAMI 26(5), 530–549 (2004)Google Scholar
  19. 19.
    Mortensen, E.N., Barrett, W.A.: Intelligent scissors for image composition. In: SIGGRAPH (1995)Google Scholar
  20. 20.
    Neal, R.: Markov chain sampling methods for Dirichlet process mixture models. Journal of Computational and Graphical Statistics 9(2), 249–265 (2000)CrossRefMathSciNetGoogle Scholar
  21. 21.
    Peng, B., Veksler, O.: Parameter selection for graph cut based image segmentation. In: BMVC (2008)Google Scholar
  22. 22.
    Price, B.L., Morse, B., Cohen, S.: Geodesic graph cut for interactive image segmentation. In: CVPR (2010)Google Scholar
  23. 23.
    Ren, X., Malik, J.: Learning a classification model for segmentation. In: ICCV (2003)Google Scholar
  24. 24.
    Rother, C., Kolmogorov, V., Blake, A.: Grabcut: Interactive foreground extraction using iterated graph cuts. In: SIGGRAPH (2004)Google Scholar
  25. 25.
    Settles, B.: Active learning literature survey. Tech. rep., University of Wisconsin-Madison Computer Sciences Technical Report 1648 (2009)Google Scholar
  26. 26.
    Shahbaba, B., Neal, R.: Nonlinear models using Dirichlet process mixtures. JMLR 10, 1755–1776 (2009)MathSciNetGoogle Scholar
  27. 27.
    Stein, A.N., Stepleton, T.S., Hebert, M.: Towards unsupervised whole-object segmentation: Combining automated matting with boundary detection. In: CVPR (2008)Google Scholar
  28. 28.
    Tsuda, K.: Propagating distributions on a hypergraph by dual information regularization. In: ICML (2005)Google Scholar
  29. 29.
    Veksler, O.: Star shape prior for graph-cut image segmentation. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 454–467. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  30. 30.
    Yao, Z., Yang, X., Zhu, S.C.: Introduction to a large scale general purpose ground truth dataset: Methodology, annotation tool, and benchmarks. In: Yuille, A.L., Zhu, S.-C., Cremers, D., Wang, Y. (eds.) EMMCVPR 2007. LNCS, vol. 4679, pp. 169–183. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  31. 31.
    Zhu, X., Ghahramani, Z., Lafferty, J.: Semi-supervised learning using Gaussian fields and harmonic functions. In: ICML (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Lei Ding
    • 1
  • Alper Yilmaz
    • 1
  1. 1.Photogrammetric Computer Vision LabThe Ohio State University 

Personalised recommendations