Co-Sparse Textural Similarity for Interactive Segmentation

  • Claudia Nieuwenhuis
  • Simon Hawe
  • Martin Kleinsteuber
  • Daniel Cremers
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8694)


We propose an algorithm for segmenting natural images based on texture and color information, which leverages the co-sparse analysis model for image segmentation. As a key ingredient of this method, we introduce a novel textural similarity measure, which builds upon the co-sparse representation of image patches. We propose a statistical MAP inference approach to merge textural similarity with information about color and location. Combined with recently developed convex multilabel optimization methods this leads to an efficient algorithm for interactive segmentation, which is easily parallelized on graphics hardware. The provided approach outperforms state-of-the-art interactive segmentation methods on the Graz Benchmark.


Image Segmentation Natural Image Image Patch Textural Similarity Graphic Hardware 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Boykov, Y., Jolly, M.: Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images. In: IEEE Int. Conf. on Computer Vision (2001)Google Scholar
  2. 2.
    Brox, T., Cremers, D.: On local region models and a statistical interpretation of the piecewise smooth mumford-shah functional. Int. J. of Computer Vision 84, 184–193 (2009)CrossRefGoogle Scholar
  3. 3.
    Chambolle, A., Cremers, D., Pock, T.: A convex approach for computing minimal partitions. Tech. rep., TR-2008-05, University of Bonn, Germany (2008)Google Scholar
  4. 4.
    Chan, T., Esedoḡlu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. SIAM Journal on Applied Mathematics 66(5), 1632–1648 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Chen, Y., Ranftl, R., Pock, T.: Insights into analysis operator learning: From patch-based sparse models to higher order MRFs. IEEE Trans. on Image Processing 23, 1060–1072 (2014)CrossRefMathSciNetGoogle Scholar
  6. 6.
    Dice, L.: Measures of the amount of ecologic association between species. Ecology 26, 297–302 (1945)CrossRefGoogle Scholar
  7. 7.
    Elad, M., Milanfar, P., Rubinstein, R.: Analysis versus synthesis in signal priors. Inverse Problems 3(3), 947–968 (2007)CrossRefMathSciNetGoogle Scholar
  8. 8.
    Federer, H.: Geometric Measure Theory. Springer (1996)Google Scholar
  9. 9.
    Grady, L.: Random walks for image segmentation. IEEE Trans. on Pattern Analysis and Machine Intelligence 28(11), 1768–1783 (2006)CrossRefGoogle Scholar
  10. 10.
    Hawe, S., Kleinsteuber, M., Diepold, K.: Analysis Operator Learning and Its Application to Image Reconstruction. IEEE Trans. on Image Processing 22(6), 2138–2150 (2013)CrossRefMathSciNetGoogle Scholar
  11. 11.
    Kiechle, M., Hawe, S., Kleinsteuber, M.: A joint intensity and depth co-sparse analysis model for depth map super-resolution. In: IEEE Int. Conf. on Computer Vision (2013)Google Scholar
  12. 12.
    Lellmann, J., Kappes, J., Yuan, J., Becker, F., Schnörr, C.: Convex multiclass image labeling by simplex-constrained total variation. Tech. rep., HCI, IWR, University of Heidelberg (2008)Google Scholar
  13. 13.
    Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A.: Discriminative learned dictionaries for local image analysis. In: Int. Conf. on Computer Vision and Pattern Recognition, pp. 1–8 (2008)Google Scholar
  14. 14.
    Michelot, C.: A finite algorithm for finding the projection of a point onto the canonical simplex of R n. Journal of Optimization Theory and Applications 50(1), 195–200 (1986)CrossRefzbMATHMathSciNetGoogle Scholar
  15. 15.
    Mignotte, M.: MDS-based segmentation model for the fusion of contour and texture cues in natural images. Computer Vision and Image Understanding (2012)Google Scholar
  16. 16.
    Mobahi, H., Rao, S., Yang, A., Sastry, S., Ma, Y.: Segmentation of natural images by texture and boundary compression. Int. J. of Computer Vision 95 (2011)Google Scholar
  17. 17.
    Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Communications on Pure and Applied Mathematics 42, 577–685 (1989)CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Nam, S., Davies, M.E., Elad, M., Gribonval, R.: The Cosparse Analysis Model and Algorithms. Applied and Computational Harmonic Analysis 34(1), 30–56 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  19. 19.
    Nieuwenhuis, C., Cremers, D.: Spatially varying color distributions for interactive multi-label segmentation. IEEE Trans. on Patt. Anal. and Mach. Intell. 35(5), 1234–1247 (2013)CrossRefGoogle Scholar
  20. 20.
    Nieuwenhuis, C., Toeppe, E., Cremers, D.: A survey and comparison of discrete and continuous multi-label optimization approaches for the Potts Model. Int. J. of Computer Vision 104(3), 223–240 (2013)CrossRefGoogle Scholar
  21. 21.
    Parzen, E.: On the estimation of a probability density function and the mode. Annals of Mathematical Statistics (1962)Google Scholar
  22. 22.
    Pock, T., Cremers, D., Bischof, H., Chambolle, A.: An algorithm for minimizing the piecewise smooth Mumford-Shah functional. In: IEEE Int. Conf. on Computer Vision (2009)Google Scholar
  23. 23.
    Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In: IEEE Int. Conf. on Computer Vision, pp. 1762–1769 (2011)Google Scholar
  24. 24.
    Ravishankar, S., Bresler, Y.: Learning Sparsifying Transforms. IEEE Transactions on Signal Processing 61(5), 1072–1086 (2013)CrossRefMathSciNetGoogle Scholar
  25. 25.
    Rother, C., Kolmogorov, V., Blake, A.: GrabCut: interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics (Proc. SIGGRAPH) 23(3), 309–314 (2004)CrossRefGoogle Scholar
  26. 26.
    Santner, J., Pock, T., Bischof, H.: Interactive multi-label segmentation. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part I. LNCS, vol. 6492, pp. 397–410. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  27. 27.
    Santner, J., Unger, M., Pock, T., Leistner, C., Saffari, A., Bischof, H.: Interactive texture segmentation using random forests and total variation. In: British Machine Vision Conference (2009)Google Scholar
  28. 28.
    Tai, Y., Jia, J., Tang, C.: Soft color segmentation and its applications. IEEE Trans. on Patt. Anal. and Mach. Intell. 29(9), 1520–1537 (2007)CrossRefGoogle Scholar
  29. 29.
    Tran, T.: Combining color and texture for a robust interactive segmentation algorithm. In: IEEE Int. Conf. Comp. and Comm. Techn., Research, Innov. and Vision for the Future (2010)Google Scholar
  30. 30.
    Unger, M., Pock, T., Cremers, D., Bischof, H.: TVSeg - interactive total variation based image segmentation. In: British Machine Vision Conference (2008)Google Scholar
  31. 31.
    Xiang, S., Nie, F., Zhang, C.: Texture image segmentation: An interactive framework based on adaptive features and transductive learning. In: Narayanan, P.J., Nayar, S.K., Shum, H.-Y. (eds.) ACCV 2006. LNCS, vol. 3851, pp. 216–225. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  32. 32.
    Yaghoobi, M., Nam, S., Gribonval, R., Davies, M.E.: Constrained Overcomplete Analysis Operator Learning for Cosparse Signal Modelling. IEEE Transactions on Signal Processing 61(9), 2341–2355 (2013)CrossRefGoogle Scholar
  33. 33.
    Yang, A., Wright, J., Ma, Y., Sastry, S.: Unsupervised segmentation of natural images via lossy data compression. Computer Vision and Image Understanding (2008)Google Scholar
  34. 34.
    Zach, C., Gallup, D., Frahm, J.M., Niethammer, M.: Fast global labeling for real-time stereo using multiple plane sweeps. In: Vision, Modeling and Visualization Workshop (VMV) (2008)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Claudia Nieuwenhuis
    • 1
  • Simon Hawe
    • 2
  • Martin Kleinsteuber
    • 2
  • Daniel Cremers
    • 2
  1. 1.UC BerkeleyUSA
  2. 2.Technische Universität MünchenGermany

Personalised recommendations