Analyzing the Subspace Structure of Related Images: Concurrent Segmentation of Image Sets

  • Lopamudra Mukherjee
  • Vikas Singh
  • Jia Xu
  • Maxwell D. Collins
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7575)


We develop new algorithms to analyze and exploit the joint subspace structure of a set of related images to facilitate the process of concurrent segmentation of a large set of images. Most existing approaches for this problem are either limited to extracting a single similar object across the given image set or do not scale well to a large number of images containing multiple objects varying at different scales. One of the goals of this paper is to show that various desirable properties of such an algorithm (ability to handle multiple images with multiple objects showing arbitary scale variations) can be cast elegantly using simple constructs from linear algebra: this significantly extends the operating range of such methods. While intuitive, this formulation leads to a hard optimization problem where one must perform the image segmentation task together with appropriate constraints which enforce desired algebraic regularity (e.g., common subspace structure). We propose efficient iterative algorithms (with small computational requirements) whose key steps reduce to objective functions solvable by max-flow and/or nearly closed form identities. We study the qualitative, theoretical, and empirical properties of the method, and present results on benchmark datasets.


Visual Word Multiple Object Sparse Representation Appearance Model Subspace Cluster 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Rother, C., Kolmogorov, V., Blake, A.: “GrabCut”: interactive foreground extraction using iterated graph cuts. ACM Trans. on Graphics 23, 309–314 (2004)CrossRefGoogle Scholar
  2. 2.
    Gehler, P.V., Nowozin, S.: On feature combination for multiclass object classification. In: ICCV (2009)Google Scholar
  3. 3.
    Rother, C., Minka, T., Blake, A., Kolmogorov, V.: Cosegmentation of image pairs by histogram matching: Incorporating a global constraint into MRFs. In: CVPR (2006)Google Scholar
  4. 4.
    Batra, D., Kowdle, A., Parikh, D., Luo, J., Chen, T.: iCoseg: Interactive co-segmentation with intelligent scribble guidance. In: CVPR (2010)Google Scholar
  5. 5.
    Chu, W.-S., Chen, C.-P., Chen, C.-S.: MOMI-Cosegmentation: Simultaneous Segmentation of Multiple Objects among Multiple Images. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part I. LNCS, vol. 6492, pp. 355–368. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  6. 6.
    Mu, Y., Zhou, B.: Co-segmentation of Image Pairs with Quadratic Global Constraint in MRFs. In: Yagi, Y., Kang, S.B., Kweon, I.S., Zha, H. (eds.) ACCV 2007, Part II. LNCS, vol. 4844, pp. 837–846. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  7. 7.
    Mukherjee, L., Singh, V., Dyer, C.R.: Half-integrality based algorithms for cosegmentation of images. In: CVPR (2009)Google Scholar
  8. 8.
    Vicente, S., Kolmogorov, V., Rother, C.: Cosegmentation Revisited: Models and Optimization. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part II. LNCS, vol. 6312, pp. 465–479. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  9. 9.
    Joulin, A., Bach, F., Ponce, J.: Discriminative clustering for image co-segmentation. In: CVPR (2010)Google Scholar
  10. 10.
    Hochbaum, D.S., Singh, V.: An efficient algorithm for co-segmentation. In: ICCV (2009)Google Scholar
  11. 11.
    Vicente, S., Kolmogorov, V., Rother, C.: Object cosegmentation. In: CVPR (2011)Google Scholar
  12. 12.
    Collins, M.D., Xu, J., Grady, L., Singh, V.: Random walks based multi-image segmentation: Quasiconvexity results and gpu-based solutions. In: CVPR (2012)Google Scholar
  13. 13.
    Mukherjee, L., Singh, V., Peng, J.: Scale invariant cosegmentation for image groups. In: CVPR (2011)Google Scholar
  14. 14.
    Chang, K., Liu, T., Lai, S.: From cosaliency to cosegmentation: An efficient and fully unsupervised energy minimization model. In: CVPR (2011)Google Scholar
  15. 15.
    Chai, Y., Lempitsky, V., Zisserman, A.: Bicos: A bi-level co-segmentation method for image classification. In: ICCV (2011)Google Scholar
  16. 16.
    Glasner, D., Vitaladevuni, S., Basri, R.: Contour based joint clustering of multiple segmentations. In: CVPR (2011)Google Scholar
  17. 17.
    Kim, G., Xing, E.P., Fei-Fei, L., Kanade, T.: Distributed cosegmentation vis submodular optimization on anisotropic diffusion. In: ICCV (2011)Google Scholar
  18. 18.
    Joulin, A., Bach, F., Ponce, J.: Multi-class cosegmentation. In: CVPR (2012)Google Scholar
  19. 19.
    Winn, J., Criminisi, A., Minka, T.: Object categorization by learned universal visual dictionary. In: ICCV (2005)Google Scholar
  20. 20.
    Liu, G., Lin, Z., Yu, Y.: Robust subspace segmentation by low-rank representation. In: ICML (2010)Google Scholar
  21. 21.
    Favaro, P., Vidal, R., Ravichandran, A.: A closed form solution to robust subspace estimation and clustering. In: CVPR (2011)Google Scholar
  22. 22.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. PAMI 23 (2001)Google Scholar
  23. 23.
    Boros, E., Hammer, P.: Pseudo-Boolean optimization. Disc. Appl. Math. 123, 155–225 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  24. 24.
    Rother, C., Kolmogorov, V., Lempitsky, V., Szummer, M.: Optimizing binary MRFs via extended roof duality. In: CVPR (2007)Google Scholar
  25. 25.
    Hochbaum, D.: Polynomial time algorithms for ratio regions and a variant of normalized cut. PAMI 32 (2010)Google Scholar
  26. 26.
    Cevher, V., Krause, A.: Greedy dictionary selection for sparse representation. IEEE J. of Selected Topics in Signal Processing 5, 979–983 (2011)CrossRefGoogle Scholar
  27. 27.
    Jenatton, R., Mairal, J., Obozinski, G., Bach, F.: Proximal methods for sparse hierarchical dictionary learning. In: ICML (2010)Google Scholar
  28. 28.
    Krause, A., Cevher, V.: Submodular dictionary selection for sparse representation. In: ICML (2010)Google Scholar
  29. 29.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? PAMI 26 (2004)Google Scholar
  30. 30.
    Narasimhan, M., Bilmes, J.: A submodular-supermodular procedure with applications to discriminative structure learning. In: UAI (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Lopamudra Mukherjee
    • 1
  • Vikas Singh
    • 2
  • Jia Xu
    • 2
  • Maxwell D. Collins
    • 2
  1. 1.University of Wisconsin-WhitewaterUSA
  2. 2.University of Wisconsin-MadisonUSA

Personalised recommendations