Image Clustering with Metric, Local Linear Structure, and Affine Symmetry

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3021)


This paper addresses the problem of clustering images of objects seen from different viewpoints. That is, given an unlabelled set of images of n objects, we seek an unsupervised algorithm that can group the images into n disjoint subsets such that each subset only contains images of a single object. We formulate this clustering problem under a very broad geometric framework. The theme is the interplay between the geometry of appearance manifolds and the symmetry of the 2D affine group. Specifically, we identify three important notions for image clustering: the L 2 distance metric of the image space, the local linear structure of the appearance manifolds, and the action of the 2D affine group in the image space. Based on these notions, we propose a new image clustering algorithm. In a broad outline, the algorithm uses the metric to determine a neighborhood structure in the image space for each input image. Using local linear structure, comparisons (affinities) between images are computed only among the neighbors. These local comparisons are agglomerated into an affinity matrix, and a spectral clustering algorithm is used to yield the final clustering result. The technical part of the algorithm is to make all of these compatible with the action of the 2D affine group. Using human face images and images from the COIL database, we demonstrate experimentally that our algorithm is effective in clustering images (according to ojbect identity) where there is a large range of pose variation.


Cluster Algorithm Cluster Result Image Space Quotient Space Spectral Cluster 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Li, S., Lv, X., Zhang, H.: View-based clustering of object appearances based on independent subspace analysis. In: Proceedings of IEEE International Conference on Computer Vision, pp. 295–300 (2001)Google Scholar
  2. 2.
    Saux, B.L., Boujemaa, N.: Unsupervised robust clustering for image database categorization. In: International Conference on Pattern Recognition, vol. 1, pp. 259–262 (2002)Google Scholar
  3. 3.
    Frigui, H., Boujemaa, N., Lim, S.: Unsupervised clustering and feature discrimination with application to image database categorization. In: Joint 9th IFSA World Congress and 20th NAFIPS Conference (2001)Google Scholar
  4. 4.
    Basri, R., Roth, D., Jacobs, D.: Clustering appearances of 3D objects. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 414–420 (1998)Google Scholar
  5. 5.
    Fitzgibbon, A.W., Zisserman, A.: On affine invariant clustering and automatic cast listing in movies. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 304–320. Springer, Heidelberg (2002)Google Scholar
  6. 6.
    Murase, H., Nayar, S.K.: Visual learning and recognition of 3-D objects from appearance. International Journal of Computer Vision 14, 5–24 (1995)CrossRefGoogle Scholar
  7. 7.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 888–905 (2000)CrossRefGoogle Scholar
  8. 8.
    Ng, A., Jordan, M., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: Ditterich, T., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems 15, pp. 849–856. MIT Press, Cambridge (2002)Google Scholar
  9. 9.
    Weiss, Y.: Segmentation using eigenvectors: A unifying view. In: Proceedings of IEEE International Conference on Computer Vision, vol. 2, pp. 975–982 (1999)Google Scholar
  10. 10.
    Chung, F.R.K.: Spectral Graph Theory. American Mathematical Society, Providence (1997)zbMATHGoogle Scholar
  11. 11.
    Ho, J., Yang, M.H., Lim, J., Lee, K.C., Kriegman, D.: Clustering appearances of objects under varying illumination conditions. In: IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, pp. 11–18 (2003)Google Scholar
  12. 12.
    Graham, D.B., Allinson, N.M.: Norm2-based face recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 586–591 (1999)Google Scholar
  13. 13.
    Raytchev, B., Murase, H.: Unsupervised face recognition from image sequences based on clustering with attraction and repulsion. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 25–30 (2001)Google Scholar
  14. 14.
    Fitzgibbon, A.W., Zisserman, A.: Joint manifold distance: a new approach to appearance based clustering. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 26–33 (2003)Google Scholar
  15. 15.
    Simard, P., Cun, Y.L., Denker, J., Victorri, B.: Transformation invariance in pattern recognition - tangent distance and tangent propagation. In: Orr, G.B., Müller, K.-R. (eds.) NIPS-WS 1996. LNCS, vol. 1524, pp. 239–274. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  16. 16.
    Mumford, D., Kirwan, F., Fogarty, J.: Geometric Invariant Thoery. Springer, Heidelberg (1994)Google Scholar
  17. 17.
    Frey, B., Jojic, N.: Fast, large-scale transformation-invariant clustering. Advances in Neural Information Processing Systems 14, 721–727 (2001)Google Scholar
  18. 18.
    Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 781–791 (1999)Google Scholar
  19. 19.
    Hager, G.D., Belhumeur, P.N.: Efficient region tracking with parametric models of geometry and illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 1025–1039 (1998)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  1. 1.University of Illinois at Urbana-ChampaignUrbanaUSA
  2. 2.University of California at San DiegoLa JollaUSA
  3. 3.Honda Research InstituteMountain ViewUSA

Personalised recommendations