Clustering Complex Data with Group-Dependent Feature Selection

  • Yen-Yu Lin
  • Tyng-Luh Liu
  • Chiou-Shann Fuh
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6316)


We describe a clustering approach with the emphasis on detecting coherent structures in a complex dataset, and illustrate its effectiveness with computer vision applications. By complex data, we mean that the attribute variations among the data are too extensive such that clustering based on a single feature representation/descriptor is insufficient to faithfully divide the data into meaningful groups. The proposed method thus assumes the data are represented with various feature representations, and aims to uncover the underlying cluster structure. To that end, we associate each cluster with a boosting classifier derived from multiple kernel learning, and apply the cluster-specific classifier to feature selection across various descriptors to best separate data of the cluster from the rest. Specifically, we integrate the multiple, correlative training tasks of the cluster-specific classifiers into the clustering procedure, and cast them as a joint constrained optimization problem. Through the optimization iterations, the cluster structure is gradually revealed by these classifiers, while their discriminant power to capture similar data would be progressively improved owing to better data labeling.


Feature Selection Local Binary Pattern Spectral Cluster Normalize Mutual Information Weak Learner 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Dueck, D., Frey, B.: Non-metric affinity propagation for unsupervised image categorization. In: ICCV (2007)Google Scholar
  2. 2.
    Tuzel, O., Porikli, F., Meer, P.: Kernel methods forweakly supervised mean shift clustering. In: ICCV (2009)Google Scholar
  3. 3.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. TPAMI (2000)Google Scholar
  4. 4.
    Comaniciu, D., Meer, P.: Mean shift: A robust approach toward feature space analysis. TPAMI (2002)Google Scholar
  5. 5.
    Roth, V., Lange, T.: Feature selection in clustering problems. In: NIPS (2003)Google Scholar
  6. 6.
    Ye, J., Zhao, Z., Wu, M.: Discriminative k-means for clustering. In: NIPS (2007)Google Scholar
  7. 7.
    Lanckriet, G., Cristianini, N., Bartlett, P., Ghaoui, L., Jordan, M.: Learning the kernel matrix with semidefinite programming. JMLR (2004)Google Scholar
  8. 8.
    Ng, A., Jordan, M., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: NIPS (2001)Google Scholar
  9. 9.
    Frey, B., Dueck, D.: Clustering by passing messages between data points. Science (2007)Google Scholar
  10. 10.
    Cheng, H., Hua, K., Vu, K.: Constrained locally weighted clustering. In: VLDB (2008)Google Scholar
  11. 11.
    Domeniconi, C., Al-Razgan, M.: Weighted cluster ensembles: Methods and analysis. TKDD (2009)Google Scholar
  12. 12.
    Berg, A., Malik, J.: Geometric blur for template matching. In: CVPR (2001)Google Scholar
  13. 13.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. IJCV (2004)Google Scholar
  14. 14.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: CVPR (2006)Google Scholar
  15. 15.
    Bosch, A., Zisserman, A., Muñoz, X.: Representing shape with a spatial pyramid kernel. In: CIVR (2007)Google Scholar
  16. 16.
    Xu, L., Neufeld, J., Larson, B., Schuurmans, D.: Maximum margin clustering. In: NIPS (2004)Google Scholar
  17. 17.
    Zhao, B., Wang, F., Zhang, C.: Efficient multiclass maximum margin clustering. In: ICML (2008)Google Scholar
  18. 18.
    Strehl, A., Ghosh, J.: Cluster ensembles – A knowledge reuse framework for combining multiple partitions. JMLR (2002)Google Scholar
  19. 19.
    Fred, A., Jain, A.: Combining multiple clusterings using evidence accumulation. TPAMI (2005)Google Scholar
  20. 20.
    Xing, E., Ng, A., Jordan, M., Russell, S.: Distance metric learning with application to clustering with side-information. In: NIPS (2002)Google Scholar
  21. 21.
    Mutch, J., Lowe, D.: Multiclass object recognition with sparse, localized features. In: CVPR (2006)Google Scholar
  22. 22.
    Zhang, H., Berg, A., Maire, M., Malik, J.: SVM-KNN: Discriminative nearest neighbor classification for visual category recognition. In: CVPR (2006)Google Scholar
  23. 23.
    Lin, Y.-Y., Liu, T.-L., Fuh, C.-S.: Local ensemble kernel learning for object category recognition. In: CVPR (2007)Google Scholar
  24. 24.
    Lin, Y.-Y., Tsai, J.-F., Liu, T.-L.: Efficient discriminative local learning for object recognition. In: ICCV (2009)Google Scholar
  25. 25.
    Moghaddam, B., Shakhnarovich, G.: Boosted dyadic kernel discriminants. In: NIPS (2002)Google Scholar
  26. 26.
    Collins, M., Schapire, R., Singer, Y.: Logistic regression, AdaBoost and Bregman distances. ML (2002)Google Scholar
  27. 27.
    Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: A statistical view of boosting. Annals of Statistics (2000)Google Scholar
  28. 28.
    Wolsey, L.: Integer Programming. John Wiley & Sons, Chichester (1998)zbMATHGoogle Scholar
  29. 29.
    The MOSEK Optimization Software,
  30. 30.
    Fei-Fei, L., Fergus, R., Perona, P.: Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In: CVPR Workshop on Generative-Model Based Vision (2004)Google Scholar
  31. 31.
    Shechtman, E., Irani, M.: Matching local self-similarities across images and videos. In: CVPR (2007)Google Scholar
  32. 32.
    Cox, T., Cox, M.: Multidimentional Scaling. Chapman & Hall, London (1994)Google Scholar
  33. 33.
    Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression database. TPAMI (2005)Google Scholar
  34. 34.
    Gross, R., Brajovic, V.: An image preprocessing algorithm for illumination invariant face recognition. In: Kittler, J., Nixon, M.S. (eds.) AVBPA 2003. LNCS, vol. 2688, Springer, Heidelberg (2003)CrossRefGoogle Scholar
  35. 35.
    Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. TPAMI (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Yen-Yu Lin
    • 1
    • 2
  • Tyng-Luh Liu
    • 1
  • Chiou-Shann Fuh
    • 2
  1. 1.Institute of Information ScienceAcademia SinicaTaipeiTaiwan
  2. 2.Department of CSIENational Taiwan UniversityTaipeiTaiwan

Personalised recommendations