Skip to main content

Advertisement

SpringerLink
Log in
Menu
Find a journal Publish with us
Search
Cart
Book cover

Joint European Conference on Machine Learning and Knowledge Discovery in Databases

ECML PKDD 2012: Machine Learning and Knowledge Discovery in Databases pp 207–222Cite as

  1. Home
  2. Machine Learning and Knowledge Discovery in Databases
  3. Conference paper
Hypergraph Spectra for Semi-supervised Feature Selection

Hypergraph Spectra for Semi-supervised Feature Selection

  • Zhihong Zhang20,
  • Edwin R. Hancock20 &
  • Xiao Bai21 
  • Conference paper
  • 4504 Accesses

  • 2 Citations

Part of the Lecture Notes in Computer Science book series (LNAI,volume 7523)

Abstract

In many data analysis tasks, one is often confronted with the problem of selecting features from very high dimensional data. Most existing feature selection methods focus on ranking individual features based on a utility criterion, and select the optimal feature set in a greedy manner. However, the feature combinations found in this way do not give optimal classification performance, since they neglect the correlations among features. While the labeled data required by supervised feature selection can be scarce, there is usually no shortage of unlabeled data. In this paper, we propose a novel hypergraph based semi-supervised feature selection algorithm to select relevant features using both labeled and unlabeled data. There are two main contributions in this paper. The first is that by incorporating multidimensional interaction information (MII) for higher order similarities measure, we establish a novel hypergraph framework which is used for characterizing the multiple relationships within a set of samples. Thus, the structural information latent in the data can be more effectively modeled. Secondly, we derive a hypergraph subspace learning view of feature selection which casting the feature discriminant analysis into a regression framework that considers the correlations among features. As a result, we can evaluate joint feature combinations, rather than being confined to consider them individually. Experimental results demonstrate the effectiveness of our feature selection method on a number of standard face data-sets.

Keywords

  • Hypergraph representation
  • Semi-supervised subspace learning

Download conference paper PDF

References

  1. Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. In: Proc. Syst. (1993)

    Google Scholar 

  2. Scholkopf, B., Smola, A., Muller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. In: Neural Computation, pp. 1299–1319 (1998)

    Google Scholar 

  3. He, X., Niyogi, P.: Locality preserving projections. Advances in Neural Information Processing Systems (2004)

    Google Scholar 

  4. He, X., Cai, D., Yan, S., Zhang, H.J.: Neighborhood preserving embedding. In: Tenth IEEE International Conference on Computer Vision, pp. 1208–1213 (2005)

    Google Scholar 

  5. Belkin, M., Niyogi, P.: Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Advances in Neural Information Processing Systems, pp. 585–592 (2002)

    Google Scholar 

  6. Battiti, R.: Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on Neural Networks, 537–550 (2002)

    Google Scholar 

  7. Peng, H., Long, F., Ding, C.: Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1226–1238 (2005)

    Google Scholar 

  8. Duda, R.O., Hart, P.E., Stork, D.G.: Pattern classification. Wiley, New York (2001)

    MATH  Google Scholar 

  9. Nie, F., Xiang, S., Jia, Y., Zhang, C., Yan, S.: Trace ratio criterion for feature selection. In: Proceedings of the 23rd National Conference on Artificial Intelligence, pp. 671–676 (2008)

    Google Scholar 

  10. Robnik-Šikonja, M., Kononenko, I.: Theoretical and empirical analysis of ReliefF and RReliefF. In: Machine Learning, pp. 23–69 (2003)

    Google Scholar 

  11. Zhao, Z., Liu, H.: Spectral feature selection for supervised and unsupervised learning. In: Proceedings of the 24th International Conference on Machine Learning, pp. 1151–1157 (2007)

    Google Scholar 

  12. He, X., Cai, D., Niyogi, P.: Laplacian score for feature selection. In: Advances in Neural Information Processing Systems (2005)

    Google Scholar 

  13. Belhumeur, P.N., Kriegman, D.J.: What is the set of images of an object under all possible illumination conditions? International Journal of Computer Vision, 245–260 (1998)

    Google Scholar 

  14. Agarwal, S., Branson, K., Belongie, S.: Higher order learning with graphs. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 17–24 (2006)

    Google Scholar 

  15. Agarwal, S., Lim, J., Zelnik-Manor, L., Perona, P., Kriegman, D., Belongie, S.: Beyond pairwise clustering. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 838–845 (2005)

    Google Scholar 

  16. Chung, F.: The Laplacian of a hypergraph. AMS DIMACS Series in Discrete Mathematics and Theoretical Computer Science, pp. 21–36 (1993)

    Google Scholar 

  17. Li, W.C.W., Solé, P.: Spectra of regular graphs and hypergraphs and orthogonal polynomials. European Journal of Combinatorics, 461–477 (1996)

    Google Scholar 

  18. Seeger, M.W.: Bayesian inference and optimal design for the sparse linear model. The Journal of Machine Learning Research, 759–813 (2008)

    Google Scholar 

  19. Argyriou, A., Evgeniou, T., Pontil, M.: Convex multi-task feature learning. In: Machine Learning, pp. 243–272 (2008)

    Google Scholar 

  20. Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. In: The Annals of Statistics, pp. 407–499 (2004)

    Google Scholar 

  21. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001)

    Google Scholar 

  22. Cai, D., Zhang, C., He, X.: Unsupervised feature selection for multi-cluster data. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 333–342 (2010)

    Google Scholar 

  23. Yang, Y., Shen, H.T., Ma, Z., Huang, Z., Zhou, X.: L21-norm regularized discriminative feature selection for unsupervised learning. In: International Joint Conferences on Artificial Intelligence, pp. 1589–1594 (2011)

    Google Scholar 

  24. Jacobs, D.W., Belhumeur, P.N., Basri, R.: Comparing images under variable illumination. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 610–617 (1998)

    Google Scholar 

  25. Nie, F., Xiang, S., Liu, Y., Zhang, C.: A general graph-based semi-supervised learning with novel class discovery. In: Neural Computing & Applications, pp. 549–555 (2010)

    Google Scholar 

  26. Zhang, Z., Hancock, E.R.: Feature selection for gender classification. In: 5th Iberian Conference on Pattern Recognition and Image Analysis, pp. 76–83 (2011)

    Google Scholar 

  27. Zhang, Z., Hancock, E.R.: Hypergraph based Information-theoretic Feature Selection. Pattern Recognition Letters (2012)

    Google Scholar 

  28. Yang, A.Y., Wright, J., Ma, Y., Sastry, S.S.: Feature selection in face recognition: A sparse representation perspective. IEEE Transactions on Pattern Analysis and Machine Intelligence (2007)

    Google Scholar 

  29. Nie, F., Xu, D., Li, X., Xiang, S.: Semi-supervised dimensionality reduction and classification through virtual label regression. IEEE Transactions on Systems, Man, and Cybernetics, 1–11 (2011)

    Google Scholar 

  30. Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Review, 455–500 (2009)

    Google Scholar 

  31. Shashua, A., Zass, R., Hazan, T.: Multi-way Clustering Using Super-Symmetric Non-negative Tensor Factorization. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3954, pp. 595–608. Springer, Heidelberg (2006)

    CrossRef  Google Scholar 

  32. Björck, A.: Numberical methods for least squares problems. In: Proc. SIAM (1996)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of York, UK

    Zhihong Zhang & Edwin R. Hancock

  2. School of Computer Science and Engineering, Beihang University, China

    Xiao Bai

Authors
  1. Zhihong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Edwin R. Hancock
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiao Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Intelligent Systems Laboratory, University of Bristol, Merchant Venturers Building, Woodland Road, BS8 1UB, Bristol, UK

    Peter A. Flach, Tijl De Bie & Nello Cristianini,  & 

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Zhang, Z., Hancock, E.R., Bai, X. (2012). Hypergraph Spectra for Semi-supervised Feature Selection. In: Flach, P.A., De Bie, T., Cristianini, N. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2012. Lecture Notes in Computer Science(), vol 7523. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33460-3_19

Download citation

  • .RIS
  • .ENW
  • .BIB
  • DOI: https://doi.org/10.1007/978-3-642-33460-3_19

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-33459-7

  • Online ISBN: 978-3-642-33460-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Search

Navigation

  • Find a journal
  • Publish with us

Discover content

  • Journals A-Z
  • Books A-Z

Publish with us

  • Publish your research
  • Open access publishing

Products and services

  • Our products
  • Librarians
  • Societies
  • Partners and advertisers

Our imprints

  • Springer
  • Nature Portfolio
  • BMC
  • Palgrave Macmillan
  • Apress
  • Your US state privacy rights
  • Accessibility statement
  • Terms and conditions
  • Privacy policy
  • Help and support

167.114.118.210

Not affiliated

Springer Nature

© 2023 Springer Nature