Advertisement

On the Relationship Between Dimensionality Reduction and Spectral Clustering from a Kernel Viewpoint

Conference paper
First Online:
  • 1.4k Downloads
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 474)

Abstract

This paper presents the development of a unified view of spectral clustering and unsupervised dimensionality reduction approaches within a generalized kernel framework. To do so, the authors propose a multipurpose latent variable model in terms of a high-dimensional representation of the input data matrix, which is incorporated into a least-squares support vector machine to yield a generalized optimization problem. After solving it via a primal-dual procedure, the final model results in a versatile projected matrix able to represent data in a low-dimensional space, as well as to provide information about clusters. Specifically, our formulation yields solutions for kernel spectral clustering and weighted-kernel principal component analysis.

Keywords

Dimensionality reduction Generalized kernel formulation Kernel PCA Spectral clustering Support vector machine 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Belanche Muñoz, L.A.: Developments in kernel design. In: European Symposium on Artificial Neural Networks (ESANN 2013). Computational Intelligence and Machine Learning, Bruges, Belgium, pp. 369–378 (2013)Google Scholar
  2. 2.
    Aldrich, C., Auret, L.: Statistical learning theory and kernel-based methods. In: Unsupervised Process Monitoring and Fault Diagnosis with Machine Learning Methods, pp. 117–181. Springer (2013)Google Scholar
  3. 3.
    Wu, Y., Ma, W., Gong, M., Li, H., Jiao, L.: Novel fuzzy active contour model with kernel metric for image segmentation. Applied Soft Computing (2015)Google Scholar
  4. 4.
    Molina-Giraldo, S., Álvarez-Meza, A., Peluffo-Ordóñez, D., Castellanos-Domínguez, G.: Image segmentation based on multi-kernel learning and feature relevance analysis. In: Advances in Artificial Intelligence–IBERAMIA 2012, pp. 501–510 (2012)Google Scholar
  5. 5.
    Langone, R., Alzate, C., Suykens, J.A.: Kernel spectral clustering with memory effect. Physica A: Statistical Mechanics and its Applications (2013)Google Scholar
  6. 6.
    Peluffo-Ordónez, D., Garcia-Vega, S., Langone, R., Suykens, J., Castellanos-Dominguez, G., et al.: Kernel spectral clustering for dynamic data using multiple kernel learning. In: The 2013 International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE (2013)Google Scholar
  7. 7.
    Harchaoui, Z., Bach, F., Cappé, O., Moulines, E.: Kernel-based methods for hypothesis testing: a unified view. IEEE Signal Processing Magazine 30(4), 87–97 (2013)CrossRefGoogle Scholar
  8. 8.
    Filippone, M., Camastra, F., Masulli, F., Rovetta, S.: A survey of kernel and spectral methods for clustering. Pattern recognition 41(1), 176–190 (2008)CrossRefzbMATHGoogle Scholar
  9. 9.
    Alvarado-Pérez, J.C., Peluffo-Ordóńez, D.H.: Artificial and natural intelligence integration. In: 12th International Conference on Distributed Computing and Artificial Intelligence (DCAI 2016), pp. 167–173. Springer (2015)Google Scholar
  10. 10.
    Alvarado-Pérez, J.C., Peluffo-Ordóñez, D.H., Therón, R.: Bridging the gap between human knowledge and machine learning. ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal 4(1), 54–64 (2015)CrossRefGoogle Scholar
  11. 11.
    Peluffo-Ordóñez, D.H., Lee, J.A., Verleysen, M.: Recent methods for dimensionality reduction: a brief comparative analysis. In: European Symposium on Artificial Neural Networks (ESANN). Citeseer (2014)Google Scholar
  12. 12.
    Ham, J., Lee, D.D., Mika, S., Schölkopf, B.: A kernel view of the dimensionality reduction of manifolds. In: Proceedings of the Twenty-First International Conference on Machine Learning, p. 47. ACM (2004)Google Scholar
  13. 13.
    Peluffo-Ordonez, D.H., Aldo Lee, J., Verleysen, M.: Generalized kernel framework for unsupervised spectral methods of dimensionality reduction. In: 2014 IEEE Symposium on Computational Intelligence and Data Mining (CIDM), pp. 171–177. IEEE (2014)Google Scholar
  14. 14.
    Peluffo-Ordónez, D.H., Alvarado-Pérez, J.C., Lee, J.A., Verleysen, M.: Geometrical homotopy for data visualization. In: European Symposium on Artificial Neural Networks (ESANN 2015). Computational Intelligence and Machine Learning (2015)Google Scholar
  15. 15.
    Peluffo-Ordónez, D.H., Alzate, C., Suykens, J.A.K., Castellanos-Domínguez, G.: Optimal data projection for kernel spectral clustering. In: European Symposium on Artificial Neural Networks. Computational Intelligence and Machine Learning, pp. 553–558 (2014)Google Scholar
  16. 16.
    Wolf, L., Bileschi, S.: Combining variable selection with dimensionality reduction. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 2, pp. 801 – 806, June 2005Google Scholar
  17. 17.
    Peluffo, D.H., Lee, J.A., Verleysen, M., Rodríguez-Sotelo, J.L., Castellanos-Domínguez, G.: Unsupervised relevance analysis for feature extraction and selection: a distance-based approach for feature relevance. In: International Conference on Pattern Recognition, Applications and Methods - ICPRAM (2014)Google Scholar
  18. 18.
    Wolf, L., Shashua, A.: Feature selection for unsupervised and supervised inference: The emergence of sparsity in a weight-based approach. Journal of machine learning 6, 1855–1887 (2005)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Alzate, C., Suykens, J.A.K.: Multiway spectral clustering with out-of-sample extensions through weighted kernel pca. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(2), 335–347 (2010)CrossRefGoogle Scholar
  20. 20.
    Seeland, M., Karwath, A., Kramer, S.: A structural cluster kernel for learning on graphs. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 516–524. ACM (2012)Google Scholar
  21. 21.
    Domeniconi, C., Peng, J., Yan, B.: Composite kernels for semi-supervised clustering. Knowledge and information systems 28(1), 99–116 (2011)CrossRefGoogle Scholar
  22. 22.
    Schölkopf, B., Smola, A.J.: Learning with Kernels (2002)Google Scholar
  23. 23.
    Peluffo-Ordóñez, D.H., Lee, J.A., Verleysen, M., Rodrıguez, J.L., Castellanos-Domınguez, G.: Unsupervised relevance analysis for feature extraction and selection. ICPRAM 2014, 310–315 (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Universidad Técnica del NorteIbarraEcuador
  2. 2.Universidad de NariñoPastoColombia
  3. 3.Institución Universitaria Salazar y HerreraMedellinColombia
  4. 4.Research Center of the Instituto Tecnológico MetropolitanoMedellinColombia
  5. 5.Universidad de SalamancaSalamancaSpain
  6. 6.Coorporación Universitaria Autónoma de NariñoPastoColombia
  7. 7.Universidad SurcolombianaNeivaColombia
  8. 8.Universidad Tecnológica de PereiraPereiraColombia

Personalised recommendations

Cite paper

Buy options