Abstract
Very often, multivariate data analysis problems require dimensionality reduction (DR) stages to either improve analysis performance or represent the data in an intelligible fashion. Traditionally DR techniques are developed under different frameworks and settings what makes their comparison a non-trivial task. In this sense, generalized DR approaches are of great interest as they enable both to power and compare the DR techniques in a proper and fair manner. This work introduces a generalized spectral dimensionality reduction (GSDR) approach able to represent DR spectral techniques and enhance their representation ability. To do so, GSDR exploits the use of kernel-based representations as an initial nonlinear transformation to obtain a new space. Then, such a new space is used as an input for a feature extraction process based on principal component analysis. As remarkable experimental results, GSDR shows to be able to outperform the conventional implementation of well-known spectral DR techniques (namely, classical multidimensional scaling and Laplacian eigenmaps) in terms of the scaled version of the average agreement rate. Additionally, relevant insights and theoretical developments to understand the effect of data structure preservation at local and global levels are provided.
M. Velez-Falconi—This work is supported by SDAS research group (www.sdas-group.com).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Borg, I.: Modern Multidimensional Scaling: Theory and Applications. Springer, New York (2005)
Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15(6), 1373–1396 (2003)
Belanche Muñoz, L.A.: Developments in kernel design. In: ESANN 2013 proceedings: European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning: Bruges (Belgium), 24–26 April 2013, pp. 369–378 (2013)
Bagchi, A.: Lecture notes: Efficient approximation of kernel functions (2020)
Ramon, E., Belanche-Muñoz, L., Molist, F., Quintanilla, R., Perez-Enciso, M., Ramayo-Caldas, Y.: kernint: a kernel framework for integrating supervised and unsupervised analyses in spatio-temporal metagenomic datasets. Front. Microbiol. 12, 60 (2021)
Porro-Muñoz, D., Duin, R.P., Talavera, I., Orozco-Alzate, M.: Classification of three-way data by the dissimilarity representation. Sig. Proc. 91(11), 2520–2529 (2011)
Peluffo-Ordonez, D.H., Aldo Lee, J., Verleysen, M.: Generalized kernel framework for unsupervised spectral methods of dimensionality reduction. In: Computational Intelligence and Data Mining (CIDM), 2014 IEEE Symposium on, pp. 171–177. IEEE (2014)
Peluffo, D., Lee, J., Verleysen, M., Rodríguez, J., Castellanos-Domínguez, G.: Unsupervised relevance analysis for feature extraction and selection: a distance-based approach for feature relevance. In: ICPRAM 2014 - Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods (2014)
Ham, J., Lee, D.D., Mika, S., Schölkopf, B.: A kernel view of the dimensionality reduction of manifolds. In: Proceedings of the Twenty-First International Conference on Machine Learning, vol. 47 ACM (2004)
Cook, J., Sutskever, I., Mnih, A., Hinton, G.E.: Visualizing similarity data with a mixture of maps. In: International Conference on Artificial Intelligence and Statistics, pp. 67–74 (2007)
Lee, J.A., Renard, E., Bernard, G., Dupont, P., Verleysen, M.: Type 1 and 2 mixtures of kullback-leibler divergences as cost functions in dimensionality reduction based on similarity preservation. Neurocomputing 112, 92–108 (2013)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Nene, S.A., Nayar, S.K., Murase, H.: Columbia object image library (coil-20). Dept. Comput. Sci. Columbia Univ. New York. 62 (1996). http://www.cs.columbia.edu/CAVE/coil-20.html
Rodríguez-Sotelo, J.L., Peluffo-Ordonez, D., Cuesta-Frau, D., Castellanos-Domínguez, G.: Unsupervised feature relevance analysis applied to improve ECG heartbeat clustering. Comput. Methods Programs Biomed. 108(1), 250–261 (2012)
Blanco Valencia, X.P., Becerra, M., Castro Ospina, A., Ortega Adarme, M., Viveros Melo, D., Peluffo Ordóñez, D.H., et al.: Kernel-based framework for spectral dimensionality reduction and clustering formulation: a theoretical study. ADCAIJ: Adv. Distrib. Comput. Artif. Intell. J. 6(1) (2017)
Acknowledgments
This work is supported by the research project “Proyecto PN223LH010-005 Desarrollo de nuevos modelos y métodos matemáticos para la toma de decisiones”. Authors thank the valuable support given by the SDAS Research Group (www.sdas-group.com).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Ortega-Bustamante, M.C., Hasperué, W., Peluffo-Ordóñez, D.H., González-Vergara, J., Marín-Gaviño, J., Velez-Falconi, M. (2021). Generalized Spectral Dimensionality Reduction Based on Kernel Representations and Principal Component Analysis. In: Gervasi, O., et al. Computational Science and Its Applications – ICCSA 2021. ICCSA 2021. Lecture Notes in Computer Science(), vol 12952. Springer, Cham. https://doi.org/10.1007/978-3-030-86973-1_36
Download citation
DOI: https://doi.org/10.1007/978-3-030-86973-1_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86972-4
Online ISBN: 978-3-030-86973-1
eBook Packages: Computer ScienceComputer Science (R0)