Although linear principal component analysis (PCA) originates from the work of Sylvester [67] and Pearson [51], the development of nonlinear counterparts has only received attention from the 1980s. Work on nonlinear PCA, or NLPCA, can be divided into the utilization of autoassociative neural networks, principal curves and manifolds, kernel approaches or the combination of these approaches. This article reviews existing algorithmic work, shows how a given data set can be examined to determine whether a conceptually more demanding NLPCA model is required and lists developments of NLPCA algorithms. Finally, the paper outlines problem areas and challenges that require future work to mature the NLPCA research field.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Abdel-Qadar, I., Pashaie-Rad, S., Abudayeh, O., and Yehia, S.: PCA-based algorithm for unsupervised bridge crack detection. Advances in Engineering Software, 37 (12), 771-778 (2006)
Anderson, T. W.: An Introduction into Multivariate Statistical Analysis. John Wiley & Sons, New York, (1958)
Bakshi, B. R., Multiscale pca with application to multivariate statistical process monitoring. AIChE Journal, 44 (7), 1596-1610 (1998)
Banfield, J. D. and Raftery A. E.: Ice floe identification in satellite images using mathematical morphology and clustering about principal curves. Journal of the American Statistical Association, 87 (417), 7-16 (1992)
Barnett, V.: Interpreting Multivariate Data. John Wiley & Sons, New York (1981)
Sevensen M., Bishop, C. M., and Williams C. K. I.: GTM: The generative topo-graphic mapping. Neural Computation, 10, 215-234 (1998)
Chang, K. and Ghosh, J.: Principal curve classifier -a nonlinear approach to pattern classification. In: IEEE International Joint Conference on Neural Networks, 4-9 May 1998, 695-700, Anchorage, Alaska (1998)
Chang, K. and Ghosh, J.: A unified model for probabilistic principal surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23 (1), 22-41 (2001)
Chen, D., Zhang, J., Tang, S., and Wang J.: Freeway traffic stream modelling based on principal curves and its analysis. IEEE Transactions on Intelligent Transportation Systems, 5 (4), 246-258 (2004)
Chen, P. and Suter, D.: An analysis of linear subspace approaches for computer vision and pattern recognition. International Journal of Computer Vision, 68 (1), 83-106 (2006)
Chennubhotla, C. and Jepson, A.: Sparse pca extracting multi-scale structure from data. In: Proceedings of the IEEE International Conference on Computer Vision, vol. 1, 641-647 (2001)
Cho, H. W.: Nonlinear feature extraction and classification of multivariate process data in kernel feature space. Expert Systems with Applications, 32 (2), 534-542 (2007)
Choi, S. W. and Lee, I. -B.: Nonlinear dynamic process monitoring based on dynamic kernel pca. Chemical Engineering Science, 59 (24), 5897-5908 (2004)
Delicado, P.: Another look at principal curves and surfaces. Journal of Multi-variate Analysis, 77 (1), 84-116 (2001)
Denoeux, T. and Masson, M. -H.: Principal component analysis of fuzzy data using autoassociative neural networks. IEEE Transactions on Fuzzy Systems, 12(3), 336-349 (2004)
Dong, D. and McAvoy, T. J.: Nonlinear principal component analysis-based on principal curves and neural networks. Computers & Chemical Engineering, 20 (1), 65-78 (1996)
Du, Q. and Chang, C.: Linear mixture analysis-based compression for hyperspectal image analysis. IEEE Transactions on Geoscience and Remote Sensing, 42(4), 875-891 (2004)
Duchamp, T. and Stuetzle, W.: Extremal properties of principal curves in the plane. Annals of Statistics, 24 (4), 1511-1520 (1996)
Duchamp, T. and Stuetzle, W.: Geometric Properties of Principal Curves in the Plane. In: Robust statistics, data analysis, and computer intensive methods: in honor of Peter Huber’s 60th birthday, ( Lecture Notes in Statistics), vol. 109, 135-152. Springer, New York (1996)
Esbensen, K. and Geladi, P.: Strategy of multivariate image analysis (MIA). Chemometrics & Intelligent Laboratory Systems, 7 (1-2), 67-86 (1989)
Fisher, R. and MacKenzie, W.: Studies in crop variation, ii, the manurial re-sponse of different potato varieties. Journal of Agricultural Science, 13, 411-444 (1923)
Golub, G. H. and van Loan, C. F.: Matrix Computation. John Hopkins, Baltimore, (1996)
Gomez, J. C. and Baeyens, E.: Subspace-based identification algorithms for hammerstein and wiener models. European Journal of Control, 11 (2), 127-136 (2005)
Hastie, T.: Principal curves and surfaces. Technical report no. 11, Department of Statistics, Stanford University (1984)
Hastie, T. and Stuetzle, W.: Principal curves. Journal of the American Statistical Association 84 (406), 502-516 (1989)
Hermann, T, Meinicke, P., and Ritter, H.: Principal curve sonification. In: Proceedings of International Conference on Auditory Display, 2-5 April 2000, Atlanta, Georgia, 81-86 (2000)
Hertz, J., Krogh, A., and Palmer, R. G.: Introduction to the Theory of Neural Computing. Addison-Wesley, Redwood City, CA (1991)
Hotelling, H.: Analysis of a complex of statistical variables into principal com-ponents. Journal of Educational Psychology, 24 417-441 (1933)
Jackson, J. E.: Principal components and factor analysis: Part III: What is factor analysis. Journal of Quality Technology, 13 (2), 125-130 (1981)
Jackson, J. E.: A Users Guide to Principal Components. Wiley Series in Prob-ability and Mathematical Statistics, John Wiley & Sons, New York (1991)
Jia, F., Martin, E. B., and Morris, A. J.: Non-linear principal component analysis for process fault detection. Computers & Chemical Engineering, 22 (Supple-ment), S851-S854 (1998)
Joliffe, I. T.: Principal Component Analysis. Springer, New York, (1986)
Kégl, B.: Principal Curves: Learning, Design and Applications. PhD thesis, Department of Computer Science, Concordia University, Montréal, Québec, Canada, 2000
Kégl, B., Krzyzak, A., Linder, T., and Zeger, K.: A polygonal line algorithm for constructing principal curves. In: Neural Information Processing (NIPS ’98), Denver, CO, 1-3 December 1998, 501-507. MIT Press (1998)
Kégl, B., Krzyzak, A., Linder, T., and Zeger, K.: Learning and design of principal curves. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(3), 281-297 (2000)
Kim, K. I., Jung, K., and Kim, H. J.: Face recognition using kernel principal component analysis. IEEE Signal Processing Letters, 9 (2), 40-42 (2002)
Kramer, M. A.: Nonlinear principal component analysis using autoassociative neural networks. AIChE Journal, 37 (3), 233-243 (1991)
Kruger, U., Antory, D., Hahn, J., Irwin, G. W., and McCullough, G.: Introduc-tion of a nonlinearity measure for principal component models. Computers & Chemical Engineering, 29 (11-12), 2355-2362 (2005)
Krzanowski, W. J.: Cross-validatory choice in principal component analysis: Some sampling results. Journal of Statistical Computation and Simulation, 18, 299-314 (1983)
Kwok, J. T. Y. and Tsang, I. W. H.: The pre-image problem in kernel methods. IEEE Transactions on Neural Networks, 15 (6), 1517-1525 (2004)
Lacy, S. L. and Bernstein, D. S.: Subspace identification for non-linear systems with measured input non-linearities. International Journal of Control, 78 (12), 906-921 (2005)
Leeuw J. d.: Nonlinear principal component analysis. In: Caussinus, H., Ettinger, P., and Tomassone, R. (eds) Proceedings in Computational Statistics (COMPSTAT 1982) October 30 -September 3, Toulouse, France 1982. Physica-Verlag, Wien (1982)
Lovera, M., Gustafsson, T., and Verhaegen, M.: Recursive subspace identifica-tion of linear and nonlinear wiener type models. Automatica, 36 (11), 1639-1650 (2000)
Malinowski, E. R.: Factor Analysis in Chemistry. John Wiley & Sons, New York (2002)
Mardia, K. V., Kent, J. T., and Bibby, J. M.: Multivariate Analysis. Probability and Mathematical Statistics. Academic Press, London, (1979)
Morales, M.: Geometric Data Fitting. PhD thesis, University of Washington (1998)
Nara, Y., Jianming Y., and Suematsu, Y.: Face recognition using improved principal component analysis. In: Proceedings of 2003 International Symposium on Micromechatronics and Human Science (IEEE Cat. No. 03TH8717), 77-82 (2003)
Nomikos, P. and MacGregor, J. F.: Monitoring of batch processes using multiway principal component analysis. AIChE Journal, 40 (8), 1361-1375 (1994)
Paluš, M. and Dvořák, I.: Singular-value decomposition in attractor recon-struction: Pitfals and precautions. Physica D: Nonlinear Phenomena, 5 (1-2), 221-234 (1992)
Parsopoulos, K. E. and Vrahatis, M. N.: Recent approaches to global optimiza-tion problems through particle swarm optimization. Natural Computing, 1 (2-3), 235-306 (2002)
Pearson, C.: On lines and planes of closest fit to systems of points in space. Phil. Mag., Series B., 2 (11), 559-572 (1901)
Qin, S. J., Valle, S., and Piovoso, M. J.: On unifying multi-block analysis with application to decentralized process monitoring. Journal of Chemometrics, 10, 715-742 (2001)
Reinhard, K. and Niranjan, M.: Subspace models for speech transitions using principal curves. Proceedings of Institute of Acoustics, 20 (6), 53-60 (1998)
Reinhard, K. and Niranjan, M.: Parametric subspace modeling of speech tran-sitions. Speech Communication, 27 (1), 19-42 (1999)
Sandilya, S. and Kulkarni, S. R.: Principal curves with bounded turn. In: Pro-ceedings of the IEEE International Symposium on Information Theory, Sorento, 25-30 June 2000, Sorento, Italy (2000)
Schölkopf, B. and Smola, A. J.: Learning with Kernels: Support Vector Ma-chines, Regularization, Optimization, and Beyond. The MIT Press, Cambridge, MA (2002)
Schölkopf, B. and Smola, A. J., and Müller, K. : Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10 (5), 1299-1319 (1998)
Shanmugam, R. and Johnson, C.: At a crossroad of data envelopment and principal component analyses. Omega, 35 (4), 351-364 (2007)
Shawe-Taylor, J. and Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, West Nyack, NY (2004)
Silverman, B. W.: Some aspects of spline smoothing. Journal of the Royal Statistical Society, Series B, 47 (1), 1-52 (1985)
Smola, A. J., Williamson, R. C., Mika, S., and Schölkopf, B.: Regularized princi-pal manifolds. In: Fischer, P., and Simon, H. U. (eds. ) Computational Learning Theory (EuroCOLT ’99), Lecture Notes in Artificial Intelligence, vol. 1572, 214-229, Springer, Heidelberg (1999)
Socas-Navarro, H.: Feature extraction techniques for the analysis of spectral polarization profiles. Astrophysical Journal, 620, 517-522 (2005)
Srinivas, M. and Patnaik, L. M.: Genetic algorithms: A survey. Computer, 27 (6), 17-26 (1994)
Stoica, P., Eykhoff, P., Janssen, P., and Söderström, T.: Model structure se-lection by cross-validation. International Journal of Control, 43 (6), 1841-1878 (1986)
Stone, M.: Cross-validatory choice and assessment of statistical prediction (with discussion). Journal of the Royal Statistical Society (Series B), 36, 111-133 (1974)
Stoyanova, R. and Brown, T. R.: Nmr spectral quantitation by principal component analysis. NMR in Biomedicine, 14 (4), 271-277 (2001)
Sylvester, J. J.: On the reduction of a bilinear quantic of the nth order to the form of a sum of n products by a double orthogonal substitution. Messenger of Mathematics, 19, 42-46 (1889)
Tan, S. and Mavrovouniotis, M. L.: Reducing data dimensionality through opti-mizing neural network inputs. AIChE Journal, 41 (6), 1471-1480 (1995)
Tibshirani, R.: Principal curves revisited. Statistics and Computation, 2 (4), 183-190 (1992)
Trafalis, T. B., Richman, M. B., White, A., and Santosa, B.: Data mining tech-niques for improved wsr-88d rainfall estimation. Computers & Industrial Engi-neering, 43 (4), 775-786 (2002)
Huffel, S. van and Vandewalle, J.: The Total Least Squares Problem: Compu-tational Aspects and Analysis. SIAM, Philadelphia (1991)
Vaswani, N. and Chellappa, R.: Principal components null space analysis for image and video classification. IEEE Transactions on Image Processing, 15 (7), 1816-1830 (2006)
’ose, B.: K-segments algorithm for finding principal curves. Technical Report IAS-UVA-00-11, Institute of Computer Science, University of Amsterdam (2000)
Verhaegen, M. and Westwick, D.: Identifying mimo hammerstein systems in the context of subspace model identification methods. International Journal of Control, 63 (2), 331-350 (1996)
Wax, M. and Kailath, T.: Detection of signals by information theoretic criteria. IEEE Transactions on Acoustics, Speech & Signal Processing, 33 (2), 387-392 (1985)
Westwick, D. and Verhaegen, M.: Identifying mimo wiener systems using sub-space model identification methods. Signal Processing, 52 (2), 235-258 (1996)
Wise, B. M. and Ricker, N. L.: Identification of finite impulse respnose models by principal components regression: Frequency response properties. Process Control & Quality, 4, 77-86 (1992)
Wold, H.: Estimation of principal components and related models by itera-tive least squares. In: Krishnaiah, P. R. (ed. ) Multivariate Analysis, 391-420. Academic Press, N. Y. (1966)
Wold, S.: Cross validatory estimation of the number of principal components in factor and principal component models. Technometrics, 20 (4), 397-406 (1978)
Wold, S., Esbensen, K., and Geladi, P.: Principal component analysis. Chemo-metrics and Intelligent Laboratory Systems, 2, 37-52 (1987)
Yoo, C. K. and Lee, I.: Nonlinear multivariate filtering and bioprocess monitoring for supervising nonlinear biological processes. Process Biochemistry, 41 (8), 1854-1863 (2006)
Zeng, Z. and Zou, X.: Application of principal component analysis to champ radio occultation data for quality control and a diagnostic study. Monthly Weather Review, 134 (11), 3263-3282 (2006)
Zhang, J. and Chen, D.: Constraint k-segment principal curves. In: Huang, De-Sh., Li, K. and Irwin, G. W. (eds. ) Intelligent Computing, Lecture Notes in Computer Sciences, vol. 4113, 345-350. Springer, Berlin Heidelberg New York (2006)
Zhang, J., Chen, D., and Kruger, U.: Constrained k-segments principal curves and its applications in intelligent transportation systems. Technical report, Department of Computer Science and Engineering, Fudan University, Shanghai, P. R. China (2007)
Zhang, J. and Wang, J.: An overview of principal curves (in chinese). Chinese Journal of Computers, 26 (2), 137-148 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kruger, U., Zhang, J., Xie, L. (2008). Developments and Applications of Nonlinear Principal Component Analysis – a Review. In: Gorban, A.N., Kégl, B., Wunsch, D.C., Zinovyev, A.Y. (eds) Principal Manifolds for Data Visualization and Dimension Reduction. Lecture Notes in Computational Science and Enginee, vol 58. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73750-6_1
Download citation
DOI: https://doi.org/10.1007/978-3-540-73750-6_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73749-0
Online ISBN: 978-3-540-73750-6
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)