Skip to main content
SpringerLink
Log in
Book cover

Principal Manifolds for Data Visualization and Dimension Reduction pp 1–43Cite as

Developments and Applications of Nonlinear Principal Component Analysis – a Review

  • Conference paper

Part of the book series: Lecture Notes in Computational Science and Enginee ((LNCSE,volume 58))

Although linear principal component analysis (PCA) originates from the work of Sylvester [67] and Pearson [51], the development of nonlinear counterparts has only received attention from the 1980s. Work on nonlinear PCA, or NLPCA, can be divided into the utilization of autoassociative neural networks, principal curves and manifolds, kernel approaches or the combination of these approaches. This article reviews existing algorithmic work, shows how a given data set can be examined to determine whether a conceptually more demanding NLPCA model is required and lists developments of NLPCA algorithms. Finally, the paper outlines problem areas and challenges that require future work to mature the NLPCA research field.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   189.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abdel-Qadar, I., Pashaie-Rad, S., Abudayeh, O., and Yehia, S.: PCA-based algorithm for unsupervised bridge crack detection. Advances in Engineering Software, 37 (12), 771-778 (2006)

    Article  Google Scholar 

  2. Anderson, T. W.: An Introduction into Multivariate Statistical Analysis. John Wiley & Sons, New York, (1958)

    Google Scholar 

  3. Bakshi, B. R., Multiscale pca with application to multivariate statistical process monitoring. AIChE Journal, 44 (7), 1596-1610 (1998)

    Article  Google Scholar 

  4. Banfield, J. D. and Raftery A. E.: Ice floe identification in satellite images using mathematical morphology and clustering about principal curves. Journal of the American Statistical Association, 87 (417), 7-16 (1992)

    Article  Google Scholar 

  5. Barnett, V.: Interpreting Multivariate Data. John Wiley & Sons, New York (1981)

    MATH  Google Scholar 

  6. Sevensen M., Bishop, C. M., and Williams C. K. I.: GTM: The generative topo-graphic mapping. Neural Computation, 10, 215-234 (1998)

    Article  Google Scholar 

  7. Chang, K. and Ghosh, J.: Principal curve classifier -a nonlinear approach to pattern classification. In: IEEE International Joint Conference on Neural Networks, 4-9 May 1998, 695-700, Anchorage, Alaska (1998)

    Google Scholar 

  8. Chang, K. and Ghosh, J.: A unified model for probabilistic principal surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23 (1), 22-41 (2001)

    Article  Google Scholar 

  9. Chen, D., Zhang, J., Tang, S., and Wang J.: Freeway traffic stream modelling based on principal curves and its analysis. IEEE Transactions on Intelligent Transportation Systems, 5 (4), 246-258 (2004)

    Article  Google Scholar 

  10. Chen, P. and Suter, D.: An analysis of linear subspace approaches for computer vision and pattern recognition. International Journal of Computer Vision, 68 (1), 83-106 (2006)

    Article  Google Scholar 

  11. Chennubhotla, C. and Jepson, A.: Sparse pca extracting multi-scale structure from data. In: Proceedings of the IEEE International Conference on Computer Vision, vol. 1, 641-647 (2001)

    Google Scholar 

  12. Cho, H. W.: Nonlinear feature extraction and classification of multivariate process data in kernel feature space. Expert Systems with Applications, 32 (2), 534-542 (2007)

    Article  Google Scholar 

  13. Choi, S. W. and Lee, I. -B.: Nonlinear dynamic process monitoring based on dynamic kernel pca. Chemical Engineering Science, 59 (24), 5897-5908 (2004)

    Article  Google Scholar 

  14. Delicado, P.: Another look at principal curves and surfaces. Journal of Multi-variate Analysis, 77 (1), 84-116 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  15. Denoeux, T. and Masson, M. -H.: Principal component analysis of fuzzy data using autoassociative neural networks. IEEE Transactions on Fuzzy Systems, 12(3), 336-349 (2004)

    Article  Google Scholar 

  16. Dong, D. and McAvoy, T. J.: Nonlinear principal component analysis-based on principal curves and neural networks. Computers & Chemical Engineering, 20 (1), 65-78 (1996)

    Article  Google Scholar 

  17. Du, Q. and Chang, C.: Linear mixture analysis-based compression for hyperspectal image analysis. IEEE Transactions on Geoscience and Remote Sensing, 42(4), 875-891 (2004)

    Article  Google Scholar 

  18. Duchamp, T. and Stuetzle, W.: Extremal properties of principal curves in the plane. Annals of Statistics, 24 (4), 1511-1520 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  19. Duchamp, T. and Stuetzle, W.: Geometric Properties of Principal Curves in the Plane. In: Robust statistics, data analysis, and computer intensive methods: in honor of Peter Huber’s 60th birthday, ( Lecture Notes in Statistics), vol. 109, 135-152. Springer, New York (1996)

    Google Scholar 

  20. Esbensen, K. and Geladi, P.: Strategy of multivariate image analysis (MIA). Chemometrics & Intelligent Laboratory Systems, 7 (1-2), 67-86 (1989)

    Article  Google Scholar 

  21. Fisher, R. and MacKenzie, W.: Studies in crop variation, ii, the manurial re-sponse of different potato varieties. Journal of Agricultural Science, 13, 411-444 (1923)

    Google Scholar 

  22. Golub, G. H. and van Loan, C. F.: Matrix Computation. John Hopkins, Baltimore, (1996)

    Google Scholar 

  23. Gomez, J. C. and Baeyens, E.: Subspace-based identification algorithms for hammerstein and wiener models. European Journal of Control, 11 (2), 127-136 (2005)

    Article  MathSciNet  Google Scholar 

  24. Hastie, T.: Principal curves and surfaces. Technical report no. 11, Department of Statistics, Stanford University (1984)

    Google Scholar 

  25. Hastie, T. and Stuetzle, W.: Principal curves. Journal of the American Statistical Association 84 (406), 502-516 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  26. Hermann, T, Meinicke, P., and Ritter, H.: Principal curve sonification. In: Proceedings of International Conference on Auditory Display, 2-5 April 2000, Atlanta, Georgia, 81-86 (2000)

    Google Scholar 

  27. Hertz, J., Krogh, A., and Palmer, R. G.: Introduction to the Theory of Neural Computing. Addison-Wesley, Redwood City, CA (1991)

    Google Scholar 

  28. Hotelling, H.: Analysis of a complex of statistical variables into principal com-ponents. Journal of Educational Psychology, 24 417-441 (1933)

    Article  Google Scholar 

  29. Jackson, J. E.: Principal components and factor analysis: Part III: What is factor analysis. Journal of Quality Technology, 13 (2), 125-130 (1981)

    Google Scholar 

  30. Jackson, J. E.: A Users Guide to Principal Components. Wiley Series in Prob-ability and Mathematical Statistics, John Wiley & Sons, New York (1991)

    Book  Google Scholar 

  31. Jia, F., Martin, E. B., and Morris, A. J.: Non-linear principal component analysis for process fault detection. Computers & Chemical Engineering, 22 (Supple-ment), S851-S854 (1998)

    Article  Google Scholar 

  32. Joliffe, I. T.: Principal Component Analysis. Springer, New York, (1986)

    Google Scholar 

  33. Kégl, B.: Principal Curves: Learning, Design and Applications. PhD thesis, Department of Computer Science, Concordia University, Montréal, Québec, Canada, 2000

    Google Scholar 

  34. Kégl, B., Krzyzak, A., Linder, T., and Zeger, K.: A polygonal line algorithm for constructing principal curves. In: Neural Information Processing (NIPS ’98), Denver, CO, 1-3 December 1998, 501-507. MIT Press (1998)

    Google Scholar 

  35. Kégl, B., Krzyzak, A., Linder, T., and Zeger, K.: Learning and design of principal curves. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(3), 281-297 (2000)

    Article  Google Scholar 

  36. Kim, K. I., Jung, K., and Kim, H. J.: Face recognition using kernel principal component analysis. IEEE Signal Processing Letters, 9 (2), 40-42 (2002)

    Article  Google Scholar 

  37. Kramer, M. A.: Nonlinear principal component analysis using autoassociative neural networks. AIChE Journal, 37 (3), 233-243 (1991)

    Article  Google Scholar 

  38. Kruger, U., Antory, D., Hahn, J., Irwin, G. W., and McCullough, G.: Introduc-tion of a nonlinearity measure for principal component models. Computers & Chemical Engineering, 29 (11-12), 2355-2362 (2005)

    Article  Google Scholar 

  39. Krzanowski, W. J.: Cross-validatory choice in principal component analysis: Some sampling results. Journal of Statistical Computation and Simulation, 18, 299-314 (1983)

    Article  Google Scholar 

  40. Kwok, J. T. Y. and Tsang, I. W. H.: The pre-image problem in kernel methods. IEEE Transactions on Neural Networks, 15 (6), 1517-1525 (2004)

    Article  Google Scholar 

  41. Lacy, S. L. and Bernstein, D. S.: Subspace identification for non-linear systems with measured input non-linearities. International Journal of Control, 78 (12), 906-921 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  42. Leeuw J. d.: Nonlinear principal component analysis. In: Caussinus, H., Ettinger, P., and Tomassone, R. (eds) Proceedings in Computational Statistics (COMPSTAT 1982) October 30 -September 3, Toulouse, France 1982. Physica-Verlag, Wien (1982)

    Google Scholar 

  43. Lovera, M., Gustafsson, T., and Verhaegen, M.: Recursive subspace identifica-tion of linear and nonlinear wiener type models. Automatica, 36 (11), 1639-1650 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  44. Malinowski, E. R.: Factor Analysis in Chemistry. John Wiley & Sons, New York (2002)

    Google Scholar 

  45. Mardia, K. V., Kent, J. T., and Bibby, J. M.: Multivariate Analysis. Probability and Mathematical Statistics. Academic Press, London, (1979)

    MATH  Google Scholar 

  46. Morales, M.: Geometric Data Fitting. PhD thesis, University of Washington (1998)

    Google Scholar 

  47. Nara, Y., Jianming Y., and Suematsu, Y.: Face recognition using improved principal component analysis. In: Proceedings of 2003 International Symposium on Micromechatronics and Human Science (IEEE Cat. No. 03TH8717), 77-82 (2003)

    Google Scholar 

  48. Nomikos, P. and MacGregor, J. F.: Monitoring of batch processes using multiway principal component analysis. AIChE Journal, 40 (8), 1361-1375 (1994)

    Article  Google Scholar 

  49. Paluš, M. and Dvořák, I.: Singular-value decomposition in attractor recon-struction: Pitfals and precautions. Physica D: Nonlinear Phenomena, 5 (1-2), 221-234 (1992)

    Google Scholar 

  50. Parsopoulos, K. E. and Vrahatis, M. N.: Recent approaches to global optimiza-tion problems through particle swarm optimization. Natural Computing, 1 (2-3), 235-306 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  51. Pearson, C.: On lines and planes of closest fit to systems of points in space. Phil. Mag., Series B., 2 (11), 559-572 (1901)

    Article  Google Scholar 

  52. Qin, S. J., Valle, S., and Piovoso, M. J.: On unifying multi-block analysis with application to decentralized process monitoring. Journal of Chemometrics, 10, 715-742 (2001)

    Article  Google Scholar 

  53. Reinhard, K. and Niranjan, M.: Subspace models for speech transitions using principal curves. Proceedings of Institute of Acoustics, 20 (6), 53-60 (1998)

    Google Scholar 

  54. Reinhard, K. and Niranjan, M.: Parametric subspace modeling of speech tran-sitions. Speech Communication, 27 (1), 19-42 (1999)

    Article  Google Scholar 

  55. Sandilya, S. and Kulkarni, S. R.: Principal curves with bounded turn. In: Pro-ceedings of the IEEE International Symposium on Information Theory, Sorento, 25-30 June 2000, Sorento, Italy (2000)

    Google Scholar 

  56. Schölkopf, B. and Smola, A. J.: Learning with Kernels: Support Vector Ma-chines, Regularization, Optimization, and Beyond. The MIT Press, Cambridge, MA (2002)

    Google Scholar 

  57. Schölkopf, B. and Smola, A. J., and Müller, K. : Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10 (5), 1299-1319 (1998)

    Article  Google Scholar 

  58. Shanmugam, R. and Johnson, C.: At a crossroad of data envelopment and principal component analyses. Omega, 35 (4), 351-364 (2007)

    Article  Google Scholar 

  59. Shawe-Taylor, J. and Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, West Nyack, NY (2004)

    Google Scholar 

  60. Silverman, B. W.: Some aspects of spline smoothing. Journal of the Royal Statistical Society, Series B, 47 (1), 1-52 (1985)

    MATH  MathSciNet  Google Scholar 

  61. Smola, A. J., Williamson, R. C., Mika, S., and Schölkopf, B.: Regularized princi-pal manifolds. In: Fischer, P., and Simon, H. U. (eds. ) Computational Learning Theory (EuroCOLT ’99), Lecture Notes in Artificial Intelligence, vol. 1572, 214-229, Springer, Heidelberg (1999)

    Google Scholar 

  62. Socas-Navarro, H.: Feature extraction techniques for the analysis of spectral polarization profiles. Astrophysical Journal, 620, 517-522 (2005)

    Article  Google Scholar 

  63. Srinivas, M. and Patnaik, L. M.: Genetic algorithms: A survey. Computer, 27 (6), 17-26 (1994)

    Article  Google Scholar 

  64. Stoica, P., Eykhoff, P., Janssen, P., and Söderström, T.: Model structure se-lection by cross-validation. International Journal of Control, 43 (6), 1841-1878 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  65. Stone, M.: Cross-validatory choice and assessment of statistical prediction (with discussion). Journal of the Royal Statistical Society (Series B), 36, 111-133 (1974)

    MATH  Google Scholar 

  66. Stoyanova, R. and Brown, T. R.: Nmr spectral quantitation by principal component analysis. NMR in Biomedicine, 14 (4), 271-277 (2001)

    Article  Google Scholar 

  67. Sylvester, J. J.: On the reduction of a bilinear quantic of the nth order to the form of a sum of n products by a double orthogonal substitution. Messenger of Mathematics, 19, 42-46 (1889)

    Google Scholar 

  68. Tan, S. and Mavrovouniotis, M. L.: Reducing data dimensionality through opti-mizing neural network inputs. AIChE Journal, 41 (6), 1471-1480 (1995)

    Article  Google Scholar 

  69. Tibshirani, R.: Principal curves revisited. Statistics and Computation, 2 (4), 183-190 (1992)

    Article  Google Scholar 

  70. Trafalis, T. B., Richman, M. B., White, A., and Santosa, B.: Data mining tech-niques for improved wsr-88d rainfall estimation. Computers & Industrial Engi-neering, 43 (4), 775-786 (2002)

    Article  Google Scholar 

  71. Huffel, S. van and Vandewalle, J.: The Total Least Squares Problem: Compu-tational Aspects and Analysis. SIAM, Philadelphia (1991)

    Google Scholar 

  72. Vaswani, N. and Chellappa, R.: Principal components null space analysis for image and video classification. IEEE Transactions on Image Processing, 15 (7), 1816-1830 (2006)

    Article  Google Scholar 

  73. ’ose, B.: K-segments algorithm for finding principal curves. Technical Report IAS-UVA-00-11, Institute of Computer Science, University of Amsterdam (2000)

    Google Scholar 

  74. Verhaegen, M. and Westwick, D.: Identifying mimo hammerstein systems in the context of subspace model identification methods. International Journal of Control, 63 (2), 331-350 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  75. Wax, M. and Kailath, T.: Detection of signals by information theoretic criteria. IEEE Transactions on Acoustics, Speech & Signal Processing, 33 (2), 387-392 (1985)

    Article  MathSciNet  Google Scholar 

  76. Westwick, D. and Verhaegen, M.: Identifying mimo wiener systems using sub-space model identification methods. Signal Processing, 52 (2), 235-258 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  77. Wise, B. M. and Ricker, N. L.: Identification of finite impulse respnose models by principal components regression: Frequency response properties. Process Control & Quality, 4, 77-86 (1992)

    Google Scholar 

  78. Wold, H.: Estimation of principal components and related models by itera-tive least squares. In: Krishnaiah, P. R. (ed. ) Multivariate Analysis, 391-420. Academic Press, N. Y. (1966)

    Google Scholar 

  79. Wold, S.: Cross validatory estimation of the number of principal components in factor and principal component models. Technometrics, 20 (4), 397-406 (1978)

    Article  MATH  Google Scholar 

  80. Wold, S., Esbensen, K., and Geladi, P.: Principal component analysis. Chemo-metrics and Intelligent Laboratory Systems, 2, 37-52 (1987)

    Article  Google Scholar 

  81. Yoo, C. K. and Lee, I.: Nonlinear multivariate filtering and bioprocess monitoring for supervising nonlinear biological processes. Process Biochemistry, 41 (8), 1854-1863 (2006)

    Article  Google Scholar 

  82. Zeng, Z. and Zou, X.: Application of principal component analysis to champ radio occultation data for quality control and a diagnostic study. Monthly Weather Review, 134 (11), 3263-3282 (2006)

    Article  Google Scholar 

  83. Zhang, J. and Chen, D.: Constraint k-segment principal curves. In: Huang, De-Sh., Li, K. and Irwin, G. W. (eds. ) Intelligent Computing, Lecture Notes in Computer Sciences, vol. 4113, 345-350. Springer, Berlin Heidelberg New York (2006)

    Google Scholar 

  84. Zhang, J., Chen, D., and Kruger, U.: Constrained k-segments principal curves and its applications in intelligent transportation systems. Technical report, Department of Computer Science and Engineering, Fudan University, Shanghai, P. R. China (2007)

    Google Scholar 

  85. Zhang, J. and Wang, J.: An overview of principal curves (in chinese). Chinese Journal of Computers, 26 (2), 137-148 (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronics Electrical Engineering and Computer Science, Queen's University Belfast, BT9 5AH, Belfast, UK

    Uwe Kruger

  2. Department of Computer Science and Engineering, Fudan University, 200433, Shanghai, China

    Junping Zhang

  3. National Key Laboratory of Industrial Control Technology, Zhejiang University, 310027, Hangzhou, China

    Lei Xie

Authors
  1. Uwe Kruger
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junping Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lei Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Leicester, University Road, LE1 7RH, Leicester, UK

    Alexander N. Gorban

  2. University of Paris-Sud - CNRS, Bâtiment 2000, 91898, Orsay, France

    Balázs Kégl

  3. University of Missouri - Rolla, 1870 Miner Circle, 65409, Rolla, MO, USA

    Donald C. Wunsch

  4. Institut Curie Service Bioinformatique, rue d'Ulm 26, 75248, Paris, France

    Andrei Y. Zinovyev

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kruger, U., Zhang, J., Xie, L. (2008). Developments and Applications of Nonlinear Principal Component Analysis – a Review. In: Gorban, A.N., Kégl, B., Wunsch, D.C., Zinovyev, A.Y. (eds) Principal Manifolds for Data Visualization and Dimension Reduction. Lecture Notes in Computational Science and Enginee, vol 58. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73750-6_1

Download citation

Publish with us

Policies and ethics

Search

Navigation