Skip to main content
SpringerLink
Log in
Book cover

International Statistical and Optimization Perspectives Workshop "Subspace, Latent Structure and Feature Selection"

SLSFS 2005: Subspace, Latent Structure and Feature Selection pp 34–51Cite as

Overview and Recent Advances in Partial Least Squares

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 3940))

Abstract

Partial Least Squares (PLS) is a wide class of methods for modeling relations between sets of observed variables by means of latent variables. It comprises of regression and classification tasks as well as dimension reduction techniques and modeling tools. The underlying assumption of all PLS methods is that the observed data is generated by a system or process which is driven by a small number of latent (not directly observed or measured) variables. Projections of the observed data to its latent structure by means of PLS was developed by Herman Wold and coworkers [48,49,52].

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Almøy, T.: A simulation study on comparison of prediction models when only a few components are relevant. Computational Statistics and Data Analysis 21, 87–107 (1996)

    Article  MATH  Google Scholar 

  2. Aronszajn, N.: Theory of reproducing kernels. Transactions of the American Mathematical Society 68, 337–404 (1950)

    Article  MathSciNet  MATH  Google Scholar 

  3. Baffi, G., Martin, E.B., Morris, A.J.: Non-linear projection to latent structures revisited (the neural network PLS algorithm). Computers Chemical Engineering 23, 1293–1307 (1999)

    Article  Google Scholar 

  4. Barker, M., Rayens, W.S.: Partial least squares for discrimination. Journal of Chemometrics 17, 166–173 (2003)

    Article  Google Scholar 

  5. Bartlett, M.S.: Further aspects of the theory of multiple regression. In: Proceedings of the Cambridge Philosophical Society, vol. 34, pp. 33–40 (1938)

    Google Scholar 

  6. De Bie, T., Cristianini, N., Rosipal, R.: Eigenproblems in Pattern Recognition. In: Bayro-Corrochano, E. (ed.) Handbook of Geometric Computing: Applications in Pattern Recognition, Computer Vision, Neuralcomputing, and Robotics, pp. 129–170. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  7. Butler, N.A., Denham, M.C.: The peculiar shrinkage properties of partial least squares regression. Journal of the Royal Statistical Society: B 62, 585–593 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  8. de Jong, S.: SIMPLS: an alternative approach to partial least squares regression. Chemometrics and Intelligent Laboratory Systems 18, 251–263 (1993)

    Article  Google Scholar 

  9. de Jong, S., Wise, B.M., Ricker, N.L.: Canonical partial least squares and continuum power regression. Journal of Chemometrics 15, 85–100 (2001)

    Article  Google Scholar 

  10. Frank, I.E.: A nonlinear PLS model. Chemolab 8, 109–119 (1990)

    Google Scholar 

  11. Frank, I.E., Friedman, J.H.: A Statistical View of Some Chemometrics Regression Tools. Technometrics 35, 109–147 (1993)

    Article  MATH  Google Scholar 

  12. Goutis, C.: Partial least squares yields shrinkage estimators. The Annals of Statistics 24, 816–824 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  13. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, Heidelberg (2001)

    Book  MATH  Google Scholar 

  14. Helland, I.S.: Some theoretical aspects of partial least squares regression. Chemometrics and Intelligent Laboratory Systems 58, 97–107 (1999)

    Article  Google Scholar 

  15. Hestenes, M., Stiefel, E.: Methods for conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards 49, 409–436 (1952)

    Article  MathSciNet  MATH  Google Scholar 

  16. Hoerl, A.E., Kennard, R.W.: Ridge regression: bias estimation for nonorthogonal problems. Technometrics 12, 55–67 (1970)

    Article  MATH  Google Scholar 

  17. Höskuldsson, A.: PLS Regression Methods. Journal of Chemometrics 2, 211–228 (1988)

    Article  Google Scholar 

  18. Hulland, J.S.: Use of partial least squares (PLS) in strategic management research: A review of four recent studies. Strategic Management Journal 20, 195–204 (1999)

    Article  Google Scholar 

  19. Lanczos, C.: An iteration method for the solution of the eigenvalue problem of linear differential and integral operators. Journal of Research of the National Bureau of Standards 45, 225–280 (1950)

    Article  MathSciNet  Google Scholar 

  20. Lewi, P.J.: Pattern recognition, reflection from a chemometric point of view. Chemometrics and Intelligent Laboratory Systems 28, 23–33 (1995)

    Article  Google Scholar 

  21. Lingjærde, O.C., Christophersen, N.: Shrinkage Structure of Partial Least Squares. Scandinavian Journal of Statistics 27, 459–473 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  22. Lobaugh, N.J., West, R., McIntosh, A.R.: Spatiotemporal analysis of experimental differences in event-related potential data with partial least squares. Psychophysiology 38, 517–530 (2001)

    Article  Google Scholar 

  23. Manne, R.: Analysis of Two Partial-Least-Squares Algorithms for Multivariate Calibration. Chemometrics and Intelligent Laboratory Systems 2, 187–197 (1987)

    Article  Google Scholar 

  24. Mardia, K.V., Kent, J.T., Bibby, J.M.: Multivariate Analysis. Academic Press, London (1997)

    MATH  Google Scholar 

  25. Martens, M., Martens, H.: Partial Least Squares Regression. In: Piggott, J.R. (ed.) Statistical Procedures in Food Research, pp. 293–359. Elsevier Applied Science, London (1986)

    Google Scholar 

  26. Massy, W.F.: Principal components regression in exploratory statistical research. Journal of the American Statistical Association 60, 234–256 (1965)

    Article  Google Scholar 

  27. Momma, M.: Efficient Computations via Scalable Sparse Kernel Partial Least Squares and Boosted Latent Features. In: Proceedings of SIGKDD International Conference on Knowledge and Data Mining, Chicago, IL, pp. 654–659 (2005)

    Google Scholar 

  28. Nguyen, D.V., Rocke, D.M.: Tumor classification by partial least squares using microarray gene expression data. Bioinformatics 18, 39–50 (2002)

    Article  Google Scholar 

  29. Nilsson, J., de Jong, S., Smilde, A.K.: Multiway Calibration in 3D QSAR. Journal of Chemometrics 11, 511–524 (1997)

    Article  Google Scholar 

  30. Parlett, B.: The symmetric eigenvalue problem. SIAM, Philadelphia (1998)

    Book  MATH  Google Scholar 

  31. Phatak, A., de Hoog, F.: Exploiting the connection between PLS, Lanczos, and conjugate gradients: Alternative proofs of some properties of PLS. Journal of Chemometrics 16, 361–367 (2003)

    Article  Google Scholar 

  32. Phatak, A., Rilley, P.M., Penlidis, A.: The asymptotic variance of the univariate PLS estimator. Linear Algebra and its Applications 354, 245–253 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  33. Rännar, S., Lindgren, F., Geladi, P., Wold, S.: A PLS kernel algorithm for data sets with many variables and fewer objects. Part 1: Theory and algorithm. Chemometrics and Intelligent Laboratory Systems 8, 111–125 (1994)

    Google Scholar 

  34. Rosipal, R., Trejo, L.J.: Kernel Partial Least Squares Regression in Reproducing Kernel Hilbert Space. Journal of Machine Learning Research 2, 97–123 (2001)

    MATH  Google Scholar 

  35. Rosipal, R., Trejo, L.J., Matthews, B.: Kernel PLS-SVC for Linear and Nonlinear Classification. In: Proceedings of the Twentieth International Conference on Machine Learning, Washington, DC, pp. 640–647 (2003)

    Google Scholar 

  36. Sampson, P.D., Streissguth, A.P., Barr, H.M., Bookstein, F.L.: Neurobehavioral effects of prenatal alcohol: Part II. Partial Least Squares analysis. Neurotoxicology and tetralogy 11, 477–491 (1989)

    Article  Google Scholar 

  37. Saunders, C., Gammerman, A., Vovk, V.: Ridge Regression Learning Algorithm in Dual Variables. In: Proceedings of the 15th International Conference on Machine Learning, Madison, WI, pp. 515–521 (1998)

    Google Scholar 

  38. Schölkopf, B., Smola, A.J.: Learning with Kernels – Support Vector Machines, Regularization, Optimization and Beyond. The MIT Press, Cambridge (2002)

    Google Scholar 

  39. Smilde, A., Bro, R., Geladi, P.: Multi-way Analysis: Applications in the Chemical Sciences. Wiley, Chichester (2004)

    Book  Google Scholar 

  40. Stone, M., Brooks, R.J.: Continuum Regression: Cross-validated Sequentially Constructed Prediction Embracing Ordinary Least Squares, Partial Least Squares and Principal Components Regression. Journal of the Royal Statistical Society: B 52, 237–269 (1990)

    MathSciNet  MATH  Google Scholar 

  41. Vinod, H.D.: Canonical ridge and econometrics of joint production. Journal of Econometrics 4, 147–166 (1976)

    Article  MATH  Google Scholar 

  42. Wahba, G.: Splines Models of Observational Data. Series in Applied Mathematics, vol. 59. SIAM, Philadelphia (1990)

    Book  MATH  Google Scholar 

  43. Wallerius, J., Trejo, L.J., Matthew, R., Rosipal, R., Caldwell, J.A.: Robust feature extraction and classification of EEG spectra for real-time classification of cognitive state. In: Proceedings of 11th International Conference on Human Computer Interaction, Las Vegas, NV (2005)

    Google Scholar 

  44. Wangen, L.E., Kowalsky, B.R.: A multiblock partial least squares algorithm for investigating complex chemical systems. Journal of Chemometrics 3, 3–20 (1989)

    Article  Google Scholar 

  45. Wegelin, J.A.: A survey of Partial Least Squares (PLS) methods, with emphasis on the two-block case. Technical report, Department of Statistics, University of Washington, Seattle (2000)

    Google Scholar 

  46. Westerhuis, J., Kourti, T., MacGregor, J.: Analysis of multiblock and hierarchical PCA and PLS models. Journal of Chemometrics 12, 301–321 (1998)

    Article  Google Scholar 

  47. Wold, H.: Path models with latent variables: The NIPALS approach. In: Blalock, H.M., et al. (eds.) Quantitative Sociology: International perspectives on mathematical and statistical model building, pp. 307–357. Academic Press, London (1975)

    Chapter  Google Scholar 

  48. Wold, H.: Soft modeling: the basic design and some extensions. In: Jöreskog, J.-K., Wold, H. (eds.) Systems Under Indirect Observation, vol. 2, pp. 1–53. North Holland, Amsterdam (1982)

    Google Scholar 

  49. Wold, H.: Partial least squares. In: Kotz, S., Johnson, N.L. (eds.) Encyclopedia of the Statistical Sciences, vol. 6, pp. 581–591. John Wiley & Sons, Chichester (1985)

    Google Scholar 

  50. Wold, S.: Nonlinear partial least squares modeling II, Spline inner relation. Chemolab 14, 71–84 (1992)

    Google Scholar 

  51. Wold, S., Kettaneh-Wold, N., Skagerberg, B.: Nonlinear PLS modelling. Chemometrics and Intelligent Laboratory Systems 7, 53–65 (1989)

    Article  Google Scholar 

  52. Wold, S., Ruhe, H., Wold, H., Dunn III., W.J.: The collinearity problem in linear regression. The partial least squares (PLS) approach to generalized inverse. SIAM Journal of Scientific and Statistical Computations 5, 735–743 (1984)

    Article  MATH  Google Scholar 

  53. Worsley, K.J.: An overview and some new developments in the statistical analysis of PET and fMRI data. Human Brain Mapping 5, 254–258 (1997)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Austrian Research Institute for Artificial Intelligence, Freyung 6/6, A-1010, Vienna, Austria

    Roman Rosipal

  2. Department of Computer Science and Electrical Engineering, TU Berlin, Franklinstraße 28/29, D-10587, Berlin, Germany

    Nicole Krämer

Authors
  1. Roman Rosipal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nicole Krämer
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. ISIS Research Group, University of Southampton, Southampton, U.K.

    Craig Saunders

  2. Dept. of Knowledge Technologies, Jozef Stefan Institute, Jamova 39, 1000, Ljubljana, Slovenia

    Marko Grobelnik

  3. School of Electronics and Computer Science, University of Southampton, Building 1, Highfield Campus, SO17 1BJ, Southampton, UK

    Steve Gunn

  4. The Centre for Computational Statistics and Machine Learning Department of Computer Science, University College London, Gower St., WC1E 6BT, London, UK

    John Shawe-Taylor

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Rosipal, R., Krämer, N. (2006). Overview and Recent Advances in Partial Least Squares. In: Saunders, C., Grobelnik, M., Gunn, S., Shawe-Taylor, J. (eds) Subspace, Latent Structure and Feature Selection. SLSFS 2005. Lecture Notes in Computer Science, vol 3940. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11752790_2

Download citation

  • DOI: https://doi.org/10.1007/11752790_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-34137-6

  • Online ISBN: 978-3-540-34138-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation