Bayesian Reduced Rank Regression for Classification

  • Heinz SchmidliEmail author
Part of the Studies in Classification, Data Analysis, and Knowledge Organization book series (STUDIES CLASS)


Many classical multivariate analysis methods are special cases of reduced rank regression, including canonical correlation analysis, redundancy analysis, and R. A. Fisher’s linear discriminant analysis. The latter classifies an object based on a few linear combinations of its multivariate measurements. Classical inference for linear discriminant analysis may be based on asymptotic theory or resampling methods. A Bayesian linear discriminant analysis is proposed using Bayesian reduced rank regression as a starting point. The model can be implemented in Bayesian software with Markov chain Monte Carlo approaches and is easily extendable.


  1. Aggarwal, C. C. (Ed.). (2014). Data classification: Algorithms and applications. Boca Raton: CRC Press.Google Scholar
  2. Aitchison, J., & Dunsmore, I. (1975). Statistical prediction analysis. Cambridge: University Press.CrossRefGoogle Scholar
  3. Anderson, T. W. (1951). Estimating linear restrictions on regression coefficients for multivariate normal distributions. Annals of Mathematical Statistics, 22, 327–351.MathSciNetCrossRefGoogle Scholar
  4. Anderson, T. W. (2002). Canonical correlation analysis and reduced rank regression in autoregressive models. Annals of Statistics, 30, 1134–1154.MathSciNetCrossRefGoogle Scholar
  5. Ashby, D. (2006). Bayesian statistics in medicine: A 25 year review. Statistics in Medicine, 25, 3589–3631.MathSciNetCrossRefGoogle Scholar
  6. Bennett, J. E., & Wakefield, J. C. (1996). A comparison of a Bayesian population method with two methods as implemented in commercially available software. Journal of Pharmacokinetics and Biopharmaceutics, 24, 403–432.CrossRefGoogle Scholar
  7. Bornkamp, B., Ohlssen, D., Magnusson, B. P., & Schmidli, H. (2017). Model averaging for treatment effect estimation in subgroups. Pharmaceutical Statistics, 16, 133–142.CrossRefGoogle Scholar
  8. Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., et al. (2017). Stan: A probabilistic programming language. Journal of Statistical Software, 76(1).Google Scholar
  9. Dempster, A., Selwyn, M., & Weeks, B. (1983). Combining historical and randomized controls for assessing trends in proportions. Journal of the American Statistical Association, 78, 221–227.CrossRefGoogle Scholar
  10. Dudek, A. Z., Arodz, T., & Galvez, J. (2006). Computational methods in developing quantitative structure-activity relationships (QSAR): A review. Combinatorial Chemistry and High Throughput Screening, 9, 213–228.CrossRefGoogle Scholar
  11. Dudoit, S., Fridlyand, J., & Speed, T. P. (2002). Comparison of discrimination methods for the classification of tumors using gene expression data. Journal of the American Statistical Association, 97, 77–87.MathSciNetCrossRefGoogle Scholar
  12. Fisch, R., Jones, I., Jones, J., Kerman, J., Rosenkranz, G. K., & Schmidli, H. (2015). Bayesian design of Proof-of-Concept trials. Therapeutical Innovations and Regulatory Science, 49, 155–162.CrossRefGoogle Scholar
  13. Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7, 179–188.CrossRefGoogle Scholar
  14. Gelfand, A. E., & Smith, A. F. M. (1990). Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association, 85, 398–406.MathSciNetCrossRefGoogle Scholar
  15. Gelfand, A. E., Hills, S. E., Racine-Poon, A., & Smith, A. F. M. (1990). Illustration of Bayesian inference in normal data models using Gibbs sampling. Journal of the American Statistical Association, 85, 972–985.CrossRefGoogle Scholar
  16. Geweke, J. (1996). Bayesian reduced rank regression in econometrics. Journal of Econometrics, 75, 121–146.MathSciNetCrossRefGoogle Scholar
  17. Geweke, J. (2004). Getting it right: Joint distribution tests of posterior simulators. Journal of the American Statistical Association, 99, 799–804.MathSciNetCrossRefGoogle Scholar
  18. Gsponer, T., Gerber, F., Bornkamp, B., Ohlssen, D., Vandemeulebroecke, M., & Schmidli, H. (2014). A practical guide to Bayesian group sequential designs. Pharmaceutical Statistics, 13, 71–80.CrossRefGoogle Scholar
  19. Gsteiger, S., Neuenschwander, B., Mercier, F., & Schmidli, H. (2013). Using historical control information for the design and analysis of clinical trials with overdispersed count data. Statistics in Medicine, 32, 3609–3622.MathSciNetCrossRefGoogle Scholar
  20. Grieve, A. P., & Krams, M. (2005). ASTIN: A Bayesian adaptive dose-response trial in acute stroke. Clinical Trials, 2, 340–351.CrossRefGoogle Scholar
  21. Grieve, A. P. (2007). 25 years of Bayesian methods in the pharmaceutical industry: A personal, statistical bummel. Pharmaceutical Statistics, 6, 261–281.CrossRefGoogle Scholar
  22. Holzhauer, B., Wang, C., & Schmidli, H. (2018). Evidence synthesis from aggregate recurrent event data for clinical trial design and analysis. Statistics in Medicine, 37, 867–882.MathSciNetCrossRefGoogle Scholar
  23. Izenman, A. J. (1986). Reduced rank regression procedures for discriminant analysis. American Statistical Association Proceedings of the Statistical Computing Section, 249–253.Google Scholar
  24. Karlsson, S. (2017). Corrigendum to Bayesian reduced rank regression. Journal of Econometrics, 201, 170–171.MathSciNetCrossRefGoogle Scholar
  25. Krzanowski, W. J. (1989). On confidence regions in canonical variate analysis. Biometrika, 76, 107–116.MathSciNetCrossRefGoogle Scholar
  26. Krzanowski, W. J., & Radley, D. (1989). Nonparametric confidence and tolerance regions in canonical variate analysis. Biometrics, 45, 1163–1173.MathSciNetCrossRefGoogle Scholar
  27. Lange, M. R., & Schmidli, H. (2015). Analysis of clinical trials with biologics using dose-time-response models. Statistics in Medicine, 34, 3017–3028.MathSciNetCrossRefGoogle Scholar
  28. Lindley, D. (1975). The future of statistics—a Bayesian 21st century. Supplement Advances in Applied Probability, 7, 106–115.CrossRefGoogle Scholar
  29. Lunn, D., Jackson, C., Best, N., Thomas, A., Spiegelhalter, D. (2012). The BUGS book: A practical introduction to Bayesian analysis. Chapman and Hall/CRCGoogle Scholar
  30. Madsen, R., Lundstedt, T., & Trygg, J. (2010). Chemometrics in metabolomics—a review in human disease diagnosis. Analytica Chimica Acta, 659, 23–33.CrossRefGoogle Scholar
  31. Mardia, K. V., Kent, J. T., & Bibby, J. M. (1979). Multivariate Analysis. London: Academic.zbMATHGoogle Scholar
  32. Mielke, J., Schmidli, H., & Jones, B. (2018). Incorporating historical information in biosimilar trials: Challenges and a hybrid Bayesian-frequentist approach. Biometrical Journal, 60, 564–582.MathSciNetCrossRefGoogle Scholar
  33. Neuenschwander, B., Branson, M., & Gsponer, T. (2008). Critical aspects of the Bayesian approach to phase I cancer trials. Statistics in Medicine, 27, 2420–2439.MathSciNetCrossRefGoogle Scholar
  34. Neuenschwander, B., Capkun-Niggli, G., Branson, M., & Spiegelhalter, D. J. (2010). Summarizing historical information on controls in clinical trials. Clinical Trials, 7, 5–18.CrossRefGoogle Scholar
  35. Pozzi, L., Schmidli, H., Gasparini, M., & Racine-Poon, A. (2013). A Bayesian adaptive dose selection procedure with an overdispersed count endpoint. Statistics in Medicine, 32, 5008–5027.MathSciNetCrossRefGoogle Scholar
  36. Racine, A., Grieve, A. P., Fluehler, H., & Smith, A. F. M. (1986). Bayesian methods in practice: Experiences in the pharmaceutical industry (with Discussion). Applied Statistics, 35, 93–150.MathSciNetCrossRefGoogle Scholar
  37. Racine-Poon, A. (1986). A Bayesian approach to nonlinear random effects models. Biometrics, 4, 1015–1023.MathSciNetzbMATHGoogle Scholar
  38. Racine-Poon, A., Weihs, C., & Smith, A. F. M. (1991). Estimation of relative potency with sequential dilution errors in radioimmunoassay. Biometrics, 47, 1235–1246.CrossRefGoogle Scholar
  39. R Core Team. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing. Vienna, Austria.
  40. Ryan, D. A. J., Hubert, J. J., Carter, E. M., Sprague, J. B., & Parrott, J. (1992). A reduced rank multivariate regression approach to aquatic joint toxicity experiments. Biometrics, 48, 155–162.MathSciNetCrossRefGoogle Scholar
  41. Seewald, W. (1994). Time trend in historical controls for tumour incidences in long-term animal studies. Journal of the Royal Statistical Society Series C (Applied Statistics), 43, 127–137.zbMATHGoogle Scholar
  42. Schmidli, H. (1995). Reduced rank regression with applications to quantitative structure-activity relationships. Heidelberg: Springer.CrossRefGoogle Scholar
  43. Schmidli, H. (1996). Bayesian analysis of reduced rank regression. Test, 51, 159–186.MathSciNetCrossRefGoogle Scholar
  44. Schmidli, H., Bretz, F., & Racine-Poon, A. (2007). Bayesian predictive power for interim adaptation in seamless phase II/III trials where the endpoint is survival up to some specified timepoint. Statistics in Medicine, 26, 4925–4938.MathSciNetCrossRefGoogle Scholar
  45. Schmidli, H., Wandel, S., & Neuenschwander, B. (2013). The network meta-analytic-predictive approach to non-inferiority trials. Statistical Methods in Medical Research, 22, 219–240.MathSciNetCrossRefGoogle Scholar
  46. Schmidli, H., Gsteiger, S., Roychoudhury, S., O’Hagan, A., Spiegelhalter, D., & Neuenschwander, B. (2014). Robust meta-analytic-predictive priors in clinical trials with historical control information. Biometrics, 70, 1023–1032.MathSciNetCrossRefGoogle Scholar
  47. Schmidli, H., Neuenschwander, B., & Friede, T. (2017). Meta-analytic-predictive use of historical variance data for the design and analysis of clinical trials. Computational Statistics and Data Analysis, 113, 100–110.MathSciNetCrossRefGoogle Scholar
  48. Tso, M. K.-S. (1981). Reduced rank regression and canonical analysis. Journal of the Royal Statistical Society B, 43, 183–189.MathSciNetzbMATHGoogle Scholar
  49. Velu, R. P., Reinsel, G. C., & Wichern, D. W. (1986). Reduced rank models for multiple times series. Biometrika, 73, 109–118.MathSciNetCrossRefGoogle Scholar
  50. van der Leeden, R. (1990). Reduced rank regression with structured residuals. Leiden: DSWO Press.Google Scholar
  51. Weber, S., Gelman, A., Lee, D., Betancourt, M., Vehtari, A., & Racine-Poon, A. (2018). Bayesian aggregation of average data: An application in drug development. Annals of Applied Statistics, 3, 1583–1604.MathSciNetCrossRefGoogle Scholar
  52. Weihs, C., & Schmidli, H. (1990). OMEGA (Online Multivariate Exploratory Graphical Analysis): Routine searching for structure (with discussion). Statistical Science, 5, 175–226.MathSciNetCrossRefGoogle Scholar
  53. Weihs, C., & Schmidli, H. (1991). Multivariate exploratory data-analysis in chemical industry. Mikrochimika Acta, 2, 467–482.CrossRefGoogle Scholar
  54. Weihs, C., Baumeister, W., & Schmidli, H. (1993). Classification methods for multivariate quality parameters. Journal of Chemometrics, 7, 131–142.CrossRefGoogle Scholar
  55. Weihs, C. (1993). Canonical discriminant analysis: Comparison of resampling methods and convex-hull approximation. In O. Opitz, B. Lausen, & R. Klar (Eds.), Information and classification. Dortmund: Springer.Google Scholar
  56. Weihs, C., Ligges, U., Moerchen, F., et al. (2007). Classification in music research. Advances in Data Analysis and Classification, 1, 255–291.MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Novartis Statistical MethodologyBaselSwitzerland

Personalised recommendations