Skip to main content
SpringerLink
Log in

Families of Parsimonious Finite Mixtures of Regression Models

  • Conference paper
Advances in Statistical Models for Data Analysis

Abstract

Finite mixtures of regression (FMR) models offer a flexible framework for investigating heterogeneity in data with functional dependencies. These models can be conveniently used for unsupervised learning on data with clear regression relationships. We extend such models by imposing an eigen-decomposition on the multivariate error covariance matrix. By constraining parts of this decomposition, we obtain families of parsimonious mixtures of regressions and mixtures of regressions with concomitant variables. These families of models account for correlations between multiple responses. An expectation-maximization algorithm is presented for parameter estimation and performance is illustrated on simulated and real data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Banfield, J.D., Raftery, A.E.: Model-based Gaussian and non-Gaussian clustering. Biometrics 49(3), 803–821 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  2. Böhning, D., Dietz, E., Schaub, R., Schlattmann, P., Lindsay, B.G.: The distribution of the likelihood ratio for mixtures of densities from the one-parameter exponential family. Ann. Inst. Stat. Math. 46(2), 373–388 (1994)

    Article  MATH  Google Scholar 

  3. Browne, R.P., McNicholas, P.D.: ‘mixture’: Mixture Models for Clustering and Classification. R package version 1.0. (2013)

    Google Scholar 

  4. Campbell, N.A., Mahon, R.J.: A multivariate study of variation in two species of rock crab of the genus leptograpsus. Aust. J. Zool. 22(3), 417–425 (1974)

    Article  Google Scholar 

  5. Celeux, G., Govaert, G.: Gaussian parsimonious clustering models. Pattern Recogn. 28(5), 781–793 (1995)

    Article  Google Scholar 

  6. Dang, U.J., Punzo, A., McNicholas, P.D., Ingrassia, S., Browne, R.P.: Multivariate response and parsimony for Gaussian cluster-weighted models [arXiv preprint arXiv:1411.0560] (2014)

    Google Scholar 

  7. Dasgupta, A., Raftery, A.E.: Detecting features in spatial point processes with clutter via model-based clustering. J. Am. Stat. Assoc. 93(441), 294–302 (1998)

    Article  MATH  Google Scholar 

  8. DeSarbo, W.S., Cron, W.L.: A maximum likelihood methodology for clusterwise linear regression. J. Classif. 5(2), 249–282 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  9. Fraley, C., Raftery, A.E.: Model-based clustering, discriminant analysis, and density estimation. J. Am. Stat. Assoc. 97(458), 611–631 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  10. Galimberti, G., Soffritti, G.: A multivariate linear regression analysis using finite mixtures of t distributions. Comput. Stat. Data Anal. 71, 138–150 (2014)

    Article  MathSciNet  Google Scholar 

  11. Gershenfeld, N.: Nonlinear inference and cluster-weighted modeling. Ann. N. Y. Acad. Sci. 808(1), 18–24 (1997)

    Article  Google Scholar 

  12. Hartigan, J.A., Wong, M.A.: A k-means clustering algorithm. J. R. Stat. Soc. C App. 28(1), 100–108 (1979)

    MATH  Google Scholar 

  13. Hubert, L., Arabie, P.: Comparing partitions. J. Classif. 2(1), 193–218 (1985)

    Article  Google Scholar 

  14. Keribin, C.: Consistent estimation of the order of mixture models. Sankhya Ser. A 62, 49–66 (2000)

    MathSciNet  MATH  Google Scholar 

  15. Leisch, F.: FlexMix: a general framework for finite mixture models and latent class regression in R. J. Stat. Softw. 11(8), 1–18 (2004)

    Google Scholar 

  16. Lindsay, B.G.: Mixture models: theory, geometry and applications. In: NSF-CBMS Regional Conference Series in Probability and Statistics, vol. 5 (1995)

    Google Scholar 

  17. McNicholas, P.D., Murphy, T.B., McDaid, A.F., Frost, D.: Serial and parallel implementations of model-based clustering via parsimonious Gaussian mixture models. Comput. Stat. Data Anal. 54(3), 711–723 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  18. R Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. Vienna, Austria (2014)

    Google Scholar 

  19. Rand, W.M.: Objective criteria for the evaluation of clustering methods. J. Am. Stat. Assoc. 66(336), 846–850 (1971)

    Article  Google Scholar 

  20. Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6, 461–464 (1978)

    Article  MATH  Google Scholar 

  21. Soffritti, G., Galimberti, G.: Multivariate linear regression with non-normal errors: a solution based on mixture models. Stat. Comput. 21(4), 523–536 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  22. Titterington, D.M., Smith, A.F.M., Makov, U,E.: Statistical Analysis of Finite Mixture Distributions, vol. 7. Wiley, New York (1985)

    Google Scholar 

  23. Venables, W.N., Ripley, B.D.: Modern Applied Statistics with S, 4th edn. Springer, New York (2002)

    Book  MATH  Google Scholar 

  24. Wedel, M.: Concomitant variables in finite mixture models. Statistica Neerlandica 56(3), 362–375 (2002)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work is supported by a Alexander Graham Bell Canada Graduate Scholarship (CGS-D; Dang) as well as a Discovery Grant from the Natural Sciences and Engineering Research Council of Canada (McNicholas).

Author information

Authors and Affiliations

  1. Department of Biology, McMaster University, Hamilton, ON, Canada

    Utkarsh J. Dang

  2. Department of Mathematics & Statistics, McMaster University, Hamilton, ON, Canada

    Paul D. McNicholas

Authors
  1. Utkarsh J. Dang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul D. McNicholas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Utkarsh J. Dang .

Editor information

Editors and Affiliations

  1. Department of Economics "Marco Biagi", University of Modena & Reggio Emilia, Modena, Italy

    Isabella Morlini

  2. Department of Communication and Economics, University of Modena & Reggio Emilia, Reggio Emilia, Italy

    Tommaso Minerva

  3. Department of Statistics, University of Rome “La Sapienza”, Roma, Italy

    Maurizio Vichi

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Dang, U.J., McNicholas, P.D. (2015). Families of Parsimonious Finite Mixtures of Regression Models. In: Morlini, I., Minerva, T., Vichi, M. (eds) Advances in Statistical Models for Data Analysis. Studies in Classification, Data Analysis, and Knowledge Organization. Springer, Cham. https://doi.org/10.1007/978-3-319-17377-1_9

Download citation

Publish with us

Policies and ethics

Search

Navigation