Finite Mixture of Linear Regression Models: An Adaptive Constrained Approach to Maximum Likelihood Estimation

Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 456)


In order to overcome the problems due to the unboundedness of the likelihood, constrained approaches to maximum likelihood estimation in the context of finite mixtures of univariate and multivariate normals have been presented in the literature. One main drawback is that they require a knowledge of the variance and covariance structure. We propose a fully data-driven constrained method for estimation of mixtures of linear regression models. The method does not require any prior knowledge of the variance structure, it is invariant under change of scale in the data and it is easy and ready to implement in standard routines.


Likelihood Function Linear Regression Model Finite Mixture Scale Balance Likelihood Principle 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Biernacki C (2004) An asymptotic upper bound of the likelihood to prevent Gaussian mixtures from degenerating. Technical Report, Université de Franche-ComtéGoogle Scholar
  2. 2.
    Biernacki C, Chrétien S (2003) Degeneracy in the maximum likelihood estimation of univariate Gaussian mixtures with the EM. Stat Probab Lett 61:373–382MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Chen J, Tan X, Zhang R (2008) Inference for normal mixtures in mean and variance. Statistica Sinica 18(2):443MathSciNetzbMATHGoogle Scholar
  4. 4.
    Ciuperca G, Ridolfi A, Idier J (2003) Penalized maximum likelihood estimator for normal mixtures. Scand J Stat 30(1):45–59MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Day NE (1969) Estimating the components of a mixture of two normal distributions. Biometrika 56:463–474MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc: Ser B (Stat Methodol) 39:1–38MathSciNetzbMATHGoogle Scholar
  7. 7.
    DeSarbo WS, Cron WL (1988) A maximum likelihood methodology for clusterwise linear regression. J Classif 5(2):249–282MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Eggermont PPB, LaRiccia VN (2001) Maximum penalized likelihood estimation, vol 1. Springer, New YorkzbMATHGoogle Scholar
  9. 9.
    Gallegos MT, Ritter G (2009) Trimmed ML estimation of contaminated mixtures. Sankhya: Indian J Stat Ser A (2008-):164–220Google Scholar
  10. 10.
    García-Escudero LA, Gordaliza A, Matran C, Mayo-Iscar A (2008) A general trimming approach to robust cluster analysis. Ann Stat 36:1324–1345MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    García-Escudero LA, Gordaliza A, San Martń R, Van Aelst S, Zamar R (2009) Robust linear clustering. J R Stat Soc: Ser B (Stat Methodol) 71(1):301–318MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    García-Escudero LA, Gordaliza A, Mayo-Iscar A, San Martń R (2010) Robust clusterwise linear regression through trimming. Comput Stat Data Anal 54(12):3057–3069MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Green PJ (1990) On use of the EM for penalized likelihood estimation. J R Stat Soc: Ser B (Stat Methodol) 443–452Google Scholar
  14. 14.
    Hathaway RJ (1985) A constrained formulation of maximum-likelihood estimation for normal mixture distributions. Ann Stat 13:795–800MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Hennig C (2000) Identifiablity of models for clusterwise linear regression. J Classif 17(2):273–296MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Huber PJ (1967) The behavior of maximum likelihood estimates under nonstandard conditions. In: Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, vol 1, no 1, pp 221–233Google Scholar
  17. 17.
    Huber PJ (1981) Robust statistics. Wiley, New YorkCrossRefzbMATHGoogle Scholar
  18. 18.
    Ingrassia S (2004) A likelihood-based constrained algorithm for multivariate normal mixture models. Stat Methods Appl 13:151–166MathSciNetCrossRefGoogle Scholar
  19. 19.
    Ingrassia S, Rocci R (2007) A constrained monotone EM algorithm for finite mixture of multivariate Gaussians. Comput Stat Data Anal 51:5339–5351MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Kiefer NM (1978) Discrete parameter variation: efficient estimation of a switching regression model. Econometrica 46:427–434MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Kiefer J, Wolfowitz J (1956) Consistency of the maximum likelihood estimator in the presence of infinitely many incidental parameters. Ann Math Stat 27:886906MathSciNetzbMATHGoogle Scholar
  22. 22.
    McLachlan GJ, Peel D (2000) Finite mixture models. John Wiley and Sons, New YorkGoogle Scholar
  23. 23.
    Quandt RE (1972) A new approach to estimating switching regressions. J Am Stat Assoc 67(338):306–310CrossRefzbMATHGoogle Scholar
  24. 24.
    Quandt RE, Ramsey JB (1978) Estimating mixtures of normal distributions and switching regressions. J Am Stat Assoc 73(364):730–738MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Ritter G (2014) Robust cluster analysis and variable selection. Monographs on statistics and applied probability, vol 137. CRC PressGoogle Scholar
  26. 26.
    Snoussi H, Mohammad-Djafari A (2001) Penalized maximum likelihood for multivariate Gaussian mixture. In: Fry RL (ed) MaxEnt workshops: Bayesian inference and maximum entropy methods. pp 36–46, Aug 2001Google Scholar
  27. 27.
    Seo B, Kim D (2012) Root selection in normal mixture models. Comput Stat Data Anal 56:2454–2470MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Tan X, Chen J, Zhang R (2007) Consistency of the constrained maximum likelihood estimator in finite normal mixture models. In: Proceedings of the American Statistical Association, American Statistical Association, Alexandria, VA, 2007, pp 2113–2119 [CD-ROM]Google Scholar
  29. 29.
    Xu J, Tan X, Zhang R (2010) A note on Phillips (1991): “A constrained maximum likelihood approach to estimating switching regressions”. J Econom 154:35–41MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  1. 1.DEF, University of Tor VergataRomeItaly
  2. 2.DiSFPEQ, University G. d’AnnunzioChieti-PescaraItaly

Personalised recommendations