Abstract
We consider finite mixtures of generalized linear models with binary output. We prove that cross moments (between the output and the regression variables) up to order three are sufficient to identify all parameters of the model. We propose a least-squares estimation method based on those moments and we prove the consistency and the Gaussian asymptotic behavior of the estimator. We provide simulation results and comparisons with likelihood methods. Numerical experiments were conducted using the R-package morpheus that we developed for our least-squares moment method and with the R-package flexmix for likelihood methods. We then give some possible extensions to finite mixtures of regressions with binary output including both continuous and categorical covariates, and possibly longitudinal data.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Afsari B (2008) Sensitivity analysis for the problem of matrix joint diagonalization. SIAM J Matrix Anal Appl 30(3):1148–1171. https://doi.org/10.1137/060655997
Anandkumar A, Ge R, Hsu D, Kakade SM, Telgarsky M (2014) Tensor decompositions for learning latent variables models. J Mach Learn 15:2773–2832
Auder B, Loum MA (2018) Morpheus: an R package to estimate parameters of logistic regressions mixtures. CRAN. https://cran.r-project.org/web/packages/morpheus
Dang UJ, Punzo A, McNicholas PD, Ingrassia S, Browne RP (2017) Multivariate response and parsimony for Gaussian cluster-weighted models. J Classif 34(1):4–34. https://doi.org/10.1007/s00357-017-9221-2
Follmann DA, Lambert D (1991) Identifiability of finite mixtures of logistic regression models. J Stat Plan Inference 27(3):375–381. https://doi.org/10.1016/0378-3758(91)90050-O
Frühwirth-Schnatter S (2006) Finite mixture and Markov switching models. Springer series in statistics. Springer, New York
Frühwirth-Schnatter S, Celeux G, Robert CP (2019) Handbook of mixture analysis. In: Frühwirth-Schnatter S (ed) Chapman & Hall/CRC Handbooks of modern statistical methods. CRC Press, Boca Raton
Grün B, Leisch F (2007) Flexmix: an R package for finite mixture modelling. R News 7(1):8–13
Grün B, Leisch F (2008) Identifiability of finite mixtures of multinomial logit models with varying and fixed effects. J Classif 25(2):225–247. https://doi.org/10.1007/s00357-008-9022-8
Grün B, Leisch F (2008) Finite mixtures of generalized linear regression models. In: Shalabh CH (ed) Recent advances in linear models and related areas. Springer, Heidelberg, pp 205–230. https://doi.org/10.1007/978-3-7908-2064-5_11
Hansen LP (1982) Large sample properties of generalized method of moments estimators. Econometrica 50(4):1029–1054. https://doi.org/10.2307/1912775
Hennig C (2000) Identifiablity of models for clusterwise linear regression. J Classif 17(2):273–296. https://doi.org/10.1007/s003570000022
Ingrassia S, Punzo A (2016) Decision boundaries for mixtures of regressions. J Korean Stat Soc 45(2):295–306. https://doi.org/10.1016/j.jkss.2015.11.005
Ingrassia S, Punzo A, Vittadini G, Minotti SC (2015) The generalized linear mixed cluster-weighted model. J Classif 32(1):85–113. https://doi.org/10.1007/s00357-015-9175-1
Komárek A, Komárková L (2013) Clustering for multivariate continuous and discrete longitudinal data. Ann Appl Stat 7(1):177–200. https://doi.org/10.1214/12-AOAS580
Maruotti A, Punzo A (2017) Model-based time-varying clustering of multivariate longitudinal data with covariates and outliers. Comput Stat Data Anal 113:475–496. https://doi.org/10.1016/j.csda.2016.05.024
Mazza A, Punzo A (2017) Mixtures of multivariate contaminated normal regression models. Stat Pap. https://doi.org/10.1007/s00362-017-0964-y
Mazza A, Punzo A, Ingrassia S (2018) flexCWM: a flexible framework for cluster-weighted models. J Stat Softw 86(2):1–30. https://doi.org/10.18637/jss.v086.i02
Punzo A, McNicholas PD (2017) Robust clustering in regression analysis via the contaminated gaussian cluster-weighted model. J Classif 34(2):249–293. https://doi.org/10.1007/s00357-017-9234-x
Punzo A, Ingrassia S, Maruotti A (2018) Multivariate generalized hidden markov regression models with random covariates: physical exercise in an elderly population. Stat Med 37:2797–2808. https://doi.org/10.1002/sim.7687
Sedghi H, Janzamin M, Anandkumar A (2016) Provable tensor methods for learning mixtures of generalized linear models. In: Gretton A, Robert CC (eds) Proceedings of the 19th international conference on artificial intelligence and statistics. Proceedings of machine learning research, vol 51. PMLR, Cadiz, Spain, pp 1223–1231
Vaart AW (1998) Asymptotic statistics. Cambridge series in statistical and probabilistic mathematics, Cambridge edition. Cambridge University Press, Cambridge. https://doi.org/10.1017/CBO9780511802256
Wang S, Yao W, Huang M (2014) A note on the identifiability of nonparametric and semiparametric mixtures of GLMs. Stat Probab Lett 93:41–45. https://doi.org/10.1016/j.spl.2014.06.010
Wu CFJ (1983) On the convergence properties of the EM algorithm. Ann Stat 11(1):95–103
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Auder, B., Gassiat, E. & Loum, M.A. Least squares moment identification of binary regression mixture models. Metrika 84, 561–593 (2021). https://doi.org/10.1007/s00184-020-00787-x
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00184-020-00787-x