Abstract
A new regression method based on constructing optimal convex combinations of simple linear regressions of the least squares method (LSM regressions) built from original regressors is presented. It is shown that, in fact, this regression method is equivalent to a modification of the LSM including the additional requirement of the coincidence of the sign of the regression parameter with that of the correlation coefficient between the corresponding regressor and the response. A method for constructing optimal convex combinations based on the concept of nonexpandable irreducible ensembles is described. Results of experiments comparing the developed method with the known glmnet algorithm are presented, which confirm the efficiency of the former.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
R. Tibshirani, “Regression shrinkage and selection via the lasso,” J. R. Stat. Soc. 58, 267–288 (1996).
H. Zou, T. Hastie, B. Efron, and T. Hastie, “Regularization and variable selection via the elastic net,” J. R. Stat. Soc. 67(2), 301–320 (2005).
B. Efron, T. Hastie, I. Jonnstone, and R. Tibshirani, “Least angle regression,” Ann. Stat. 32(2), 407–499 (2004).
Yu. I. Zhuravlev, V. V. Ryazanov, and O. V. Senko, Pattern Recognition: Mathematical Methods, Software System, and Applications (Fazis, Moscow, 2006) [in Russian].
V. A. Kuznetsov and O. V. Senko, et al., “Recognition of fuzzy systems by method of statistically weighed syndromes and its using for immunological and hematological norm and chronic pathology,” Chem. Phys. 15(1), 81–100 (1996).
L. Breiman, “Random forests-random features,” Technical Report 567 (Statistics Department, Univ. of California, Berkley, 1999); www.boosting.org.
L. I. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms (Wiley, Hoboken, N.J., 2004).
Yu. I. Zhuravlev, A. V. Kuznetsova, V. V. Ryazanov, O. V. Senko, M. A. Botvin, “The use of pattern recognition methods in tasks of biomedical diagnostics and forecasting,” Pattern Recogn. Image Anal. 18(2), 195–200 (2008).
G. Brown, J. L. Wyatt, and P. Tino, “Managing diversity in regression ensembles,” J. Machine Learning Res. 6, 1621–1650 (2005).
O. V. Senko, “An optimal ensemble of predictors in convex correcting procedures,” Pattern Recogn. Image Anal. 19(3), 465–468 (2009).
O. Senko and A. Dokukin, “Optimal forecasting based on convex correcting procedures,” in New Trends in Classification and Data Mining (ITHEA, Sofia, 2010), pp. 62–72.
A. A. Dokukin and O. V. Senko, “Optimal convex correcting procedures in problems of high dimension,” Comput. Math. Math. Phys. 51(9), 1644–1652 (2011).
O. Senko and A. Dokukin, “Correlation maximization in regression models based on convex combinations,” Inf. Theor. Appl. 18(3), 224–231 (2011).
J. H. Friedman, T. Hastie, and R. Tibshirani, “Regularization paths for generalized linear models via coordinate descent,” J. Stat. Software 33(1), 1–22 (2010).
Author information
Authors and Affiliations
Corresponding author
Additional information
Original Russian Text © A.A. Dokukin, O.V. Senko, 2015, published in Zhurnal Vychislitel’noi Matematiki i Matematicheskoi Fiziki, 2015, Vol. 55, No. 3, pp. 530–544.
Rights and permissions
About this article
Cite this article
Dokukin, A.A., Senko, O.V. Regression model based on convex combinations best correlated with response. Comput. Math. and Math. Phys. 55, 526–539 (2015). https://doi.org/10.1134/S0965542515030045
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S0965542515030045