Abstract
In maximum penalized or regularized methods, it is important to select a tuning parameter appropriately. This paper proposes a direct plug-in method for tuning parameter selection. The tuning parameters selected using a generalized information criterion (Konishi and Kitagawa, Biometrika, 83, 875–890, 1996) and cross-validation (Stone, Journal of the Royal Statistical Society, Series B, 58, 267–288, 1974) are shown to be asymptotically equivalent to those selected using the proposed method, from the perspective of estimation of an optimal tuning parameter. Because of its directness, the proposed method is superior to the two selection methods mentioned above in terms of computational cost. Some numerical examples which contain the penalized spline generalized linear model regressions are provided.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akaike H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control 19: 716–723
DiCiccio T.J. and Efron B. (1992). More accurate confidence intervals in exponential families. Biometrika 79: 231–245
DiCiccio T.J. and Efron B. (1996). Bootstrap confidence intervals (with discussion). Statistical Science 11: 189–228
DiCiccio T.J. and Monti A.C. (2001). Accurate confidence limits for scalar functions of vector M-estimands. Biometrika 89: 437–451
Fan J. and Li R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96: 1348–1360
Gilbert. P. numDeriv: Accurate Numerical Derivatives. R package Version 2006.4-1. http://www.bank-banque-canada.ca/pgilbert.
Good I.J. and Gaskins R.A. (1971). Nonparametric roughness penalties for probability densities. Biometrika 58: 255–277
Green P.J. and Silverman B.W. (1994). Nonparametric regression and generalized linear models, a roughness penalty approach. Chapman and Hall, London
Imoto S. and Konishi S. (2003). Selection of smoothing parameters in B-spline nonparametric regression models using information criteria. Annals of the Institute of Statistical Mathematics 55: 671–687
Konishi S. and Kitagawa G. (1996). Generalised information criteria in model selection. Biometrika 83: 875–890
Konishi S. and Kitagawa G. (2003). Asymptotic theory for information criteria in model selection – Functional approach. Journal of Statistical Planning and Inference 114: 45–61
Konishi S., Ando T. and Imoto S. (2004). Bayesian information criteria and smoothing parameter selection in radial basis function networks. Biometrika 91: 27–43
Nonaka Y. and Konishi S. (2005). Nonlinear regression modeling using regularized local likelihood method. Annals of the Institute of Statistical Mathematics 57: 617–635
Shimodaira H. (2000). Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of Statistical Planning and Inference 90: 227–244
Stone M. (1974). Cross-validatory choice and assessment of statistical predictions (with discussion). Journal of the Royal Statistical Society, Series B 36: 111–147
Takeuchi K. (1976). Distribution of informational statistics and a criterion of model fitting. Suri-Kagaku (Mathematical Sciences) 153: 12–18 (in Japanese)
Tibshirani R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B 58: 267–288
Wand M.P. (1999). On the optimal amount of smoothing in penalised spline regression. Biometrika 86: 936–940
White H. (1982). Maximum likelihood estimation of misspecified models. Econometrika 50: 1–26
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Ueki, M., Fueda, K. Optimal tuning parameter estimation in maximum penalized likelihood method. Ann Inst Stat Math 62, 413–438 (2010). https://doi.org/10.1007/s10463-008-0186-0
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10463-008-0186-0