Statistics and Computing

, 19:355 | Cite as

Penalized semiparametric density estimation

  • Ying YangEmail author


In this article we propose a penalized likelihood approach for the semiparametric density model with parametric and nonparametric components. An efficient iterative procedure is proposed for estimation. Approximate generalized maximum likelihood criterion from Bayesian point of view is derived for selecting the smoothing parameter. The finite sample performance of the proposed estimation approach is evaluated through simulation. Two real data examples, suicide study data and Old Faithful geyser data, are analyzed to demonstrate use of the proposed method.


Density estimation Penalized likelihood estimation Generalized maximum likelihood criterion Reproducing kernel Hilbert space Smoothing splines 


  1. Abramowitz, M., Stegun, I.A.: Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. Dover, New York (1972) zbMATHGoogle Scholar
  2. Aronszajn, N.: Theory of reproducing kernels. Trans. Am. Math. Soc. 68, 337–807 (1950) zbMATHCrossRefMathSciNetGoogle Scholar
  3. Azzalini, A., Bowman, A.W.: A look at some data on the Old Faithful geyser. Appl. Stat. J. R. Stat. Soc. Ser. C 39(3), 357–365 (1990) zbMATHGoogle Scholar
  4. Cheng, K., Chu, C.: Semiparametric density estimation under a two-sample density ratio model. Bernoulli 10(4), 583–604 (2004) zbMATHCrossRefMathSciNetGoogle Scholar
  5. Copas, J.B., Fryer, M.J.: Density estimation and suicide risks in psychiatric treatment. J. R. Stat. Soc. A 143 (1980) Google Scholar
  6. Eggermont, P.P.B., Lariccia, V.N.: Maximum Penalized Likelihood Estimation, vol. I: Density Estimation. Springer, New York (2001) Google Scholar
  7. Fokianos, K.: Merging information for semiparametric density estimation. J. R. Stat. Soc. Ser. B 66, 941 (2004) zbMATHCrossRefMathSciNetGoogle Scholar
  8. Good, I.J., Gaskins, R.A.: Nonparametric roughness penalties for probability densities. Biometrika 58, 255–277 (1971) zbMATHCrossRefMathSciNetGoogle Scholar
  9. Gu, C.: Smoothing spline density estimation: a dimensionless automatic algorithm. J. Am. Stat. Assoc. 88, 495–504 (1993) zbMATHCrossRefGoogle Scholar
  10. Gu, C.: Smoothing Spline: ANOVA Models. Springer, New York (2002) zbMATHGoogle Scholar
  11. Gu, C., Kim, Y.J.: Penalized likelihood regression: general formulation and efficient approximation. Can. J. Stat. 30, 619–628 (2002) zbMATHCrossRefMathSciNetGoogle Scholar
  12. Gu, C., Qiu, C.: Penalized likelihood density estimation: theory. Ann. Stat. 21, 217–234 (1993) zbMATHCrossRefMathSciNetGoogle Scholar
  13. Kohn, R., Ansley, C.F., Tharm, D.: The performance of cross validation and maximum likelihood estimators of spline smoothing parameters. J. Am. Stat. Assoc. 86, 1042–1050 (1991) CrossRefMathSciNetGoogle Scholar
  14. Lenk, P.J.: The logistic normal-distribution for Bayesian, nonparametric predicitve densities. J. Am. Stat. Assoc. 83, 509–516 (1988) zbMATHCrossRefMathSciNetGoogle Scholar
  15. Lenk, P.J.: A bayesian nonparametric density estimator. J. Nonparametr. Stat. 3, 53–69 (1993) zbMATHCrossRefMathSciNetGoogle Scholar
  16. Lenk, P.J.: Bayesian semiparametric density estimation and model verification using a logistic-gaussian model. J. Comput. Graph. Stat. 12, 548–565 (2003) CrossRefMathSciNetGoogle Scholar
  17. Leonard, T.: Density estimation, stochastic processes and prior information (with discussion). J. R. Stat. Soc. Ser. B. 40, 113–146 (1978) zbMATHMathSciNetGoogle Scholar
  18. Qin, J., Zhang, B.: Density estimation under a two-sample semiparametric model. J. Nonparametr. Stat. 17(6), 665–683 (2005) zbMATHCrossRefMathSciNetGoogle Scholar
  19. Sain, S.R., Scott, D.W.: On locally adaptive density estimation. J. Am. Stat. Assoc. 91(436), 1525–1534 (1996) zbMATHCrossRefMathSciNetGoogle Scholar
  20. Silverman, B.W.: On the estimation of a probability density function by the maximum penalized likelihood method. Ann. Stat. 10, 795–810 (1982) zbMATHCrossRefGoogle Scholar
  21. Silverman, B.W.: Density Estimation for Statistics and Data Analysis. Chapman and Hall, New York (1986) zbMATHGoogle Scholar
  22. Tapia, R.A., Thompson, J.R.: Nonparametric Probability Density Estimation. Johns Hopkins University Press, Baltimore (1978) zbMATHGoogle Scholar
  23. Wahba, G.: Improper priors, spline smoothing and the problem of guarding against model errors in regression. J. R. Stat. Soc. Ser. B. 40, 364–372 (1978) zbMATHMathSciNetGoogle Scholar
  24. Wahba, G.: A comparison of GCV and GML for choosing the smoothing parameter in the generalized spline smoothing problem. Ann. Stat. 13, 1378–1402 (1985) zbMATHCrossRefMathSciNetGoogle Scholar
  25. Wahba, G.: Spline Modeling for Observational Data. CBMS-NSF Regional Conference Series in Applied Mathematics, vol. 59. SIAM, Philadelphia (1990) Google Scholar
  26. Wang, Y.: Smoothing spline models with correlated random errors. J. Am. Stat. Assoc. 93, 341–348 (1998) zbMATHCrossRefGoogle Scholar
  27. Weinberger, H.F.: Variational Methods for Eigenvalue Approximation. CBMS-NSF Regional Conference Series in Applied Mathematics, vol. 15. SIAM, Philadelphia (1974) zbMATHGoogle Scholar
  28. Wolfinger, R.: Laplace’s approximation for nonlinear mixed models. Biometrika 80, 791–795 (1993) zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.Department of Mathematical SciencesTsinghua UniversityBeijingChina

Personalised recommendations