Empirical Bayes Estimation in Wavelet Nonparametric Regression

  • Merlise A. Clyde
  • Edward I. George
Part of the Lecture Notes in Statistics book series (LNS, volume 141)


Bayesian methods based on hierarchical mixture models have demonstrated excellent mean squared error properties in constructing data dependent shrinkage estimators in wavelets, however, subjective elicitation of the hyperparameters is challenging. In this chapter we use an Empirical Bayes approach to estimate the hyperparameters for each level of the wavelet decomposition, bypassing the usual difficulty of hyperparameter specification in the hierarchical model. The EB approach is computationally competitive with standard methods and offers improved MSE performance over several Bayes and classical estimators in a wide variety of examples.


Discrete Wavelet Transformation Shrinkage Estimator Wavelet Shrinkage Posterior Model Probability Conditional Maximum Likelihood Estimate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Abramovich, F., Sapatinas, T., and Silverman, B.W. (1998). “Wavelet Thresholding via a Bayesian Approach,” Journal of the Royal Statistics Society, Series B, 60, 725–749.MathSciNetzbMATHCrossRefGoogle Scholar
  2. Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In 2nd International Symposium on Information Theory, Eds B.N. Petrov and F. Csaki, pp. 267–281. Budapest: Akademia Kiado.Google Scholar
  3. Box, G.E.P. and Tiao, G.C. (1973). Bayesian Inference in Statistical Analysis, Wiley, NY.Google Scholar
  4. Chipman, H., Kolaczyk, E., and McCulloch, R. (1997). “Adaptive Bayesian Wavelet Shrinkage”, Journal of the American Statistical Association, 92, 1413–1421.zbMATHCrossRefGoogle Scholar
  5. Clyde, M. and George, E.I. (1998). “Robust Empirical Bayes Estimation in Wavelets”, ISDS Discussion Paper 98-21/ Scholar
  6. Clyde, M., Parmigiani, G., Vidakovic, B. (1998). “Multiple Shrinkage and Subset Selection in Wavelets,” Biometrika, 85, 391–402.MathSciNetzbMATHCrossRefGoogle Scholar
  7. Dempster, A.P. Laird, N.M. and Rubin, D.B. (1977). Maximum likelihood estimation from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society, Series B, 39, 1–38.MathSciNetzbMATHGoogle Scholar
  8. Donoho, D.L., and Johnstone, I.M., (1994). “Ideal spatial adaptation by wavelet shrinkage,” Biometrika, 81, 256–425.MathSciNetCrossRefGoogle Scholar
  9. Donoho, D. and Johnstone, I. (1995). “Adapting to Unknown Smoothness via Wavelet Shrinkage,” Journal of the American Statistical Association, 90, 1200–1224.MathSciNetzbMATHCrossRefGoogle Scholar
  10. Donoho, D., Johnstone, I., Kerkyacharian, G., and Picard, D. (1995). “Wavelet shrinkage: Asymptopia?” Journal of the Royal Statistical Society, Series B, 57, 301–369.MathSciNetzbMATHGoogle Scholar
  11. Foster, D. and George, E. (1994). The risk inflation criterion for multiple regression. Annals of Statistics 22, 1947–1975.MathSciNetzbMATHCrossRefGoogle Scholar
  12. George, E.I. (1986). “Minimax multiple shrinkage estimation,” Annals of Statistics, 14, 188–205.MathSciNetzbMATHCrossRefGoogle Scholar
  13. George, E.I. and Foster, D.P. (1997). “Empirical Bayes Variable Selection”, Tech Report, University of Texas at Austin.Google Scholar
  14. George, E.I. and McCulloch, R. (1993). Variable selection via Gibbs sampling. Journal of the American Statistical Association 88, 881–889.CrossRefGoogle Scholar
  15. Johnstone, I.M. and Silverman, B.W. (1997). “Wavelet Threshold Estimators for Data with Correlated Noise,” Journal of the Royal Statistical Society, Series B, 59, 319–351.MathSciNetzbMATHCrossRefGoogle Scholar
  16. Johnstone, I.M. and Silverman, B.W. (1998). “Empirical Bayes approaches to mixture problems and wavelet regression”, Technical report. Department of Mathematics, University of Bristol.Google Scholar
  17. Neal, R. M. and Hinton, G. E. (1998) “A view of the EM algorithm that justifies incremental, sparse, and other variants”, in M. I. Jordan (editor) Learning in Graphical Models, Dordrecht: Kluwer Academic Publishers, pages 355–368.CrossRefGoogle Scholar
  18. Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics 6, 461–464.MathSciNetzbMATHCrossRefGoogle Scholar
  19. Tanner, M.A. (1996). Tools for statistical inference: methods for the exploration of posterior distributions and likelihood functions. New York: Springer, 3rd Edition. Chapter 4, pages 64–89.zbMATHGoogle Scholar
  20. Yau, P. and Kohn, R. (1999). “Wavelet Nonparametric Regression Using Basis Averaging”. To appear in Bayesian Inference in Wavelet Based Models eds P. Müller and B. Vidakovic. Springer-Verlag.Google Scholar

Copyright information

© Springer Science+Business Media New York 1999

Authors and Affiliations

  • Merlise A. Clyde
  • Edward I. George

There are no affiliations available

Personalised recommendations