Skip to main content
SpringerLink
Log in

Geometric Sensitivity Measures for Bayesian Nonparametric Density Estimation Models

  • Published:
Sankhya A Aims and scope Submit manuscript

Abstract

We propose a geometric framework to assess global sensitivity in Bayesian nonparametric models for density estimation. We study sensitivity of nonparametric Bayesian models for density estimation, based on Dirichlet-type priors, to perturbations of either the precision parameter or the base probability measure. To quantify the different effects of the perturbations of the parameters and hyperparameters in these models on the posterior, we define three geometrically-motivated global sensitivity measures based on geodesic paths and distances computed under the nonparametric Fisher-Rao Riemannian metric on the space of densities, applied to posterior samples of densities: (1) the Fisher-Rao distance between density averages of posterior samples, (2) the log-ratio of Karcher variances of posterior samples, and (3) the norm of the difference of scaled cumulative eigenvalues of empirical covariance operators obtained from posterior samples. We validate our approach using multiple simulation studies, and consider the problem of sensitivity analysis for Bayesian density estimation models in the context of three real datasets that have previously been studied.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bean, A., Xu, X. and MacEachern, S. (2016). Transformations and Bayesian density estimation. Electr. J. Statist. 10, 2, 3355–3373.

    Article  MathSciNet  MATH  Google Scholar 

  • Berger, J.O. (1982). The robust Bayesian viewpoint, Technical Report, Purdue University Department of Statistics.

  • Berger, J.O. (1990). Robust Bayesian analysis: Sensitivity to the prior. Journal of Statistical Planning and Inference 25, 3, 303–328.

    Article  MathSciNet  MATH  Google Scholar 

  • Berger, J.O. (1994). An overview of robust Bayesian analysis. Test 3, 1, 5–124.

    Article  MathSciNet  MATH  Google Scholar 

  • Bhattacharyya, A. (1943). On a measure of divergence between two statistical populations defined by their probability distribution. Bull. Calcutta Math. Soc. 35, 99–109.

    MathSciNet  MATH  Google Scholar 

  • Blackwell, D. and MacQueen, J.B. (1973). Ferguson distributions via Pólya urn schemes. Ann. Statist. 1, 2, 353–355.

    Article  MathSciNet  MATH  Google Scholar 

  • Bush, C.A., Lee, J. and MacEachern, S.N. (2010). Minimally informative prior distributions for non-parametric Bayesian analysis. J. R. Statist. Soc. Series B 72, 2, 253–268.

    Article  MathSciNet  MATH  Google Scholar 

  • Cencov, N.N. (1982). Statistical decision rules and optimal inference, no. 53. American Mathematical Society.

  • Dey, D.D., Müller, P. and Sinha, D. (1998). Practical nonparametric and semiparametric Bayesian statistics, vol. 133 Springer.

  • Dryden, I.L. and Mardia, K.V. (1998). Statistical shape analysis, 4. Wiley, Chichester.

    MATH  Google Scholar 

  • Escobar, M.D. (1994). Estimating normal means with a Dirichlet process prior. J. Am. Stat. Assoc. 89, 425, 268–277.

    Article  MathSciNet  MATH  Google Scholar 

  • Escobar, M.D. and West, M. (1995). Bayesian density estimation and inference using mixtures. J. Am. Stat. Assoc. 90, 430, 577–588.

    Article  MathSciNet  MATH  Google Scholar 

  • Fearnhead, P. (2004). Particle filters for mixture models with an unknown number of components. Stat. Comput. 14, 1, 11–21.

    Article  MathSciNet  Google Scholar 

  • Ferguson, T.S. (1973). A Bayesian analysis of some nonparametric problems. Ann. Statist. 1, 209–230.

    Article  MathSciNet  MATH  Google Scholar 

  • Ferguson, T.S. (1983). Bayesian density estimation by mixtures of normal distributions. In Recent advances in statistics. Elsevier, pp. 287–302.

  • Gershman, S.J. and Blei, D.M. (2012). A tutorial on Bayesian nonparametric models. J. Math. Psychol. 56, 1, 1–12.

    Article  MathSciNet  MATH  Google Scholar 

  • Ghosal, S. and Van Der Vaart, A.W. (2001). Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities. Ann. Statist. 29, 5, 1233–1263.

    Article  MathSciNet  MATH  Google Scholar 

  • Ghosh, J.K. and Ramamoorthi, R.V. (2003). Bayesian nonparametrics. Springer.

  • Görür, D. and Rasmussen, C.E. (2010). Dirichlet process Gaussian mixture models: Choice of the base distribution. J. Comput. Sci. Technol. 25, 4, 653–664.

    Article  MathSciNet  Google Scholar 

  • Griffin, J.E. (2010). Default priors for density estimation with mixture models. Bayesian Anal. 5, 1, 45–64.

    Article  MathSciNet  MATH  Google Scholar 

  • Griffin, J.E. and Steel, M.F. (2004). Semiparametric Bayesian inference for stochastic frontier models. J. Econometr. 123, 1, 121–152.

    Article  MathSciNet  MATH  Google Scholar 

  • Gustafson, P. (1996). Local sensitivity of inferences to prior marginals. J. Am. Stat. Assoc. 91, 434, 774–781.

    Article  MathSciNet  MATH  Google Scholar 

  • Gustafson, P. (2000). Local robustness in Bayesian analysis. In Robust Bayesian analysis. Springer, pp. 71–88.

  • Gustafson, P. and Wasserman, L. (1995). Local sensitivity diagnostics for Bayesian inference. Ann. Statist. 23, 6, 2153–2167.

    Article  MathSciNet  MATH  Google Scholar 

  • Hjort, N.L., Holmes, C., Müller, P. and Walker, S.G. (2010). Bayesian nonparametrics, vol. 28. Cambridge University Press.

  • Insua, D.R. and Ruggeri, F. (2012). Robust Bayesian analysis, vol. 152. Springer.

  • Ishwaran, H. and James, L.F. (2001). Gibbs sampling methods for stick-breaking priors. J. Am. Stat. Assoc. 96, 453, 161–173.

    Article  MathSciNet  MATH  Google Scholar 

  • Kass, R.E. and Vos, P.W. (1997). Geometrical foundations of asymptotic inference. Wiley.

  • Kass, R.E., Tierney, L. and Kadane, J.B. (1989). Approximate methods for assessing influence and sensitivity in Bayesian analysis. Biometrika 76, 4, 663–674.

    Article  MathSciNet  MATH  Google Scholar 

  • Kurtek, S. (2017). A geometric approach to pairwise Bayesian alignment of functional data using importance sampling. Electronic Journal of Statistics 11, 1, 502–531.

    Article  MathSciNet  MATH  Google Scholar 

  • Kurtek, S. and Bharath, K. (2015). Bayesian sensitivity analysis with Fisher–Rao metric. Biometrika 102, 3, 601–616.

    Article  MathSciNet  MATH  Google Scholar 

  • Lang, S. (2012). Fundamentals of differential geometry, vol. 191. Springer Science & Business Media.

  • Lee, J., MacEachern, S.N., Lu, Y., Mills, G.B. et al. (2014). Local-mass preserving prior distributions for nonparametric Bayesian models. Bayesian Anal. 9, 2, 307–330.

    Article  MathSciNet  MATH  Google Scholar 

  • MacEachern, S.N. (1994). Estimating normal means with a conjugate style Dirichlet process prior. Commun. Stat. Simul. Comput. 23, 3, 727–741.

    Article  MathSciNet  MATH  Google Scholar 

  • MacEachern, S.N. (1998). Computational methods for mixture of Dirichlet process models. In Practical nonparametric and semiparametric Bayesian statistics. Springer, pp. 23–43.

  • MacEachern, S.N. (2016). Nonparametric Bayesian methods: A gentle introduction and overview. Communications for Statistical Applications and Methods 23, 6, 445–466.

    Article  Google Scholar 

  • MacEachern, S.N. and Müller, P. (1998). Estimating mixture of Dirichlet process models. J. Comput. Graph. Stat. 7, 2, 223–238.

    Google Scholar 

  • MacEachern, S.N., Clyde, M. and Liu, J.S. (1999). Sequential importance sampling for nonparametric Bayes models: The next generation. Can. J. Stat. 27, 2, 251–267.

    Article  MathSciNet  MATH  Google Scholar 

  • Millar, R.B. and Stewart, W.S. (2007). Assessment of locally influential observations in Bayesian models. Bayesian Anal. 2, 2, 365–383.

    Article  MathSciNet  MATH  Google Scholar 

  • Müller, P. and Quintana, F.A. (2004). Nonparametric Bayesian data analysis. Stat. Sci. 19, 1, 95–110.

    Article  MathSciNet  MATH  Google Scholar 

  • Müller, P. and Rodriguez, A. (2013). Nonparametric Bayesian inference. Institute of Mathematical Statistics.

  • Müller, P., Quintana, F.A., Jara, A. and Hanson, T. (2015). Bayesian nonparametric data analysis. Springer.

  • Neal, R.M. (2000). Markov chain sampling methods for Dirichlet process mixture models. J. Comput. Graph. Stat. 9, 2, 249–265.

    MathSciNet  Google Scholar 

  • Newton, M.A. and Zhang, Y. (1999). A recursive algorithm for nonparametric analysis with missing data. Biometrika 86, 1, 15–26.

    Article  MathSciNet  MATH  Google Scholar 

  • Nieto-Barajas, L.E. and Prünster, I. (2009). A sensitivity analysis for Bayesian nonparametric density estimators. Stat. Sin. 19, 685–705.

    MathSciNet  MATH  Google Scholar 

  • Oakley, J.E. and O’Hagan, A. (2004). Probabilistic sensitivity analysis of complex models: A Bayesian approach. J. R. Statist. Soc. Series B 66, 3, 751–769.

    Article  MathSciNet  MATH  Google Scholar 

  • Pitman, J. (2002). Combinatorial stochastic processes, Technical Report 621, University of California, Berkeley, Department of Statistics.

  • Rao, C.R. (1945). Information and accuracy attainable in the estimation of statistical parameters. Bull. Calcutta Math. Soc. 37, 81–91.

    MathSciNet  MATH  Google Scholar 

  • Richardson, S. and Green, P.J. (1997). On Bayesian analysis of mixtures with an unknown number of components (with discussion). J. R. Statist. Soc. Series B 59, 4, 731–792.

    Article  MATH  Google Scholar 

  • Roeder, K. and Wasserman, L. (1997). Practical Bayesian density estimation using mixtures of normals. J. Am. Stat. Assoc. 92, 439, 894–902.

    Article  MathSciNet  MATH  Google Scholar 

  • Roos, M., Martins, T.G., Held, L. and Rue, H. (2015). Sensitivity analysis for Bayesian hierarchical models. Bayesian Anal. 10, 2, 321–349.

    Article  MathSciNet  MATH  Google Scholar 

  • Ruggeri, F. (2008). Bayesian robustness, European working group. Multiple Criteria Decision Aiding 3, 17, 6.

    Google Scholar 

  • Ruggeri, F. and Sivaganesan, S. (2000). On a global sensitivity measure for Bayesian inference. Sankhyā Series A 62, 110–127.

    MathSciNet  MATH  Google Scholar 

  • Sethuraman, J. (1994). A constructive definition of Dirichlet priors. Stat. Sin. 4, 639–650.

    MathSciNet  MATH  Google Scholar 

  • Sivaganesan, S. (2000). Global and local robustness approaches: Uses and limitations. In Robust Bayesian analysis. Springer, pp. 89–108.

  • Walker, S. (2004). New approaches to Bayesian consistency. Ann. Statist. 32, 5, 2028–2043.

    Article  MathSciNet  MATH  Google Scholar 

  • Walker, S.G. and Mallick, B.K. (1997). A note on the scale parameter of the Dirichlet process. Can. J. Stat. 25, 4, 473–479.

    Article  MathSciNet  MATH  Google Scholar 

  • Walker, S.G., Damien, P., Laud, P.W. and Smith, A.F. (1999). Bayesian nonparametric inference for random distributions and related functions. J. R. Statist. Soc. Series B 61, 3, 485–527.

    Article  MathSciNet  MATH  Google Scholar 

  • Yang, L. and Marron, J.S. (1999). Iterated transformation–kernel density estimation. J. Am. Stat. Assoc. 94, 446, 580–589.

    MathSciNet  MATH  Google Scholar 

  • Zhu, H., Ibrahim, J.G. and Tang, N. (2011). Bayesian influence analysis: A geometric approach. Biometrika 98, 2, 307–323.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Karthik Bharath for valuable discussions and suggestions. SK was partially supported by NSF DMS 1613054, NSF CCF 1740761 and NIH R01-CA214955.

Author information

Authors and Affiliations

  1. Department of Statistics, The Ohio State University, Columbus, OH, USA

    Abhijoy Saha & Sebastian Kurtek

Authors
  1. Abhijoy Saha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sebastian Kurtek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abhijoy Saha.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saha, A., Kurtek, S. Geometric Sensitivity Measures for Bayesian Nonparametric Density Estimation Models. Sankhya A 81, 104–143 (2019). https://doi.org/10.1007/s13171-018-0145-7

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13171-018-0145-7

Keywords and phrases.

AMS (2000) subject classification.

Search

Navigation