A Weighted Bootstrap Procedure for Divergence Minimization Problems

  • Michel BroniatowskiEmail author
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 193)


Sanov-type results hold for some weighted versions of empirical measures, and the rates for those Large Deviation principles can be identified as divergences between measures, which in turn characterize the form of the weights. This correspondence is considered within the range of the Cressie–Read family of statistical divergences, which covers most of the usual statistical criterions. We propose a weighted bootstrap procedure in order to estimate these rates. To any such rate we produce an explicit procedure which defines the weights, therefore replacing a variational problem in the space of measures by a simple Monte Carlo procedure.


Divergence Optimization Bootstrap Monte Carlo Large deviation Weighted empirical measure Conditional Sanov theorem 


  1. 1.
    Broniatowski, M., Keziou, A.: Divergences and duality for estimation and test under moment condition models. J. Stat. Plann. Inference 142(9), 2554–2573 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Broniatowski, M., Decurninge, A.: Estimation for models defined by conditions on their L-moments. IEEE Trans. Inf. Theory. doi: 10.1109/TIT.2016.2586085
  3. 3.
    Ruschendorf, L.: Projections of probability measures. Statistics 18(1), 123–129 (1987)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Csiszár, I.: Eine informationstheoretische Ungleichung und ihre Anwendung auf den Beweis der Ergodizität von Markoffschen Ketten. (German) Magyar Tud. Akad. Mat. Kutató Int. Közl. 8, 85–108 (1963)Google Scholar
  5. 5.
    Csiszár, I., Gamboa, F., Gassiat, E.: MEM pixel correlated solutions for generalized moment and interpolation problems. IEEE Trans. Inf. Theory 45(7), 2253–2270 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Csiszár, I.: Information-type measures of difference of probability distributions and indirect observations. Studia Sci. Math. Hungar. 2, 299–318 (1967)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Csiszár, I.: On topology properties of f-divergences. Studia Sci. Math. Hungar. 2, 329–339 (1967)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Liese, F., Vajda, I.: Convex statistical distances. With German, French and Russian summaries. Teubner-Texte zur Mathematik [Teubner Texts in Mathematics], 95. BSB B. G. Teubner Verlagsgesellschaft, pp. 224. Leipzig (1987). ISBN: 3-322-00428-7Google Scholar
  9. 9.
    Broniatowski, M., Keziou, A.: Minimization of \(\varphi \)-divergences on sets of signed measures. Studia Sci. Math. Hungar. 43(4), 403–442 (2006)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Cressie, N., Read, T.R.C.: Multinomial goodness-of-fit tests. J. Roy. Stat. Soc. Ser. B 46(3), 440–464 (1984)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Broniatowski, M., Keziou, A.: Parametric estimation and tests through divergences and the duality technique. J. Multivar. Anal. 100(1), 16–36 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Keziou, A.: Dual representation of \(\varphi \)-divergences and applications. C. R. Math. Acad. Sci. Paris 336(10), 857–862 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Owen, A.: Empirical likelihood ratio confidence regions. Ann. Stat. 18(1), 90–120 (1990)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Bertail, P.: Empirical likelihood in some semiparametric models. Bernoulli 12(2), 299–331 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Jiménez, R., Shao, Y.: On robustness and efficiency of minimum divergence estimators. Test 10(2), 241–248 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Beran, R.: Minimum Hellinger distance estimates for parametric models. Ann. Stat. 5(3), 445–463 (1977)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Lindsay, B.G.: Efficiency versus robustness: the case for minimum Hellinger distance and related methods. Ann. Stat. 22(2), 1081–1114 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Rockafellar, R.: Tyrrell Convex Analysis, Princeton Mathematical Series, vol. 28, pp. xviii+451. Princeton University Press, Princeton, NJ (1970)Google Scholar
  19. 19.
    Barndorff-Nielsen, O.: Information and Exponential Families in Statistical Theory. Wiley Series in Probability and Mathematical Statistics, pp. ix+238. Wiley, Ltd., Chichester (1978)Google Scholar
  20. 20.
    Trashorras, J., Wintenberger, O.: Large deviations for bootstrapped empirical measures. Bernoulli 20(4), 1845–1878 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Najim, J.: A Cramer type theorem for weighted random variables. Electron. J. Probab. 7 (2002)Google Scholar
  22. 22.
    Broniatowski, M. Weighted sampling, maximum likelihood and minimum divergence estimators. In: Geometric science of information, Lecture Notes in Computer Science, vol. 8085, pp. 467–478. Springer, Heidelberg (2013)Google Scholar
  23. 23.
    Barbe, P., Bertail, P.: The Weighted Bootstrap, Lecture Notes in Statistics. Springer, New York (1995)Google Scholar
  24. 24.
    Letac, G., Mora, M.: Natural real exponential families with cubic variance functions. Ann. Stat. 18(1), 1–37 (1990)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Tweedie, M.C.K.: Functions of a statistical variate with given means, with special reference to Laplacian distributions. Proc. Camb. Philos. Soc. 43, 41–49 (1947)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Jørgensen, B.: Exponential dispersion models. With discussion and a reply by the author. J. Roy. Stat. Soc. Ser. B 49(2), 127–162 (1987)Google Scholar
  27. 27.
    Morris, C.N.: Natural exponential families with quadratic variance functions. Ann. Stat. 10(1), 65–80 (1982)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Bar-Lev, S.K., Enis, P.: Reproducibility and natural exponential families with power variance functions. Ann. Stat. 14(4), 1507–1522 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Feller, W.: An Introduction to Probability Theory and its Applications, Vol. II, 2nd edn., pp. xxiv+669. Wiley, Inc., New York (1971)Google Scholar
  30. 30.
    Groeneboom, P., Oosterhoff, J., Ruymgaart, F.H.: Large deviation theorems for empirical probability measures. Ann. Probab. 7(4), 553–586 (1979)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Read, T.R.C., Cressie, N.A.C.: Goodness-of-fit Statistics for Discrete Multivariate Data. Springer Series in Statistics, pp. xii+211. Springer, New York (1988). ISBN: 0-387-96682-XGoogle Scholar
  32. 32.
    Mason, D.M., Newton, M.A.: A rank statistic approach to the consistency of a general bootstrap. Ann. Stat. 20, 1611–1624 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Hoadley, A.B.: On the probability of large deviations of functions of several empirical cdf’s. Ann. Math. Stat. 38, 360–381 (1967)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Withers, C.S., Nadarajah, S.: On the compound Poisson-gamma distribution. Kybernetika (Prague) 47(1), 15–37 (2011)MathSciNetzbMATHGoogle Scholar
  35. 35.
    Several Applications of Divergence Criteria in Continuous Families, to appear Kybernetika (2012)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Laboratoire de Statistique Théorique et AppliquéeUniversité Pierre et Marie CurieParis Cedex 05France

Personalised recommendations