Jackknife multiplier bootstrap: finite sample approximations to the U-process supremum with applications

  • Xiaohui ChenEmail author
  • Kengo Kato


This paper is concerned with finite sample approximations to the supremum of a non-degenerate U-process of a general order indexed by a function class. We are primarily interested in situations where the function class as well as the underlying distribution change with the sample size, and the U-process itself is not weakly convergent as a process. Such situations arise in a variety of modern statistical problems. We first consider Gaussian approximations, namely, approximate the U-process supremum by the supremum of a Gaussian process, and derive coupling and Kolmogorov distance bounds. Such Gaussian approximations are, however, not often directly applicable in statistical problems since the covariance function of the approximating Gaussian process is unknown. This motivates us to study bootstrap-type approximations to the U-process supremum. We propose a novel jackknife multiplier bootstrap (JMB) tailored to the U-process, and derive coupling and Kolmogorov distance bounds for the proposed JMB method. All these results are non-asymptotic, and established under fairly general conditions on function classes and underlying distributions. Key technical tools in the proofs are new local maximal inequalities for U-processes, which may be useful in other problems. We also discuss applications of the general approximation results to testing for qualitative features of nonparametric functions based on generalized local U-processes.


Gaussian approximation Jackknife multiplier bootstrap Coupling U-process Local maximal inequality 

Mathematics Subject Classification

60F17 62E17 62F40 62G10 



The authors would like to thank the anonymous referees and an Associate Editor for their constructive comments that improve the quality of this paper.


  1. 1.
    Abrevaya, J., Jiang, W.: A nonparametric approach to measuring and testing curvature. J. Bus. Econ. Stat. 23(1), 1–19 (2005)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Adamczak, R.: Moment inequalities for U-statistics. Ann. Probab. 34(6), 2288–2314 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Arcones, M., Giné, E.: On the bootstrap of \(U\)- and \(V\)-statistics. Ann. Stat. 20(2), 655–674 (1992)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Arcones, M., Giné, E.: Limit theorems for \(U\)-processes. Ann. Probab. 21(3), 1495–1542 (1993)MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Arcones, M., Giné, E.: U-processes indexed by Vapnik–Červonenkis classes of functions with applications to asymptotics and bootstrap of U-statistics with estimated parameters. Stoch. Process. Appl. 52(1), 17–38 (1994)zbMATHCrossRefGoogle Scholar
  6. 6.
    Bickel, P.J., Freedman, D.A.: Some asymptotic theory for the bootstrap. Ann. Stat. 9(6), 1196–1217 (1981)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Blundell, R., Gosling, A., Ichimura, H., Meghir, C.: Changes in the distribution of male and female wages accounting for employment composition using bounds. Econometrica 75(2), 323–363 (2007)zbMATHCrossRefGoogle Scholar
  8. 8.
    Borovskikh, Y.V.: U-Statistics in Banach Spaces. V.S.P. Intl Science, Zeist (1996)zbMATHGoogle Scholar
  9. 9.
    Bretagnolle, J.: Lois limits du Bootstrap de certaines functionnelles. Annales de l’Institut Henri Poincaré Section B XIX(3), 281–296 (1983)zbMATHGoogle Scholar
  10. 10.
    Callaert, H., Veraverbeke, N.: The order of the normal approximation for a Studentized \(U\)-statistic. Ann. Stat. 9(1), 360–375 (1981)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Chen, X.: Gaussian and bootstrap approximations for high-dimensional U-statistics and their applications. Ann. Stat. 46(2), 642–678 (2018)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Chernozhukov, V., Chetverikov, D., Kato, K.: Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors. Ann. Stat. 41(6), 2786–2819 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Chernozhukov, V., Chetverikov, D., Kato, K.: Anti-concentration and honest, adaptive confidence bands. Ann. Stat. 42(5), 1787–1818 (2014)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Chernozhukov, V., Chetverikov, D., Kato, K.: Gaussian approximation of suprema of empirical processes. Ann. Stat. 42(4), 1564–1597 (2014)MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Chernozhukov, V., Chetverikov, D., Kato, K.: Empirical and multiplier bootstraps for suprema of empirical processes of increasing complexity, and related gaussian couplings. Stoch. Process. Appl. 126(12), 3632–3651 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Chetverikov, D.: Testing regression monotonicity in econometric models. arXiv:1212.6757 (2012)
  17. 17.
    Davydov, Y., Lifshits, M., Smorodina, N.: Local Properties of Distributions of Stochastic Functions (Transaction of Mathematical Monographs, Vol. 173). American Mathematical Society, New York (1998)Google Scholar
  18. 18.
    de la Peña, V., Giné, E.: Decoupling: From Dependence to Independence. Springer, Berlin (1999)CrossRefGoogle Scholar
  19. 19.
    Dehling, H., Mikosch, T.: Random quadratic forms and the bootstrap for \(U\)-statistics. J. Multivar. Anal. 51(2), 392–413 (1994)MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Dudley, R.M.: Real Analysis and Probability. Cambridge University Press, Cambridge (2002)zbMATHCrossRefGoogle Scholar
  21. 21.
    Dümbgen, L.: Application of local rank tests to nonparametric regression. J. Nonparametric Stat. 14(5), 511–537 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Einmahl, U., Mason, D.M.: Uniform in bandwidth consistency of kernel-type function estimators. Ann. Stat. 33(3), 1380–1403 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Ellison, G., Ellison, S.F.: Strategic entry deterrence and the behavior of pharmaceutical incumbents prior to patent expiration. Am. Econ. J. Microecon. 3(1), 1–36 (2011)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Frees, E.W.: Estimating densities of functions of observations. J. Am. Stat. Assoc. 89(426), 517–525 (1994)MathSciNetzbMATHCrossRefGoogle Scholar
  25. 25.
    Ghosal, S., Sen, A., van der Vaart, A.: Testing monotonicity of regression. Ann. Stat. 28(4), 1054–1082 (2000)MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Giné, E., Latała, R., Zinn, J.: Exponential and moment inequalities for \(U\)-statistics. High Dimensional Probability II. Springer, Berlin (2000)Google Scholar
  27. 27.
    Giné, E., Mason, D.M.: On local \(U\)-statistic processes and the estimation of densities of functions of several sample variables. Ann. Stat. 35(3), 1105–1145 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  28. 28.
    Giné, E., Nickl, R.: Uniform limit theorems for wavelet density estimators. Ann. Probab. 37(4), 1605–1646 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Giné, E., Nickl, R.: Mathematical Foundations of Infinite-Dimensional Statistical Models. Cambridge University Press, Cambridge (2016)zbMATHCrossRefGoogle Scholar
  30. 30.
    Hall, P.: On convergence rates of suprema. Probab. Theory Relat. Fields 89(4), 447–455 (1991)MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Hoeffding, W.: A class of statistics with asymptotically normal distributions. Ann. Math. Stat. 19(3), 293–325 (1948)MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Huškova, M., Janssen, P.: Consistency of the generalized bootstrap for degenerate \(U\)-statistics. Ann. Stat. 21(4), 1811–1823 (1993)MathSciNetzbMATHCrossRefGoogle Scholar
  33. 33.
    Hušková, M., Janssen, P.: Generalized bootstrap for studentized \(U\)-statistics: a rank statistic approach. Stat. Probab. Lett. 16(3), 225–233 (1993)MathSciNetzbMATHCrossRefGoogle Scholar
  34. 34.
    Janssen, P.: Weighted bootstrapping of \(U\)-statistics. J. Stat. Plann. Inference 38(1), 31–42 (1994)MathSciNetzbMATHCrossRefGoogle Scholar
  35. 35.
    Koltchinskii, V.I.: Komlos–Major–Tusnády approximation for the general empirical process and Haar expansions of classes of functions. J. Theor. Probab. 7(1), 73–118 (1994)zbMATHCrossRefGoogle Scholar
  36. 36.
    Komlós, J., Major, P., Tusnády, G.: An approximation of partial sums of independent rv’s and the sample df. I. Z. Wahrscheinlichkeitstheor. Verw. Geb. 32(1–2), 111–131 (1975)MathSciNetzbMATHCrossRefGoogle Scholar
  37. 37.
    Ledoux, M., Talagrand, M.: Probability in Banach Spaces: Isoperimetry and Processes. Springer, New York (1991)zbMATHCrossRefGoogle Scholar
  38. 38.
    Lee, S., Linton, O., Whang, Y.-J.: Testing for stochastic monotonicity. Econometrica 77(2), 585–602 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  39. 39.
    Albert, Y.L.: A large sample study of the Bayesian bootstrap. Ann. Stat. 15(1), 360–375 (1987)MathSciNetzbMATHCrossRefGoogle Scholar
  40. 40.
    Mason, D.M., Newton, M.A.: A rank statistics approach to the consistency of a general bootstrap. Ann. Stat. 20(3), 1611–1624 (1992)MathSciNetzbMATHCrossRefGoogle Scholar
  41. 41.
    Massart, P.: Strong approximation for multivariate empirical and related processes, via KMT constructions. Ann. Probab. 17(1), 266–291 (1989)MathSciNetzbMATHCrossRefGoogle Scholar
  42. 42.
    Monrad, D., Philipp, W.: Nearby variables with nearby conditional laws and a strong approximation theorem for Hilbert space valued martingales. Probab. Theory Relat. Fields 88(3), 381–404 (1991)MathSciNetzbMATHCrossRefGoogle Scholar
  43. 43.
    Nolan, D., Pollard, D.: \(U\)-processes: rates of convergence. Ann. Stat. 15(2), 780–799 (1987)MathSciNetzbMATHCrossRefGoogle Scholar
  44. 44.
    Nolan, D., Pollard, D.: Functional limit theorems for \(U\)-processes. Ann. Probab. 16(3), 1291–1298 (1988)MathSciNetzbMATHCrossRefGoogle Scholar
  45. 45.
    Piterberg, V.I.: Asymptotic Methods in the Theory of Gaussian Processes and Fields. American Mathematical Society, New York (1996)Google Scholar
  46. 46.
    Resnick, S.I.: Extreme Values, Regular Variation, and Point Processes. Springer, Berlin (1987)zbMATHCrossRefGoogle Scholar
  47. 47.
    Rio, E.: Local invariance principles and their application to density estimation. Probab. Theory Relat. Fields 98(1), 21–45 (1994)MathSciNetzbMATHCrossRefGoogle Scholar
  48. 48.
    Rubin, D.B.: The Bayesian bootstrap. Ann. Stat. 9(1), 130–134 (1981)MathSciNetCrossRefGoogle Scholar
  49. 49.
    Serfling, R.J.: Approximation Theorems of Mathematical Statistics. Wiley, New York (1980)zbMATHCrossRefGoogle Scholar
  50. 50.
    Sherman, R.P.: Limiting distribution of the maximal rank correlation estimator. Econometrica 61(1), 123–137 (1993)MathSciNetzbMATHCrossRefGoogle Scholar
  51. 51.
    Sherman, R.P.: Maximal inequalities for degenerate \(U\)-processes with applications to optimization estimators. Ann. Stat. 22(1), 439–459 (1994)MathSciNetzbMATHCrossRefGoogle Scholar
  52. 52.
    Solon, G.: Intergenerational income mobility in the United States. Am. Econ. Rev. 82(3), 393–408 (1992)Google Scholar
  53. 53.
    van der Vaart, A., Wellner, J.A.: Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, Berlin (1996)zbMATHCrossRefGoogle Scholar
  54. 54.
    van der Vaart, A., Wellner, J.A.: A local maximal inequality under uniform entropy. Electron. J. Stat. 5, 192–203 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  55. 55.
    Wang, Q., Jing, B.-Y.: Weighted bootstrap for \(U\)-statistics. J. Multivar. Anal. 91(2), 177–198 (2004)MathSciNetzbMATHCrossRefGoogle Scholar
  56. 56.
    Zhang, D.: Bayesian bootstraps for U-processes, hypothesis tests and convergence of Dirichlet U-processes. Stat. Sin. 11(2), 463–478 (2001)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of StatisticsUniversity of Illinois at Urbana-ChampaignChampaignUSA
  2. 2.Department of Statistical ScienceCornell UniversityIthacaUSA

Personalised recommendations