Abstract
Bootstrap inference is a powerful tool for obtaining robust inference for quantiles and difference-in-quantiles estimators. The computationally intensive nature of bootstrap inference has made it infeasible in large-scale experiments. In this paper, the theoretical properties of the Poisson bootstrap algorithm and quantile estimators are used to derive alternative resampling-free algorithms for Poisson bootstrap inference that reduce the computational complexity substantially without additional assumptions. These findings are connected to existing literature on analytical confidence intervals for quantiles based on order statistics. The results unlock bootstrap inference for difference-in-quantiles for almost arbitrarily large samples. At Spotify, we can now easily calculate bootstrap confidence intervals for quantiles and difference-in-quantiles in A/B tests with hundreds of millions of observations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bezanson, J., Edelman, A., Karpinski, S., Shah, V.B.: Julia: A fresh approach to numerical computing. SIAM review, 59(1), pp. 65–98 (2017)
Chamandy, N., Muralidharan, O., Najmi, A., Naidu, S.: Estimating Uncertainty for Massive Data Streams. Technical report, Google (2012)
Chen, J., Revels, J.: Robust benchmarking in noisy environments. arXiv e-prints, arXiv:1608.04295 (2016)
Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms, Third Edition. The MIT Press, 3rd edition (2009)
David, H.A., Nagaraja, H.N.: Order statistics. John Wiley & Sons (2004)
Dean, J., Ghemawat, S.: Mapreduce: Simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)
Efron, B.: Bootstrap methods: another look at the jackknife. Ann. Stat. 7(1), 1–26 (1979)
Falk, M., Reiss, R.-D.: Weak convergence of smoothed and nonsmoothed bootstrap quantile estimates. Ann. Probab. 17(1), 362–371 (1989)
Ghosh, M., Parr, W.C., Singh, K., Babu, G.J.: A Note on Bootstrapping the Sample Median. The Annals of Stat. 12(3), 1130–1135 (1984)
Gibbons, J.D., Chakraborti, S.: Nonparametric statistical inference. CRC press (2014)
Hanley, J.A., MacGibbon, B.: Creating non-parametric bootstrap samples using poisson frequencies. Comput. Methods Programs Biomed. 83(1), 57–62 (2006)
Hutson, A.D.: Calculating nonparametric confidence intervals for quantiles using fractional order statistics. J. Appl. Stat. 26(3), 343–353 (1999)
Kleiner, A., Talwalkar, A., Sarkar, P., Jordan, M.I.: A scalable bootstrap for massive data. J. Royal Stat. Soc.: Series B (Statistical Methodology) 76(4), 795–816 (2014)
Liu, M., Sun, X., Varshney, M., Xu, Y.: Large-Scale Online Experimentation with Quantile Metrics. arXiv e-prints, arXiv:1903.08762 (2019)
Nyblom, J.: Note on interpolated order statistics. Stat. Probab. Lett. 14(2), 129–131 (1992)
Rao, C.R., Statistiker, M.: Linear statistical inference and its applications vol. 2, Wiley New York (1973)
Scheffe, H., Tukey, J.W.: Non-Parametric Estimation. I. Validation of Order Statistics. Ann. Math. Stat. 16(2), 187–192 (1945)
Acknowledgments
The authors gratefully acknowledge help and feedback from Anton Muratov, Shaobo Jin, Thommy Perlinger and Claire Detilleux.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Schultzberg, M., Ankargren, S. (2023). Resampling-Free Bootstrap Inference for Quantiles. In: Arai, K. (eds) Proceedings of the Future Technologies Conference (FTC) 2022, Volume 1. FTC 2022 2022. Lecture Notes in Networks and Systems, vol 559. Springer, Cham. https://doi.org/10.1007/978-3-031-18461-1_36
Download citation
DOI: https://doi.org/10.1007/978-3-031-18461-1_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-18460-4
Online ISBN: 978-3-031-18461-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)