Entropy, Randomization, Derandomization, and Discrepancy

Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 23)

Abstract

The star discrepancy is a measure of how uniformly distributed a finite point set is in the d-dimensional unit cube. It is related to high-dimensional numerical integration of certain function classes as expressed by the Koksma-Hlawka inequality. A sharp version of this inequality states that the worst-case error of approximating the integral of functions from the unit ball of some Sobolev space by an equal-weight cubature is exactly the star discrepancy of the set of sample points. In many applications, as, e.g., in physics, quantum chemistry or finance, it is essential to approximate high-dimensional integrals. Thus with regard to the Koksma-Hlawka inequality the following three questions are very important:
  1. 1.

    What are good bounds with explicitly given dependence on the dimension d for the smallest possible discrepancy of any n-point set for moderate n?

     
  2. 2.

    How can we construct point sets efficiently that satisfy such bounds?

     
  3. 3.

    How can we calculate the discrepancy of given point sets efficiently?

     

We want to discuss these questions and survey and explain some approaches to tackle them relying on metric entropy, randomization, and derandomization.

Keywords

Infinite Sequence Star Discrepancy Random Experiment Hammersley Point Small Discrepancy Sample 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

The author would like to thank two anonymous referees for their helpful suggestions.

Part of the work on this book chapter was done while the author was research fellow at the Department of Computer Science of Columbia University in the City of New York.

He gratefully acknowledges support from the German Science Foundation DFG under grant GN91/3-1 and GN91/4-1.

References

  1. 1.
    Aistleitner, C.: Covering numbers, dyadic chaining and discrepancy. J. Complexity 27, 531–540 (2011)Google Scholar
  2. 2.
    Atanassov, E.: On the discrepancy of the Halton sequences. Math. Balkanica (N.S.) 18, 15–32 (2004)Google Scholar
  3. 3.
    Beck, J.: Some upper bounds in the theory of irregularities of distribution. Acta Arith. 43, 115–130 (1984)Google Scholar
  4. 4.
    Beck, J.: A two-dimensional van Aardenne-Ehrenfest theorem in irregularities of distribution. Composito Math. 72, 269–339 (1989)Google Scholar
  5. 5.
    Beck, J., Chen, W.W.L.: Irregularities of Distribution. Cambridge University Press, Cambridge (1987)Google Scholar
  6. 6.
    Bilyk, D., Lacey, M.T.: On the small ball inequality in three dimensions. Duke Math. J. 143, 81–115 (2008)Google Scholar
  7. 7.
    Bilyk, D., Lacey, M.T., Vagharshakyan, A.: On the small ball inequality in all dimensions. J. Funct. Anal. 254, 2470–2502 (2008)Google Scholar
  8. 8.
    Bundschuh, P., Zhu, Y.C.: A method for exact calculation of the discrepancy of low-dimensional point sets I. Abh. Math. Sem. Univ. Hamburg 63, 115–133 (1993)Google Scholar
  9. 9.
    Chari, S., Rohatgi, P., Srinivasan, A.: Improved algorithms via approximation of probability distributions. J. Comput. System Sci. 61, 81–107 (2000)Google Scholar
  10. 10.
    de Clerck, L.: A method for exact calculation of the star-discrepancy of plane sets applied to the sequence of Hammersley. Monatsh. Math. 101, 261–278 (1986)Google Scholar
  11. 11.
    van der Corput, J.G.: Verteilungsfunktionen I. Nederl. Akad. Wetensch. Proc. Ser. B 38, 813–821 (1935)Google Scholar
  12. 12.
    van der Corput, J.G.: Verteilungsfunktionen II. Nederl. Akad. Wetensch. Proc. Ser. B 38, 1058–1066 (1935)Google Scholar
  13. 13.
    Davenport, H.: Note on irregularities of distribution. Matematika 3, 131–135 (1956)Google Scholar
  14. 14.
    Dick, J.: A note on the existence of sequences with small star discrepancy. J. Complexity 23, 649–652 (2007)Google Scholar
  15. 15.
    Dick, J.: Koksma-Hlawka type inequalities of fractional order. Ann. Mat. Pura Appl. 187, 385–403 (2008)Google Scholar
  16. 16.
    Dick, J., Pillichshammer, F.: Digital Nets and Sequences. Cambridge University Press, Cambridge (2010)Google Scholar
  17. 17.
    Dobkin, D.P., Eppstein, D., Mitchell, D.P.: Computing the discrepancy with applications to supersampling patterns. ACM Trans. Graph. 15, 354–376 (1996)Google Scholar
  18. 18.
    Doerr, B.: Generating randomized roundings with cardinality constraints and derandomizations. In: B. Durand, W. Thomas (eds.) STACS 2006, 23rd Annual Symposium on Theoretical Aspects of Computer Science, Lecture Notes in Computer Science, vol. 3884, pp. 571–583. Springer (2006)Google Scholar
  19. 19.
    Doerr, B., Gnewuch, M., Kritzer, P., Pillichshammer, F.: Component-by-component construction of low-discrepancy point sets of small size. Monte Carlo Methods Appl. 14, 129–149 (2008)Google Scholar
  20. 20.
    Doerr, B., Gnewuch, M., Srivastav, A.: Bounds and construction for the star discrepancy via δ-covers (2004). Berichtsreihe des Mathematischen Seminars der Christian-Albrechts-Universität zu Kiel, Report 04–13 (Preliminary preprint version of [21])Google Scholar
  21. 21.
    Doerr, B., Gnewuch, M., Srivastav, A.: Bounds and constructions for the star discrepancy via δ-covers. J. Complexity 21, 691–709 (2005)Google Scholar
  22. 22.
    Doerr, B., Gnewuch, M., Wahlström, M.: Implementation of a component-by-component algorithm to generate low-discrepancy samples. In: P. L’Ecuyer, A.B. Owen (eds.) Monte Carlo and Quasi-Monte Carlo Methods 2008, pp. 323–338. Springer, Berlin Heidelberg (2009)Google Scholar
  23. 23.
    Doerr, B., Gnewuch, M., Wahlström, M.: Algorithmic construction of low-discrepancy point sets via dependent randomized rounding. J. Complexity 26, 490–507 (2010)Google Scholar
  24. 24.
    Doerr, B., Wahlström, M.: Randomized rounding in the presence of a cardinality constraint. In: I. Finocchi, J. Hershberger (eds.) Proceedings of the 10th Workshop on Algorithm Engineering and Experiments (ALENEX 2009), pp. 162–174. SIAM, New York, USA (2009)Google Scholar
  25. 25.
    Drmota, M., Tichy, R.F.: Sequences, Discrepancies, and Applications, Lecture Notes in Math., vol. 1651. Springer, Berlin Heidelberg New York (1997)Google Scholar
  26. 26.
    Dueck, G., Scheuer, T.: Threshold accepting: A general purpose algorithm appearing superior to simulated annealing. J. Comp. Phys. 90 (1990)Google Scholar
  27. 27.
    Even, G., Goldreich, O., Luby, M., Nisan, N., Veličković, B.: Approximations of general independent distributions. In: Proceedings of the 24th ACM Symposium on Theory of Computing (STOC), pp. 10–16 (1992)Google Scholar
  28. 28.
    Faure, H.: Discrépance de suites associées à un système de numération (en dimension s). Acta Arith. 41, 337–351 (1982)Google Scholar
  29. 29.
    Frank, K., Heinrich, S.: Computing discrepancies of Smolyak quadrature rules. J. Complexity 12, 287–314 (1996)Google Scholar
  30. 30.
    Frolov, K.K.: An upper estimate for the discrepancy in the L p-metric, 2 ≤ p ≤ . Soviet. Math. Dokl. 21, 840–842 (1980)Google Scholar
  31. 31.
    Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman and Co., San Francisco (1979)Google Scholar
  32. 32.
    Giannopoulos, P., Knauer, C., Wahlström, M., Werner, D.: Hardness of discrepancy computation and ε-net verification in high dimension. J. Complexity 28, 162–176 (2012)Google Scholar
  33. 33.
    Gnewuch, M.: Weighted geometric discrepancies and numerical integration on reproducing kernel Hilbert spaces. J. Complexity 28, 2–17 (2012)Google Scholar
  34. 34.
    Gnewuch, M.: Bounds for the average L p-extreme and the L -extreme discrepancy. Electron. J. Combin. 12, Research Paper 54 (2005)Google Scholar
  35. 35.
    Gnewuch, M.: Bracketing numbers for axis-parallel boxes and applications to geometric discrepancy. J. Complexity 24, 154–172 (2008)Google Scholar
  36. 36.
    Gnewuch, M.: Construction of minimal bracketing covers for rectangles. Electron. J. Combin. 15, Research Paper 95 (2008)Google Scholar
  37. 37.
    Gnewuch, M.: On probabilistic results for the discrepancy of a hybrid-Monte Carlo sequence. J. Complexity 23, 312–317 (2009)Google Scholar
  38. 38.
    Gnewuch, M., Srivastav, A., Winzen, C.: Finding optimal volume subintervals with k points and calculating the star discrepancy are NP-hard problems. J. Complexity 25, 115–127 (2009)Google Scholar
  39. 39.
    Gnewuch, M., Wahlström, M., Winzen, C.: A new randomized algorithm to approximate the star discrepancy based on threshold-accepting. SIAM J. Numer. Anal. 50, 781–807 (2012)Google Scholar
  40. 40.
    Halton, J.H.: On the efficiency of certain quasi-random sequences of points in evaluating multidimensional integrals. Numer. Math. 2, 84–90 (1960)Google Scholar
  41. 41.
    Hardy, G.H., Littlewood, J.E.: Some problems of diophantine approximation: the lattice points of a right-angled triangle. Part I. Proc. London Math. Soc. 20, 212–249 (1922)Google Scholar
  42. 42.
    Haussler, D.: Sphere packing numbers for subsets of the Boolean n-cube with bounded Vapnik-Chervonenkis dimension. J. Combin. Theory A 69, 279–302 (1995)Google Scholar
  43. 43.
    Heinrich, S.: Efficient algorithms for computing the L 2 discrepancy. Math. Comp. 65, 1621–1633 (1996)Google Scholar
  44. 44.
    Heinrich, S.: Some open problems concerning the star-discrepancy. J. Complexity 19, 416–419 (2003)Google Scholar
  45. 45.
    Heinrich, S., Novak, E., Wasilkowski, G.W., Woźniakowski, H.: The inverse of the star-discrepancy depends linearly on the dimension. Acta Arith. 96, 279–302 (2001)Google Scholar
  46. 46.
    Hickernell, F.J.: A generalized discrepancy and quadrature error bound. Math. Comp. 67, 299–322 (1998)Google Scholar
  47. 47.
    Hickernell, F.J., Sloan, I.H., Wasilkowski, G.W.: On tractability of weighted integration over bounded and unbounded regions in \({\mathbb{R}}^{s}\). Math. Comp. 73, 1885–1901 (2004)Google Scholar
  48. 48.
    Hickernell, F.J., Wang, X.: The error bounds and tractability of quasi-Monte Carlo algorithms in infinite dimensions. Math. Comp. 71, 1641–1661 (2001)Google Scholar
  49. 49.
    Hinrichs, A.: Covering numbers, Vapnik-Červonenkis classes and bounds for the star-discrepancy. J. Complexity 20, 477–483 (2004)Google Scholar
  50. 50.
    Hinrichs, A., Pillichshammer, F., Schmid, W.C.: Tractability properties of the weighted star discrepancy. J. Complexity 23, 134–143 (2008)Google Scholar
  51. 51.
    Hlawka, E.: Funktionen von beschränkter Variation in der Theorie der Gleichverteilung. Ann. Mat. Pura Appl. 54, 325–333 (1961)Google Scholar
  52. 52.
    Hoeffding, W.: Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58, 13–30 (1963)Google Scholar
  53. 53.
    Hromković, J.: Algorithms for Hard Problems. Springer, Berlin Heidelberg (2003). 2nd editionGoogle Scholar
  54. 54.
    Johnson, D.S.: The NP-completeness column: an ongoing guide. J. Algorithms 8, 438–448 (1987)Google Scholar
  55. 55.
    Kirckpatrick, S., Gelatt, C., Vecchi, M.: Optimization by simulated annealing. Science 20, 671–680 (1983)Google Scholar
  56. 56.
    Koksma, J.F.: Een algemeene stelling uit de theorie der gelijkmatige verdeeling modulo 1. Mathematica B (Zutphen) 11, 7–11 (1942/3)Google Scholar
  57. 57.
    Lerch, M.: Question 1547. L’Intermédiaire des Mathématiciens 11, 145–146 (1904)Google Scholar
  58. 58.
    Matoušek, J.: Geometric Discrepancy. Springer-Verlag, Berlin Heidelberg (1999). Revised second printing 2010.Google Scholar
  59. 59.
    Mhaskar, H.N.: On the tractability of multivariate integration and approximation by neural networks. J. Complexity 20, 561–590 (2004)Google Scholar
  60. 60.
    Niederreiter, H.: Discrepancy and convex programming. Ann. Mat. Pura Appl. 93, 89–97 (1972)Google Scholar
  61. 61.
    Niederreiter, H.: Methods for estimating discrepancy. In: S.K. Zaremba (ed.) Applications of number theory to numerical analysis, pp. 203–236. Academic Press, New York (1972)Google Scholar
  62. 62.
    Niederreiter, H.: Random Number Generation and Quasi-Monte Carlo Methods. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA (1992)Google Scholar
  63. 63.
    Niederreiter, H.: On the discrepancy of some hybrid sequences. Acta Arith. 138, 373–398 (2009)Google Scholar
  64. 64.
    Niederreiter, H.: Further discrepancy bounds and an Erdös-Turán-Koksma inequality for hybrid sequences. Monatsh. Math. 161, 193–222 (2010)Google Scholar
  65. 65.
    Niederreiter, H., Xing, C.: Low-discrepancy sequences and global function fields with many rational places. Finite Fields and their Applications 2, 241–273 (1996)Google Scholar
  66. 66.
    Novak, E., Woźniakowski, H.: Tractability of Multivariate Problems, Volume I: Linear Information. European Mathematical Society, Freiburg (2008)Google Scholar
  67. 67.
    Novak, E., Woźniakowski, H.: L 2 discrepancy and multivariate integration. In: W.W.L. Chen, W.T. Gowers, H. Halberstam, W.M. Schmidt, R.C. Vaughan (eds.) Essays in honour of Klaus Roth, pp. 359–388. Cambridge Univ. Press, Cambridge (2009)Google Scholar
  68. 68.
    Novak, E., Woźniakowski, H.: Tractability of Multivariate Problems, Volume II: Standard Information for Functionals. European Mathematical Society, Freiburg (2010)Google Scholar
  69. 69.
    Ökten, G.: A probabilistic result on the discrepancy of a hybrid Monte Carlo sequence and applications. Monte Carlo Methods Appl. 2, 255–270 (1996)Google Scholar
  70. 70.
    Ökten, G., Shah, M., Goncharov, Y.: Random and deterministic digit permutations of the Halton sequence. In: L. Plaskota, H. Woźniakowski (eds.) Monte Carlo and Quasi-Monte Carlo Methods 2010, pp. 609–622. Springer-Verlag, Berlin (2012)Google Scholar
  71. 71.
    Ökten, G., Tuffin, B., Burago, V.: A central limit theorem and improved error bounds for a hybrid-Monte Carlo sequence with applications in computational finance. J. Complexity 22, 435–458 (2006)Google Scholar
  72. 72.
    Ostrowski, A.: Bemerkungen zur Theorie der Diophantischen Approximationen I. Abh. Math. Semin. Univ. Hamburg 1, 77–98 (1922)Google Scholar
  73. 73.
    Ostrowski, A.: Bemerkungen zur Theorie der Diophantischen Approximationen II. Abh. Math. Semin. Univ. Hamburg 1, 250–251 (1922)Google Scholar
  74. 74.
    Panconesi, A., Srinivasan, A.: Randomized distributed edge coloring via an extension of the Chernoff–Hoeffding bounds. SIAM J. Comput. 26, 350–368 (1997)Google Scholar
  75. 75.
    Raghavan, P.: Probabilistic construction of deterministic algorithms: approximating packing integer programs. J. Comput. Syst. Sci. 37, 130–143 (1988)Google Scholar
  76. 76.
    Roth, K.F.: On irregularities of distribution. Mathematika 1, 73–79 (1954)Google Scholar
  77. 77.
    Roth, K.F.: On irregularities of distribution III. Acta Arith. 35, 373–384 (1979)Google Scholar
  78. 78.
    Roth, K.F.: On irregularities of distribution IV. Acta Arith. 37, 67–75 (1980)Google Scholar
  79. 79.
    Schmidt, W.M.: On irregularities of distribution VII. Acta Arith. 21, 45–50 (1972)Google Scholar
  80. 80.
    Shah, M.: A genetic algorithm approach to estimate lower bounds of the star discrepancy. Monte Carlo Methods Appl. 16, 379–398 (2010)Google Scholar
  81. 81.
    Sloan, I.H., Joe, S.: Lattice Methods for Multiple Integration. Oxford University Press, New York (1994)Google Scholar
  82. 82.
    Sloan, I.H., Woźniakowski, H.: When are quasi-Monte Carlo algorithms efficient for high dimensional integrals? J. Complexity 14, 1–33 (1998)Google Scholar
  83. 83.
    Spanier, J.: Quasi-Monte Carlo methods for particle transport problems. In: H. Niederreiter, P.J.S. Shiue (eds.) Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing, Lecture Notes in Statistic, vol. 106, pp. 121–148. Springer, Berlin (1995)Google Scholar
  84. 84.
    Srinivasan, A.: Distributions on level-sets with applications to approximation algorithms. In: Proceedings of FOCS’01, pp. 588–597 (2001)Google Scholar
  85. 85.
    Steinerberger, S.: The asymptotic behavior of the average L p-discrepancies and a randomized discrepancy. Electron. J. Combin. 17(1), Research paper 106 (2010)Google Scholar
  86. 86.
    Talagrand, M.: Sharper bounds for Gaussian and empirical processes. Ann. Probab. 22, 28–76 (1994)Google Scholar
  87. 87.
    Thiémard, E.: Computing bounds for the star discrepancy. Computing 65, 169–186 (2000)Google Scholar
  88. 88.
    Thiémard, E.: An algorithm to compute bounds for the star discrepancy. J. Complexity 17, 850–880 (2001)Google Scholar
  89. 89.
    Thiémard, E.: Optimal volume subintervals with k points and star discrepancy via integer programming. Math. Meth. Oper. Res. 54, 21–45 (2001)Google Scholar
  90. 90.
    van der Vaart, A.W., Wellner, J.A.: Weak Convergence and Empirical Processes. Springer Series in Statistics. Springer, New York (1996)Google Scholar
  91. 91.
    Warnock, T.T.: Computational investigations of low-discrepancy point sets. In: S.K. Zaremba (ed.) Applications of number theory to numerical analysis, pp. 319–343. Academic Press, New York (1972)Google Scholar
  92. 92.
    Winker, P., Fang, K.T.: Application of threshold-accepting to the evaluation of the discrepancy of a set of points. SIAM J. Numer. Anal. 34, 2028–2042 (1997)Google Scholar
  93. 93.
    Woźniakowski, H.: Average case complexity of multivariate integration. Bull. Amer. Math. Soc. (N. S.) 24, 185–191 (1991)Google Scholar
  94. 94.
    Zaremba, S.K.: Some applications of multidimensional integration by parts. Ann. Polon. Math. 21, 85–96 (1968)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.Institut für InformatikChristian-Albrechts-Universität zu KielKielGermany

Personalised recommendations