Computable Exchangeable Sequences Have Computable de Finetti Measures

  • Cameron E. Freer
  • Daniel M. Roy
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5635)

Abstract

We prove a uniformly computable version of de Finetti’s theorem on exchangeable sequences of real random variables. In the process, we develop machinery for computably recovering a distribution from its sequence of moments, which suffices to prove the theorem in the case of (almost surely) continuous directing random measures. In the general case, we give a proof inspired by a randomized algorithm which succeeds with probability one. Finally, we show how, as a consequence of the main theorem, exchangeable stochastic processes in probabilistic functional programming languages can be rewritten as procedures that do not use mutation.

Keywords

de Finetti exchangeability computable probability theory probabilistic programming languages mutation 

References

  1. [AES00]
    Alvarez-Manilla, M., Edalat, A., Saheb-Djahromi, N.: An extension result for continuous valuations. J. London Math. Soc. 61(2), 629–640 (2000)MathSciNetCrossRefMATHGoogle Scholar
  2. [Ald81]
    Aldous, D.J.: Representations for partially exchangeable arrays of random variables. J. Multivariate Analysis 11(4), 581–598 (1981)MathSciNetCrossRefMATHGoogle Scholar
  3. [Ald85]
    Aldous, D.J.: Exchangeability and related topics. In: École d’été de probabilités de Saint-Flour, XIII—1983. Lecture Notes in Math., vol. 1117, pp. 1–198. Springer, Berlin (1985)CrossRefGoogle Scholar
  4. [Aus08]
    Austin, T.: On exchangeable random variables and the statistics of large graphs and hypergraphs. Probab. Surv. 5, 80–145 (2008)MathSciNetCrossRefMATHGoogle Scholar
  5. [Bau05]
    Bauer, A.: Realizability as the connection between constructive and computable mathematics. In: CCA 2005: Second Int. Conf. on Comput. and Complex in Analysis (2005)Google Scholar
  6. [BG07]
    Brattka, V., Gherardi, G.: Borel complexity of topological operations on computable metric spaces. In: Cooper, S.B., Löwe, B., Sorbi, A. (eds.) CiE 2007. LNCS, vol. 4497, pp. 83–97. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  7. [Bil95]
    Billingsley, P.: Probability and measure, 3rd edn. John Wiley & Sons Inc., New York (1995)MATHGoogle Scholar
  8. [Bos08]
    Bosserhoff, V.: Notions of probabilistic computability on represented spaces. J. of Universal Comput. Sci. 14(6), 956–995 (2008)MathSciNetMATHGoogle Scholar
  9. [dF31]
    de Finetti, B.: Funzione caratteristica di un fenomeno aleatorio. Atti della R. Accademia Nazionale dei Lincei, Ser. 6. Memorie, Classe di Scienze Fisiche, Matematiche e Naturali 4, 251–299 (1931)Google Scholar
  10. [dF75]
    de Finetti, B.: Theory of probability, vol. 2. John Wiley & Sons Ltd., London (1975)MATHGoogle Scholar
  11. [DF84]
    Diaconis, P., Freedman, D.: Partial exchangeability and sufficiency. In: Statistics: applications and new directions (Calcutta, 1981), pp. 205–236. Indian Statist. Inst., Calcutta (1984)Google Scholar
  12. [DJ08]
    Diaconis, P., Janson, S.: Graph limits and exchangeable random graphs. Rendiconti di Matematica, Ser. VII 28(1), 33–61 (2008)MathSciNetMATHGoogle Scholar
  13. [Eda95]
    Edalat, A.: Domain theory and integration. Theoret. Comput. Sci. 151(1), 163–193 (1995)MathSciNetCrossRefMATHGoogle Scholar
  14. [Eda96]
    Edalat, A.: The Scott topology induces the weak topology. In: 11th Ann. IEEE Symp. on Logic in Comput. Sci., pp. 372–381. IEEE Comput. Soc. Press, Los Alamitos (1996)CrossRefGoogle Scholar
  15. [GG05]
    Griffiths, T.L., Ghahramani, Z.: Infinite latent feature models and the Indian buffet process. In: Adv. in Neural Inform. Processing Syst., vol. 17, pp. 475–482. MIT Press, Cambridge (2005)Google Scholar
  16. [GMR+08]
    Goodman, N.D., Mansinghka, V.K., Roy, D.M., Bonawitz, K., Tenenbaum, J.B.: Church: a language for generative models. In: Uncertainty in Artificial Intelligence (2008)Google Scholar
  17. [GSW07]
    Grubba, T., Schröder, M., Weihrauch, K.: Computable metrization. Math. Logic Q. 53(4-5), 381–395 (2007)MathSciNetCrossRefMATHGoogle Scholar
  18. [Hoo79]
    Hoover, D.N.: Relations on probability spaces and arrays of random variables, Institute for Advanced Study. Princeton, NJ (preprint) (1979)Google Scholar
  19. [HS55]
    Hewitt, E., Savage, L.J.: Symmetric measures on Cartesian products. Trans. Amer. Math. Soc. 80, 470–501 (1955)MathSciNetCrossRefMATHGoogle Scholar
  20. [JP89]
    Jones, C., Plotkin, G.: A probabilistic powerdomain of evaluations. In: Proc. of the Fourth Ann. Symp. on Logic in Comp. Sci., pp. 186–195. IEEE Press, Los Alamitos (1989)CrossRefGoogle Scholar
  21. [Kal02]
    Kallenberg, O.: Foundations of modern probability, 2nd edn. Springer, New York (2002)CrossRefMATHGoogle Scholar
  22. [Kal05]
    Kallenberg, O.: Probabilistic symmetries and invariance principles. Springer, New York (2005)MATHGoogle Scholar
  23. [Kin78]
    Kingman, J.F.C.: Uses of exchangeability. Ann. Probability 6(2), 183–197 (1978)MathSciNetCrossRefMATHGoogle Scholar
  24. [Koz81]
    Kozen, D.: Semantics of probabilistic programs. J. Comp. System Sci. 22(3), 328–350 (1981)MathSciNetCrossRefMATHGoogle Scholar
  25. [KTG+06]
    Kemp, C., Tenenbaum, J., Griffiths, T., Yamada, T., Ueda, N.: Learning systems of concepts with an infinite relational model. In: Proc. of the 21st Nat. Conf. on Artificial Intelligence (2006)Google Scholar
  26. [Lau84]
    Lauritzen, S.L.: Extreme point models in statistics. Scand. J. Statist. 11(2), 65–91 (1984)MathSciNetMATHGoogle Scholar
  27. [Man09]
    Mansinghka, V.K.: Natively Probabilistic Computing. PhD thesis, Massachusetts Institute of Technology (2009)Google Scholar
  28. [Mül99]
    Müller, N.T.: Computability on random variables. Theor. Comput. Sci. 219(1-2), 287–299 (1999)MathSciNetCrossRefMATHGoogle Scholar
  29. [Pfe01]
    Pfeffer, A.: IBAL: A probabilistic rational programming language. In: Proc. of the 17th Int. Joint Conf. on Artificial Intelligence, pp. 733–740. Morgan Kaufmann Publ., San Francisco (2001)Google Scholar
  30. [PPT08]
    Park, S., Pfenning, F., Thrun, S.: A probabilistic language based on sampling functions. ACM Trans. Program. Lang. Syst. 31(1), 1–46 (2008)CrossRefGoogle Scholar
  31. [PR89]
    Pour-El, M.B., Richards, J.I.: Computability in analysis and physics. Springer, Berlin (1989)CrossRefMATHGoogle Scholar
  32. [RMG+08]
    Roy, D.M., Mansinghka, V.K., Goodman, N.D., Tenenbaum, J.B.: A stochastic programming perspective on nonparametric Bayes. In: Nonparametric Bayesian Workshop, Int. Conf. on Machine Learning (2008)Google Scholar
  33. [RN57]
    Ryll-Nardzewski, C.: On stationary sequences of random variables and the de Finetti’s equivalence. Colloq. Math. 4, 149–156 (1957)MathSciNetMATHGoogle Scholar
  34. [Rog67]
    Rogers, Jr., H.: Theory of recursive functions and effective computability. McGraw-Hill, New York (1967)MATHGoogle Scholar
  35. [RP02]
    Ramsey, N., Pfeffer, A.: Stochastic lambda calculus and monads of probability distributions. In: Proc. of the 29th ACM SIGPLAN-SIGACT Symp. on Principles of Program. Lang., pp. 154–165 (2002)Google Scholar
  36. [RT09]
    Roy, D.M., Teh, Y.W.: The Mondrian process. In: Adv. in Neural Inform. Processing Syst., vol. 21 (2009)Google Scholar
  37. [Sch07]
    Schröder, M.: Admissible representations for probability measures. Math. Logic Q. 53(4-5), 431–445 (2007)MathSciNetCrossRefMATHGoogle Scholar
  38. [Set94]
    Sethuraman, J.: A constructive definition of Dirichlet priors. Statistica Sinica 4, 639–650 (1994)MathSciNetMATHGoogle Scholar
  39. [Soa87]
    Soare, R.I.: Recursively enumerable sets and degrees. Springer, Berlin (1987)CrossRefMATHGoogle Scholar
  40. [SS06]
    Schröder, M., Simpson, A.: Representing probability measures using probabilistic processes. J. Complex. 22(6), 768–782 (2006)MathSciNetCrossRefMATHGoogle Scholar
  41. [TGG07]
    Teh, Y.W., Görür, D., Ghahramani, Z.: Stick-breaking construction for the Indian buffet process. In: Proc. of the 11th Conf. on A.I. and Stat. (2007)Google Scholar
  42. [TJ07]
    Thibaux, R., Jordan, M.I.: Hierarchical beta processes and the Indian buffet process. In: Proc. of the 11th Conf. on A.I. and Stat. (2007)Google Scholar
  43. [Wei99]
    Weihrauch, K.: Computability on the probability measures on the Borel sets of the unit interval. Theoret. Comput. Sci. 219(1–2), 421–437 (1999)MathSciNetCrossRefMATHGoogle Scholar
  44. [Wei00]
    Weihrauch, K.: Computable analysis: an introduction. Springer, Berlin (2000)CrossRefMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Cameron E. Freer
    • 1
  • Daniel M. Roy
    • 2
  1. 1.Department of MathematicsMassachusetts Institute of TechnologyUSA
  2. 2.Computer Science and Artificial Intelligence LaboratoryMassachusetts Institute of TechnologyUSA

Personalised recommendations