, Volume 90, Issue 2, pp 205–232 | Cite as

Predicting the unpredictable

  • S. L. Zabell


A major difficulty for currently existing theories of inductive inference involves the question of what to do when novel, unknown, or previously unsuspected phenomena occur. In this paper one particular instance of this difficulty is considered, the so-called sampling of species problem.

The classical probabilistic theories of inductive inference due to Laplace, Johnson, de Finetti, and Carnap adopt a model of simple enumerative induction in which there are a prespecified number of types or species which may be observed. But, realistically, this is often not the case. In 1838 the English mathematician Augustus De Morgan proposed a modification of the Laplacian model to accommodate situations where the possible types or species to be observed are not assumed to be known in advance; but he did not advance a justification for his solution.

In this paper a general philosophical approach to such problems is suggested, drawing on work of the English mathematician J. F. C. Kingman. It then emerges that the solution advanced by De Morgan has a very deep, if not totally unexpected, justification. The key idea is that although ‘exchangeable’ random sequences are the right objects to consider when all possible outcome-types are known in advance, exchangeable random partitions are the right objects to consider when they are not. The result turns out to be very satisfying. The classical theory has several basic elements: a representation theorem for the general exchangeable sequence (the de Finetti representation theorem), a distinguished class of sequences (those employing Dirichlet priors), and a corresponding rule of succession (the continuum of inductive methods). The new theory has parallel basic elements: a representation theorem for the general exchangeable random partition (the Kingman representation theorem), a distinguished class of random partitions (the Poisson-Dirichlet process), and a rule of succession which corresponds to De Morgan's rule.


Basic Element Representation Theorem Distinguished Class Major Difficulty Inductive Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aldous, D. J.: 1985, ‘Exchangeability and Related Topics’, in P. L. Hennequin (ed.), École d'Été de Probabilités de Saint-Flour XIII — 1983, Lecture Notes in Mathematics 1117, 1–198.Google Scholar
  2. Antoniak, C. E.: 1974, ‘Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems’, Annals of Statistics 2, 1152–74.Google Scholar
  3. Bayes, T.: 1764, ‘An Essay Towards Solving a Problem in the Doctrine of Chances’, Philosophical Transactions of the Royal Society of London 53, 370–418 (reprinted: 1958, Biometrika 45, 293–315 (page citations in the text are to this edition)).Google Scholar
  4. Blackwell, D. and MacQueen, J. B.: 1973, ‘Ferguson Distributions via Polya Urn Schemes’, Annals of Statistics 1, 353–55.Google Scholar
  5. Carnap, Rudolph: 1950, Logical Foundations of Probability, University of Chicago Press, Chicago.Google Scholar
  6. Carnap, R.: 1980, ‘A Basic System of Inductive Logic, Part II’, in R. C. Jeffrey (ed.), Studies in Inductive Logic and Probability, Vol. 2, University of California Press, Berkeley and Los Angeles, pp. 7–155.Google Scholar
  7. De Finetti, B.: 1937, ‘La prevision: ses lois logiques, ses sources subjectives’, Annales de l'Institut Henri Poincaré 7, 1–68.Google Scholar
  8. De Morgan, Augustus: 1838, An Essay on Probabilities, and on their Application to Life Contingencies and Insurance Offices, Longman et al., London.Google Scholar
  9. De Morgan, A.: 1845, ‘Theory of Probabilities’, in Encyclopedia Metropolitana, Volume 2: Pure Mathematics, B. Fellowes et al., London, pp. 393–490.Google Scholar
  10. Diaconis, P. and Freedman, D.: 1980, ‘De Finetti's Generalizations of Exchangeability’, in R. C. Jeffrey (ed.), Studies in Inductive Logic and Probability, Vol. 2, University of California Press, Berkeley and Los Angeles, pp. 233–50.Google Scholar
  11. Donnelly, P.: 1986, ‘Partition Structures, Polya Urns, the Ewens Sampling Formula, and the Ages of Alleles’, Theoretical Population Biology 30, 271–88.Google Scholar
  12. Efron, B. and Thisted, R.: 1976, ‘Estimating the Number of Unseen Species: How Many Words did Shakespeare Know?’, Biometrika 63, 435–47.Google Scholar
  13. Ewens, W. J.: 1972, ‘The Sampling Theory of Selectively Neutral Alleles’, Theoretical Population Biology 3, 87–112.Google Scholar
  14. Feller, William: 1968, An Introduction to Probability Theory and its Applications, Vol. 1, 3d ed., Wiley, New York.Google Scholar
  15. Fisher, R. A., Corbet, A. S. and Williams, C. B.: 1943, ‘The Relation Between the Number of Species and the Number of Individuals in a Random Sample of an Animal Population’, Journal of Animal Ecology 12, 42–58.Google Scholar
  16. Good, I. J.: 1953, ‘On the Population Frequencies of Species and the Estimation of Population Parameters’, Biometrika 40, 237–64.Google Scholar
  17. Good, I. J. and Toulmin, G. H.: 1956, ‘The Number of New Species, and the Increase in Population Coverage, When a Sample is Increased’, Biometrika 43, 45–63.Google Scholar
  18. Good, I. J.: 1965, The Estimation of Probabilities: An Essay on Modern Bayesian Methods, M.I.T. Press, Cambridge MA.Google Scholar
  19. Good, I. J.: 1979, ‘Turing's Statistical Work in World War II’, Biometrika 66, 393–96.Google Scholar
  20. Hill, B.: 1968, ‘Posterior Distribution of Percentiles: Bayes's Theorem for Sampling from a Finite Population’, Journal of the American Statistical Association 63, 677–91.Google Scholar
  21. Hill, B.: 1970, ‘Zipf's Law and Prior Distributions for the Composition of a Population’, Journal of the American Statistical Association 65, 1220–32.Google Scholar
  22. Hill, B.: 1979, ‘Posterior Moments of the Number of Species in a Finite Population, and the Posterior Probability of Finding a New Species’, Journal of the American Statistical Association 74, 668–73.Google Scholar
  23. Hill, B.: 1988, ‘Parametric Models for An: Splitting Processes and Mixtures’, unpublished manuscript.Google Scholar
  24. Hintikka, J. and Niiniluoto, I.: 1980, ‘An Axiomatic Foundation for the Logic of Inductive Generalization’, in R. C. Jeffrey (ed.), Studies in Inductive Logic and Probability, Vol. 2, University of California Press, Berkeley and Los Angeles, pp. 157–82.Google Scholar
  25. Hoppe, F.: 1984, ‘Polya-Like Urns and the Ewens Sampling Formula’, Journal of Mathematical Biology 20, 91–94.Google Scholar
  26. Hoppe, F.: 1987, ‘The Sampling Theory of Neutral Alleles and an Urn Model in Population Genetics’, Journal of Mathematical Biology 25, 123–59.Google Scholar
  27. Jeffrey, R. C. (ed.): 1980, Studies in Inductive Logic and Probability, Vol. 2, University of California Press, Berkeley and Los Angeles.Google Scholar
  28. Johnson, William Ernest: 1924, Logic, Part III: The Logical Foundations of Science, Cambridge University Press, Cambridge.Google Scholar
  29. Johnson, William Ernest: 1932, ‘Probability: the Deductive and Inductive Problems’, Mind 49, 409–23.Google Scholar
  30. Kingman, J. F. C.: 1975, ‘Random Discrete Distributions’, Journal of the Royal Statistical Society B37, 1–22.Google Scholar
  31. Kingman, J. F. C.: 1978a, ‘Random Partitions in Population Genetics’, Proceedings of the Royal Society A361, 1–20.Google Scholar
  32. Kingman, J. F. C.: 1978b, ‘The Representation of Partition Structures’, Journal of the London Mathematical Society 18, 374–80.Google Scholar
  33. Kingman, J. F. C.: 1980, The Mathematics of Genetic Diversity, SIAM, Philadelphia.Google Scholar
  34. Kuipers, T. A. F.: 1973, ‘A Generalization of Carnap's Inductive Logic’, Synthese 25, 334–36.Google Scholar
  35. Kuipers, T. A. F.: 1986, ‘Some Estimates of the Optimum Inductive Method’, Erkenntnis 24, 37–46.Google Scholar
  36. Laplace, P. S., Marquis de: 1781, ‘Mémoire sur les probabilités’, Mem. Acad. Sci. Paris 1778, 227–32 (Oeuvres complètes, Vol. 9, pp. 383–485).Google Scholar
  37. Quetelet, A.: 1846, Lettres à S.A.R. le Duc Régnant de Saxe-Cobourg et Gotha, sur la théorie des probabilités, appliquée aux sciences morales et politiques, Hayez, Brussels.Google Scholar
  38. Thisted, R. and Efron, B.: 1987, ‘Did Shakespeare Write a Newly-Discovered Poem?’, Biometrika 74, 445–55.Google Scholar
  39. Zabell, S. L.: 1982, ‘W. E. Johnson's “Sufficientness” Postulate’, Annals of Statistics 10, 1091–99.Google Scholar
  40. Zabell, S. L.: 1988, ‘Symmetry and its Discontents’, in B. Skyrms and W. L. Harper (eds.), Causation, Chance, and Credence, Vol. 1, Kluwer, Dordrecht, pp. 155–90.Google Scholar
  41. Zabell, S. L.: 1989, ‘The Rule of Succession’, Erkenntnis 31, 283–321.Google Scholar

Copyright information

© Kluwer Academic Publishers 1992

Authors and Affiliations

  • S. L. Zabell
    • 1
  1. 1.Department of MathematicsNorthwestern UniversityEvanstonUSA

Personalised recommendations