Predicting the unpredictable
26
Citations
180
Downloads
Abstract A major difficulty for currently existing theories of inductive inference involves the question of what to do when novel, unknown, or previously unsuspected phenomena occur. In this paper one particular instance of this difficulty is considered, the so-called sampling of species problem .

The classical probabilistic theories of inductive inference due to Laplace, Johnson, de Finetti, and Carnap adopt a model of simple enumerative induction in which there are a prespecified number of types or species which may be observed. But, realistically, this is often not the case. In 1838 the English mathematician Augustus De Morgan proposed a modification of the Laplacian model to accommodate situations where the possible types or species to be observed are not assumed to be known in advance; but he did not advance a justification for his solution.

In this paper a general philosophical approach to such problems is suggested, drawing on work of the English mathematician J. F. C. Kingman. It then emerges that the solution advanced by De Morgan has a very deep, if not totally unexpected, justification. The key idea is that although ‘exchangeable’ random sequences are the right objects to consider when all possible outcome-types are known in advance, exchangeable random partitions are the right objects to consider when they are not. The result turns out to be very satisfying. The classical theory has several basic elements: a representation theorem for the general exchangeable sequence (the de Finetti representation theorem), a distinguished class of sequences (those employing Dirichlet priors), and a corresponding rule of succession (the continuum of inductive methods). The new theory has parallel basic elements: a representation theorem for the general exchangeable random partition (the Kingman representation theorem), a distinguished class of random partitions (the Poisson-Dirichlet process), and a rule of succession which corresponds to De Morgan's rule.

References Aldous, D. J.: 1985, ‘Exchangeability and Related Topics’, in P. L. Hennequin (ed.),

École d'Été de Probabilités de Saint-Flour XIII — 1983, Lecture Notes in Mathematics
1117 , 1–198.

Google Scholar Antoniak, C. E.: 1974, ‘Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems’,

Annals of Statistics
2 , 1152–74.

Google Scholar Bayes, T.: 1764, ‘An Essay Towards Solving a Problem in the Doctrine of Chances’,

Philosophical Transactions of the Royal Society of London
53 , 370–418 (reprinted: 1958,

Biometrika
45 , 293–315 (page citations in the text are to this edition)).

Google Scholar Blackwell, D. and MacQueen, J. B.: 1973, ‘Ferguson Distributions via Polya Urn Schemes’,

Annals of Statistics
1 , 353–55.

Google Scholar Carnap, Rudolph: 1950,

Logical Foundations of Probability , University of Chicago Press, Chicago.

Google Scholar Carnap, R.: 1980, ‘A Basic System of Inductive Logic, Part II’, in R. C. Jeffrey (ed.),

Studies in Inductive Logic and Probability , Vol. 2, University of California Press, Berkeley and Los Angeles, pp. 7–155.

Google Scholar De Finetti, B.: 1937, ‘La prevision: ses lois logiques, ses sources subjectives’,

Annales de l'Institut Henri Poincaré
7 , 1–68.

Google Scholar De Morgan, Augustus: 1838,

An Essay on Probabilities, and on their Application to Life Contingencies and Insurance Offices , Longman et al., London.

Google Scholar De Morgan, A.: 1845, ‘Theory of Probabilities’, in

Encyclopedia Metropolitana, Volume 2: Pure Mathematics , B. Fellowes et al., London, pp. 393–490.

Google Scholar Diaconis, P. and Freedman, D.: 1980, ‘De Finetti's Generalizations of Exchangeability’, in R. C. Jeffrey (ed.),

Studies in Inductive Logic and Probability , Vol. 2, University of California Press, Berkeley and Los Angeles, pp. 233–50.

Google Scholar Donnelly, P.: 1986, ‘Partition Structures, Polya Urns, the Ewens Sampling Formula, and the Ages of Alleles’,

Theoretical Population Biology
30 , 271–88.

Google Scholar Efron, B. and Thisted, R.: 1976, ‘Estimating the Number of Unseen Species: How Many Words did Shakespeare Know?’,

Biometrika
63 , 435–47.

Google Scholar Ewens, W. J.: 1972, ‘The Sampling Theory of Selectively Neutral Alleles’,

Theoretical Population Biology
3 , 87–112.

Google Scholar Feller, William: 1968,

An Introduction to Probability Theory and its Applications , Vol. 1, 3d ed., Wiley, New York.

Google Scholar Fisher, R. A., Corbet, A. S. and Williams, C. B.: 1943, ‘The Relation Between the Number of Species and the Number of Individuals in a Random Sample of an Animal Population’,

Journal of Animal Ecology
12 , 42–58.

Google Scholar Good, I. J.: 1953, ‘On the Population Frequencies of Species and the Estimation of Population Parameters’,

Biometrika
40 , 237–64.

Google Scholar Good, I. J. and Toulmin, G. H.: 1956, ‘The Number of New Species, and the Increase in Population Coverage, When a Sample is Increased’,

Biometrika
43 , 45–63.

Google Scholar Good, I. J.: 1965,

The Estimation of Probabilities: An Essay on Modern Bayesian Methods , M.I.T. Press, Cambridge MA.

Google Scholar Good, I. J.: 1979, ‘Turing's Statistical Work in World War II’,

Biometrika
66 , 393–96.

Google Scholar Hill, B.: 1968, ‘Posterior Distribution of Percentiles: Bayes's Theorem for Sampling from a Finite Population’,

Journal of the American Statistical Association
63 , 677–91.

Google Scholar Hill, B.: 1970, ‘Zipf's Law and Prior Distributions for the Composition of a Population’,

Journal of the American Statistical Association
65 , 1220–32.

Google Scholar Hill, B.: 1979, ‘Posterior Moments of the Number of Species in a Finite Population, and the Posterior Probability of Finding a New Species’,

Journal of the American Statistical Association
74 , 668–73.

Google Scholar Hill, B.: 1988, ‘Parametric Models for A

_{n} : Splitting Processes and Mixtures’, unpublished manuscript.

Google Scholar Hintikka, J. and Niiniluoto, I.: 1980, ‘An Axiomatic Foundation for the Logic of Inductive Generalization’, in R. C. Jeffrey (ed.),

Studies in Inductive Logic and Probability , Vol. 2, University of California Press, Berkeley and Los Angeles, pp. 157–82.

Google Scholar Hoppe, F.: 1984, ‘Polya-Like Urns and the Ewens Sampling Formula’,

Journal of Mathematical Biology
20 , 91–94.

Google Scholar Hoppe, F.: 1987, ‘The Sampling Theory of Neutral Alleles and an Urn Model in Population Genetics’,

Journal of Mathematical Biology
25 , 123–59.

Google Scholar Jeffrey, R. C. (ed.): 1980,

Studies in Inductive Logic and Probability , Vol. 2, University of California Press, Berkeley and Los Angeles.

Google Scholar Johnson, William Ernest: 1924,

Logic, Part III: The Logical Foundations of Science , Cambridge University Press, Cambridge.

Google Scholar Johnson, William Ernest: 1932, ‘Probability: the Deductive and Inductive Problems’,

Mind
49 , 409–23.

Google Scholar Kingman, J. F. C.: 1975, ‘Random Discrete Distributions’,

Journal of the Royal Statistical Society
B37 , 1–22.

Google Scholar Kingman, J. F. C.: 1978a, ‘Random Partitions in Population Genetics’,

Proceedings of the Royal Society
A361 , 1–20.

Google Scholar Kingman, J. F. C.: 1978b, ‘The Representation of Partition Structures’,

Journal of the London Mathematical Society
18 , 374–80.

Google Scholar Kingman, J. F. C.: 1980,

The Mathematics of Genetic Diversity , SIAM, Philadelphia.

Google Scholar Kuipers, T. A. F.: 1973, ‘A Generalization of Carnap's Inductive Logic’,

Synthese
25 , 334–36.

Google Scholar Kuipers, T. A. F.: 1986, ‘Some Estimates of the Optimum Inductive Method’,

Erkenntnis
24 , 37–46.

Google Scholar Laplace, P. S., Marquis de: 1781, ‘Mémoire sur les probabilités’,

Mem. Acad. Sci. Paris 1778 , 227–32 (

Oeuvres complètes , Vol. 9, pp. 383–485).

Google Scholar Quetelet, A.: 1846,

Lettres à S.A.R. le Duc Régnant de Saxe-Cobourg et Gotha, sur la théorie des probabilités, appliquée aux sciences morales et politiques , Hayez, Brussels.

Google Scholar Thisted, R. and Efron, B.: 1987, ‘Did Shakespeare Write a Newly-Discovered Poem?’,

Biometrika
74 , 445–55.

Google Scholar Zabell, S. L.: 1982, ‘W. E. Johnson's “Sufficientness” Postulate’,

Annals of Statistics
10 , 1091–99.

Google Scholar Zabell, S. L.: 1988, ‘Symmetry and its Discontents’, in B. Skyrms and W. L. Harper (eds.),

Causation, Chance, and Credence , Vol. 1, Kluwer, Dordrecht, pp. 155–90.

Google Scholar Zabell, S. L.: 1989, ‘The Rule of Succession’,

Erkenntnis
31 , 283–321.

Google Scholar © Kluwer Academic Publishers 1992

Authors and Affiliations 1. Department of Mathematics Northwestern University Evanston USA