Journal of Logic, Language and Information

, Volume 21, Issue 2, pp 163–188

Probability as a Measure of Information Added



Some propositions add more information to bodies of propositions than do others. We start with intuitive considerations on qualitative comparisons of information added. Central to these are considerations bearing on conjunctions and on negations. We find that we can discern two distinct, incompatible, notions of information added. From the comparative notions we pass to quantitative measurement of information added. In this we borrow heavily from the literature on quantitative representations of qualitative, comparative conditional probability. We look at two ways to obtain a quantitative conception of information added. One, the most direct, mirrors Bernard Koopman’s construction of conditional probability: by making a strong structural assumption, it leads to a measure that is, transparently, some function of a function P which is, formally, an assignment of conditional probability (in fact, a Popper function). P reverses the information added order and mislocates the natural zero of the scale so some transformation of this scale is needed but the derivation of P falls out so readily that no particular transformation suggests itself. The Cox–Good–Aczél method assumes the existence of a quantitative measure matching the qualitative relation, and builds on the structural constraints to obtain a measure of information that can be rescaled as, formally, an assignment of conditional probability. A classical result of Cantor’s, subsequently strengthened by Debreu, goes some way towards justifying the assumption of the existence of a quantitative scale. What the two approaches give us is a pointer towards a novel interpretation of probability as a rescaling of a measure of information added.


Information Probability Comparative probability Koopman Cox 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aczél, J. (1966). Lectures on functional equations and their applications. New York and London: Academic Press. (Reprinted 2006, Mineola NY: Dover). (Supplemented English translation of Vorlesungen über Funktionalgleichungen und ihre Anwendungen. Basel: Birkhäuser, 1961.)Google Scholar
  2. Aczél J., Daróczy Z. (1975) On measures of information and their characterization, Volume 115 of Mathematics in science and engineering. Academic Press, New York and LondonGoogle Scholar
  3. Aczél J., Pfanzagl J. (1966) Remarks on the measurement of subjective probability and information. Metrika 2: 91–105Google Scholar
  4. Arnborg, S., & Sjödin, G. (2001). On the foundations of Bayesianism. In A. Mohammad-Djarafi (Ed.), Bayesian inference and maximum entropy methods in science and engineering, 20th international workshop, Gif-sur-Yvette (France), 2000. Vol. 568 of AIP conference proceedings (pp. 61–71). American Institute of Physics.Google Scholar
  5. Bar-Hillel, Y. (1952). Semantic information and its measures. In Transactions of the tenth conference on cybernetics (pp. 33–48). New York: Josiah Macy Jr., Foundation. (Reprinted In Language and information: Selected essays on their theory and application, pp. 298–310, Y. Bar-Hillel, Ed., 1964, Reading, MA: Addison-Wesley.)Google Scholar
  6. Carnap R., Bar-Hillel Y. (1953) Semantic information. British Journal for the Philosophy of Science 4: 147–157Google Scholar
  7. Csiszár I. (2008) Axiomatic characterizations of information measures. Entropy 10: 261–273CrossRefGoogle Scholar
  8. Debreu G. (1954) Representation of preference ordering by a numerical function. In: Thrall R., Coombs C., David R. (eds) Decision processes. Wiley, New York, pp 159–165Google Scholar
  9. Dummett, M. A. E. (1976). What is a theory of meaning? (II). In G. Evans, J. McDowell (Eds.), Truth and meaning: Essays in semantics (pp. 67–137). Oxford: Oxford University Press. (Reprinted in Dummett, The seas of language, pp. 34–93, 1996, Oxford: Oxford University Press, Page reference to the reprint.)Google Scholar
  10. Dummett M. A. E. (1978) Truth and other enigmas. Duckworth, LondonGoogle Scholar
  11. Edgington, D. (1986). Do conditionals have truth conditions? Crítica 18, 3–39. (Reprinted In Conditionals, pp. 176–201, F. Jackson, (Ed.), 1991, Oxford: Oxford University Press.)Google Scholar
  12. Good, I. J. (1962). Subjective probability as the measure of an unmeasurable set. In E. Nagel, P. Suppes, & A. Tarski (Eds.), Logic, methodology and philosophy of science (pp. 319–329). Stanford: Stanford University Press. (Reprinted In Studies in subjective probability, second edition, pp. 133–146, H. Kyburg, & H. Smokler, Eds., 1980, Huntington NY: Krieger, Page reference to the reprint.)Google Scholar
  13. Hacking I. (1965) The logic of statistical inference. Cambridge University Press, CambridgeGoogle Scholar
  14. Ingarden R. S., Urbanik K. (1962) Information without probability. Colloquium Mathematicum 9: 131–150Google Scholar
  15. Joyce, J. M. (2009). Accuracy and coherence: Prospects for an alethic epistemology of partial belief. In F. Huber & C. Schmidt-Petri (Eds.), Degrees of belief, Vol. 342 of synthese library (pp. 263–266) New York: Springer.Google Scholar
  16. Keynes J. M. (1921) A treatise on probability. Macmillan, London (Reprinted 2004, Mineola NY: Dover.)Google Scholar
  17. Kolmogorov, A. (1929). General measure theory and probability calculus. Sbornik rabot Matematicheskogo Razdela. Kommunisticheskaya Akademiya, Sektsiya Estestvennikh i Tochnikh Nauk 1, 8–21. (In Russian. English translation in A. N. Shiryayev (Ed.), Selected Works of A. N. Kolmogorov. Vol. II, Probability theory and mathematical statistics (pp. 48–59). (G. Lundquist, Trans.). Dordrecht: Kluwer (1992).)Google Scholar
  18. Koopman B. O. (1940a) The axioms and algebra of intuitive probability. Annals of Mathematics 41: 269–292CrossRefGoogle Scholar
  19. Koopman, B. O. (1940b). The bases of probability. Bulletin of the American Mathematical Society. 46, 763–774. (Reprinted In: H. Kyburg, H. Smokler (Eds.), Studies in subjective probability second ed., pp. 117–131, 1980, Huntington NY: Krieger.)Google Scholar
  20. Luce R.D., Suppes P., Tversky A. (1971) Foundations of measurement, Vol. I, additive and polynomial representations. Academic Press, San Diego and LondonGoogle Scholar
  21. Malmnäs P.-E. (1981) From qualitative to quantitative probability, Vol. 7 of Stockholm Studies in Philosophy. Almqvist and Wiksell, StockholmGoogle Scholar
  22. Milne, P. (2011). On measures of confirmation. British Journal for the Philosophy of Science, (to appear).Google Scholar
  23. Morgan C., Mares E. (1995) Conditionals, probability, and non-triviality. Journal of Philosophical Logic 24: 455–467CrossRefGoogle Scholar
  24. Osteyee, D. B. & Good, I. J. (1974). Information, weight of evidence, the singularity between probability measures and signal detection, Vol. 376 of Lecture notes in mathematics. Berlin, Heidelberg and New York: Springer.Google Scholar
  25. Paris , J. B. (1994) The uncertain reasoner’s companion: A mathematical perspective, Vol. 39 of Cambridge tracts in theoretical computer science. Cambridge University Press, CambridgeGoogle Scholar
  26. Popper, K. R. (1959). Logic of scientific discovery. London: Hutchinson. (Expanded English translation of Logik der Forschung, Vienna: Springer, 1935.)Google Scholar
  27. Popper, K. R. (1972). Conjectures and refutations (fourth ed.). London: Routledge and Kegan Paul. First edition, 1963.Google Scholar
  28. Roeper P., Leblanc H. (1991) Indiscernibility and identity in probability theory. Notre Dame Journal of Formal Logic 32: 1–46CrossRefGoogle Scholar
  29. Schroeder M. J. (2004) An alternative to entropy in the measurement of information. Entropy 6: 388–412CrossRefGoogle Scholar
  30. Szpilrajn E. (1930) Sur l’expansion de l’ordre partiel. Fundamenta Mathematicae 16: 386–389Google Scholar
  31. Van Horn K. S. (2003) Constructing a logic of plausible inference: A guide to Cox’s theorem. International Journal of Approximate Reasoning 34: 3–24CrossRefGoogle Scholar
  32. Weirich P. (1983) Conditional probabilities and probabilities given knowledge of a condition. Philosophy of Science 50: 82–95CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  1. 1.University of StirlingStirlingUnited Kingdom

Personalised recommendations