Abstract
Categorical logic has shown that modern logic is essentially the logic of subsets (or “subobjects”). In “subset logic,” predicates are modeled as subsets of a universe and a predicate applies to an individual if the individual is in the subset. Partitions are dual to subsets so there is a dual logic of partitions where a “distinction” [an ordered pair of distinct elements (u, u′) from the universe U] is dual to an “element”. A predicate modeled by a partition π on U would apply to a distinction if the pair of elements was distinguished by the partition π, i.e., if u and u′ were in different blocks of π. Subset logic leads to finite probability theory by taking the (Laplacian) probability as the normalized size of each subset-event of a finite universe. The analogous step in the logic of partitions is to assign to a partition the number of distinctions made by a partition normalized by the total number of ordered |U|2 pairs from the finite universe. That yields a notion of “logical entropy” for partitions and a “logical information theory.” The logical theory directly counts the (normalized) number of distinctions in a partition while Shannon’s theory gives the average number of binary partitions needed to make those same distinctions. Thus the logical theory is seen as providing a conceptual underpinning for Shannon’s theory based on the logical notion of “distinctions.”
Similar content being viewed by others
References
Aczél J. and Daróczy Z. (1975). On measures of information and their characterization. Academic Press, New York
Adelman M.A. (1969). Comment on the H concentration measure as a numbers-equivalent. Review of Economics and Statistics 51: 99–101
Baclawski, K., & Rota, G.-C. (1979). An introduction to probability and random processes. Unpublished typescript. 467 pages. Download available at: http://www.ellerman.org.
Bhargava T.N. and Uppuluri V.R.R. (1975). On an axiomatic derivation of gini diversity, with applications. Metron 33: 41–53
Birkhoff G. (1948). Lattice theory. American Mathematical Society, New York
Cover T. and Thomas J. (1991). Elements of information theory. John Wiley, New York
Finberg D., Mainetti M. and Rota G.C. (1996). The logic of commuting equivalence relations. In: Aldo, U. and Agliano, P. (eds) Logic and algebra, pp 69–96. Marcel Dekker, New York
Friedman W.F. (1922). The index of coincidence and its applications in cryptography. Riverbank Laboratories, Geneva, IL
Ganeshaiah K.N., Chandrashekara K. and Kumar A.R.V. (1997). Avalanche Index: A new measure of biodiversity based on biological heterogeneity of communities. Current Science 73: 128–133
Gini C. (1912). Variabilitá e mutabilitá. Tipografia di Paolo Cuppini, Bologna
Gini C. (1955). Variabilitá e mutabilitá. In: Pizetti, E. and Salvemini, T. (eds) Memorie di metodologica statistica, pp. Libreria Eredi Virgilio Veschi, Rome
Good I.J. (1979). A. M. Turing’s statistical work in World War II. Biometrika 66(2): 393–396
Good I.J. (1982). Comment (on Patil and Taillie: Diversity as a concept and its measurement). Journal of the American Statistical Association 77(379): 561–563
Gray R.M. (1990). Entropy and information theory. Springer-Verlag, New York
Hartley R.V.L. (1928). Transmission of information. Bell System Technical Journal 7(3): 535–563
Havrda J.H. and Charvat F. (1967). Quantification methods of classification processes: Concept of structural α-entropy. Kybernetika (Prague) 3: 30–35
Herfindahl O.C. (1950). Concentration in the U.S. Steel Industry. Unpublished doctoral dissertation, Columbia University
Hirschman A.O. (1945). National power and the structure of foreign trade. University of California Press, Berkeley
Hirschman A.O. (1964). The Paternity of an Index. American Economic Review 54(5): 761–762
Kapur J.N. (1994). Measures of information and their applications. Wiley Eastern, New Delhi
Kolmogorov A.N. (1956). Foundations of the theory of probability. Chelsea, New York
Kullback S. (1976). Statistical methods in cryptanalysis. Aegean Park Press, Walnut Creek CA
Lawvere F.W. and Rosebrugh R. (2003). Sets for mathematics. Cambridge University Press, Cambridge
MacArthur R.H. (1965). Patterns of species diversity. Biology Review 40: 510–533
Patil G.P. and Taillie C. (1982). Diversity as a concept and its measurement. Journal of the American Statistical Association 77(379): 548–561
Rao C.R. (1982). Diversity and dissimilarity coefficients: A unified approach. Theoretical Population Biology 21: 24–43
Rejewski M. (1981). How polish mathematicians deciphered the enigma. Annals of the History of Computing 3: 213–234
Rényi A. (1965). On the theory of random search. Bulletin of American Mathematical Society 71: 809–828
Rényi A. (1970). Probability theory (trans: Vekerdi, L.). North-Holland, Amsterdam
Rényi, A. (1976). In P. Turan (Ed.), Selected Papers of Alfréd Rényi, Volumes 1,2, and 3. Budapest: Akademiai Kiado.
Ricotta C. and Szeidl L. (2006). Towards a unifying approach to diversity measures: Bridging the gap between the Shannon entropy and Rao’s quadratic index. Theoretical Population Biology 70: 237–243
Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27, 379–423; 623–656.
Simpson E.H. (1949). Measurement of diversity. Nature 163: 688
Stigler S.M. (1999). Statistics on the table. Harvard University Press, Cambridge
Tsallis C. (1988). Possible generalization for Boltzmann-Gibbs statistics. Journal of Statistical Physics 52: 479–487
Vajda I. (1969). A contribution to informational analysis of patterns. In: Watanabe, S. (eds) Methodologies of pattern recognition, pp 509–519. Academic Press, New York
Author information
Authors and Affiliations
Corresponding author
Additional information
This paper is dedicated to the memory of Gian-Carlo Rota—mathematician, philosopher, mentor, and friend.
Rights and permissions
About this article
Cite this article
Ellerman, D. Counting distinctions: on the conceptual foundations of Shannon’s information theory. Synthese 168, 119–149 (2009). https://doi.org/10.1007/s11229-008-9333-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11229-008-9333-7