A Survey of Some Quantitative Approaches to the Notion of Information

  • Aldo de Luca
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2300)


We survey some formalizations of the intuitive notion of information which have been formulated in different mathematical and conceptual frames. The existence of different formalizations reflects the different aspects of the notion of information which are strongly related to the mechanisms of information processing of the receiver. Finally, some considerations and remarks on information in Physics and Biology are made.


Energy Measure Entropy Measure Random Experiment Source Message Conditional Information 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Alsina, C., Trillas, E.: Sur les mesures du degré de flou. Stocastica 3 (1979) 81–84zbMATHMathSciNetGoogle Scholar
  2. [2]
    Backer, E.: A Non Statistical Type of Uncertainty in Fuzzy Events, in: Topics in Information Theory, Colloquia Mathematica Societatis Janos Bolyai, vol. 16, pp. 53–73. Amsterdam: North-Holland 1977Google Scholar
  3. [3]
    Bedrosian, S.D., Xie, W.X.: An Information Measure for Fuzzy Sets. IEEE Transactions on Systems, Man, and Cybernetics 14 (1984) 151–156MathSciNetGoogle Scholar
  4. [4]
    Blum, M.: A Machine Independent Theory of the Complexity of Recursive Functions. J. of ACM 14 (1967) 322–336zbMATHCrossRefMathSciNetGoogle Scholar
  5. [5]
    Boltzmann, L.: Vorlesungen über Gas Theorie, vol.1. J. A. Barth Leipzig, 1896, English Transl. Lectures on Gas Theory. Berkeley Calif. 1964Google Scholar
  6. [6]
    Brillouin, L.: La Science et La Théorie de l’Information. Paris: Masson 1959Google Scholar
  7. [7]
    Capocelli, R., de Luca, A.: Fuzzy Sets and Decision Theory. Information and Control 23 (1973) 446–473CrossRefMathSciNetzbMATHGoogle Scholar
  8. [8]
    Carnap, R., Bar-Hillel, Y.: An Outline of a Theory of Semantic Information. Tech. Rep. 247, M.I.T., Research Laboratory of Electronics, 1952, Reprinted in Bar-Hillel, Y.: Language and Information. Reading, MA: Addison Wesley 1962Google Scholar
  9. [9]
    Carpi, A., de Luca, A.: Words and Special Factors. Theoretical Computer Science 259 (2001) 145–182zbMATHCrossRefMathSciNetGoogle Scholar
  10. [10]
    Carpi, A., de Luca, A.: Words and Repeated Factors. Séminaire Lotharingien de Combinatoire B42l (1998) 24 pp. and in: Foata, D., Han, G. (Eds.) Andrews Festschrift, pp. 231–251. Berlin: Springer 2001Google Scholar
  11. [11]
    Carpi, A., de Luca, A.: Repetitions and Boxes in Words and Pictures, in: Karhumäki, J., Maurer, H., Păun, G., Rozenberg, G. (Eds.) Jewels Are Forever, pp. 295–306. Berlin: Springer 1999Google Scholar
  12. [12]
    Carpi, A., de Luca, A.: Special Factors, Periodicity, and an Application to Sturmian Words. Acta Informatica 36 (2000) 983–1006zbMATHCrossRefMathSciNetGoogle Scholar
  13. [13]
    Carpi, A., de Luca, A.: Periodic-like Words, Periodicity, and Boxes. Acta Informatica 37 (2001) 597–618zbMATHCrossRefMathSciNetGoogle Scholar
  14. [14]
    Carpi, A., de Luca, A., Varricchio, S.: Special Factors and Uniqueness Conditions in Rational Trees. Theory of Computing Systems (to appear)Google Scholar
  15. [15]
    Carpi, A., de Luca, A., Varricchio, S.: Words, Univalent Factors, and Boxes. Preprint 41/2000, Dipartimento di Matematica dell’Università di Roma ‘La Sapienza’, 2000Google Scholar
  16. [16]
    Chaitin, G.J.: On the Length of Programs for Computing Finite Binary Sequences. J. of ACM 13 (1966) 547–569zbMATHCrossRefMathSciNetGoogle Scholar
  17. [17]
    Chaitin, G.J.: Information-Theoretic Computational Complexity. IEEE Trans. on Information Theory 20 (1974) 10–15zbMATHCrossRefMathSciNetGoogle Scholar
  18. [18]
    Colosimo, A., de Luca, A.: Special Factors in Biological Strings. J. Theor. Biol. 204 (2000) 29–46CrossRefGoogle Scholar
  19. [19]
    Davis, M.: Computability and Unsolvability. New York: McGraw Hill 1958zbMATHGoogle Scholar
  20. [20]
    de Luca, A.: Complexity and Information Theory, in: Longo, G. (Ed.) Coding and Complexity, CISM Courses and Lectures No. 216, pp. 207–270. Wien: Springer 1975Google Scholar
  21. [21]
    de Luca, A.: On the Entropy of a Formal Languages, Lecture Notes in Computer Science, vol. 33, pp. 103–109. Berlin: Springer 1975Google Scholar
  22. [22]
    de Luca, A.: The Mind as a Mechanism of the Brain. (1982) (manuscript)Google Scholar
  23. [23]
    de Luca, A.: Dispersion Measures of Fuzzy Sets, in: Gupta, M.M., Kandel, A., Bandler, W., Kiszka, J.B. (Eds.) Approximate Reasoning in Expert Systems, pp. 199–216. Amsterdam: North-Holland 1985Google Scholar
  24. [24]
    de Luca, A.: On the Combinatorics of FiniteWords. Theoretical Computer Science 218 (1999) 13–39zbMATHCrossRefMathSciNetGoogle Scholar
  25. [25]
    de Luca, A., Fischetti, E.: Outline of a New Logical Approach to Information Theory, in Caianiello, E.R. (Ed.) New Concepts and Technologies in Parallel Processing, Nato Advanced Study Institutes Series, Series E: Applied Sciences-No.9, pp. 359–380. Leyden: Noordho. International Publ. 1975Google Scholar
  26. [26]
    de Luca, A., Termini, S.: A Definition of a Non-probabilistic Entropy in the Setting of Fuzzy Sets Theory. Information and Control 20 (1972) 301–312CrossRefMathSciNetzbMATHGoogle Scholar
  27. [27]
    de Luca, A., Termini, S.: Algebraic Properties of Fuzzy Sets. J. Math. Anal. Appl. 40 (1972) 373–386CrossRefMathSciNetGoogle Scholar
  28. [28]
    de Luca, A., Termini, S.: Entropy of L-fuzzy Sets. Information and Control 24 (1974) 55–73CrossRefMathSciNetzbMATHGoogle Scholar
  29. [29]
    de Luca, A., Termini, S.: Entropy and Energy Measures of a Fuzzy Set, in: Gupta, M.M., Ragade, R.K., Yager, R.R. (Eds.) Advances in Fuzzy Set Theory and Applications, pp. 321–338. Amsterdam: North-Holland 1979Google Scholar
  30. [30]
    de Luca, A., Termini, S.: Superposition Sets and Their Entropies, Proc. s IEEE Int. Conf. on Systems, Man and Cybernetics, Delhi/Bombay, India, pp. 493–497. New York: IEEE 1983Google Scholar
  31. [31]
    de Luca, A., Termini, S.: Entropy Measures in the Theory of Fuzzy Sets, in Singh, M.G. (Ed.) Encyclopedia of Systems and Control, pp. 1467–1473. Oxford: Pergamon-Press 1983Google Scholar
  32. [32]
    Ebanks, B.R.: On Measures of Fuzziness and Their Representations. J. Math. Anal. Appl. 94 (1983) 24–37zbMATHCrossRefMathSciNetGoogle Scholar
  33. [33]
    Hartley, R.V.L.: Transmission of Information. Bell Syst. Tech. J. 7 (1928) 535–563Google Scholar
  34. [34]
    Khinchin, A.I.: Mathematical Foundations of Information Theory. New York: Dover Publ. 1957zbMATHGoogle Scholar
  35. [35]
    Klir, G.J. (Ed.): Measures of Uncertainty. Fuzzy Sets and Systems (Special Issue) 24 (1987) n.2Google Scholar
  36. [36]
    Knopfmacher, J.: On Measures of Fuzziness. J. Math. Anal. Appl. 49 (1975) 529–534zbMATHCrossRefMathSciNetGoogle Scholar
  37. [37]
    Kolmogorov, A.N.: Three Approaches to the Quantitative Definition of Information. Problemi Peredachi Informatsii 1 (1965) 3–11MathSciNetzbMATHGoogle Scholar
  38. [38]
    Li, M., Vitanyi, P.: An Introduction to Kolmogorov Complexity and Its Applications. Berlin: Springer 1993zbMATHGoogle Scholar
  39. [39]
    Lothaire, M.: Combinatorics on Words. Reading, MA: Addison-Wesley 1983zbMATHGoogle Scholar
  40. [40]
    Lothaire, M.: Algebraic Combinatorics on Words. Cambridge: Cambridge University Press (to appear)Google Scholar
  41. [41]
    Martin-Löf, P.: The Definition of Random Sequences. Information and Control 9 (1966) 602–619CrossRefMathSciNetGoogle Scholar
  42. [42]
    Nyquist, H.: Certain Factors Affecting Telegraph Speed. Bell Syst. Tech. J. 3 (1924) 324Google Scholar
  43. [43]
    Pal, S.K.: A Note on the Quantitative Measure of Image Enhancement Through Fuzziness. IEEE Trans. on Pattern Analysis and Machine Intelligence 4 (1982) 204–208zbMATHCrossRefGoogle Scholar
  44. [44]
    Papadimitriou, C.H.: Computational Complexity. Reading MA: Addison-Wesley 1994zbMATHGoogle Scholar
  45. [45]
    Shannon, C.E.: A Mathematical Theory of Communication. Bell Syst. Tech. J. 27 (1948) 379–423, 623–656MathSciNetzbMATHGoogle Scholar
  46. [46]
    Shannon C.E., Weaver, W.: The Mathematical Theory of Communication. Urbana ILL: University of Illinois Press 1949zbMATHGoogle Scholar
  47. [47]
    Shimizu, A.: Approximate Reasoning Based on Fuzzy Entropy’s Theory, in: Piera Carreté, N., Singh, M.G. (Eds.) Qualitative Reasoning and Decision Technologies, pp. 662–671. Barcelona: CIMNE 1983Google Scholar
  48. [48]
    Solomonov, R.J.: A Formal Theory of Inductive Inference. Part I. Information and Control 7 (1964) 1–22CrossRefMathSciNetGoogle Scholar
  49. [49]
    Szilard, L.: Über die Entropieverminderung in einem thermodynamischen System bei Eingriffen intelligenter Wesen. Zeitschrift f. Physik 53 (1929) 840–856, Reprinted: On Decrease of Entropy in Thermodynamic System by the Intervention of Intelligent Beings. Behavioral Science 9 (1964) 301–310CrossRefGoogle Scholar
  50. [50]
    Taneja, I.J.: On Generalized Entropies with Applications, in Ricciardi, L.M. (Ed.) Lectures in Applied Mathematics and Informatics, pp. 107–169. Manchester: Manchester University Press 1990Google Scholar
  51. [51]
    Trillas, E., Riera, T.: Entropies in Finite Fuzzy Sets. Information Sciences 15 (1978) 159–168zbMATHCrossRefMathSciNetGoogle Scholar
  52. [52]
    Wiener, N.: The Extrapolation, Interpolation and Smoothing of Stationary Time Series. New York: J. Wiley and Sons 1949Google Scholar
  53. [53]
    Zadeh, L.A.: Fuzzy Sets. Information and Control 8 (1965) 338–353zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Aldo de Luca
    • 1
    • 2
  1. 1.Dipartimento di Matematica dell’Università di Roma “La Sapienza”RomaItaly
  2. 2.Centro Interdisciplinare ‘B. Segre’Accademia dei LinceiRomaItaly

Personalised recommendations