Skip to main content
SpringerLink
Log in

Entropy Optimization Principles and Their Applications

  • Chapter
Entropy and Energy Dissipation in Water Resources

Part of the book series: Water Science and Technology Library ((WSTL,volume 9))

Abstract

A state-of-the-art description of the theory and applications of the various entropy optimization principles is given. These principles include Jaynes’ maximum entropy principle (MaxEnt), Kullback’s minimum cross-entropy principle (MinxEnt), generalised maximum entropy and minimum cross-entropy principles, inverse entropy optimization principles, minimum interdependence principle, minimax entropy principle and finally, the dual entropy optimization principles. The relation between information-theoretic entropy and thermodynamic entropy is specially recalled in the context of the more general relationship that exist between what are designated as primary and secondary entropies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. R. Christensen (1981) Entropy Minimax Source Book, 1–4, Entropy Ltd., Lincoln, Mass.

    Google Scholar 

  2. I. Csiszer (1972) “A class of measures of informativity of Observation channels”, Periodic Math. Hungarica, Vol. 2, pp. 191–213.

    Article  Google Scholar 

  3. J.G. Erickson and C.R. Smith (editors) (1988) Maximum Entropy and Bayesian Methods in Science and Engineering, Vol. 1 (Foundations), Vol. 2 ( Applications). Kluwer Academic Publishers, New York.

    Google Scholar 

  4. S. Erlander (1978) Optimal Interaction and the Gravity Models, Springer-Verlag, New York.

    Google Scholar 

  5. B. Forte and C. Sempi (1976) “Maximizing conditional Entropy: A Derivation of Quantal Statistics”, Rendi Conta de Mathematics, Vol. 9, pp. 551–566.

    Google Scholar 

  6. P.F. Fourgere (ed) (1990) Maximum Entropy and Bayesian Methods, Kluwer Academic Press, New York.

    Google Scholar 

  7. N. Georgescu-Rogen (1971) The Entropy Law and The Economic Process, Harvard University Press, Camb.

    Google Scholar 

  8. S. Guissu, R. Lablanc and L. Reischer (1982) “On the principle of minimum interdependence”, J. Inf. Opt. Sci. Vol. 3, pp. 149–172.

    Google Scholar 

  9. J.H. Havrda and F. Charvat (1967) “Quantification methods of classification Processes: Concepts of Structural Entropy”, Kybernatica, Vol. 3, pp. 30–35.

    Google Scholar 

  10. E.T. Jaynes (1957) “Information theory and statistical mechanics”, Physical Reviews, Vol. 106, pp. 620–630.

    Article  Google Scholar 

  11. J.H. Justice (editor) (1986) Maximum Entropy and Bayesian Methods in Applied Statistics, Cambridge University Press, Boston.

    Google Scholar 

  12. J.N. Kapur (1972) “Measures of Uncertainty, Mathematical programming and Physics”, Jour. Ind. Soc. Agri. Stat., Vol. 24, pp. 46–66.

    Google Scholar 

  13. J.N. Kapur (1983) “Derivation of logistic law of population growth from maximum entropy principle”, Nat. Acad. Sci. Letters, Vol. 6, No. 12, pp. 429–433.

    Google Scholar 

  14. J.N. Kapur (1983) “Comparative assessment of various measures of entropy”, Jour. Inf. and Opt. Sci. Vol. 4, No. 1, pp. 207–232.

    Google Scholar 

  15. J.N. Kapur (1983) “Non-additive measures of entropy and distributions of Statistical Mechanics”, Ind. Jour., Pure and App. Math., Vol. 1, No. 11, pp. 1372–1384.

    Google Scholar 

  16. J.N. Kapur (1983) “Twenty-five years of maximum entropy”, Jour. Math. Phy. Sci. Vol. 17, No. 2, pp. 103–156.

    Google Scholar 

  17. J.N. Kapur (1984) “The role of maximum entropy and minimum discrimination information principles in statistics”, Jour. Ind. Soc. Agri. Stat., Vol. 36, No. 3, pp. 12–55.

    Google Scholar 

  18. J.N. Kapur (1984) “A comparative assessment of various measures of directed divergences”, Advances in Management studies, Vol. 3, pp. 1–16.

    Article  Google Scholar 

  19. J.N. Kapur (1984) “On maximum entropy principle and its applications to science and engineering”, Proc. Nat. Symposium on Mathematical Modelling, MRI, Allahabad, India, pp. 75–78.

    Google Scholar 

  20. J.N. Kapur (1984) “On minimum interdependence principle”, Ind. Jour. Pure and App. Math. 15 (9), 968–977.

    Google Scholar 

  21. J.N. Kapur (1984) “Maximum entropy models in science and engineering”, Proc. Nat. Acad. Sciences, (Presidential Address, Physical Sciences Section ), Annual number, pp. 35–57.

    Google Scholar 

  22. J.N. Kapur, P.K. Sahoo and A.K.C. Wong (1985) “A new method of grey level picture thresholding using entropy of the histogram”, Computer vision, Graphics and Image Processing, Vol. 29, pp. 273–28?.

    Google Scholar 

  23. J.N. Kapur (1986) “Application of entropie measures of stochastic dependence”, Pattern Recognition, Vol. 19, pp. 473–476.

    Article  Google Scholar 

  24. J.N. Kapur (1985) “Some thoughts on scientific and philosophical foundation of the maximum entropy principle”, Bull. Math. Ass. Ind., 17, pp. 15–40.

    Google Scholar 

  25. J.N. Kapur (1986) “Four families of measures of entropy”, Ind. Jour. Pure and App. Maths., Vol. 17, No. 4, pp. 429–449.

    Google Scholar 

  26. J.N. Kapur (1990) Maximum-Entropy Models in Science and Engineering, John Wiley, New York.

    Google Scholar 

  27. J.N. Kapur “Applications of generalised maximum entropy principle to population dynamics, innovation diffusion models and chemical reactions”. To appear in Journ. Math. Phy. Sciences.

    Google Scholar 

  28. J.N. Kapur “Entropy optimization principles and their applications”. To appear in Mathematics Student.

    Google Scholar 

  29. J.N. Kapur and H.K. Kesavan (1990) “Inverse MaxEnt or MinxEnt principles and their applications”, In Maximum Entropy and Bayesian Methods, edited by P. Fourgere, pp. 433–450.

    Google Scholar 

  30. J.N. Kapur (1989) Maximum entropy principle, large-scale systems and cybernetics, In Artificial Intelligence, Ed. by A. Ghoshal et al., South Asia Publishers, New Delhi.

    Google Scholar 

  31. H.K. Kesavan and J.N. Kapur (1989) “The generalised maximum entropy principle”, IEEE Trans. Syst. Man. Cyb., 19, pp. 1042–1052.

    Article  Google Scholar 

  32. J.N. Kapur and H.K. Kesavan (1988) Generalised Maximum Entropy Principle (with Applications), PP. 225, Sandford Educational Press, University of Waterloo.

    Google Scholar 

  33. J.N. Kapur and H.K. Kesavan (1991) Entropy Optimization Principles and their Applications, (book under publication).

    Google Scholar 

  34. J.N. Kapur and H.K. Kesavan (1989) “Generalised maximum entropy principle”, Proc. Int. Conf. Math. Mod., Vol. 2, IIT Madras, pp. 1–11.

    Google Scholar 

  35. H.K. Kesavan and J.N. Kapur (1990) “On the families of solutions of generalised maximum and minimum cross-entropy models”, Int. Jour. Systems, Vol. 16, pp. 199–219.

    Article  Google Scholar 

  36. J.N. Kapur and H.K. Kesavan (1990) “Maximum entropy and minimum cross entropy principles: need for new perspectives”, In Maximum Entropy and Bayesian Methods, edited by P. Fougere, Kluwer Press, pp. 419–432.

    Google Scholar 

  37. S. Kullback and R.A. Leibler (1951) “On information and sufficiency”, Ann. Math. Stat., Vol. 22, pp. 79–86.

    Article  Google Scholar 

  38. S. Kullback (1958) On Information Theory and Statistics, John Wiley, New York.

    Google Scholar 

  39. A. Renyi (1961) “On measures of entropy and information”, Proc. 4th Berkeley Symposium, Maths. Stat. Prob., Vol. 1, pp. 547–561.

    Google Scholar 

  40. J. Rifkin (1980) Entropy: A New World View, Vikas Press.

    Google Scholar 

  41. A.K. Seth (1989) “Prof. J.N. Kapur’s Views on Entropy Optimization Principles”, Bull. Math. Ass. Ind., Vol. 21, pp. 1–38, Vol. 22, 1–42.

    Google Scholar 

  42. C.E. Shannon (1948) “A mathematical theory of communication”, Bell System, Tech. J., Vol$127, pp. 379–423, 623–659.

    Google Scholar 

  43. J. Skilling (Editor) (1989) Maximum Entropy and Bayesian Methods, Kluwer Academic Publishers, New York.

    Google Scholar 

  44. C.R. Smith and W.T. Grandy, Jr. (1985) (eds) Maximum Entropy and Bayesian Methods in Inverse Problem, D. Reidel, Doedrecht, Holland.

    Google Scholar 

  45. C.R. Smith and J.G Erickson (eds) (1989) Maximum-Entropy and Bayesian Spectral Analysis and Estimation Problems, D. Reidel and Kluwer Academic Publishers, New York.

    Google Scholar 

  46. M. Tribus (1979) “Thirty years of information theory”, In Maximum Entropy Formalism, (ed) R.D. Levine and M. Tribus, MIT Press.

    Google Scholar 

  47. S. Watanabe (1981) “Pattern recognition as a quest for minimum entropy”, Pattern Recognition, Vol. 13, pp. 381–387.

    Article  Google Scholar 

  48. S. Watanabe (1969) Knowing and Guessing, John Wiley, New York.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Systems Design Department, University of Waterloo, Waterloo, Ontario, Canada, N2L 3G1

    J. N. Kapur & H. K. Kesavan

Authors
  1. J. N. Kapur
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. H. K. Kesavan
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Civil Engineering, Louisiana State University, Baton Rouge, USA

    V. P. Singh

  2. Department of Environmental Engineering and Physics, University of Basilicata, Potenza, Italy

    M. Fiorentino

Rights and permissions

Reprints and permissions

Copyright information

© 1992 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Kapur, J.N., Kesavan, H.K. (1992). Entropy Optimization Principles and Their Applications. In: Singh, V.P., Fiorentino, M. (eds) Entropy and Energy Dissipation in Water Resources. Water Science and Technology Library, vol 9. Springer, Dordrecht. https://doi.org/10.1007/978-94-011-2430-0_1

Download citation

  • DOI: https://doi.org/10.1007/978-94-011-2430-0_1

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-94-010-5072-2

  • Online ISBN: 978-94-011-2430-0

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation