Skip to main content
SpringerLink
Log in

Entropy Maximization

  • Chapter
  • First Online:
Book cover Statistical Decision Problems

Part of the book series: Springer Optimization and Its Applications ((SOIA,volume 85))

  • 2281 Accesses

Abstract

The previous chapter showed that given independent observations of a random variable X, the probability distribution of X can be estimated based on the maximum likelihood principle. However, if no observations of X are available, but some integral characteristics of the distribution of X are known, for example, mean μ and standard deviation σ, the main principle for finding the distribution in question is, arguably, the one of maximum entropy. This principle, also known as MaxEnt, originated from the information theory and statistical mechanics (see [22]) and determines the “most unbiased” probability distribution for X subject to any constraints on X (prior information). Nowadays, it is widely used in financial engineering and statistical decision problems [4, 11, 56]. Estimation of probability distributions through entropy maximization and through relative entropy minimization subject to various constraints on unknown distributions is the subject of this chapter.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    \(\mathcal{D}\) is comonotone if \(\mathcal{D}(X + Y ) = \mathcal{D}(X) + \mathcal{D}(Y )\) for any two comonotone random variables \(X \in {\mathcal{L}}^{p}(\Omega )\) and \(Y \in {\mathcal{L}}^{p}(\Omega )\), whereas the random variables X and Y are comonotone, if there exists a set \(A \subseteq \Omega \) such that \(\mathbb{P}[A] = 1\) and \((X(\omega _{1}) - X(\omega _{2}))(Y (\omega _{1}) - Y (\omega _{2})) \geq 0\) for all \(\omega _{1},\omega _{2} \in A\).

References

  1. Acerbi, C.: Spectral measures of risk: a coherent representation of subjective risk aversion. J. Bank. Financ. 26(7), 1487–1503 (2002)

    Article  Google Scholar 

  2. Artzner, P., Delbaen, F., Eber, J.M., Heath, D.: Coherent measures of risk. Math. Financ. 9(3), 203–228 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  3. Bonami, P., Lejeune, M.A.: An exact solution approach for portfolio optimization problems under stochastic and integer constraints. Oper. Res. 57(3), 650–670 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  4. Buckley, J.J.: Entropy principles in decision making under risk. Risk Anal. 5(4), 303–313 (1979)

    Article  MathSciNet  Google Scholar 

  5. Chang, C.C., Lin, C.J.: Training ν-support vector classifiers: theory and algorithms. Neural Comput. 13, 2119–2147 (2001)

    Article  MATH  Google Scholar 

  6. Chekhlov, A., Uryasev, S., Zabarankin, M.: Portfolio Optimization with Drawdown Constraints, pp. 263–278. Risk Books, London (2003)

    Google Scholar 

  7. Chekhlov, A., Uryasev, S., Zabarankin, M.: Drawdown measure in portfolio optimization. Int. J. Theor. Appl. Financ. 8(1), 13–58 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  8. Cortes, C., Vapnik, V.: Support vector networks. Mach. Learn. 20, 273–297 (1995)

    MATH  Google Scholar 

  9. Costa, J., Hero, A., Vignat, C.: On Solutions to Multivariate Maximum-entropy Problems, vol. 2683, pp. 211–228. Springer, Berlin (2003)

    Google Scholar 

  10. Cover, T.M., Thomas, J.A.: Elements of Information Theory, 2nd edn. Wiley, New York (2006)

    MATH  Google Scholar 

  11. Cozzolino, J.M., Zahner, M.J.: The maximum-entropy distribution of the future market price of a stock. Oper. Res. 21(6), 1200–1211 (1973)

    Article  MATH  MathSciNet  Google Scholar 

  12. Crisp, D.J., Burges, C.J.C.: A geometric interpretation of ν-SVM classifiers. Neural Inf. Process. Syst. 12, 244–250 (2000)

    Google Scholar 

  13. Fölmer, H., Schied, A.: Stochastic Finance, 2nd edn. Walter de Gruyter GmbH & Co., Berlin (2004)

    Book  Google Scholar 

  14. Grauer, R.R.: Introduction to asset pricing theory and tests. In: Roll, R. (ed.) The International Library of Critical Writings in Financial Economics. Edward Elgar Publishing Inc., Cheltenham (2001)

    Google Scholar 

  15. Grechuk, B., Zabarankin, M.: Inverse portfolio problem with mean-deviation model. Eur. J. Oper. Res. (2013, to appear)

    Google Scholar 

  16. Grechuk, B., Molyboha, A., Zabarankin, M.: Maximum entropy principle with general deviation measures. Math. Oper. Res. 34(2), 445–467 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  17. Grechuk, B., Molyboha, A., Zabarankin, M.: Chebyshev’s inequalities with law invariant deviation measures. Probab. Eng. Informational Sci. 24, 145–170 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  18. Hardy, G.E., Littlewood, J.E., Pólya, G.: Inequalities, 2nd edn. Cambridge University Press, New York (1952)

    MATH  Google Scholar 

  19. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. Springer, New York (2008)

    Google Scholar 

  20. Hull, J.C., White, A.D.: Valuing credit derivatives using an implied copula approach. J. Derivatives 14(2), 8–28 (2006)

    Article  Google Scholar 

  21. Iscoe, I., Kreinin, A., Mausser, H., Romanko, A.: Portfolio credit-risk optimization. J. Bank. Financ. 36(6), 1604–1615 (2012)

    Article  Google Scholar 

  22. Jaynes, E.T.: Information theory and statistical mechanics. Phys. Rev. 106(4), 620–630 (1957)

    Article  MATH  MathSciNet  Google Scholar 

  23. Jensen, J.L.: Surles fonctions convexes et les inégalités entre les valeurs moyennes. Acta Math. 30(1), 175–193 (1906)

    Article  MATH  MathSciNet  Google Scholar 

  24. Johnson, O., Vignat, C.: Some results concerning maximum Rényi entropy distributions. Annales de l’Institut Henri Poincare (B) Probab. Stat. 43(3), 339–351 (2007)

    Google Scholar 

  25. Kalinchenko, K., Uryasev, S., Rockafellar, R.T.: Calibrating risk preferences with generalized CAPM based on mixed CVaR deviation. J. Risk 15(1), 45–70 (2012)

    Google Scholar 

  26. Koenker, R., Bassett, G.: Regression quantiles. Econometrica 46, 33–50 (1978)

    Article  MATH  MathSciNet  Google Scholar 

  27. Kurdila, A., Zabarankin, M.: Convex Functional Analysis. Birkhauser, Switzerland (2005)

    MATH  Google Scholar 

  28. Levy, H.: Stochastic dominance and expected utility: survey and analysis. Manag. Sci. 38(4), 555–593 (1992)

    Article  MATH  Google Scholar 

  29. Markowitz, H.M.: Portfolio selection. J. Financ. 7(1), 77–91 (1952)

    Google Scholar 

  30. Markowitz, H.M.: Foundations of portfolio theory. J. Financ. 46, 469–477 (1991)

    Article  Google Scholar 

  31. Mercer, J.: Functions of positive and negative type, and their connection with the theory of integral equations. Philos. Trans. Roy. Soc. London 209(441–458), 415–446 (1909)

    Article  MATH  Google Scholar 

  32. Molyboha, A., Zabarankin, M.: Stochastic optimization of sensor placement for diver detection. Oper. Res. 60(2), 292–312 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  33. Ogryczak, W., Ruszczyński, A.: On consistency of stochastic dominance and mean-semideviation models. Math. Program. 89, 217–232 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  34. Ogryczak, W., Ruszczyński, A.: Dual stochastic dominance and related mean-risk models. SIAM J. Optim. 13(1), 60–78 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  35. Perez-Cruz, F., Weston, J., Hermann, D.J.L., Schölkopf, B.: Extension of the ν-SVM range for classification. Adv. Learn. Theory Method. Models Appl. 190, 179–196 (2003)

    Google Scholar 

  36. Rockafellar, R.T.: Convex Analysis, Princeton Mathematics Series, vol. 28. Princeton University Press, Princeton (1970)

    Google Scholar 

  37. Rockafellar, R.T.: Coherent approaches to risk in optimization under uncertainty. In: Gray, P. (ed.) Tutorials in Operations Research, pp. 38–61. INFORMS, Hanover (2007)

    Google Scholar 

  38. Rockafellar, R.T., Royset, J.O.: On buffered failure probability in design and optimization of structures. Reliab. Eng. Syst. Saf. 95, 499–510 (2011)

    Article  Google Scholar 

  39. Rockafellar, R.T., Uryasev, S.: Optimization of conditional value-at-risk. J. Risk 2, 21–41 (2000)

    Google Scholar 

  40. Rockafellar, R.T., Uryasev, S.: Conditional value-at-risk for general loss distributions. J. Bank. Financ. 26(7), 1443–1471 (2002)

    Article  Google Scholar 

  41. Rockafellar, R.T., Uryasev, S., Zabarankin, M.: Deviation measures in risk analysis and optimization. Technical Report 2002–7. ISE Department, University of Florida, Gainesville, FL (2002)

    Google Scholar 

  42. Rockafellar, R.T., Uryasev, S., Zabarankin, M.: Generalized deviations in risk analysis. Financ. Stoch. 10(1), 51–74 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  43. Rockafellar, R.T., Uryasev, S., Zabarankin, M.: Master funds in portfolio analysis with general deviation measures. J. Bank. Financ. 30(2), 743–778 (2006)

    Article  MathSciNet  Google Scholar 

  44. Rockafellar, R.T., Uryasev, S., Zabarankin, M.: Optimality conditions in portfolio analysis with general deviation measures. Math. Program. 108(2–3), 515–540 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  45. Rockafellar, R.T., Uryasev, S., Zabarankin, M.: Equilibrium with investors using a diversity of deviation measures. J. Bank. Financ. 31(11), 3251–3268 (2007)

    Article  Google Scholar 

  46. Rockafellar, R.T., Uryasev, S., Zabarankin, M.: Risk tuning with generalized linear regression. Math. Oper. Res. 33(3), 712–729 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  47. Roell, A.: Risk aversion in Quiggin and Yaari’s rank-order model of choice under uncertainty. Econ. J. 97(Issue Supplement: Conference papers), 143–159 (1987)

    Google Scholar 

  48. Rousseeuw, P.J., Driessen, K.: Computing LTS regression for large data sets. Data Min. Knowl. Discov. 12(1), 29–45 (2006)

    Article  MathSciNet  Google Scholar 

  49. Rousseeuw, P., Leroy, A.: Robust Regression and Outlier Detection. Wiley, New York (1987)

    Book  MATH  Google Scholar 

  50. Roy, A.D.: Safety first and the holding of assets. Econometrica 20(3), 431–449 (1952)

    Article  MATH  Google Scholar 

  51. Ruszczyński, A.: Nonlinear Optimization. Princeton University Press, Princeton (2006)

    MATH  Google Scholar 

  52. Schölkopf, B., Smola, A., Williamson, R., Bartlett, P.: New support vector algorithms. Neural Comput. 12, 1207–1245 (2000)

    Article  Google Scholar 

  53. Sharpe, W.F.: Capital asset prices: a theory of market equilibrium under conditions of risk. J. Financ. 19, 425–442 (1964)

    Google Scholar 

  54. Sharpe, W.F.: Capital asset prices with and without negative holdings. J. Financ. 46, 489–509 (1991)

    Article  Google Scholar 

  55. Takeda, A., Sugiyama, M.: ν-support vector machine as conditional value-at-risk minimization. In: Proceedings of the 25th International Conference on Machine Learning (ICML 2008), pp. 1056–1063. Morgan Kaufmann, Montreal, Canada (2008)

    Google Scholar 

  56. Thomas, M.U.: A generalized maximum entropy principle. Oper. Res. 27(6), 1188–1196 (1979)

    Article  MATH  Google Scholar 

  57. Tobin, J.: Liquidity preference as behavior towards risk. Rev. Econ. Stud. 25(2), 65–86 (1958)

    Article  Google Scholar 

  58. Tsyurmasto, P., Zabarankin, M., Uryasev, S.: Value-at-risk support vector machine: stability to outliers. J. Comb. Optim. (2014, to appear)

    Google Scholar 

  59. Venables, W., Ripley, B.: Modern Applied Statistics with S-PLUS, 4th edn. Springer, New York (2002)

    Google Scholar 

  60. van der Waerden, B.: Mathematische Statistik. Springer, Berlin (1957)

    MATH  Google Scholar 

  61. Wets, R.J.B.: Statistical estimation from an optimization viewpoint. Ann. Oper. Res. 85, 79–101 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  62. Yaari, M.E.: The dual theory of choice under risk. Econometrica 55(1), 95–115 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  63. Zabarankin, M., Pavlikov, K., Uryasev, S.: Capital asset pricing model (CAPM) with drawdown measure. Eur. J. Oper. Res. (2013, to appear)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematical Sciences, Stevens Institute of Technology, Hoboken, NJ, USA

    Michael Zabarankin

  2. Department of Industrial and Systems Engineering, University of Florida, Gainesville, FL, USA

    Stan Uryasev

Authors
  1. Michael Zabarankin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stan Uryasev
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media, New York

About this chapter

Cite this chapter

Zabarankin, M., Uryasev, S. (2014). Entropy Maximization. In: Statistical Decision Problems. Springer Optimization and Its Applications, vol 85. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-8471-4_5

Download citation

Publish with us

Policies and ethics

Search

Navigation