Skip to main content
SpringerLink
Log in

On Solutions to Multivariate Maximum α-Entropy Problems

  • Conference paper
Book cover Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR 2003)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2683))

Abstract

Entropy has been widely employed as an optimization function for problems in computer vision and pattern recognition. To gain insight into such methods it is important to characterize the behavior of the maximum-entropy probability distributions that result from the entropy optimization. The aim of this paper is to establish properties of multivariate distributions maximizing entropy for a general class of entropy functions, called Rényi’s α-entropy, under a covariance constraint. First we show that these entropy-maximizing distributions exhibit interesting properties, such as spherical invariance, and have a stochastic Gaussian-Gamma mixture representation. We then turn to the question of stability of the class of entropy-maximizing distributions under addition.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abramowitz, M., Stegun, I.: Handbook of mathematical functions with formulas, graphs, and mathematical tables, U.S. Govt. Print. Off. Applied Mathematics Series, vol. 55 (1964)

    Google Scholar 

  2. Chu, K.C.: Estimation and decision for linear systems with elliptical random processes. IEEE Trans. on Automatic Control 18, 499–505 (1973)

    Article  MATH  Google Scholar 

  3. Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (1987)

    MATH  Google Scholar 

  4. Csiszár, I.: Information-type measures of difference of probability distributions and indirect observations. Studia Sci. Math. Hungar. 2, 299–318 (1967)

    MATH  MathSciNet  Google Scholar 

  5. Feller, W.: An introduction to probability theory and its applications, 3rd edn., vol. I. John Wiley & Sons, Chichester (1968)

    MATH  Google Scholar 

  6. Geman, D., Jedynak, B.: An active testing model for tracking roads in satellite images. IEEE Trans. on Pattern Anal. and Machine Intell. 1, 10–17 (1996)

    Google Scholar 

  7. Gullberg, G.T., Tsui, B.M.W.: Maximum entropy reconstruction with constraints: Iterative algorithms for solving the primal and dual programs. In: de Graaf, C.N., Viergever, M.A. (eds.) Information Processing in Medical Imaging,  ch. 23. Plenum Press, New York (1988)

    Google Scholar 

  8. Heemuchwala, H., Hero, A.O., Carson, P.: Image registration using entropy measures and entropic graphs. To appear in European Journal of Signal Processing, special Issue on Content-based Visual Information Retrieval (December 2003)

    Google Scholar 

  9. Hero, A.O., Ma, B., Michel, O., Gorman, J.: Applications of entropic spanning graphs. IEEE Signal Processing Magazine 19(5), 85–95 (2002)

    Article  Google Scholar 

  10. Kapur, J.N.: Generalised Cauchy and Student’s distributions as maximumentropy distributions. Proc. Nat. Acad. Sci. India Sect. A 58(2), 235–246 (1988)

    MATH  MathSciNet  Google Scholar 

  11. McEliece, R.J., Rodemich, E.R., Swanson, L.: An entropy maximization problem elated to optical communication. IEEE Trans. on Inform. Theory 32, 322–325 (1986)

    Article  MATH  Google Scholar 

  12. Mendes, R.S., Tsallis, C.: Renormalization group approach to nonextensive statistical mechanics. Phys. Lett. A 285(5-6), 273–278 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  13. Miller, M.I., Snyder, D.L.: The role of likelihood and entropy in incompletedata problems: applications to estimating point-process intensities and Toeplitz constrained covariances. In: IEEE Proceedings, 75th edn., pp. 892–907 (1987)

    Google Scholar 

  14. Moriguti, S.: A lower bound for a probability moment of any absolutely continuous distribution with finite variance. Ann. Math. Statistics 23, 286–289 (1952)

    Article  MATH  MathSciNet  Google Scholar 

  15. Oliveira, F.A., Mello, B.A., Xavier Jr., I.M.: Scaling transformation of random walk distributions in a lattice. Physical Review. E 61(6, Part B), 7200–7203 (2000)

    Article  Google Scholar 

  16. Rényi, A.: On measures of entropy and information. In: Proc. 4th Berkeley Symp. Math. Stat. and Prob., vol. 1, pp. 547–561 (1961)

    Google Scholar 

  17. Silvey, S.D., Ali, S.M.: A general class of coefficients of divergence of one distribution from another. J. Roy. Statist. Soc. Ser. B 28, 131–142 (1966)

    MATH  MathSciNet  Google Scholar 

  18. Vignat, C., Costa, J., Hero, A.: Characterization of the multivariate distributions maximazing Tsallis entropy under covariance constraint. Technical report (January 2003)

    Google Scholar 

  19. Walker, G.A., Saw, J.G.: The distribution of linear combinations of t-variables. J. Amer. Statist. Assoc. 73(364), 876–878 (1978)

    Article  MATH  MathSciNet  Google Scholar 

  20. Witkovsky, V.: On the exact computation of the density and of the quantiles of linear combinations of t and F random variables. J. Statist. Plann. Inference 94(1), 1–13 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  21. Zografos, K.: On maximum entropy characterization of Pearson’s type II and VII multivariate distributions. Journal of Multivariate Analysis 71(1), 67–75 (1999)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, 48109-2122, USA

    Jose Costa & Alfred Hero

  2. Laboratoire Systèmes de Communications, Université Marne la Vallée, France

    Christophe Vignat

Authors
  1. Jose Costa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alfred Hero
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christophe Vignat
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of CISE, University of Florida, Gainesville, FL, USA

    Anand Rangarajan

  2. Instituto de Telecomunicações, Instituto Superior Técnico, T.U. Lisbon, Av. Rovisco Pais, 1049-001, Lisboa, Portugal

    Mário Figueiredo

  3. ARIANA Research Group, INRIA/I3S, Sophia Antipolis, France

    Josiane Zerubia

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Costa, J., Hero, A., Vignat, C. (2003). On Solutions to Multivariate Maximum α-Entropy Problems. In: Rangarajan, A., Figueiredo, M., Zerubia, J. (eds) Energy Minimization Methods in Computer Vision and Pattern Recognition. EMMCVPR 2003. Lecture Notes in Computer Science, vol 2683. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45063-4_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-45063-4_14

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-40498-9

  • Online ISBN: 978-3-540-45063-4

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation