Image Registration and Segmentation by Maximizing the Jensen-Rényi Divergence

  • A. Ben Hamza
  • Hamid Krim
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2683)


Information theoretic measures provide quantitative entropic divergences between two probability distributions or data sets. In this paper, we analyze the theoretical properties of the Jensen-Rényi divergence which is defined between any arbitrary number of probability distributions. Using the theory of majorization, we derive its maximum value, and also some performance upper bounds in terms of the Bayes risk and the asymptotic error of the nearest neighbor classifier. To gain further insight into the robustness and the application of the Jensen-Rényi divergence measure in imaging, we provide substantial numerical experiments to show the power of this entopic measure in image registration and segmentation.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ali, S., Silvey, S.: A general class of coefficients of divergence of one distribution from another. J. Roy. Soc. 28, 131–142 (1966)MATHMathSciNetGoogle Scholar
  2. 2.
    Kullback, S., Liebler, R.: On information and sufficiency. Ann. Math. Statist. 22, 79–86 (1951)MATHCrossRefGoogle Scholar
  3. 3.
    Stoica, R., Zerubia, J.: Image retrieval and indexing: A hierarchical approach in computing the distance between textured images. In: IEEE Int. Conf. on Image Processing, Chicago (1998)Google Scholar
  4. 4.
    Hero, A.O., Ma, B., Michel, O., Gorman, J.: Applications of entropic spanning graphs. IEEE Signal Processing Magazine 19(5), 85–95 (2002)CrossRefGoogle Scholar
  5. 5.
    Rényi, A.: On Measures of Entropy and Information. Selected Papers of Alfréd Rényi 2, 525–580 (1961)Google Scholar
  6. 6.
    Lin, J.: Divergence Measures Based on the Shannon Entropy. IEEE Trans. Information Theory 37(1), 145–151 (1991)MATHCrossRefGoogle Scholar
  7. 7.
    Gomez, J.F., Martinez, J., Robles, A.M., Roman, R.: An analysis of edge detection by using the Jensen-Shannon divergence. Journal of Mathematical Imaging and Vision 13, 35–56 (2000)MATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Roman, R., Bernaola, P., Oliver, J.L.: Sequence compositional complexity of DNA through an entropic segmentation method. Physical Review Letters 80(6), 1344–1347 (1998)CrossRefGoogle Scholar
  9. 9.
    Viola, P., Wells, W.M.: A lignment by maximization of mutual information. International Journal of Computer Vision 24(2), 154–173 (1997)CrossRefGoogle Scholar
  10. 10.
    Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P.: Multimodality image registration by maximization of mutual information. IEEE Trans. on Medical Imaging 16(2), 187–198 (1997)Google Scholar
  11. 11.
    He, Y., Ben Hamza, A., Krim, H., Chen, V.C.: An information theoretic measure for ISAR imagery focusing. In: Proc. SPIE, San Diego, vol. 4116 (2000)Google Scholar
  12. 12.
    He, Y., Ben Hamza, A., Krim, H.: A generalized divergence measure for robust image registration. IEEE Trans. Signal Processing 51(5) (2003)Google Scholar
  13. 13.
    Marshall, A.W., Olkin, I.: Inequalities: Theory of Majorization and Its Applications. Academic Press, London (1979)MATHGoogle Scholar
  14. 14.
    Devroye, L., Gyorfi, L., Lugosi, G.: A probabilistic theory of pattern recognition, New York. Springer, New York (1996)Google Scholar
  15. 15.
    Figueiredo, M.A., Jain, A.K.: Unsupervised learning of finite mixture models. IEEE Trans. on pattern analysis and machine intelligence 24(3), 381–396 (2002)CrossRefGoogle Scholar
  16. 16.
    Paragios, N., Deriche, R.: Geo desic active contours and level sets for the detection and tracking of moving objects. IEEE Trans. on pattern analysis and machine intelligence 22(3), 266–280 (2000)CrossRefGoogle Scholar
  17. 17.
    Hellman, M., Raviv, J.: Probability of error, equivocation, and the Chernoff bound. IEEE Trans. Information Theory 16, 368–372 (1970)MATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Cover, T.M., Hart, P.E.: Nearest neighbor pattern classification. IEEE Trans. Inform. Theory 13, 21–27 (1967)MATHCrossRefGoogle Scholar
  19. 19.
    Katuri, R., Jain, R.C.: Computer Vision: Principles. IEEE Computer Society Press, Los Alamitos (1991)Google Scholar
  20. 20.
    Jensen, J.R.: Introductory digital image processing: a remote sensing perspective, 2nd edn. Prentice Hall, Upper Saddle River (1996)Google Scholar
  21. 21.
    Van den Elsen, P.A., Pol, E.J.D., Viergever, M.A.: Medical image matching-a review with classification. IEEE Engineering in Medicine and Biology Magazine 12(1), 26–39 (1993)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • A. Ben Hamza
    • 1
  • Hamid Krim
    • 1
  1. 1.Department of Electrical and Computer EngineeringNorth Carolina State UniversityRaleighUSA

Personalised recommendations