An Algebraic Implicitization and Specialization of Minimum KL-Divergence Models

  • Ambedkar Dukkipati
  • Joel George Manathara
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6244)


In this paper we study representation of KL-divergence minimization, in the cases where integer sufficient statistics exists, using tools from polynomial algebra. We show that the estimation of parametric statistical models in this case can be transformed to solving a system of polynomial equations. In particular, we also study the case of Kullback-Csisźar iteration scheme. We present implicit descriptions of these models and show that implicitization preserves specialization of prior distribution. This result leads us to a Gröbner bases method to compute an implicit representation of minimum KL-divergence models.


Gröbner Bases statistical models elimination 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Kullback, S.: Information Theory and Statistics. Wiley, New York (1959)zbMATHGoogle Scholar
  2. 2.
    Csiszár, I., Shields, P.: Information Theory and Statistics: A Tutorial. Foundations and Trends in Communications and Information Theory, vol. 1. Now Publications (2004)Google Scholar
  3. 3.
    Diaconis, P., Sturmfels, B.: Algebraic algorithms for sampling from conditional distributions. Annals of Statistics 26, 363–397 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Pistone, G., Riccomagno, E., Wynn, H.: Algebraic Statistics. In: Computational Commutative Algebra in Statistics. Chapman and Hall, New York (2001)Google Scholar
  5. 5.
    Pachter, L., Sturmfels, B.: Algebraic Statistics and Computational Biology. Cambridge University Press, Cambridge (2005)CrossRefzbMATHGoogle Scholar
  6. 6.
    Jaynes, E.T.: Prior probabilities. IEEE Transactions on Systems Science and Cybernetics sec-4(3), 227–241 (1968)CrossRefzbMATHGoogle Scholar
  7. 7.
    Darroch, J.N., Ratcliff, D.: Generalized iterative scaling for log-linear models. The Annals of Mathematical Statistics 43(5), 1470–1480 (1972)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Dukkipati, A.: On parametric and implicit algebraic descriptions of maximum entropy models. Technical Report 2008-035, EURANDOM, Eindhoven, The Netherlands (2008)Google Scholar
  9. 9.
    Ireland, C., Kullback, S.: Contingency tables with given marginals. Biometrika 55, 179–188 (1968)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Kullback, S.: Probability densities with given marginals. The Annals of Mathematical Statistics 39(4), 1236–1243 (1968)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Csiszár, I.: I-divergence of probability distributions and minimization problems. Ann. Prob. 3(1), 146–158 (1975)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Cox, D., Little, J., O’Shea, D.: Ideals, Varieties, and Algorithms, 2nd edn. Springer, New York (1991)zbMATHGoogle Scholar
  13. 13.
    Adams, W.W., Loustaunau, P.: An Introduction to Gröbner Bases. Graduate Studies in Mathematics, vol. 3. American Mathematical Society, Providence (1994)zbMATHGoogle Scholar
  14. 14.
    Dukkipati, A.: On embedding maximum entropy models in algebraic varieties by Gröbner bases methods. In: Proceedings of IEEE International Symposium on Information Theory (ISIT), pp. 1904–1908. IEEE Press, Los Alamitos (2009)Google Scholar
  15. 15.
    Gianni, P.: Properties of Gröbner bases under specialization. In: Davenport, J.H. (ed.) ISSAC 1987 and EUROCAL 1987. LNCS, vol. 378, pp. 293–297. Springer, Heidelberg (1989)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Ambedkar Dukkipati
    • 1
  • Joel George Manathara
    • 2
  1. 1.Department of Computer Science and AutomationIndian Institute of ScienceIndia
  2. 2.Department of Aerospace EngineeringIndian Institute of ScienceIndia

Personalised recommendations