Abstract
In this paper we study representation of KL-divergence minimization, in the cases where integer sufficient statistics exists, using tools from polynomial algebra. We show that the estimation of parametric statistical models in this case can be transformed to solving a system of polynomial equations. In particular, we also study the case of Kullback-Csisźar iteration scheme. We present implicit descriptions of these models and show that implicitization preserves specialization of prior distribution. This result leads us to a Gröbner bases method to compute an implicit representation of minimum KL-divergence models.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Kullback, S.: Information Theory and Statistics. Wiley, New York (1959)
Csiszár, I., Shields, P.: Information Theory and Statistics: A Tutorial. Foundations and Trends in Communications and Information Theory, vol. 1. Now Publications (2004)
Diaconis, P., Sturmfels, B.: Algebraic algorithms for sampling from conditional distributions. Annals of Statistics 26, 363–397 (1998)
Pistone, G., Riccomagno, E., Wynn, H.: Algebraic Statistics. In: Computational Commutative Algebra in Statistics. Chapman and Hall, New York (2001)
Pachter, L., Sturmfels, B.: Algebraic Statistics and Computational Biology. Cambridge University Press, Cambridge (2005)
Jaynes, E.T.: Prior probabilities. IEEE Transactions on Systems Science and Cybernetics sec-4(3), 227–241 (1968)
Darroch, J.N., Ratcliff, D.: Generalized iterative scaling for log-linear models. The Annals of Mathematical Statistics 43(5), 1470–1480 (1972)
Dukkipati, A.: On parametric and implicit algebraic descriptions of maximum entropy models. Technical Report 2008-035, EURANDOM, Eindhoven, The Netherlands (2008)
Ireland, C., Kullback, S.: Contingency tables with given marginals. Biometrika 55, 179–188 (1968)
Kullback, S.: Probability densities with given marginals. The Annals of Mathematical Statistics 39(4), 1236–1243 (1968)
Csiszár, I.: I-divergence of probability distributions and minimization problems. Ann. Prob. 3(1), 146–158 (1975)
Cox, D., Little, J., O’Shea, D.: Ideals, Varieties, and Algorithms, 2nd edn. Springer, New York (1991)
Adams, W.W., Loustaunau, P.: An Introduction to Gröbner Bases. Graduate Studies in Mathematics, vol. 3. American Mathematical Society, Providence (1994)
Dukkipati, A.: On embedding maximum entropy models in algebraic varieties by Gröbner bases methods. In: Proceedings of IEEE International Symposium on Information Theory (ISIT), pp. 1904–1908. IEEE Press, Los Alamitos (2009)
Gianni, P.: Properties of Gröbner bases under specialization. In: Davenport, J.H. (ed.) ISSAC 1987 and EUROCAL 1987. LNCS, vol. 378, pp. 293–297. Springer, Heidelberg (1989)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dukkipati, A., Manathara, J.G. (2010). An Algebraic Implicitization and Specialization of Minimum KL-Divergence Models. In: Gerdt, V.P., Koepf, W., Mayr, E.W., Vorozhtsov, E.V. (eds) Computer Algebra in Scientific Computing. CASC 2010. Lecture Notes in Computer Science, vol 6244. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15274-0_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-15274-0_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15273-3
Online ISBN: 978-3-642-15274-0
eBook Packages: Computer ScienceComputer Science (R0)