Abstract
Divergences, also known as contrast functions, are distance-like quantities defined on manifolds of non-negative or probability measures. Using the duality in optimal transport, we introduce and study the one-parameter family of \(L^{(\pm \alpha )}\)-divergences. They extrapolate between the Bregman divergence corresponding to the Euclidean quadratic cost, and the L-divergence introduced by Pal and the author in connection with portfolio theory and a logarithmic cost function. They admit natural generalizations of exponential family that are closely related to the \(\alpha \)-family and q-exponential family. In particular, the \(L^{(\pm \alpha )}\)-divergences of the corresponding potential functions are Rényi divergences. Using this unified framework we prove that the induced geometries are dually projectively flat with constant sectional curvatures, and a generalized Pythagorean theorem holds true. Conversely, we show that if a statistical manifold is dually projectively flat with constant curvature \(\pm \alpha \) with \(\alpha > 0\), then it is locally induced by an \(L^{(\mp \alpha )}\)-divergence. We define in this context a canonical divergence which extends the one for dually flat manifolds.
Notes
The author thanks an anonymous referee for pointing this out.
The author thanks Shun-ichi Amari for pointing this out.
References
Adams, S., Dirr, N., Peletier, M.A., Zimmer, J.: From a large-deviations principle to the Wasserstein gradient flow: a new micro-macro passage. Commun. Math. Phys. 307(3), 791 (2011)
Amari, S., Nagaoka, H.: Methods of Information Geometry. Translations of Mathematical Monographs, vol. 191. Am. Math. Soc./Oxford University Press, Providence/London (2000)
Amari, S.I.: Information Geometry and Its Applications. Springer, Berlin (2016)
Amari, S.I., Karakida, R., Oizumi, M.: Information geometry connecting Wasserstein distance and Kullback–Leibler divergence via the entropy-relaxed transportation problem (2018) (to appear in Information Geometry)
Amari, S.I., Ohara, A.: Geometry of \(q\)-exponential family of probability distributions. Entropy 13(6), 1170–1185 (2011)
Ambrosio, L., Gigli, N.: A User’s Guide to Optimal Transport. In: Modelling and Optimisation of Flows on Networks. Lecture Notes in Mathematics, vol 2062. Springer, Berlin, Heidelberg (2013). https://link.springer.com/chapter/10.1007/978-3-642-32160-3_1
Ambrosio, L., Gigli, N., Savaré, G.: Gradient Flows in Metric Spaces and in the Spaces of Probability Measures, Second Edition. Lectures in Mathematics ETH Zürich. Birkhäuser Verlag (2008). https://www.springer.com/us/book/9783764387211
Ay, N., Amari, S.I.: A novel approach to canonical divergences within information geometry. Entropy 17(12), 8111–8129 (2015)
Ay, N., Jost, J., Lê, H., Schwachhöfer, L.: Information Geometry. Springer, Berlin (2017)
Banerjee, A., Merugu, S., Dhillon, I.S., Ghosh, J.: Clustering with bregman divergences. J. Mach. Learn. Res. 6(Oct), 1705–1749 (2005)
Brenier, Y.: Polar factorization and monotone rearrangement of vector-valued functions. Commun. Pure Appl. Math. 44(4), 375–417 (1991)
Calin, O., Udrişte, C.: Geometric modeling in probability and statistics. Springer, Berlin (2014)
Chen, Y., Li, W.: Natural gradient in Wasserstein statistical manifold (2018). arXiv preprint arXiv:1805.08380
Chizat, L., Peyré, G., Schmitzer, B., Vialard, F.X.: Unbalanced optimal transport: geometry and kantorovich formulation (2015). arXiv preprint arXiv:1508.05216
Dillen, F., Nomizu, K., Vranken, L.: Conjugate connections and radon’s theorem in affine differential geometry. Monatshefte für Mathematik 109(3), 221–235 (1990)
Eguchi, S.: Second order efficiency of minimum contrast estimators in a curved exponential family. Ann. Stat. 11(3), 793–803 (1983)
Eguchi, S.: Geometry of minimum contrast. Hiroshima Math. J. 22(3), 631–647 (1992)
Erbar, M., Kuwada, K., Sturm, K.T.: On the equivalence of the entropic curvature–dimension condition and Bochner’s inequality on metric measure spaces. Invent. Math. 201(3), 993–1071 (2015)
Felice, D., Ay, N.: Towards a canonical divergence within information geometry (2018). arXiv preprint arXiv:1806.11363
Jordan, R., Kinderlehrer, D., Otto, F.: The variational formulation of the Fokker–Planck equation. SIAM J. Math. Anal. 29(1), 1–17 (1998)
Kurose, T.: Dual connections and affine geometry. Mathematische Zeitschrift 203(1), 115–121 (1990)
Kurose, T.: On the divergences of 1-conformally flat statistical manifolds. Tohoku Math. J. Second Ser. 46(3), 427–433 (1994)
Léonard, C.: From the Schrödinger problem to the Monge–Kantorovich problem. J. Funct. Anal. 262(4), 1879–1920 (2012)
Li, W., Montufar, G.: Ricci curvature for parametric statistics via optimal transport (2018). arXiv preprint arXiv:1807.07095
Matsuzoe, H.: Geometry of contrast functions and conformal geometry. Hiroshima Math. J. 29(1), 175–191 (1999)
Moreau, J.J.: Inf-convolutions, sous-additive, convexite des fonctions numeriques. J. Math. Anal. Appl. 49, 109–154 (1970)
Nagaoka, H., Amari, S.I.: Differential geometry of smooth families of probability distributions. Tech. Rep. METR 82–7. University of Tokyo, Tokyo (1982)
Naudts, J.: The \(q\)-exponential family in statistical physics. In: Journal of Physics: Conference Series, vol. 201, p. 012003. IOP Publishing (2010). http://iopscience.iop.org/article/10.1088/1742-6596/201/1/012003/meta
Naudts, J., Zhang, J.: Rho-tau embedding and gauge freedom in information geometry (2018). https://doi.org/10.1007/s41884-018-0004-6
Pal, S.: Embedding optimal transports in statistical manifolds. Indian J. Pure Appl. Math. 48(4), 541–550 (2017)
Pal, S., Wong, T.K.L.: Energy, entropy, and arbitrage (2013). arXiv preprint arXiv:1308.5376
Pal, S., Wong, T.K.L.: The geometry of relative arbitrage. Math. Fin. Econ. 10, 263–293 (2016)
Pal, S., Wong, T.K.L.: Exponentially concave functions and a new information geometry. Ann. Probab. 46(2), 1070–1113 (2018)
Pal, S., Wong, T.K.L.: Multiplicative Schrödinger problem and the Dirichlet transport (2018). arXiv preprint arXiv:1807.05649
Rachev, S.T., Rüschendorf, L.: Mass Transportation Problems: Volume I: Theory, vol. 1. Springer, Berlin (1998)
Rényi, A.: On measures of entropy and information. In: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics. The Regents of the University of California (1961)
Santambrogio, F.: Optimal Transport for Applied Mathematicians. Springer, Berlin (2015)
Van Erven, T., Harremos, P.: Rényi divergence and Kullback–Leibler divergence. IEEE Trans. Inf. Theory 60(7), 3797–3820 (2014)
Villani, C.: Topics in Optimal Transportation. American Mathematical Society, Rhode Island (2003)
Villani, C.: Optimal Transport: Old and New. Springer, Berlin (2008)
Wong, T.K.L.: Optimization of relative arbitrage. Ann. Financ. 11(3–4), 345–382 (2015)
Wong, T.K.L.: On portfolios generated by optimal transport (2017). arXiv preprint arXiv:1709.03169
Zhang, J.: Divergence function, duality, and convex analysis. Neural Comput. 16(1), 159–195 (2004)
Zhang, J.: Referential duality and representational duality on statistical manifolds. In: Proceedings of the Second International Symposium on Information Geometry and Its Applications, Tokyo, Japan, vol. 1216, p. 5867 (2005)
Zhang, J.: Reference duality and representation duality in information geometry. In: AIP Conference Proceedings, vol. 1641, pp. 130–146. AIP (2015)
Acknowledgements
Some of the results were obtained when the author was teaching a PhD topics course on optimal transport and information geometry at the University of Southern California. He thanks his students for their interest and feedbacks. He also thanks John Man-shun Ma, Jeremy Toulisse and Soumik Pal for helpful discussions. He is grateful to Shun-ichi Amari for his insightful comments and for pointing out the connection with the \(\alpha \)-divergence. Finally, he thanks the anonymous referees for their helpful comments which improved the paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wong, TK.L. Logarithmic divergences from optimal transport and Rényi geometry. Info. Geo. 1, 39–78 (2018). https://doi.org/10.1007/s41884-018-0012-6
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41884-018-0012-6