Skip to main content
SpringerLink
Log in

When optimal transport meets information geometry

  • Survey Paper
  • Published:
Information Geometry Aims and scope Submit manuscript

Abstract

Information geometry and optimal transport are two distinct geometric frameworks for modeling families of probability measures. During the recent years, there has been a surge of research endeavors that cut across these two areas and explore their links and interactions. This paper is intended to provide an (incomplete) survey of these works, including entropy-regularized transport, divergence functions arising from c-duality, density manifolds and transport information geometry, the para-Kähler and Kähler geometries underlying optimal transport and the regularity theory for its solutions. Some outstanding questions that would be of interest to audience of both these two disciplines are posed. Our piece also serves as an introduction to the Special Issue on Optimal Transport of the journal Information Geometry.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Data Availability

Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.

Notes

  1. Items ii. and iii. are often known collectively as the Twist condition.

  2. There are easier ways to see that the optimal transport map might fail to be continuous. For instance, if X is connected whereas Y is disconnected (and the measures have full support), there are no continuous transport maps at all, let alone an optimal one.

  3. This dichotomy only holds in the interior. It is possible for the Monge map to be smooth in the interior, but become singular near the boundary.

  4. If the solution to this transport problem is not Monge, this should be interpreted in the sense of couplings where mass may split at the initial time along various geodesics.

  5. More precisely, most of these bounds involve both the curvature and the dimension, so are known as curvature-dimension bounds.

  6. See Sect. 2.4 for related work in the continuous setting.

  7. Originally [81] used \(\alpha \) and treated \(\alpha >0\) and \(\alpha <0\) cases separately. This is unified by [84] based on the notion of \(\lambda \)-exponential convexity.

  8. It is a general fact that the tangent bundle of any Hessian manifold admits a Kähler metric (see, e.g. [23, 68]).

  9. The proportionality constant depends on how the convex potential is used to induce the cost function.

References

  1. Altschuler, J., Niles-Weed, J., Rigollet, P.: Near-linear Time Approximation Algorithms for Optimal Transport via Sinkhorn Iteration. Advances in Neural Information Processing Systems, vol. 30 (2017)

  2. Amari, S.-I.: Differential-Geometrical Methods in Statistics, vol. 28. Springer Science & Business Media, Berlin (2012)

    MATH  Google Scholar 

  3. Amari, S.-I., Karakida, R., Oizumi, M.: Information geometry connecting Wasserstein distance and Kullback–Leibler divergence via the entropy-relaxed transportation problem. Inf. Geom. 1(1), 13–37 (2018)

    MathSciNet  MATH  Google Scholar 

  4. Amari, S.-I., Nagaoka, H.: Methods of Information Geometry, vol. 191. American Mathematical Society, Providence (2000)

    MATH  Google Scholar 

  5. Ambrosio, L., Gigli, N.: A user’s guide to optimal transport. In: Modelling and Optimisation of Flows on Networks, pp. 1–155. Springer (2013)

  6. Ay, N., Jost, J., Vân Lê, H., Schwachhöfer, L.: Information Geometry, vol. 64. Springer, Berlin (2017)

    MATH  Google Scholar 

  7. Bauer, M., Bruveris, M., Michor, P.W.: Uniqueness of the Fisher–Rao metric on the space of smooth densities. Bull. Lond. Math. Soc. 48(3), 499–506 (2016)

    MathSciNet  MATH  Google Scholar 

  8. Belavkin, R.V.: Asymmetric topologies on statistical manifolds. In: International Conference on Geometric Science of Information, pp. 203–210. Springer (2015)

  9. Benamou, J.-D., Brenier, Y.: A computational fluid mechanics solution to the Monge–Kantorovich mass transfer problem. Numer. Math. 84(3), 375–393 (2000)

    MathSciNet  MATH  Google Scholar 

  10. Benamou, J.-D., Carlier, G., Cuturi, M., Nenna, L., Peyré, G.: Iterative Bregman projections for regularized transportation problems. SIAM J. Sci. Comput. 37(2), A1111–A1138 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  11. Bertsekas, D.: Network Optimization: Continuous and Discrete Models, vol. 8. Athena Scientific, Belmont (1998)

    MATH  Google Scholar 

  12. Brenier, Y.: Décomposition polaire et réarrangement monotone des champs de vecteurs. CR Acad. Sci. Paris Sér. I Math. 305, 805–808 (1987)

    MATH  Google Scholar 

  13. Caffarelli, L.A.: The regularity of mappings with a convex potential. J. Am. Math. Soc. 5(1), 99–104 (1992)

    MathSciNet  MATH  Google Scholar 

  14. Cao, H.-D.: Deformation of Kähler matrics to Kähler–Einstein metrics on compact Kähler manifolds. Invent. Math. 81(2), 359–372 (1985)

    MathSciNet  MATH  Google Scholar 

  15. Chen, Y., Li, W.: Optimal transport natural gradient for statistical manifolds with continuous sample space. Inf. Geom. 3(1), 1–32 (2020)

    MathSciNet  MATH  Google Scholar 

  16. Chizat, L., Peyré, G., Schmitzer, B., Vialard, F.-X.: An interpolating distance between optimal transport and Fisher–Rao metrics. Found. Comput. Math. 18(1), 1–44 (2018)

    MathSciNet  MATH  Google Scholar 

  17. Cortés, V., Mayer, C., Mohaupt, T., Saueressig, F.: Special geometry of Euclidean supersymmetry 1. Vector multiplets. J. High Energy Phys. 03, 028 (2004)

    MathSciNet  Google Scholar 

  18. Csiszár, I.: Eine informationstheoretische ungleichung und ihre anwendung auf beweis der ergodizitaet von markoffschen ketten. Magyer Tud. Akad. Mat. Kutato Int. Koezl. 8, 85–108 (1964)

    MATH  Google Scholar 

  19. Cuturi, M.: Sinkhorn distances: lightspeed computation of optimal transport. Adv. Neural Inf. Process. Syst. 26 (2013)

  20. De Philippis, G., Figalli, A.: The Monge–Ampère equation and its link to optimal transportation. Bull. Am. Math. Soc. 51(4), 527–580 (2014)

    MATH  Google Scholar 

  21. De Philippis, G., Figalli, A.: Partial regularity for optimal transport maps. Publ. Math. de l’IHÉS 121(1), 81–112 (2015)

    MathSciNet  MATH  Google Scholar 

  22. Delanoë, P.: Classical solvability in dimension two of the second boundary-value problem associated with the Monge–Ampère operator. In: Annales de l’Institut Henri Poincaré C, Analyse non linéaire, vol. 8, pp. 443–457. Elsevier (1991)

  23. Dombrowski, P.: On the geometry of the tangent bundle. J. Für Die Reine Angew. Math. 210, 73–88 (1962)

    MathSciNet  MATH  Google Scholar 

  24. Figalli, A., Rifford, L., Villani, C.: Nearly round spheres look convex. Am. J. Math. 134(1), 109–139 (2012)

    MathSciNet  MATH  Google Scholar 

  25. Gangbo, W., McCann, R.J.: The geometry of optimal transportation. Acta Math. 177(2), 113–161 (1996)

    MathSciNet  MATH  Google Scholar 

  26. Guillen, N., Kitagawa, J.: On the local geometry of maps with c-convex potentials. Calc. Var. Partial Differ. Equ. 52(1), 345–387 (2015)

    MathSciNet  MATH  Google Scholar 

  27. Hamilton, R.S.: Three-manifolds with positive Ricci curvature. J. Differ. Geom. 17(2), 255–306 (1982)

    MathSciNet  MATH  Google Scholar 

  28. Jordan, R., Kinderlehrer, D., Otto, F.: The variational formulation of the Fokker–Planck equation. SIAM J. Math. Anal. 29(1), 1–17 (1998)

    MathSciNet  MATH  Google Scholar 

  29. Kantorovich, L.V.: On the translocation of masses. J. Math. Sci. 133(4), 1381–1382 (2006)

    MathSciNet  MATH  Google Scholar 

  30. Khan, G., Zhang, J.: The Kähler geometry of certain optimal transport problems. Pure Appl. Anal. 2(2), 397–426 (2020)

    MathSciNet  MATH  Google Scholar 

  31. Khan, G., Zhang, J.: A hall of statistical mirrors. arXiv preprint arXiv:2109.13809 (2021)

  32. Khan, G., Zheng, F.: Kähler–Ricci flow preserves negative anti-bisectional curvature. arXiv preprint arXiv:2011.07181 (2020)

  33. Khesin, B., Lenells, J., Misiołek, G., Preston, S.C.: Geometry of diffeomorphism groups, complete integrability and geometric statistics. Geom. Funct. Anal. 23(1), 334–366 (2013)

    MathSciNet  MATH  Google Scholar 

  34. Kim, Y.-H., Kitagawa, J.: Prohibiting isolated singularities in optimal transport. Ann. Della Scuola Norm. Super. Pisa Classe Sci. 16(1), 277–290 (2016)

    MathSciNet  MATH  Google Scholar 

  35. Kim, Y.-H., McCann, R.J.: Continuity, curvature, and the general covariance of optimal transportation. J. Eur. Math. Soc. 12(4), 1009–1040 (2010)

    MathSciNet  MATH  Google Scholar 

  36. Kim, Y.H., McCann, R.J., Warren, M.: Pseudo-Riemannian geometry calibrates optimal transportation. Math. Res. Lett. 17(5), 1183–1197 (2010)

    MathSciNet  MATH  Google Scholar 

  37. Kitagawa, J.: A parabolic flow toward solutions of the optimal transportation problem on domains with boundary. J. Für Die Reine Angew. Math. (Crelles Journal) 2012(672), 127–160 (2012)

    MathSciNet  MATH  Google Scholar 

  38. Knott, M., Smith, C.S.: On the optimal mapping of distributions. J. Optim. Theory Appl. 43(1), 39–49 (1984)

    MathSciNet  MATH  Google Scholar 

  39. Kondratyev, S., Monsaingeon, L., Vorotnikov, D.: A new optimal transport distance on the space of finite Radon measures. Adv. Differ. Equ. 21(11/12), 1117–1164 (2016)

    MathSciNet  MATH  Google Scholar 

  40. Kurose, T., Yoshizawa, S., Amari, S.-I.: Optimal transportation plans with escort entropy regularization. Inf. Geom. 5(1), 79–95 (2022)

  41. Lee, W., Li, W., Lin, B., Monod, A.: Tropical optimal transport and wasserstein distances. Inf. Geom. 5(1), 247–287 (2022)

  42. Li, W.: Transport information Bregman divergences. Inf. Geom. 4(2), 435–470 (2021)

    MathSciNet  MATH  Google Scholar 

  43. Li, W.: Transport information geometry: Riemannian calculus on probability simplex. Inf. Geom. 5(1), 161–207 (2022)

  44. Li, W., Montúfar, G.: Natural gradient via optimal transport. Inf. Geom. 1(2), 181–214 (2018)

    MathSciNet  MATH  Google Scholar 

  45. Li, W., Montúfar, G.: Ricci curvature for parametric statistics via optimal transport. Inf. Geom. 3(1), 89–117 (2020)

    MathSciNet  MATH  Google Scholar 

  46. Liero, M., Mielke, A., Savaré, G.: Optimal entropy-transport problems and a new Hellinger–Kantorovich distance between positive measures. Invent. Math. 211(3), 969–1117 (2018)

    MathSciNet  MATH  Google Scholar 

  47. Loeper, G.: On the regularity of solutions of optimal transportation problems. Acta Math. 202(2), 241–283 (2009)

    MathSciNet  MATH  Google Scholar 

  48. Loeper, G., Trudinger, N.S.: Weak formulation of the MTW condition and convexity properties of potentials. Methods Appl. Anal. 28(1), 53–60 (2021)

    MathSciNet  MATH  Google Scholar 

  49. Lott, J.: Some geometric calculations on Wasserstein space. Commun. Math. Phys. 2(277), 423–437 (2008)

    MathSciNet  MATH  Google Scholar 

  50. Ma, X.-N., Trudinger, N.S., Wang, X.-J.: Regularity of potential functions of the optimal transportation problem. Arch. Ration. Mech. Anal. 177(2), 151–183 (2005)

    MathSciNet  MATH  Google Scholar 

  51. Malago, L., Montrucchio, L., Pistone, G.: Wasserstein Riemannian geometry of Gaussian densities. Inf. Geom. 1(2), 137–179 (2018)

    MathSciNet  MATH  Google Scholar 

  52. Mallasto, A., Gerolin, A., Minh, H.Q.: Entropy-regularized 2-Wasserstein distance between Gaussian measures. Inf. Geom. 5(1), 289–323 (2022)

  53. McCann, R.J.: A convexity principle for interacting gases. Adv. Math. 128(1), 153–179 (1997)

    MathSciNet  MATH  Google Scholar 

  54. McCann, R.J., Topping, P.M.: Ricci flow, entropy and optimal transportation. Am. J. Math. 132(3), 711–730 (2010)

    MathSciNet  MATH  Google Scholar 

  55. Monge, G.: Mémoire sur la théorie des déblais et des remblais. Histoire de l’Académie Royale des Sciences de Paris (1781)

  56. Montrucchio, L., Pistone, G.: Kantorovich distance on finite metric spaces: Arens-eells norm and cut norms. Inf. Geom. 5(1), 209–245 (2022)

  57. Muzellec, B., Nock, R., Patrini, G., Nielsen, F.: Tsallis regularized optimal transport and ecological inference. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)

  58. Naudts, J.: Estimators, escort probabilities, and \(\phi \)-exponential families in statistical physics. J. Inequalities Pure Appl. Math. 5(4), 102 (2004)

    MathSciNet  MATH  Google Scholar 

  59. Naudts, J., Zhang, J.: Rho–tau embedding and gauge freedom in information geometry. Inf. Geom. 1(1), 79–115 (2018)

    MathSciNet  MATH  Google Scholar 

  60. Ohta, S.-I., Takatsu, A.: Displacement convexity of generalized relative entropies. Adv. Math. 228(3), 1742–1787 (2011)

    MathSciNet  MATH  Google Scholar 

  61. Otto, F.: The geometry of dissipative evolution equations: the porous medium equation. Commun. Partial Differ. Equ. 26, 101–174 (2001)

    MathSciNet  MATH  Google Scholar 

  62. Otto, F., Villani, C.: Generalization of an inequality by Talagrand and links with the logarithmic Sobolev inequality. J. Funct. Anal. 173(2), 361–400 (2000)

    MathSciNet  MATH  Google Scholar 

  63. Pal, S., Wong, T.-K.L.: The geometry of relative arbitrage. Math. Financial Econ. 10(3), 263–293 (2016)

    MathSciNet  MATH  Google Scholar 

  64. Pal, S., Wong, T.-K.L.: Exponentially concave functions and a new information geometry. Ann. Probab. 46(2), 1070–1113 (2018)

    MathSciNet  MATH  Google Scholar 

  65. Pal, S., Wong, T.-K.L.: Multiplicative Schröodinger problem and the Dirichlet transport. Probab. Theory Relat. Fields 178(1), 613–654 (2020)

    MATH  Google Scholar 

  66. Peyré, G., Cuturi, M.: Computational optimal transport: with applications to data science. Found. Trends. Mach. Learn. 11(5–6), 355–607 (2019)

    MATH  Google Scholar 

  67. Santambrogio, F.: Optimal transport for applied mathematicians. Birkäuser 55(58–63), 94 (2015)

    MATH  Google Scholar 

  68. Satoh, H.: Almost Hermitian structures on tangent bundles. Workshop Differ. Geom. 11, 105–118 (2007)

    MathSciNet  MATH  Google Scholar 

  69. Sei, T.: Coordinate-wise transformation of probability distributions to achieve a Stein-type identity. Inf. Geom. 5(1), 325–354 (2022)

  70. Takatsu, A.: Wasserstein geometry of gaussian measures. Osaka J. Math. 48(4), 1005–1026 (2011)

    MathSciNet  MATH  Google Scholar 

  71. Takatsu, A.: Wasserstein geometry of porous medium equation. In: Annales de l’Institut Henri Poincaré C, Analyse non linéaire, vol. 29, pp. 217–232. Elsevier (2012)

  72. Takatsu, A.: Behaviors of \(\varphi \)-exponential distributions in wasserstein geometry and an evolution equation. SIAM J. Math. Anal. 45(4), 2546–2556 (2013)

    MathSciNet  MATH  Google Scholar 

  73. Topping, P.: \(\cal{L}\)-optimal transportation for ricci flow. J. Für Die Reine Angew. Math. 2009, 636 (2009)

    MATH  Google Scholar 

  74. Trudinger, N., Wang, X.-J.: On the second boundary value problem for Monge–Ampère type equations and optimal transportation. Ann. Della Scuola Normale Super. Pisa Classe Sci. 8(1), 143–174 (2009)

    MathSciNet  MATH  Google Scholar 

  75. Tsutsui, D.: Optimal transport problems regularized by generic convex functions: a geometric and algorithmic approach. Inf. Geom. 5(1), 97–130 (2022)

  76. Urbas, J.: On the second boundary value problem for equations of Monge–Ampère type, pp. 115–124 (1997)

  77. Villani, C.: Stability of a 4th-order curvature condition arising in optimal transport theory. J. Funct. Anal. 255(9), 2683–2708 (2008)

    MathSciNet  MATH  Google Scholar 

  78. Villani, C.: Optimal Transport: Old and New, vol. 338. Springer, Berlin (2009)

    MATH  Google Scholar 

  79. Villani, C.: Synthetic theory of Ricci curvature bounds. Jpn. J. Math. 11(2), 219–263 (2016)

    MathSciNet  MATH  Google Scholar 

  80. von Renesse, M.-K., Sturm, K.-T.: Transport inequalities, gradient estimates, entropy and ricci curvature. Commun. Pure Appl. Math. 58(7), 923–940 (2005)

    MathSciNet  MATH  Google Scholar 

  81. Wong, T.-K.L.: Logarithmic divergences from optimal transport and Rényi geometry. Inf. Geom. 1(1), 39–78 (2018)

    MathSciNet  MATH  Google Scholar 

  82. Wong, T.K.L.: Information geometry in portfolio theory. In: Geometric Structures of Information, pp. 105–136. Springer (2019)

  83. Wong, T.-K.L., Yang, J.: Pseudo-Riemannian geometry encodes information geometry in optimal transport. Inf. Geom. 5(1), 131–159 (2022)

  84. Wong, T.-K.L., Zhang, J.: Tsallis and Rényi deformations linked via a new \(\lambda \)-duality. IEEE Trans. Inf. Theory. arXiv preprint arXiv:2107.11925 (2021)

  85. Zhang, J.: Divergence function, duality, and convex analysis. Neural Comput. 16(1), 159–195 (2004)

    MATH  Google Scholar 

  86. Zhang, J.: Referential duality and representational duality on statistical manifolds. In: Proceedings of the Second International Symposium on Information Geometry and Its Applications, vol. 1216, p. 5867. Tokyo, Japan (2005)

  87. Zhang, J.: Nonparametric information geometry: from divergence function to referential-representational biduality on statistical manifolds. Entropy 15(12), 5384–5418 (2013)

    MathSciNet  MATH  Google Scholar 

  88. Zhang, J., Fei, T.: Information geometry with (para-) Kähler structures. In: Information Geometry and its Applications IV, pp. 297–321. Springer (2016)

  89. Zhang, J., Khan, G.: Statistical mirror symmetry. Differ. Geom. Appl. 73, 101678 (2020)

    MathSciNet  MATH  Google Scholar 

  90. Zhang, J., Wong, T.-K.L.: \(\lambda \)-Deformed probability families with subtractive and divisive normalizations. In: Handbook of Statistics, vol. 45, pp. 187–215. Elsevier (2021)

  91. Zhang, J., Wong, T.-K.L.: \(\lambda \)-Deformation: a canonical framework for statistical manifolds of constant curvature. Entropy 24(2), 193 (2022)

    MathSciNet  Google Scholar 

  92. Zhu, H., Rohwer, R.: Information geometric measurements of generalisation. NCRC Technical Report 4350, Aston University, Birmingham UK (1995)

Download references

Acknowledgements

The authors would like to thank Ting-Kam Leonard Wong for his helpful comments on the manuscript. G.K. is partially supported by a Simons Collaboration Grant 849022 (“Kähler–Ricci flow and optimal transport”), while J.Z. is partially supported by United States Air Force Office for Scientific Research, Grant number AFOSR-FA9550-19-1-0213.

Author information

Authors and Affiliations

  1. Iowa State University, Ames, USA

    Gabriel Khan

  2. University of Michigan, Ann Arbor, USA

    Jun Zhang

Authors
  1. Gabriel Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Communicated by Nihat Ay.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khan, G., Zhang, J. When optimal transport meets information geometry. Info. Geo. 5, 47–78 (2022). https://doi.org/10.1007/s41884-022-00066-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41884-022-00066-w

Keywords

Search

Navigation