Abstract
Optimal transport plays a fundamental role in deep learning. Natural data sets have intrinsic patterns, which can be summarized as the manifold distribution principle: a natural class of data can be treated as a probability distribution on a low-dimensional manifold, embedded in a high-dimensional ambient space. A deep learning system mainly accomplishes two tasks: manifold learning and probability distribution learning. Given a manifold X, all the probability measures on X form an infinite dimensional manifold, the so-called Wasserstein space. Optimal transport assigns a Riemannian metric on the Wasserstein space, the so-called Wasserstein metric, and defines Otto’s calculus, such that variational optimization can be carried out in the Wasserstein space \(\mathcal {P}(X)\). A deep learning system learns the distribution by optimizing some functionals in the Wasserstein space \(\mathcal {P}(X)\); therefore optimal transport lays down the theoretic foundation for deep learning. This work introduces the theory of optimal transport and the profound relation between Brenier’s theorem and Alexandrov’s theorem in differential geometry via Monge-Ampère equation. We give a variational proof for Alexandrov’s theorem and convert the proof to a computational algorithm to solve the optimal transport maps. The algorithm is based on computational geometry and can be generalized to general manifold setting. Optimal transport theory and algorithms have been extensively applied in the models of generative adversarial networks (GANs). In a GAN model, the generator computes the optimal transport map (OT map), while the discriminator computes the Wasserstein distance between the generated data distribution and the real data distribution. The optimal transport theory shows the competition between the generator and the discriminator is completely unnecessary and should be replaced by collaboration. Furthermore, the regularity theory of optimal transport map explains the intrinsic reason for mode collapsing. A novel generative model is introduced, which uses an autoencoder (AE) for manifold learning and OT map for probability distribution transformation. This AE-OT model improves the theoretical rigor and transparency, as well as the computational stability and efficiency; in particular, it eliminates the mode collapsing.
References
Alexandrov, A.D.: Convex polyhedra Translated from the 1950 Russian edition by N.S. Dairbekov, S.S. Kutateladze, A.B. Sossinsky. Springer Monographs in Mathematics (2005)
An, D., Guo, Y., Lei, N., Luo, Z., Yau, S.-T., Gu, X.: Ae-ot: A new generative model based on extended semi-discrete optimal transport. In: International Conference on Learning Representations (2020)
Angenent, S., Haker, S., Tannenbaum, A.: Minimizing flows for the monge-kantorovich problem. SIAM J. Math. Ann. 35(1), 61–97 (2003)
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: ICML, pp. 214–223 (2017)
Benamous, J.-D., Brenier, Y.: A numerical method for the optimal time-continuous mass transport problem and related problems. In: Caffarelli, L.A., Milman, M. (eds.) Monge Ampère Equation: Applications to Geometry and Optimization (Deerfield Beach, FL), volume 226 of Contemporary Mathematics, pp. 1–11, Providence (1999) American Mathematics Society
Bonnotte, N.: From knothe’s rearrangement to Brenier’s optimal transport map. SIAM J. Math. Anal. 45(1), 64–87 (2013)
Brenier, Y.: Polar factorization and monotone rearrangement of vector-valued functions. Commun. Pure Appl. Math. 44(4), 375–417 (1991)
Caffarelli, L.A.: Some regularity properties of solutions of monge–ampère equation. Commun. Pure Appl. Math. 44(8–9), 965–969 (1991)
Cui, L., Qi, X., Wen, C., Lei, N., Li, X., Zhang, M., Gu, X.: Spherical optimal transportation. Comput. Aided Des. 115, 181–193 (2019)
Cuturi, M.: Sinkhorn distances: Lightspeed computation of optimal transport. In: Burges, C.J.C., Bottou, L., Welling, M., Ghahramani, Z., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 26, pp. 2292–2300. Curran Associates, Inc. (2013)
Dai, B., Wipf, D.: Diagnosing and enhancing VAE models. In: International Conference on Learning Representations (2019)
De Goes, F., Breeden, K., Ostromoukhov, V., Desbrun, M.: Blue noise through optimal transport. ACM Trans. Graph. 31(6), 171 (2012)
De Goes, F., Cohen-Steiner, D., Alliez, P., Desbrun, M.: An optimal transport approach to robust reconstruction and simplification of 2D shapes. In: Computer Graphics Forum, vol. 30, pp. 1593–1602. Wiley Online Library (2011)
Dominitz, A., Tannenbaum, A.: Texture mapping via optimal mass transport. IEEE Trans. Vis. Comput. Graph. 16(3), 419–433 (2010)
Donahue, J., Simonyan, K.: Large scale adversarial representation learning. In: https://arxiv.org/abs/1907.02544 (2019)
Figalli, A.: Regularity properties of optimal maps between nonconvex domains in the plane. Communications in Partial Differential Equations, 35(3), 465–479 (2010)
Goodfellow, I.: Nips 2016 tutorial: Generative adversarial networks. arXiv preprint arXiv:1701.00160 (2016)
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: NIPS, pp. 2672–2680 (2014)
Gu, D.X., Luo, F., Sun, J., Yau, S.-T.: Variational principles for minkowski type problems, discrete optimal transport, and discrete monge–ampère equations. Asian J. Math. 20, 383–398 (2016)
Guan, P., Wang, X.-J., et al.: On a monge-ampere equation arising in geometric optics. J. Diff. Geom. 48(48), 205–223 (1998)
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.C.: Improved training of wasserstein gans.yIn NIPS, pp. 5769–5779 (2017)
Gutiérrez, C.E., Huang, Q.: The refractor problem in reshaping light beams. Arch. Ration. Mech. Anal. 193(2), 423–443 (2009)
Gelly, S., Schoelkopf, B., Tolstikhin, I., Bousquet, O.: Wasserstein auto-encoders. In: ICLR (2018)
Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: IEEE Conference on Computer Vision and Pattern Recognition (2017)
Jain, U., Zhang, Z., Schwing, A.G.: Creativity: Generating diverse questions using variational autoencoders. In: CVPR, pp. 5415–5424 (2017)
Jeff Donahue, T.D., Krähenbühl, P.: Adversarial feature learning. In: International Conference on Learning Representations (2017)
Kantorovich, L.V.: On a problem of monge. J. Math. Sci. 133(4), 1383–1383 (2006)
Kantorovich, L.V.: On a problem of monge. Uspekhi Mat. Nauk. 3, 225–226 (1948)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
Lindbo Larsen, A.B., Sønderby, S.K., Larochelle, H., Winther, O.: Autoencoding beyond pixels using a learned similarity metric (2016)
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., Shi, W.: Photo-realistic single image super-resolution using a generative adversarial network (2017)
Lei, N., An, D., Guo, Y., Su, K., Liu, S., Luo, Z., Yau, S.-T., Gu, X.: A geometric understanding of deep learning. Engineering 6(3), 361–374 (2020)
Lei, N., Su, K., Cui, L., Yau, S.-T., Gu, X.D.: A geometric view of optimal transportation and generative model. Comput. Aided Geom. Des. 68, 1–21 (2019)
Lin, Z., Khetan, A., Fanti, G., Oh, S.: Pacgan: The power of two samples in generative adversarial networks. In: Advances in Neural Information Processing Systems, pp. 1505–1514 (2018)
Liu, H., Gu, X., Samaras, D.: Wasserstein gan with quadratic transport cost. In: ICCV (2019)
Ma, X.N., Trudinger, N.S., Wang, X.J.: Regularity of potential functions of the optimal transportation problem. Arch. Ration. Mech. Anal. 177(2), 151–183 (2005)
Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., Frey, B.: Adversarial autoencoders. arXiv preprint arXiv:1511.05644 (2015)
Mérigot, Q.: A multiscale approach to optimal transport. In: Computer Graphics Forum, vol. 30, pp. 1583–1592. Wiley Online Library (2011)
Mescheder, L.M., Nowozin, S., Geiger, A.: Which training methods for gans do actually convergence? In: International Conference on Machine Learning (ICML) (2018)
Meyron, J., Mérigot, Q., Thibert, B.: Light in power: a general and parameter-free algorithm for caustic design. In: SIGGRAPH Asia 2018 Technical Papers, p. 224. ACM (2018)
Miyato, T., Kataoka, T., Koyama, M., Yoshida, Y.: Spectral normalization for generative adversarial networks. In: ICLR (2018)
Nadeem, S., Su, Z., Zeng, W., Kaufman, A.E., Gu, X.: Spherical parameterization balancing angle and area distortions. IEEE Trans. Vis. Comput. Graph. 23(6), 1663–1676 (2017)
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. In: ICLR (2016)
Razavi, A., Oord, A., Vinyals, O.: Generating diverse high-fidelity images with vq-vae-2. In: ICLR 2019 Workshop DeepGenStruct (2019)
Rezende, D.J., Mohamed, S., Wierstra, D.: Stochastic backpropagation and approximate inference in deep generative models. arXiv preprint arXiv:1401.4082 (2014)
Salakhutdinov, R., Burda, Y., Grosse, R.: Importance weighted autoencoders. In: ICML (2015)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014)
Simonyany, K., Brock, A., Donahuey, J.: Large scale gan training for high fidelity natural image synthesis. In: International Conference on Learning Representations (2019)
Solomon, J., de Goes, F., Peyré, G., Cuturi, M., Butscher, A., Nguyen, A., Du, T., Guibas, L.: Convolutional wasserstein distances: Efficient optimal transportation on geometric domains. ACM Trans. Graph. 34, 1–11 (2015a)
Solomon, J., De Goes, F., Peyré, G., Cuturi, M., Butscher, A., Nguyen, A., Du, T., Guibas, L.: Convolutional wasserstein distances: Efficient optimal transportation on geometric domains. ACM Trans. Graph. 34(4), 66 (2015b)
Solomon, J., Rustamov, R., Guibas, L., Butscher, A.: Earth mover’s distances on discrete surfaces. ACM Trans. Graph. 33(4), 67 (2014)
Su, K., Chen, W., Lei, N., Cui, L., Jiang, J., Gu, X.D.: Measure controllable volumetric mesh parameterization. Comput. Aided Des. 78(C), 188–198 (2016)
Su, K., Chen, W., Lei, N., Zhang, J., Qian, K., Gu, X.: Volume preserving mesh parameterization based on optimal mass transportation. Comput. Aided Des. 82:42–56 (2017)
Su, K., Cui, L., Qian, K., Lei, N., Zhang, J., Zhang, M., Gu, X.D.: Area-preserving mesh parameterization for poly-annulus surfaces based on optimal mass transportation. Comput. Aided Geom. Des. 46(C):76–91 (2016)
Su, Z., Zeng, W., Shi, R., Wang, Y., Sun, J., Gu, X.: Area preserving brain mapping. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2235–2242 (2013)
Thoma, J., Acharya, D., Van Gool, L., Wu, J., Huang, Z.: Wasserstein divergence for gans. In: ECCV (2018)
ur Rehman, T., Haber, E., Pryor, G., Melonakos, J., Tannenbaum, A.: 3D nonrigid registration via optimal mass transport on the GPU. Med. Image Anal. 13(6), 931–940 (2009)
van den Oord, K.K.A., Vinyals, O.: Neural discrete representation learning. In: NeurIPS (2017)
Villani, C.: Optimal transport: Old and new, vol. 338. Springer Science & Business Media (2008)
Wang, X.-J.: On the design of a reflector antenna. Inverse Prob. 12(3), 351 (1996)
Wang, X.-J.: On the design of a reflector antenna II. Calc. Var. Partial Differ. Equ. 20(3), 329–341 (2004)
Xiao, C., Zhong, P., Zheng, C.: Bourgan: Generative networks with metric embeddings. In: NeurIPS (2018)
Yau, S.-T.: SS Chern: A great geometer of the twentieth century. International PressCo (1998)
Yu, X., Lei, N., Zheng, X., Gu, X.: Surface parameterization based on polar factorization. J. Comput. Appl. Math. 329(C), 24–36 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this entry
Cite this entry
Gu, X., Lei, N., Yau, ST. (2021). Optimal Transport for Generative Models. In: Chen, K., Schönlieb, CB., Tai, XC., Younces, L. (eds) Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging. Springer, Cham. https://doi.org/10.1007/978-3-030-03009-4_105-1
Download citation
DOI: https://doi.org/10.1007/978-3-030-03009-4_105-1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-03009-4
Online ISBN: 978-3-030-03009-4
eBook Packages: Springer Reference MathematicsReference Module Computer Science and Engineering