Skip to main content

Advertisement

SpringerLink
Log in

Mirror Descent Algorithms for Minimizing Interacting Free Energy

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

This note considers the problem of minimizing interacting free energy. Motivated by the mirror descent algorithm, for a given interacting free energy, we propose a descent dynamics with a novel metric that takes into consideration the reference measure and the interacting term. This metric naturally suggests a monotone reparameterization of the probability measure. By discretizing the reparameterized descent dynamics with the explicit Euler method, we arrive at a new mirror-descent-type algorithm for minimizing interacting free energy. Numerical results are included to demonstrate the efficiency of the proposed algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Amari, S.: Information Geometry and Its Applications, vol. 194. Springer, Berlin (2016)

    Book  Google Scholar 

  2. Arora, S., Hazan, E., Kale, S.: The multiplicative weights update method: a meta-algorithm and applications. Theory Comput. 8(1), 121164 (2012)

    Article  MathSciNet  Google Scholar 

  3. Beck, A., Teboulle, M.: Mirror descent and nonlinear projected subgradient methods for convex optimization. Oper. Res. Lett. 31(3), 167175 (2003)

    Article  MathSciNet  Google Scholar 

  4. Bessemoulin-Chatard, M., Filbet, F.: A finite volume scheme for nonlinear degenerate parabolic equations. SIAM J. Sci. Comput. 34(5), B559–B583 (2012)

    Article  MathSciNet  Google Scholar 

  5. Bubeck, S.: Introduction to online optimization. Lect. Notes 2 (2011)

  6. Bubeck, S., et al.: Convex optimization: algorithms and complexity. Found. Trends R Mach. Learn. 8(3–4), 231357 (2015)

    MATH  Google Scholar 

  7. Carrillo, J.A., McCann, R.J., Villani, C., et al.: Kinetic equilibration rates for granular media and related equations: entropy dissipation and mass transportation estimates. Rev. Mat. Iberoam. 19(3), 9711018 (2003)

    MathSciNet  MATH  Google Scholar 

  8. Carrillo, J.A., Craig, K., Patacchini, F.S.: A blob method for diffusion. Cal. Var. Partial Differ. Equ. 58(2), 53 (2019)

    Article  MathSciNet  Google Scholar 

  9. Chizat, L., Bach, F.: On the global convergence of gradient descent for over-parameterized models using optimal transport. In: Advances in Neural Information Processing Systems, pp. 3036–3046 (2018)

  10. Forsythe, G.E., Malcolm, M.A., Moler, C.B.: Computer Methods for Mathematical Computations, vol. 259. Prentice-Hall, Englewood Cliffs (1977)

    MATH  Google Scholar 

  11. Jordan, R., Kinderlehrer, D., Otto, F.: The variational formulation of the fokker-planck equation. SIAM J. Math. Anal. 29(1), 117 (1998)

    Article  MathSciNet  Google Scholar 

  12. Li, W., Lu, J., Wang, L.: Fisher information regularization schemes for wasserstein gradientows (2019). arXiv:1907.02152

  13. Li, W., Montúfar, G.: Natural gradient via optimal transport. Inf. Geom. 1(2), 181214 (2018)

    MathSciNet  MATH  Google Scholar 

  14. Liu, J.-G., Wang, L., Zhou, Z.: Positivity-preserving and asymptotic preserving method for 2d Keller–Segal equations. Math. Comput. 87(311), 11651189 (2018)

    MathSciNet  Google Scholar 

  15. Mei, S., Montanari, A., Nguyen, P.-M.: A mean field view of the landscape of two-layer neural networks. Proc. Natl. Acad. Sci. 115(33), E7665–E7671 (2018)

    Article  MathSciNet  Google Scholar 

  16. Nemirovsky, A.S., Yudin, D.B.: Problem complexity and method efficiency in optimization. A Wiley-Interscience Publication. Wiley, New York (1983). Translated from the Russian and with a preface by Dawson, E.R. Wiley-Interscience Series in Discrete Mathematics. MR702836

  17. Otto, F.: The Geometry of Dissipative Evolution Equations: The Porous Medium Equation. Taylor & Francis, London (2001)

    MATH  Google Scholar 

  18. Perthame, B.: Transport Equations in Biology. Springer, Berlin (2006)

    MATH  Google Scholar 

  19. Rotskoff, G.M., Vanden-Eijnden, E.: Neural networks as interacting particle systems: asymptotic convexity of the loss landscape and universal scaling of the approximation error (2018). arXiv:1805.00915

  20. Shalev-Shwartz, S., et al.: Online learning and online convex optimization. Found. Trends R Mach. Learn. 4(2), 107194 (2012)

    Google Scholar 

  21. Sirignano, J., Spiliopoulos, K.: Mean field analysis of neural networks (2018). arXiv:1805.01053

  22. Villani, C.: Mathematics of granular materials. J. Stat. Phys. 124(2–4), 781822 (2006)

    MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics and ICME, Stanford University, Stanford, CA, 94305, USA

    Lexing Ying

Authors
  1. Lexing Ying
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lexing Ying.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The work of L.Y. is partially supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Scientific Discovery through Advanced Computing (SciDAC) program and also by the National Science Foundation under award DMS-1818449. The author thanks Wuchen Li and Wotao Yin for comments and suggestions. Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ying, L. Mirror Descent Algorithms for Minimizing Interacting Free Energy. J Sci Comput 84, 51 (2020). https://doi.org/10.1007/s10915-020-01303-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10915-020-01303-z

Keywords

Search

Navigation