Abstract
For applications in chemistry and physics, machine learning is generally used to solve one of three problems: interpolation, classification or clustering. These problems use information about physical systems in a certain range of parameters or variables in order to make predictions at unknown values of these variables within the same range. The present work illustrates the application of machine learning to prediction of physical properties outside the range of the training parameters. We define ‘physical extrapolation’ to refer to accurate predictions y(x ∗) of a given physical property at a point \(\boldsymbol x^\ast = \left [ x^\ast _1, \ldots , x^\ast _{\mathcal {D}} \right ]\) in the \(\mathcal {D}\)-dimensional space, if, at least, one of the variables \(x^\ast _i \in \left [ x^\ast _1, \ldots , x^\ast _{\mathcal {D}} \right ]\) is outside of the range covering the training data. We show that Gaussian processes can be used to build machine learning models capable of physical extrapolation of quantum properties of complex systems across quantum phase transitions. The approach is based on training Gaussian process models of variable complexity by the evolution of the physical functions. We show that, as the complexity of the models increases, they become capable of predicting new transitions. We also show that, where the evolution of the physical functions is analytic and relatively simple (one example considered here is a + b∕x + c∕x 3), Gaussian process models with simple kernels already yield accurate generalization results, allowing for accurate predictions of quantum properties in a different quantum phase. For more complex problems, it is necessary to build models with complex kernels. The complexity of the kernels is increased using the Bayesian Information Criterion (BIC). We illustrate the importance of the BIC by comparing the results with random kernels of various complexity. We discuss strategies to minimize overfitting and illustrate a method to obtain meaningful extrapolation results without direct validation in the extrapolated region.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
J.N. Murrell, S. Carter, S.C. Farantos, P. Huxley, A.J.C. Varandas, Molecular Potential Energy Functions (Wiley, Chichester, 1984)
T. Hollebeek, T.-S. Ho, H. Rabitz, Annu. Rev. Phys. Chem. 50, 537 (1999)
B.J. Braams, J.M. Bowman, Int. Rev. Phys. Chem. 28, 577 (2009)
M.A. Collins, Theor. Chem. Acc. 108, 313 (2002)
C.M. Handley, P.L.A. Popelier, J. Phys. Chem. A 114, 3371 (2010)
S. Manzhos, T. Carrington Jr., J. Chem. Phys. 125, 194105 (2006)
J. Cui, R.V. Krems, Phys. Rev. Lett. 115, 073202 (2015)
J. Cui, R.V. Krems, J. Phys. B 49, 224001 (2016)
R.A. Vargas-Hernández, Y. Guan, D.H. Zhang, R.V. Krems, New J. Phys. 21, 022001 (2019)
A. Kamath, R.A. Vargas-Hernández, R.V. Krems, T. Carrington Jr., S. Manzhos, J. Chem. Phys. 148, 241702 (2018)
C. Qu, Q. Yu, B.L. Van Hoozen Jr., J.M. Bowman, R.A. Vargas-Hernández, J. Chem. Theory Comp. 14, 3381 (2018)
L. Wang, Phys. Rev. B 94, 195105 (2016)
J. Carrasquilla, R.G. Melko, Nat. Phys. 13, 431 (2017)
E.P.L. van Nieuwenburg, Y.-H. Liu, S.D. Huber, Nat. Phys. 13, 435 (2017)
P. Broecker, F. Assaad, S. Trebst, (2017). arXiv:1707.00663
S.J. Wetzel, M. Scherzer, Phys. Rev. B 96, 184410 (2017)
S.J. Wetzel, Phys. Rev. E 96, 022140 (2017)
Y.-H. Liu, E.P.L. van Nieuwenburg, Phys. Rev. Lett. 120, 176401 (2018)
K. Chang, J. Carrasquilla, R.G. Melko, E. Khatami, Phys. Rev. X 7, 031038 (2017)
P. Broecker, J. Carrasquilla, R.G. Melko, S. Trebst, Sci. Rep. 7, 8823 (2017)
F. Schindler, N. Regnault, T. Neupert, Phys. Rev. B 95, 245134 (2017)
T. Ohtsuki, T. Ohtsuki, J. Phys. Soc. Jpn 85, 123706 (2016)
L.-F. Arsenault, A. Lopez-Bezanilla, O.A. von Lilienfeld, A.J. Millis, Phys. Rev. B 90, 155136 (2014)
L.-F. Arsenault, O.A. von Lilienfeld, A.J. Millis, (2015). arXiv:1506.08858
M.J. Beach, A. Golubeva, R.G. Melko, Phys. Rev. B 97, 045207 (2018)
E. van Nieuwenburg, E. Bairey, G. Refael, Phys. Rev. B 98, 060301(R) (2018)
N. Yoshioka, Y. Akagi, H. Katsura, Phys. Rev. B 97, 205110 (2018)
J. Venderley, V. Khemani, E.-A. Kim, Phys. Rev. Lett. 120, 257204 (2018)
G. Carleo, M. Troyer, Science 355, 602 (2017)
M. Schmitt, M. Heyl, SciPost Phys. 4, 013 (2018)
Z. Cai, J. Liu, Phys. Rev. B 97, 035116 (2017)
Y. Huang, J.E. Moore, (2017). arXiv:1701.06246
D.-L. Deng, X. Li, S.D. Sarma, Phys. Rev. B 96, 195145 (2017)
Y. Nomura, A. Darmawan, Y. Yamaji, M. Imada, Phys. Rev. B 96, 205152 (2017)
D.-L. Deng, X. Li, S.D. Sarma, Phys. Rev. X 7, 021021 (2017)
X. Gao, L.-M. Duan, Nat. Commun. 8, 662 (2017)
G. Torlai, G. Mazzola, J. Carrasquilla, M. Troyer, R. Melko, G. Carleo, Nat. Phys. 14, 447 (2018)
T. Hazan, T. Jaakkola, (2015). arXiv:1508.05133
A. Daniely, R. Frostig, Y. Singer, NIPS 29, 2253 (2016)
J. Lee, Y. Bahri, R. Novak, S.S. Schoenholz, J. Pennington, J. Sohl-Dickstein, Deep neural networks as Gaussian processes, in ICLR (2018)
K.T. Schütt, H. Glawe, F. Brockherde, A. Sanna, K.R. Müller, E.K.U. Gross, Phys. Rev. B 89, 205118 (2014)
L.M. Ghiringhelli, J. Vybiral, S.V. Levchenko, C. Draxl, M. Scheffler, Phys. Rev. Lett. 114, 105503 (2015)
F.A. Faber, A. Lindmaa, O.A, von Lilienfeld, R. Armient, Int. J. Quantum Chem. 115, 1094 (2015)
F.A. Faber, A. Lindmaa, O.A, von Lilienfeld, R. Armient, Phys. Rev. Lett. 117, 135502 (2016)
R.A. Vargas-Hernández, J. Sous, M. Berciu, R.V. Krems, Phys. Rev. Lett. 121, 255702 (2018)
D.J.J. Marchand, G. De Filippis, V. Cataudella, M. Berciu, N. Nagaosa, N.V. Prokof’ev, A.S. Mishchenko, P.C.E. Stamp, Phys. Rev. Lett. 105, 266605 (2010)
B. Lau, M. Berciu, G.A. Sawatzky, Phys. Rev. B 76, 174305 (2007)
F. Herrera, K.W. Madison, R.V. Krems, M. Berciu, Phys. Rev. Lett. 110, 223002 (2013)
B. Gerlach, H. Löwen, Rev. Mod. Phys. 63, 63 (1991)
P.M. Chaikin, T.C. Lubensky, Principles of Condensed Matter Physics (Cambridge University Press, Cambridge, 1998)
S. Sachdev, Quantum Phase Transitions (Cambridge University Press, Cambridge, 1999)
C.E. Rasmussen, C.K.I. Williams, Gaussian Process for Machine Learning (MIT Press, Cambridge, 2006)
G. Schwarz, Ann. Stat. 6(2), 461 (1978)
D.K. Duvenaud, H. Nickisch, C.E. Rasmussen, Adv. Neural Inf. Proces. Syst. 24, 226 (2011)
D.K. Duvenaud, J. Lloyd, R. Grosse, J.B. Tenenbaum, Z. Ghahramani, Proceedings of the 30th International Conference on Machine Learning Research, vol. 28 (2013), p. 1166
R.S. Sutton, A.G. Barto, Reinforcement Learning: An Introduction (MIT Press, Cambridge, 2016)
J. Dai, R.V. Krems, J. Chem. Theory Comput. 16(3), 1386–1395 (2020). arXiv:1907.08717
A. Christianen, T. Karman, R.A. Vargas-Hernández, G.C. Groenenboom, R.V. Krems, J. Chem. Phys. 150, 064106 (2019)
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, J. Mach. Learn. Res. 15, 1929 (2014)
R.V. Krems, Bayesian machine learning for quantum molecular dynamics. Phys. Chem. Chem. Phys. 21, 13392 (2019)
Acknowledgements
We thank Mona Berciu for the quantum results used for training and verifying the ML models for the polaron problem. We thank John Sous and Mona Berciu for the ideas that had led to work published in Ref. [45] and for enlightening discussions.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Vargas-Hernández, R.A., Krems, R.V. (2020). Physical Extrapolation of Quantum Observables by Generalization with Gaussian Processes. In: Schütt, K., Chmiela, S., von Lilienfeld, O., Tkatchenko, A., Tsuda, K., Müller, KR. (eds) Machine Learning Meets Quantum Physics. Lecture Notes in Physics, vol 968. Springer, Cham. https://doi.org/10.1007/978-3-030-40245-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-40245-7_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-40244-0
Online ISBN: 978-3-030-40245-7
eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)