PCA Meets RG
- 484 Downloads
A system with many degrees of freedom can be characterized by a covariance matrix; principal components analysis focuses on the eigenvalues of this matrix, hoping to find a lower dimensional description. But when the spectrum is nearly continuous, any distinction between components that we keep and those that we ignore becomes arbitrary; it then is natural to ask what happens as we vary this arbitrary cutoff. We argue that this problem is analogous to the momentum shell renormalization group. Following this analogy, we can define relevant and irrelevant operators, where the role of dimensionality is played by properties of the eigenvalue density. These results also suggest an approach to the analysis of real data. As an example, we study neural activity in the vertebrate retina as it responds to naturalistic movies, and find evidence of behavior controlled by a nontrivial fixed point. Applied to financial data, our analysis separates modes dominated by sampling noise from a smaller but still macroscopic number of modes described by a non-Gaussian distribution.
KeywordsRenormalization group Neural networks Financial markets
We thank D Amodei, MJ Berry II, and O Marre for making available the data of Ref.  and M Marsili for the data of Ref. . We are especially grateful to G Biroli, J–P Bouchaud, MP Brenner, CG Callan, A Cavagna, I Giardina, MO Magnasco, A Nicolis, SE Palmer, G Parisi, and DJ Schwab for helpful discussions and comments on the manuscript. Work at CUNY was supported in part by the Swartz Foundation. Work at Princeton was supported in part by Grants from the National Science Foundation (PHY-1305525, PHY-1451171, and CCF-0939370) and the Simons Foundation.
- 2.Shlens, J.: A tutorial on principal components analysis. arXiv:1404.1100 [cs.LG] (2014)
- 7.Kadanoff, L.P.: Scaling laws for Ising models near \(T_c\). Physics 2, 263–272 (1966)Google Scholar
- 9.Kadanoff, L.P., Weinblatt, H.: Public policy conclusions from urban growth models. IEEE Trans. Syst. Man Cybern. SMC–2, 139–165 (1972)Google Scholar
- 11.Halsey, T.C., Jensen, M.H., Kadanoff, L.P., Procaccia, I., Shraiman, B.I.: Fractal measures and their singularities: the characterization of strange sets. Phys. Rev. A33, 1141–1151 (1986); erratum 34, 1601 (1986)Google Scholar
- 13.Bertozzi, A., Brenner, M., Dupont, T.F., Kadanoff, L.P.: Singularities and similarities in interface flows. In: Sirovich, L.P. (ed.) Trends and Perspectives in Applied Mathatematics. Springer Verlag Applied Math Series Vol. 100, pp. 155–208 (1994)Google Scholar
- 29.Bouchaud, J.P., Potters, M: Financial applications. In: Akemann, G., Baik, J., Di Francesco, P. (eds.) The Oxford Handbook of Random Matrix Theory. Oxford University Press, Oxford (2011). arXiv:0910.1205 [q–fin.ST] (2009)
- 30.Bun, J., Allez, R., Bouchaud, J.P., Potters, M.: Rotational invariant estimator for general noisy matrices. arXiv:1502.06736 [cond–mat.stat–mech] (2015)
- 31.Bun, J., Bouchaud, J.-P., Potters, M.: Cleaning large correlation matrices: tools from random matrix theory. arXiv:1610.08104 [cond–mat.stat–mech] (2016)
- 41.Mehta, P., Schwab, D.J.: An exact mapping between the variational renormalization group and deep learning. arXiv:1410.3831 [stat.ML] (2014)