Interpreting latent variables in factor models via convex optimization
- 377 Downloads
Latent or unobserved phenomena pose a significant difficulty in data analysis as they induce complicated and confounding dependencies among a collection of observed variables. Factor analysis is a prominent multivariate statistical modeling approach that addresses this challenge by identifying the effects of (a small number of) latent variables on a set of observed variables. However, the latent variables in a factor model are purely mathematical objects that are derived from the observed phenomena, and they do not have any interpretation associated to them. A natural approach for attributing semantic information to the latent variables in a factor model is to obtain measurements of some additional plausibly useful covariates that may be related to the original set of observed variables, and to associate these auxiliary covariates to the latent variables. In this paper, we describe a systematic approach for identifying such associations. Our method is based on solving computationally tractable convex optimization problems, and it can be viewed as a generalization of the minimum-trace factor analysis procedure for fitting factor models via convex optimization. We analyze the theoretical consistency of our approach in a high-dimensional setting as well as its utility in practice via experimental demonstrations with real data.
KeywordsCanonical correlations analysis Factor analysis Log-determinant optimization Minimum-trace factor analysis Nuclear-norm relaxation Semidefinite programming
Mathematics Subject Classification90C34 90C47 90C90 90C25 62-07 62F12 62H25
The authors were supported in part by NSF Career Award CCF-1350590, by Air Force Office of Scientific Research Grant Nos. FA9550-14-1-0098 and FA9550-16-1-0210, by a Sloan research fellowship, and the Resnick Sustainability Institute at Caltech. Armeen Taeb would like to thank Yong Sheng Soh for many helpful discussions.
- 8.Fazel, M.: Matrix rank minimization with applications. Ph.D. thesis, Department of Electrical Engineering, Stanford University (2002)Google Scholar
- 20.Shapiro, A.: Identifiability of factor analysis: some results and open problems. Linear Algebra Appl. 15, 201–292 (1904)Google Scholar
- 22.Toh, K.C., Todd, M.J., Tutuncu, R.H.: SDPT3—a MATLAB software package for semidefinite-quadratic-linear programming. http://www.math.nus.edu.sg/~mattohkc/sdpt3.html