Behavior Research Methods

, Volume 46, Issue 2, pp 385–395 | Cite as

Efficient Hessian computation using sparse matrix derivatives in RAM notation

  • Timo von Oertzen
  • Timothy R. Brick


This article proposes a new, more efficient method to compute the minus two log likelihood, its gradient, and the Hessian for structural equation models (SEMs) in reticular action model (RAM) notation. The method exploits the beneficial aspect of RAM notation that the matrix derivatives used in RAM are sparse. For an SEM with K variables, P parameters, and P′ entries in the symmetrical or asymmetrical matrix of the RAM notation filled with parameters, the asymptotical run time of the algorithm is O(P ′ K 2 + P 2 K 2 + K 3). The naive implementation and numerical implementations are both O(P 2 K 3), so that for typical applications of SEM, the proposed algorithm is asymptotically K times faster than the best previously known algorithm. A simulation comparison with a numerical algorithm shows that the asymptotical efficiency is transferred to an applied computational advantage that is crucial for the application of maximum likelihood estimation, even in small, but especially in moderate or large, SEMs.


RAM notation Minus two log likelihood Hessian Sparse density algorithm 


  1. Baltes, P. B., Reese, H. W., & Nesselroade, J. R. (1988). Life-span developmental psychology: Introduction to research methods (reprint of (1977th ed.). Hillsdale: Lawrence Erlbaum Associates.Google Scholar
  2. Boker, S., Neale, M., Maes, H., Wilde, M., Spiegel, M., Brick, T.,… Fox, J. (2011). OpenMx: An open source extended structural equation modeling software. Psychometrika, 76(2), 306–317.Google Scholar
  3. Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Annals of Statistics, 7(1), 1–26.CrossRefGoogle Scholar
  4. Evans, D. (2002). The power of multivariate quantitative-trait loci linkage analysis is influenced by the correlation between variables. American Journal of Human Genetics, 70(6), 1599–1602.PubMedCentralPubMedCrossRefGoogle Scholar
  5. Fox, J. (2006). Structural equation modeling with the sem package in r. Structural Equation Modeling, 13, 465–486.CrossRefGoogle Scholar
  6. Gill, P. E., Murray, W., Saunders, M. A., Wright, M. H. (2001). User's guide for NPSOL 5.0: A Fortran package for nonlinear programming. San Diego: University of California.Google Scholar
  7. Grimm, K. J., & McArdle, J. J. (2005). A note on the computer generation of mean and covariance expectations in latent growth curve analysis. In F. Danserau & F. Yammarino (Eds.), Multi-level issues in strategy and methods (pp. 335–364). New York: Elsiver.CrossRefGoogle Scholar
  8. Jöreskog, K. G., & Sörbom, D. (1993). New features in LISREL 8. Chicago: Scientific Software.Google Scholar
  9. Kaabi, B., Gelernter, J., Woods, S., Goddard, A., Page, G., & Elston, R. (2006). Genome scan for loci predisposing to anxiety disorders using a novel multivariate approach: Strong evidence for a chromosome 4 risk locus. American Journal of Human Genetics, 78(4), 543–553.PubMedCentralPubMedCrossRefGoogle Scholar
  10. Laird, N. M., & Ware, J. H. (1982). Random effect models for longitudinal data. Biometrics, 38, 963–974.PubMedCrossRefGoogle Scholar
  11. McArdle, J., & McDonald, R. (1984). Some algebraic properties of the reticular action model for moment structures. British Journal of Mathematical and Statistical Psychology, 37, 234–251.PubMedCrossRefGoogle Scholar
  12. McArdle, J. J. (2005). The development of RAM rules for latent variable structural equation modeling. In A. Madeau (Ed.), Contemporary advances in psychometrics (pp. 225–273). Mahwah: Lawrence Erlbaum Associates, Inc.Google Scholar
  13. McArdle, J. J., & Boker, S. M. (1990). Rampath: A computer program for automatic path diagrams. Hillsdale: Lawrence Erlbaum Publishers.Google Scholar
  14. McArdle, J. J., & Epstein, D. B. (1987). Latent growth curves within developmental structural equation models. Child Development, 58, 110–133.PubMedCrossRefGoogle Scholar
  15. Muthén, L. K., & Muthén, B. O. (2004). MPlus user’s guide. Los Angeles: Muthén and Muthén.Google Scholar
  16. von Oertzen, T., Brandmaier, A. M., & Tsang, S. (2012). The onyx manual. Onyx Developer Team. (to be found at Scholar
  17. von Oertzen, T., Ghisletta, P., & Lindenberger, U. (2010). Simulating statistical power in latent growth curve modeling: A strategy for evaluating age-based changes in cognitive resources. In M. Crocker & J. Siekmann (Eds.), Resource adaptive cognitive processes (pp. 95–117). Heidelberg: Springer–Verlag.CrossRefGoogle Scholar
  18. Pinheiro, J. C., & Bates, D. M. (2000). Mixed-effects models in S and S-Plus (1st ed.). New York: Springer.CrossRefGoogle Scholar
  19. Prindle, J., & McArdle, J. (2012). An analysis of statistical power in dynamic structural equation models. Structural Equation Modeling, 19(3), 351–371.CrossRefGoogle Scholar
  20. Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36.Google Scholar
  21. Schmiedek, F., & Lindenberger, U. (2010). of cognitive training enhance broad cognitive abilities in adulthood: Findings from the cogito study. Frontiers in Aging Neuroscience, 2, 1–10.Google Scholar
  22. Schmitt, J. E., Wallce, G., Rosenthal, M., Molloy, E., Ordaz, S., Lenroot, R.,…, Giedd, J. (2007). multivariate analysis of neuroanatomic relationships in a genetically infromative pediatric sample. Neuroimage, 35, 70–82.Google Scholar
  23. Stelzl, I. (1986). Changing a causal hypothesis without changing the fit: Some rules for generating equivalent path models. Multivariate Behavioral Research, 21, 309–331.CrossRefGoogle Scholar
  24. Voelkle, M., Oud, J., von Oertzen, T., & Lindenberger, U. (2012). Maximum likelihood dynamic factor modeling for arbitrary N and T using SEM. Structural Equation Modeling, 19(3), 329–350.CrossRefGoogle Scholar
  25. Wright, S. (1934). The method of path coefficients. Annals of Mathematical Statistics, 5, 161–215.CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2013

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of VirginiaCharlottesvilleUSA
  2. 2.Center for Lifespan Psychology, Max Planck Institute for Human DevelopmentBerlinGermany

Personalised recommendations