, Volume 70, Issue 4, pp 599–617 | Cite as

Fitting Psychometric Models with Methods Based on Automatic Differentiation

  • Robert CudeckEmail author
2005 Presidential Address


Quantitative psychology is concerned with the development and application of mathematical models in the behavioral sciences. Over time, models have become more complex, a consequence of the increasing complexity of research designs and experimental data, which is also a consequence of the utility of mathematical models in the science. As models have become more elaborate, the problems of estimating them have become increasingly challenging. This paper gives an introduction to a computing tool called automatic differentiation that is useful in calculating derivatives needed to estimate a model. As its name implies, automatic differentiation works in a routine way to produce derivatives accurately and quickly. Because so many features of model development require derivatives, the method has considerable potential in psychometric work. This paper reviews several examples to demonstrate how the methodology can be applied.


Estimation model fitting differentiation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Albert, P.S., & Dodd, L.E. (2004). A cautionary note on the robustness of latent class models for estimating diagnostic error without a gold standard. Biometrics, 60, 427–435.CrossRefPubMedGoogle Scholar
  2. Birkes, D., & Dodge, Y. (1993). Alternative methods of regression. New York: Wiley.Google Scholar
  3. Burden, R.L., & Faires, J.D. (2005). Numerical analysis (8th ed.). Belmont, CA: Thompson Brooks/Cole.Google Scholar
  4. Chinchalkar, S. (1994). The application of automatic differentiation to problems in engineering analysis. Computer Methods in Applied Mechanics and Engineering, 118, 197–207.CrossRefGoogle Scholar
  5. Dayton, C.M., & Macready, G.B. (1988). Concomitant-variable latent-class models. Journal of the American Statistical Association, 83, 173–178.Google Scholar
  6. Donaldson, J.R., & Schnabel, R.B. (1987). Computational experience with confidence regions and confidence intervals for nonlinear least squares. Technometrics, 29, 67–82.Google Scholar
  7. Dunnill, M. (2000). The Plato of Praed Street: The life and times of Almroth Wright. London: Royal Society of Medicine Press.Google Scholar
  8. Fischer, H. (1993). Automatic differentiation and applications. In E. Adams, & U. Kurlisch, (Eds.), Scientific computing with automatic result verification (pp. 105–142). San Diego, CA: Academic Press.Google Scholar
  9. Froemel, E.C. (1971). A comparison of computer routines for the calculation of the tetrachoric correlation coefficient. Psychometrika, 36, 165–173.CrossRefGoogle Scholar
  10. Goetghebeur, E., Liinev, J., Boelaert, M., & Van der Stuyft, P. (2000). Diagnostic test analyses in search of their gold standard: Latent class analyses with random effects. Statistical Methods in Medical Research, 9, 231–248.CrossRefPubMedGoogle Scholar
  11. Griewank, A. (2000). Evaluating derivatives: Principles and techniques of algorithmic differentiation. Philadelphia: Society for Industrial and Applied Mathematics.Google Scholar
  12. Griewank, A., & Walther, A. (2000). Algorithm 799: Revolve: An implementation of checkpointing for the reverse or adjoint mode of computational differentiation. ACM Transactions on Mathematical Software, 26, 19–45.CrossRefGoogle Scholar
  13. Guilford, J.P., & Fruchter, B. (1973). Fundamental statistics in psychology and education (5th ed.). New York: McGraw-Hill.Google Scholar
  14. Hadgu, A., & Qu, Y. (1998). A biomedical application of latent class models with random effects. Applied Statistics, 47, 603–616.Google Scholar
  15. Hamdan, M.A. (1970). The equivalence of tetrachoric and maximum likelihood estimates of ρ in 2 × 2 tables. Biometrika, 57, 212–215.Google Scholar
  16. Hammer, R., Hocks, M., Kulisch, U., & Ratz, D. (1991). Numerical toolbox for verified computing I: Basic numerical problems. New York: Springer-Verlag.Google Scholar
  17. Hansen, J.W., Caviness, J.S., & Joseph, C. (1962). Analytic differentiation by computer. Communications of the Association for Computing Machinery, 5, 349–355.Google Scholar
  18. Hovland, P., Bischof, C., Spiegelman, D., & Casella, M. (1997). Efficient derivative codes through automatic differentiation and interface contraction: An application in biostatistics. SIAM Journal on Scientific Computing, 18, 1056–1066.CrossRefGoogle Scholar
  19. Huang, W., Zeger, S.L., Anthony, J.C., & Garrett, E. (2001). Latent variable model for joint analysis of multiple repeated measures and bivariate event times. Journal of the American Statistical Association, 96, 906–914.Google Scholar
  20. Hui, S.L., & Zhou, X.H. (1998). Evaluation of diagnostic tests without gold standards. Statistical Methods in Medical Research, 7, 354–370.CrossRefPubMedGoogle Scholar
  21. Jerrell, M.E. (1997). Automatic differentiation and interval arithmetic for estimation of disequilibrium models. Computational Economics, 10, 295–316.Google Scholar
  22. Juedes, D. (1991). A taxonomy of automatic differentiation tools. In A. Griewank & G.F. Corliss (Eds.), Automatic differentiation of algorithms: Theory, implementation, and application (pp. 315–329). Philadelphia: SIAM.Google Scholar
  23. Kalaba, R., & Tishler, A. (1984). Automatic derivative evaluation in the optimization of nonlinear models. Review of Economics and Statistics, 66, 653–660.Google Scholar
  24. Kendall, M. (1980). Multivariate analysis (2nd ed.). London: Charles Griffin.Google Scholar
  25. Lau, T.-S. (1997). The latent class model for multiple binary screening tests. Statistics in Medicine, 16, 2283–2295.CrossRefPubMedGoogle Scholar
  26. Lewis, D. (1960). Quantitative methods in psychology. New York: McGraw-Hill.Google Scholar
  27. Pearson, K. (1900). Mathematical contributions to the theory of evolution. VII. On the correlation of characters not quantitatively measurable. Philosophical Transactions of the Royal Society of London. Series A, Mathematical and Physical Sciences, 195, 1–47.Google Scholar
  28. Pearson, K. (1904, November 19). Antityphoid inoculation. British Medical Journal, 1432.Google Scholar
  29. Press, W.H., Teukolsky, S.A., Vetterling, W.T., & Flannery, B.P. (1992). Numerical recipes in C: The art of scientific computing (2nd ed.). Cambridge: Cambridge University Press.Google Scholar
  30. Qu, Y., Tan, M., & Kutner, M.H. (1996). Random effects models in latent class analysis for evaluating accuracy of diagnostic tests. Biometrics, 52, 797–810.PubMedGoogle Scholar
  31. Seber, G.A.F., & Wild, C.J. (1989). Nonlinear regression. New York: Wiley.Google Scholar
  32. Schittkowski, K. (2002). Numerical data fitting in dynamical systems. Dordrecht: Kluwer Academic.Google Scholar
  33. Simpson, R.J.S., & Pearson, K. (1904, November 5). Report on certain enteric fever inoculation statistics. British Medical Journal, 1243–1246.Google Scholar
  34. Skaug, H.J. (2002). Automatic differentiation to facilitate maximum likelihood estimation in nonlinear random effects models. Journal of Computational and Graphical Statistics, 11, 458–470.CrossRefGoogle Scholar
  35. Skaug, H.J., & Fournier, D. (2004, October). Automatic evaluation of the marginal likelihood in nonlinear hierarchical models. Unpublished research report. Bergen, Norway: Institute of Marine Research. Available at
  36. Tateneni, K. (1998). Use of automatic and numerical differentiation in the estimation of asymptotic standard errors in exploratory factor analysis. Unpublished doctoral dissertation, Columbus, OH: Psychology Department, Ohio State University.Google Scholar
  37. Vermunt, J.K. (2003). Multilevel latent class models. Sociological Methodology, 33, 213–239.CrossRefGoogle Scholar
  38. Wengert, R.E. (1964). A simple automatic derivative evaluation program. Communications of the Association for computing Machinery,7 463–464.Google Scholar
  39. Wilkins, R.D. (1964). Investigation of a new analytical method for numerical derivative evaluation. Communications of the Association for Computing Machinery, 7, 465–471.Google Scholar

Copyright information

© The Psychometric Society 2005

Authors and Affiliations

  1. 1.Psychology DepartmentOhio State UniversityColumbusUSA

Personalised recommendations