Skip to main content
SpringerLink
Log in

Baby steps in evaluating the capacities of large language models

  • Comment
  • Published:

From Nature Reviews Psychology

View current issue Sign up to alerts

Large language models show remarkable capacities, but it is unclear what abstractions support their behaviour. Methods from developmental psychology can help researchers to understand the representations used by these models, complementing standard computational approaches — and perhaps leading to insights about the nature of mind.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Brown, T. et al. Language models are few-shot learners. Adv. Neural Inf. Process. Syst. 33, 1877–1901 (2020).

    Google Scholar 

  2. Ullman, T. (2023). Large language models fail on trivial alterations to theory-of-mind tasks. Preprint at https://doi.org/10.48550/arXiv.2302.08399 (2023).

  3. Geiger, A., Carstensen, A., Frank, M. C. & Potts, C. Relational reasoning and generalization using nonsymbolic neural networks. Psychol. Rev. 130, 308–333 (2023).

    Article  PubMed  Google Scholar 

  4. Sober, E. in The Evolution of Mind (eds. Cummins, D. D. & Allen, C.) 224–242 (Oxford Univ. Press, 1998).

  5. Kominsky, J. F., Lucca, K., Thomas, A. J., Frank, M. C. & Hamlin, J. K. Simplicity and validity in infant research. Cogn. Dev. 63, 101213 (2022).

    Article  Google Scholar 

  6. Liu, S., Ullman, T. D., Tenenbaum, J. B. & Spelke, E. S. Ten-month-old infants infer the value of goals from the costs of actions. Science 358, 1038–1041 (2017).

    Article  PubMed  Google Scholar 

  7. Saffran, J. R., Aslin, R. N. & Newport, E. L. Statistical learning by 8-month-old infants. Science 274, 1926–1928 (1996).

    Article  PubMed  Google Scholar 

  8. Davidson, K., Eng, K. & Barner, D. Does learning to count involve a semantic induction? Cognition 123, 162–173 (2012).

    Article  PubMed  Google Scholar 

  9. Ruffman, T., Slade, L. & Crowe, E. The relation between children’s and mothers’ mental state language and theory-of-mind understanding. Child Dev. 73, 734–751 (2002).

    Article  PubMed  Google Scholar 

  10. Geiger, A., Lu, H., Icard, T. & Potts, C. Causal abstractions of neural networks. Adv. Neural Inf. Process. Syst. 34, 9574–9586 (2021).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, Stanford University, Stanford, CA, USA

    Michael C. Frank

Authors
  1. Michael C. Frank
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael C. Frank.

Ethics declarations

Competing interests

The author declares no competing interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Frank, M.C. Baby steps in evaluating the capacities of large language models. Nat Rev Psychol 2, 451–452 (2023). https://doi.org/10.1038/s44159-023-00211-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s44159-023-00211-x

  • Springer Nature America, Inc.

This article is cited by

Search

Navigation