Computers and the Humanities

, Volume 32, Issue 5, pp 323–352

How Variable May a Constant be? Measures of Lexical Richness in Perspective

  • Fiona J. Tweedie
  • R. Harald Baayen

DOI: 10.1023/A:1001749303137

Cite this article as:
Tweedie, F.J. & Baayen, R.H. Computers and the Humanities (1998) 32: 323. doi:10.1023/A:1001749303137


A well-known problem in the domain of quantitative linguistics and stylistics concerns the evaluation of the lexical richness of texts. Since the most obvious measure of lexical richness, the vocabulary size (the number of different word types), depends heavily on the text length (measured in word tokens), a variety of alternative measures has been proposed which are claimed to be independent of the text length. This paper has a threefold aim. Firstly, we have investigated to what extent these alternative measures are truly textual constants. We have observed that in practice all measures vary substantially and systematically with the text length. We also show that in theory, only three of these measures are truly constant or nearly constant. Secondly, we have studied the extent to which these measures tap into different aspects of lexical structure. We have found that there are two main families of constants, one measuring lexical richness and one measuring lexical repetition. Thirdly, we have considered to what extent these measures can be used to investigate questions of textual similarity between and within authors. We propose to carry out such comparisons by means of the empirical trajectories of texts in the plane spanned by the dimensions of lexical richness and lexical repetition, and we provide a statistical technique for constructing confidence intervals around the empirical trajectories of texts. Our results suggest that the trajectories tap into a considerable amount of authorial structure without, however, guaranteeing that spatial separation implies a difference in authorship.

lexical statistics Monte Carlo methods vocabulary richness 

Copyright information

© Kluwer Academic Publishers 1998

Authors and Affiliations

  • Fiona J. Tweedie
    • 1
  • R. Harald Baayen
    • 2
  1. 1.University of GlasgowUnited Kingdom
  2. 2.Max Planck Institute for PsycholinguisticsNijmegenThe Netherlands

Personalised recommendations