The Dependability of Test Scores: Generalizability Theory and Hierarchical Linear Models

Abstract

Generalizability theory (Cronbach, Gleser, Nanda & Rajaratnam, 1972; Shavelson, Webb, & Rowley, 1989) is an extension of classical reliability theory. It provides a useful and flexible framework for dealing with the dependability of psychological and educational measurement, especially with regard to observational or cognitive process data. Also with respect to test items, generalizability theory provides a useful framework for addressing some important questions concerning the quality of measurement However, generalizability theory is not suited for dealing with systematic changes over time or over instructional units (e.g., learning gains). For the latter purpose, hierarchical linear modeis are very powerful Statistical tools. Thus, in the first part of this paper some uses of generalizability theory are presented; in the second part the application of hierarchical linear modeis in modelling change is discussed.