Evaluating the Psychometric Qualities of the National Board for Professional Teaching Standards' Assessments: A Methodological Accounting

Abstract

In 1991 the National Board for Professional Teaching Standards established a Technical Analysis Group (TAG) with responsibility for conducting research on the measurement quality of its innovative performance assessments of classroom teachers. The TAG's measurement research agenda focused on four principal areas of inquiry and development—(1) validation of the Board's assessments, (2) characterizing the reliability of the Board's assessments, (3) establishing standards of performance for awarding candidate teachers National Board Certification, and (4) investigation of the presence and degree of adverse impact and bias in the Board's assessments. Because the National Board's assessments differed materially from conventional tests that had been used in the past for assessing teachers' knowledge and skills (for example, the National Teacher Examinations), textbook approaches to evaluation of their measurement properties were largely inapplicable. New measurement methodology was thus required. This article contains a summary of the measurement strategies developed and employed by the TAG. Because investigations of the degree of adverse impact and bias in the National Board's assessments are described in another contribution to this journal issue, the article considers only the first three issues mentioned above. The article begins with a brief description of the structure of the National Board's assessments. A final section of the article identifies some remaining measurement dilemmas and provides suggestions for additional inquiry.