Abstract
Standard setting panels are tasked with recommending one or more performance level standards for assessments that are used to classify students into ability categories. These assessments are sometimes developed with the performance level descriptors known and other times without these descriptors. Based on an analysis of 11 state, educational, alternative assessments, this chapter investigates the effects on the standard setting process of developing a test both with and without these descriptors. The results suggest that the standard setting panelists are more consistent with one another and more aligned with empirical data when the items were developed with the descriptors in mind.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
The Common Core State Standards are a series of academic content standards developed in the United States that have been defined for English Language Arts and Mathematics across primary, middle, and secondary grade levels.
References
Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508–600). Washington: American Council on Education.
Cizek, G. J., & Earnest, D. S. (2016). Setting performance standards on tests. In S. L. Lane, M. R. Raymond, & T. M. Haladyna (Eds.), Handbook of test development (2nd ed., pp. 212–238). New York: Routledge.
Egan, K. L., Schneider, M. C., & Ferrara, S. (2012). Performance level descriptors: History, practice, and a proposed framework. In G. J. Cizek (Ed.), Setting performance standards: Foundations, methods, and innovations (2nd ed., pp. 79–106). New York: Routledge.
Foley, B. P. (2016). Getting lucky: How guessing threatens the validity of performance classifications. Practical Assessment, Research & Evaluation, 21(3). Available online: http://pareonline.net/getvn.asp?v=21&n=3
Giraud, G., Impara, J. C., & Plake, B. S. (2005). Teachers’ conceptions of the target examinee in Angoff standard setting. Applied Measurement in Education, 18(3), 219–232.
Hambleton, R. K., & Plake, B. S. (1995). Using an extended Angoff procedure to set standards on complex performance assessments. Applied Measurement in Education, 8, 41–56.
Impara, J. C., & Plake, B. S. (1997). Standard setting: An alternative approach. Journal of Educational Measurement, 34, 353–366.
Millman, J., & Greene, J. (1989). The specification and development of tests of achievement and ability. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 335–366). New York: Macmillan Publishing Company.
Mitzel, H. C., Lewis, D. M., Patz, R. J., & Green, D. R. (2001). The bookmark procedure: Psychological perspectives. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods, and perspectives (pp. 249–281). Mahwah: Lawrence Erlbaum Associates.
Norman, R. L., & Buckendahl, C. W. (2008). Determining sufficient measurement opportunities when using multiple cut scores. Educational Measurement: Issues and Practice, 27(1), 37–46.
Plake, B. S., & Hambleton, R. K. (2001). The analytic judgment method for setting standards on complex performance assessments. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods, and perspectives (pp. 283–312). Mahwah: Lawrence Erlbaum Associates.
Plake, B. S., Huff, K., Reshetar, R. R., Kaliski, P., & Chajewski, M. (2016). Validity in the making from evidence-centered design to the validation of the interpretations of test performance. In C. S. Wells & M. Faulkner-Bond (Eds.), Educational measurement from foundations to future (pp. 62–73). New York: The Guilford Press.
Skorupski, W. P., & Hambleton, R. K. (2005). What are panelists thinking when they participate in standard-setting studies? Applied Measurement in Education, 18(3), 233–256.
Wyse, A. E. (2015). The issue of range restriction in bookmark standard setting. Educational Measurement: Issues and Practice, 34(2), 47–54.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Wolkowitz, A.A., Impara, J.C., Buckendahl, C.W. (2017). Closing the Loop: Providing Test Developers with Performance Level Descriptors So Standard Setters Can Do Their Job. In: Blömeke, S., Gustafsson, JE. (eds) Standard Setting in Education. Methodology of Educational Measurement and Assessment. Springer, Cham. https://doi.org/10.1007/978-3-319-50856-6_17
Download citation
DOI: https://doi.org/10.1007/978-3-319-50856-6_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-50855-9
Online ISBN: 978-3-319-50856-6
eBook Packages: EducationEducation (R0)