Abstract
Standard setting is a complex process that involves the application of social science, psychometrics, content expertise, politics, and economics. Over the last 60 years of the practice of standard setting, many methods have been proposed, and many implementation decisions have been made that affect the practice of standard setting. Some of these decisions have been made based on scientific studies about their impact on the standard setting results, but many have been made purely on factors of human judgment or for streamlining the process without the benefit of research to support these decisions. The purpose of this chapter is to focus on where additional research is needed to support many of the practical decisions that are found in many standard setting applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508–600). Washington, DC: American Council on Education.
Cizek, G. J., & Agger, C. A. (2012). Vertically moderated standard setting. In G. J. Cizek (Ed.), Setting performance standards: Foundations, methods, and innovations (2nd ed., pp. 467–484). New York: Routledge.
Clauser, A. L., & Wainer, H. (2016). A tale of two tests (and of two examinees). Educational Measurement: Issues and Practice, 35, 19–28.
Clauser, B. E., Mee, J., Baldwin, S. G., Margolis, M. J., & Dillon, G. F. (2009). Judges’ use of examinee performance data in an Angoff standard-setting exercise for a medical licensing examination: An experimental study. Journal of Educational Measurement, 46, 390–407.
Clauser, J. C., Margolis, M., & Clauser, B. (2014). An examination of the replicability of Angoff standard setting results within a generalizability theory framework. Journal of Educational Measurement, 51, 127–140.
Dwyer, A. C. (2016). Maintaining equivalent cut scores for small sample test forms. Journal of Educational Measurement, 53, 3–22.
Ferdous, A. A., & Plake, B. S. (2007). Item selection strategy for reducing the number of items rated in an Angoff standard setting study. Educational and Psychological Measurement, 65(2), 185–201.
Hambleton, R. K., & Pitoniak, M. J. (2006). Setting performance standards. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 433–470). Washington, DC: American Council on Education.
Hambleton, R. K., & Slater, S. C. (1997). Reliability of credentialing examinations and the impact of scoring models and standard-setting policies. Applied Measurement in Education, 10, 19–38.
Impara, J. C., & Plake, B. S. (1997). Standard setting: An alternative approach. Journal of Educational Measurement, 34, 353–366.
Kane, M. T. (2001). So much remains the same: Conception and status of validation in setting standards. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods, and perspectives (pp. 52–88). Mahwah: Lawrence Erlbaum Associates.
Lewis, D. M., Mitzel, H. C., Mercado, R. L., & Schultz, M. (2012). The bookmark standard setting procedure. In G. J. Cizek (Ed.), Setting performance standards: Foundations, methods, and innovations (2nd ed., pp. 225–254). New York: Routledge.
Loomis, S. C., & Bourque, M. L. (2001). From tradition to innovations: Standard setting on the National Assessment of Educational Progress. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods, and perspectives (pp. 172–217). Mahwah: Lawrence Erlbaum Associates.
Margolis, M., Mee, J., Clauser, B. E., & Winward, M. (2016). Effect of content knowledge on Angoff-style standard setting judgments. Educational Measurements: Issues and Practice, 35, 29–37.
Mattar, J., Hambleton, R., Copella, J., & Finger, M. (2012). Reviewing or revalidating performance standards on credentialing examinations. In G. J. Cizek (Ed.), Setting performance standards: Foundations, methods, and innovations (2nd ed., pp. 399–412). New York: Routledge.
Millman, J. (1989). If at first you don’t succeed: setting passing scores when more than one attempt is permitted. Educational Researcher, 18, 5–9.
Phillips, G. W. (2012). The benchmark method of standard setting. In G. J. Cizek (Ed.), Setting performance standards: Foundations, methods, and innovations (2nd ed., pp. 323–346). New York: Routledge.
Plake, B. S. (2008). Standard setters: Stand up and take a stand. Educational Measurement: Issues and Practices, 27(1), 3–9.
Plake, B. S., & Kane, M. T. (1991). Comparison of methods for combining minimum passing levels for individual items into a passing score for a test. Journal of Educational Measurement, 28, 248–256.
Reckase, M. D. (2001). Innovative methods for helping standard-setting participants to perform their task: The role of feedback regarding consistency, accuracy, and impact. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods, and perspectives (pp. 159–174). Mahwah: Lawrence Erlbaum Associates.
Way, W. D., & McLarty, K. L. (2012). Standard setting for computer-based assessments: A summary of mode comparability research and considerations. In G. J. Cizek (Ed.), Setting performance standards: Foundations, methods, and innovations (2nd ed., pp. 451–466). New York: Routledge.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Plake, B.S. (2017). In the Science and Practice of Standard Setting: Where Is the Science??. In: Blömeke, S., Gustafsson, JE. (eds) Standard Setting in Education. Methodology of Educational Measurement and Assessment. Springer, Cham. https://doi.org/10.1007/978-3-319-50856-6_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-50856-6_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-50855-9
Online ISBN: 978-3-319-50856-6
eBook Packages: EducationEducation (R0)