Abstract
Teachers are one of the main stakeholder groups in test development and as such need to be included as panellists in standard setting. However, their participation can have additional advantageous effects by helping to increase public acceptance of a test and spreading awareness of principles and good practice of language testing among teachers. This chapter presents a case study of involving teachers as judges in newly implemented standard setting procedures as part of the development of a standardized, nation-wide school-leaving exam. The perspectives of teachers who had participated in the standard setting training or live sessions were investigated by means of a questionnaire (N = 43). Results suggest that teacher involvement fostered a wider understanding of the standard to be implemented and that it had a positive direct effect on the attitudes of practitioners towards exam reform. Teachers who had participated in standard setting/standard setting training reported that their involvement had resulted in an increased professional understanding of assessment issues and had directly impacted on classroom testing practices.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
Bejar, I. I. (1983). Subject matter experts’ assessment of item statistics. Applied Psychological Measurement, 7, 303–310.
Bejar, I. I. (2008). Standard setting: What is it? Why is it important? R&D Connections, 1–6. Retrieved September 30, 2016, from https://www.ets.org/Media/Research/pdf/RD_Connections7.pdf
Cizek, G. J. (1993). Reconsidering standards and criteria. Journal of Educational Measurement, 30, 93–106.
Cizek, G. J. (2012). An introduction to contemporary standard setting: Concepts, characteristics, and contexts. In G. J. Cizek (Ed.), Setting performance standards: Foundations, methods, and innovations (2nd ed., p. 588). New York: Routledge.
Cizek, G. J., & Bunch, M. B. (2007). Standard setting: A guide to establishing and evaluating performance standards on tests. Thousand Oaks: Sage.
Council of Europe. (2009). Relating language examinations to the Common European Framework of Reference for Languages: Learning, teaching, assessment (CEFR). Retrieved September 30, 2016, from http://www.coe.int/t/DG4/linguistic/Source/ManualRevision-proofread-FINAL.pdf\npapers3://publication/uuid/B6F2107B-A069-4C66-8DF7-572172E80D49
Feskens, R., Jeuning, J., van Til, A., & Verheyen, R. (2014). Performance standards for the CEFR in Dutch secondary education: An international standard setting study. Arnhem: Cito.
Figueras, N., Kaftandjieva, F., & Takala, S. (2013). Relating a reading comprehension test to the CEFR levels: A case of standard setting in practice with focus on judges and items. Canadian Modern Language Review/La Revue Canadienne Des Langues Vivantes, 69, 359–385.
Hambleton, R. (2001). Setting performance standards on educational assessments and criteria for evaluating the process. In G. J. Cizek (Ed.), Setting performance standards. Concepts, methods and perspectives (pp. 89–116). New York: Routledge.
Hambleton, R. K., & Pitoniak, M. (2006). Setting performance standards. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 433–470). Westport: Praeger.
Kaftandjieva, F. (2010). Methods for setting cut scores in criterion-referenced achievement tests: A comparative analysis of six recent methods with an application to tests of reading in EFL. Arnhem: Cito.
Lissitz, R. W., & Li, F. (2011). Standard setting in complex performance assessments: An approach aligned with cognitive diagnostic models. Psychological Test and Assessment Modeling, 53, 461–485 Retrieved September 30, 2016, from http://www.psychologie-aktuell.com/fileadmin/download/ptam/4-2011_20111217/04_lissitz.pdf
MacCann, R. G., & Stanley, G. (2010). Extending participation in standard setting: An online judging proposal. Educational Assessment, Evaluation and Accountability, 22, 139–157.
McClarty, K. L., Way, W. D., Porter, A. C., Beimers, J. N., & Miles, J. A. (2013). Evidence-based standard setting: Establishing a validity framework for cut scores. Educational Researcher, 42, 78–88.
McNamara, T., & Knoch, U. (2012). The Rasch wars: The emergence of Rasch measurement in language testing. Language Testing, 29, 555–576.
Papageorgiou, S. (2009). Analyzing the decision-making process of standard setting participants. In N. Figueras & J. Noijons (Eds.), Linking to the CEFR levels: Research perspectives (pp. 75–79). Arnhem: Cito.
Reckase, M. D. (2009). Standard setting theory and practice: Issues and difficulties. In N. Figueras & J. Noijons (Eds.), Linking to the CEFR levels: Research perspectives (pp. 13–20). Arnhem: Cito.
Shepard, L. A. (1994). Implications for standard setting of the NAE evaluation of NAEP achievement levels. Paper presented at the Joint Conference on Standard Setting for Large Scale Assessments, National Center for Educational Statistics, Washington, DC
Tannenbaum, R. J., & Cho, Y. (2014). Critical factors to consider in evaluating standard-setting studies to map language test scores to frameworks of language proficiency. Language Assessment Quarterly, 11, 233–249.
Tiffin-Richards, S. P., & Pant, H. A. (2013). Setting standards for English foreign language assessment: Methodology, validation, and a degree of arbitrariness. Educational Measurement: Issues and Practice, 32, 15–25.
Zieky, M. J., Perie, M., & Livingston, S. A. (2008). Cut scores: A manual for setting standards of performance on educational and occupational tests. Princeton: ETS.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Konrad, E., Spöttl, C., Holzknecht, F., Kremmel, B. (2018). The Role of Classroom Teachers in Standard Setting and Benchmarking. In: Xerri, D., Vella Briffa, P. (eds) Teacher Involvement in High-Stakes Language Testing. Springer, Cham. https://doi.org/10.1007/978-3-319-77177-9_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-77177-9_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-77175-5
Online ISBN: 978-3-319-77177-9
eBook Packages: EducationEducation (R0)