Approaches in Aligning Language Assessments to Standards and Frameworks

Abstract

This chapter discusses how the interpretation of test scores can be facilitated by aligning these scores to proficiency levels found in language frameworks such as the Common European Framework of Reference (CEFR).

Appendices

The Research Questions

• 1. What are the factors affecting panelists’ decision making during standard setting?

• 2. How do panelist characteristics relate to decisions panelists make during standard setting?

• 3. What types of information provided by standard setting facilitators seem to improve the accuracy and consistency of the panelists’ cut score judgments?

• 4. What types of activities can improve the panelists’ understanding of the standards on which cut scores are set?

• 5. What types of data can be used to validate the cut score recommendation made by a panel?

• 6. How can information from other sources of data be consolidated with a panel’s recommendation, so that cut scores are useful to teachers and students?

• 7. How can technology be used to facilitate the implementation of standard setting procedures?

• 8. How can the association between levels of different language standards or frameworks be explored?

• 9. In what ways does standard setting support the argument for the use of a language assessment and its scores?

• 10. What evidence should be collected to demonstrate the positive consequences of a selected cut score on teachers, students and other stakeholders?

Suggested Resources

Cizek, G. J., & Bunch, M. (2007). Standard setting: A guide to establishing and evaluating performance standards on tests. London: Sage Publications.

This book is written with a focus on the practicalities and “how-to” of standard setting, as the authors point out in the preface. The first part of the book contains an overview of fundamental topics on standard setting. The second part focuses on the procedures for implementing some of the most popular standard setting methods. The third part describes some of the challenges and future directions in the practice of standard setting. This is a highly recommended book for those who need to be involved in standard setting activities.

Cizek, G. J. (Ed.) (2012). Setting performance standards: Concepts, methods, and innovations (2nd ed.). New York, NY: Routledge.

This second edition of the volume is the perhaps the most comprehensive publication on the topic of standard setting, with chapters by several well-known authors in the field. The book contains 27 chapters in four sections that offer a detailed treatment of foundational concepts, applications of several standard setting methods, and address important issues for researchers and practitioners to consider. The volume is highly recommended for readers who want to solidify their understanding of standard setting procedures.

Figueras, N., & Noijons, J. (Eds.). (2009). Linking to the CEFR levels: Research perspectives. Arnhem: CITO. Retrieved from http://www.ealta.eu.org/documents/resources/Research_Colloquium_report.pdf

This freely available edited volume includes papers from a research colloquium in which professionals from the field of language testing exchanged views and discussed their work on aligning language assessments to the CEFR levels. The papers reflect the structure of the colloquium. In the first part expert discussants present their views on critical issues related to the process of aligning assessment to the CEFR levels. In the second part, practitioners offer reports of their alignment projects. The unique combination of theoretical and applied papers is a strong feature of this volume.

Papageorgiou, S., Wu, S., Hsieh, C.-N., Tannenbaum, R. J., & Cheng, M. M. (2019). Mapping the TOEFL iBT® test scores to China’s Standards of English Language Ability: Implications for score interpretation and use (Research Report No. TOEFL-RR-89). Princeton, NJ: Educational Testing Service. Retrieved from https://doi.org/10.1002/ets2.12281

This publication reports on the research collaboration between Educational Testing Service (ETS) in the United States and the National Educational Examination Authority (NEEA) in China. The research project aimed to build an argument for aligning the scores of the TOEFL iBT test to the levels of China’s Standards of English Language Ability (CSE), a localized framework for English as a foreign language. This technical report demonstrates the different steps in building a robust alignment claim, including: establishing construct congruence between the test and the CSE; establishing recommended minimum test scores (cut scores), set by local experts, to classify language learners into the CSE proficiency levels; collection of scores by test takers and evaluations of the test takers’ proficiency levels by their teachers, based on the CSE; consideration of the results of other alignment studies in the local context. Because of the detailed description of the above steps, this report will be of interest to those engaging in similar alignment projects.

Tschirner, E. (Ed.) (2012). Aligning Frameworks of Reference in Language Testing: The ACTFL Proficiency Guidelines and the Common European Framework of Reference for Languages. Tübingen, Germany: Stauffenburg Verlag.

This is a unique edited volume in that it explores the relationship between the levels of two language standards, the CEFR and the ACTFL Proficiency Guidelines. Both language standards have been particularly influential in different geographical areas and share the common element of describing language proficiency through multi-level language proficiency scales. This common element raises the question of how the CEFR and ACTFL levels relate to each other. The book chapters are based on papers presented in two conferences on the relationship between the CEFR and the ACTFL levels held in 2010 and 2011. The book is organized in three parts, ranging from theoretical issues to empirical studies trying to establish a link between the CEFR and the ACTFL levels through tests linked to these levels. This volume makes an important contribution to the relevant literature in that it demonstrates that correspondence between the levels of the two standards is not as straightforward as it might seem.