Skip to main content

Advertisement

Log in

Examining the validity of the Arizona English Language Learners Assessment cut scores

  • Original Paper
  • Published:
Language Policy Aims and scope Submit manuscript

Abstract

The Arizona English Language Learners Assessment (AZELLA) is used by the Arizona Department of Education to determine which children should receive English support services. AZELLA results are used to determine if children are either proficient in English or have English language skills in one of four non-proficient categories (pre-emergent, emergent, basic, intermediate). Children who test at or above the proficient cut score in English are placed in mainstream classes without English language support. Children who obtain scores below the proficient cut scores receive English language support services in state-mandated Structured English Immersion classes. Whenever tests are used to make high-stakes decisions, especially about vulnerable populations (e.g., children), it is the test developers’ responsibility to ensure the instrument yields fair and valid results. When cut scores are used as the primary interpretation of the test they are key to establishing the test’s validity. This validation study found that cut scores for the AZELLA are of questionable validity. The procedure used to set the cut scores is criticized by national measurement experts as ineffective and obsolete. Further, the test developers do not adequately establish the expertise of the judges used to set the cut scores. Evidence from the cut-score-setting process indicates judges did not come to consensus at the kindergarten level. Analysis of empirical evidence suggests cut scores over-identify kindergarten children and under-identify older children. Finally, the test developers rejected 85% of the cut scores recommended by the standard-setting judges, setting cut scores higher than recommended for kindergarten and lower than recommended for older children, without describing their process or rationale.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Abedi, J. (2007). English language proficiency assessment in the nation: Current status and future practice. Davis, C.A.: University of California, Davis.

    Google Scholar 

  • American Educational Research Association, The American Psychological Association, & The National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, D.C.: American Educational Research Association, The American Psychological Association, and The National Council on Measurement in Education.

    Google Scholar 

  • Arizona Department of Education & Harcourt Assessments, Inc. (2007). Arizona English language learner assessment. Phoenix, AZ: Arizona Department of Education [ADE] and Harcourt Assessments, Inc. [Harcourt].

    Google Scholar 

  • Bourque, M. L. (2009). A history of NAEP achievement levels: Issues, implementation, and impact 1989–2009. Washington, D.C.: National Assessment Governing Board.

    Google Scholar 

  • Brown, W. (2000). Reporting NAEP by achievement levels: An analysis of policy and external reviews. In M. L. Bourque & S. Byrd (Eds.). Student performance standards on the National Assessment of Educational Progress: Affirmations and improvements. Washington, D.C.: National Assessment Governing Board.

  • Hambleton, R. K., & Pitoniak, M. J. (2007). Setting performance standards. In R. L. Brennan (Ed.), Educational measurement (Fourth ed., pp. 433–470). Westport, C.T.: Praeger.

    Google Scholar 

  • Lesaux, N. K., Rupp, A. A., & Siegel, L. S. (2007). Growth in reading skills of children from diverse linguistic backgrounds: Findings from a 5-year longitudinal study. Journal of Educational Psychology, 99, 821–834.

    Article  Google Scholar 

  • Mahoney, K., Haladyna, T., & Macswan, J. (2010). The need for multiple measures in reclassification decisions: A validity study of the Standford English language proficiency test. In P. Gandara, M. Hopkins, P. Gandara, & M. Hopkins (Eds.), Forbidden language: English learners and restrictive language policies (pp. 240–262). New York, N.Y.: Teachers College Press.

    Google Scholar 

  • Meisels, S. J. (1986). Testing four- and five-year-olds: Response to Salzer and to Shepard and Smith. Educational Leadership, 44, 90–92.

    Google Scholar 

  • Nagle, R. J. (2007). Issues in preschool assessment. In B. A. Bracken & R. J. Nagle (Eds.), Psychoeducational assessment of preschool children (Fourth ed.). Mahwah, N.J.: Lawrence Erlbaum Associates.

    Google Scholar 

  • National Association for the Education of Young Children (NAEYC) & National Association of Early Childhood Specialists in State Departments of Education (NAECS/SDE). (2003). Early childhood curriculum, assessment and program evaluation: Building an effective, accountable system in programs for children birth through age 8 (position statement with expended resources). Washington D.C.: National Association for the Education of Young Children. Retrieved from www.naeyc.org/about/positions/pdf/pscape.pdf.

  • Reckase, M. D. (2000). A survey and evaluation of recently developed procedures for setting standards on educational tests. In M. L. Bourque & S. Byrd (Eds.), Student performance standards on the National Assessment of Educational Progress: Affirmation and improvements. Washington, D.C.: National Assessment Governing Board.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ida Rose Florez.

Additional information

This work was conducted while the author was affiliated with Arizona State University. The author is now affiliated with the Arizona Early Childhood Development and Health Board.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Florez, I.R. Examining the validity of the Arizona English Language Learners Assessment cut scores. Lang Policy 11, 33–45 (2012). https://doi.org/10.1007/s10993-011-9225-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10993-011-9225-4

Keywords

Navigation