Abstract
National learning assessments have been growing in Latin America and the Caribbean (LAC) with the purpose of monitoring the mastery of the curriculum achieved in schooling and to foster improvements in achievement. Therefore, it is important to pose the question of to what extent these assessments adequately substantiate inferences about achievement levels as stated in curricular policies, to what extent their results can be interpreted as evidence of learning, and how they can be used to promote better learning. In other words, to what extent are these assessments valid, given their purposes, intended interpretations, and expected uses? National learning assessments are becoming ever more prominent in Latin America and the Caribbean (LAC). Since they are used to monitor how well students have mastered the curriculum and foster improvements in achievement, it is important to ask: How well do such assessments substantiate inferences about achievement levels as stated in curricular policies? To what extent can their results be interpreted as evidence of student progress? And, finally, how can they be used to promote better learning? In other words, we need to evaluate the validity of these assessments—given their purposes, intended interpretations, and expected uses. This chapter identifies distinct dimensions of validity evidence relevant to national learning assessments: (1) the dimension of test alignment with the official curriculum, (2) the dimension of results by performance levels, and (3) the dimension of impact of the assessments. For each of these dimensions, it offers criteria that can be used to carry out an internal review of methods and procedures, to guide external audits, or to define an agenda for validity studies, as well as other efforts associated with assessment validation.
We appreciate Elisa de Padua's valuable collaboration in collecting and analyzing information on validation practices in learning assessment programs. We also thank all the professionals of the assessment programs contacted. Finally, our thanks to Patricia Arregui (GRADE) for her valuable contributions and comments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
For the purposes of this study, we accept these stated objectives as the fundamentals of the assessment policy in the region, although there may and should be other stated and undeclared objectives for these assessments.
- 2.
Although computer tests are becoming more common.
- 3.
Responding to the principle: “If you want to measure change, do not change the measure.”
- 4.
By curricular updates, we understand adjustments that do not affect the fundamental elements of the evaluated curriculum. This is, adjustments of content, skills, or competencies to be achieved in a certain grade or educational cycle. This is usually the case when making updates or curricular reforms in LAC.
- 5.
In LAC countries, curricula are frequently written without reference to the evidence showing what is actually taught and learned in classrooms. Consequently, these curricula often have learning objectives that could hardly be attained by large percentages of the student population. This presents an additional challenge to design assessments with performance levels that provide useful information about students who do not meet curricular expectations. It is also common that in the design of the curriculum, there is no collaboration among experts in educational measurement, and curricular experts. In such cases, the curriculum is not designed to be measurable. Therefore, it is often difficult to operationalize the curriculum with acceptable levels of validity for assessment purposes.
References
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.
Boyd, D., Lankford, H., Loeb, S., & Wyckoff, J. (2008). The impact of assessment and accountability on teacher recruitment and retention: Are there unintended consequences? Public Finance Review, 36(1), 88–111. https://doi.org/10.1177/1091142106293446
Bruns, B., Filmer, D., & Patrinos, H. A. (2011). Making schools work: New evidence on accountability reforms. World Bank. https://doi.org/10.1596/978-0-8213-8679-8
Darling-Hammond, L., Herman, J., Pellegrino, J., Abedi, J., Aber, J. L., Baker, E., & Steele, C. M. (2013). Criteria for high-quality assessment. Stanford Center for Opportunity Policy in Education.
de Hoyos, R., García-Moreno, V., & Patrinos, H. A. (2017). The impact of an accountability intervention with diagnostic feedback: evidence from Mexico. Economics of Education Review, 58(C), 123–140. https://doi.org/10.1016/j.econedurev.2017.03.007
Deming, D. J., Cohodes, S., Jennings, J., & Jencks, C. (2016). When does accountability work? The Texas system had mixed effects on college graduation rates and future earnings. Education next, 16(1), 71–76.
Elacqua, G., Martínez, M., Santos, H., & Urbina, D. (2015). Short-run effects of accountability pressures on teacher policies and practices in the Chilean voucher system in Santiago, Chile. School Effectiveness and School Improvement, 27(3), 385–405. https://doi.org/10.1080/09243453.2015.1086383
Falabella, A. (2014). The performing school: The effects of market and accountability policies. Education Policy Analysis Archives, 22(70), 1–29. https://doi.org/10.14507/epaa.v22n70.2014
Kearns, L. L. (2011). High-stakes standardized testing and marginalized youth: An examination of the impact on those who fail. Canadian Journal of Education, 34(2), 112–130.
Knoester, M., & Wayne, A. (2017). Standardized testing and school segregation: Like tinder for fire? Race, Ethnicity & Education, 20(1), 1–14. https://doi.org/10.1080/13613324.2015.1121474
Ministry of Education. (2015). Towards a complete and balanced system of learning assessment in Chile. Report task force for the revision of the Simce. Retrieved from http://www.mineduc.cl/wp-content/uploads/sites/19/2015/11/Informe-Equipo-de-Tarea-Revisi%C3%B3n-Simce.pdf
Mitroff, I. I., & Featheringham, T. (1974). On systemic problem solving and the error of the third kind. Behavioral Science, 19(6), 383–393.
Mizala, A., & Urquiola, M. (2007). School markets: the impact of information approximating schools’ effectiveness. Journal of Development Economics, 103(C), 313–335. https://doi.org/10.3386/w13676
Mourshed, M., Chijioke, C., & Barber, M. (2010). How the world's most improved school systems keep getting better. McKinsey & Company.
Papay, J. P., Murnane, R. J., & Willett, J. B. (2016). The impact of test score labels on human-capital investment decisions. Journal of Human Resources, 51(2), 357–388.
Segool, N. K., Carlson, J. S., Goforth, A. N., Von der Embse, N., & Barterian, J. A. (2013). Heightened test anxiety among young children: Elementary school students’ anxious responses to high-stakes testing. Psychology in the Schools, 50(5), 489–499. https://doi.org/10.1002/pits.21689
Sempé, L., & Andrade. P. (2017). Final report: Evaluation of the use of census reports of students in the school. GRADE/FORGE.
Taut, S., Cortés, F., Sebastian, C., & Preiss, D. (2009). Evaluating school and parent reports of the national student achievement testing system (Simce) in Chile: Access, comprehension and use. Evaluation & Program Planning, 32(2), 129–137. https://doi.org/10.1016/j.evalprogplan.2008.10.004
UNESCO Institute for Statistics, & Australian Council for Educational Research. (2017). Principles of good practice in learning assessment. Recovered from http://uis.unesco.org/sites/default/files/documents/principles-good-practice-learning-assessments-2017-en.pdf
Valverde, G., & Ramírez, M. J. (2019). Contemporary practices in the curricular validation of national learning assessments in Latin America: A comparative study of cases from Chile, Mexico and Peru. In J. Manzi, M. R. García, & S. Taut (2019), Validity of educational assessment in Chile and Latin America. Ediciones UC.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Ramírez, M.J., Valverde, G.A. (2021). How to Ensure the Validity of National Learning Assessments? Priority Criteria for Latin America and the Caribbean. In: Manzi, J., García, M.R., Taut, S. (eds) Validity of Educational Assessments in Chile and Latin America. Springer, Cham. https://doi.org/10.1007/978-3-030-78390-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-78390-7_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-78389-1
Online ISBN: 978-3-030-78390-7
eBook Packages: EducationEducation (R0)