Skip to main content
SpringerLink
Log in

Linear and nonlinear growth models for value-added assessment: an application to Spanish primary and secondary schools’ progress in reading comprehension

  • Published:
Educational Assessment, Evaluation and Accountability Aims and scope Submit manuscript

Abstract

Value-added models are considered one of the best alternatives not only for accountability purposes but also to improve the school system itself. The estimates provided by these models measure the contribution of schools to students’ academic progress, once the effect of other factors outside school control are eliminated. The functional form for the growth of students’ performance is a key aspect to be considered when calculating value-added estimations, and, in this regard, there are several reasons that might suggest a nonlinear growth trajectory over time. This aspect is especially important when a variable such as reading comprehension is analyzed, since this is a skill in which nonlinear progression can be expected. This paper focuses on this issue and proposes to estimate schools’ value-added under nonlinear growth models where changes in performance follows a quadratic trajectory, analyzing differences in results with respect to those provided by linear growth models. To illustrate this point, the value-added in reading comprehension was estimated for three parallel cohorts that collect data from 153 primary and secondary schools in Madrid (Spain) and 6,755 students who were assessed at four different times during the academic years 2005–2006 and 2006–2007. Different models have been calculated using hierarchical linear models. The results show that nonlinear growth models fit better. Additionally, the inclusion of the students’ individual and family characteristics in the model provides more accurate VA measures for schools. Finally, it could be noted that the value-added is a relative magnitude that depends on the reference set.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. As indicated by Hox (2010), the equation for estimating the BIC is ambiguous in multilevel modeling because it is unclear the level to which N is referred. Therefore, to calculate BIC1 the number of schools has been considered as N, and N is the number of students in BIC2.

References

  • Al Otaiba, S., Petscher, Y., Pappamihiel, N. E., Williams, R. S., Dyrlund, A. K., & Connor, C. (2009). Modeling oral reading fluency development in Latino students: A longitudinal study across second and third grade. Journal of Educational Psychology, 101(2), 315–329.

    Article  Google Scholar 

  • American Educational Research Association/American Psychological Association/National Council on Measurement in Education (AERA/APA/NCME) (1999). Standards for educational and psychological testing. Washington, DC: Author.

  • Anderman, E. M., Anderman, L. H., Yough, M. S., & Gimbert, B. G. (2010). Value-added models of assessment: Implications for motivation and accountability. Educational Psychologist, 45(2), 123–137.

    Article  Google Scholar 

  • Arnau, J., & Balluerka, N. (2004). Análisis de datos longitudinales y curvas de crecimiento. Enfoque clásico y propuestas actuales. Psicothema, 16(1), 156–162.

    Google Scholar 

  • Baker, B. D. (2011). Take your SGP and VAMit, damn it! Retrieved June 24, from http://schoolfinance101.wordpress.com/2011/09/02/take-your-sgp-and-vamit-damn-it/.

  • Baker, E. L., Barton, P. E., Darling-Hammond, L., Haertel, E., Ladd, H. F., Linn, R. L., et al. (2010). Problems with the use of student test scores to evaluate teachers. Washington, DC: Economic Policy Institute. EPI Briefing Paper #278.

    Google Scholar 

  • Ballou, D., Sanders, W., & Wright, P. (2004). Controlling for student background in value-added assessment of Teacher. Journal of Educational and Behavioral Statistics, 29(1), 37–65.

    Article  Google Scholar 

  • Betebenner, D. W. (2008). A primer on student growth percentiles. Dover, NH: National center for the improvement of educational assessment (NCIEA). Dover, NH: National Center for the Improvement of Educational Assessment (NCIEA). Retrieved February 15, 2009, from http://www.cde.state.co.us/cdedocs/Research/PDF/Aprimeronstudentgrowthpercentiles.pdf.

  • Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for. research. Boston, MA: Houghton-Mifflin.

    Google Scholar 

  • Castro, M., Ruíz, C., & López, E. (2009). Forma básica del crecimiento en los modelos de valor añadido: Vías para la supresión de efecto de regresión. Revista de Educación, 348(1), 111–136.

    Google Scholar 

  • Choi, K., & Seltzer, M. (2010). Modeling heterogeneity in relationships between initial status and rates of change: Treating latent variable regression coefficients as random coefficients in a three-level hierarchical model. Journal of Educational and Behavioral Statistics, 35(1), 54–91.

    Article  Google Scholar 

  • Coleman, J. S., Campbell, E. Q., Hobson, C. J., McPartland, J., Mood, A. M., Weinfeld, F. D., et al. (1966). Equality of educational opportunity (2nd ed.). Washington, DC: US Government Printing Office.

    Google Scholar 

  • Council of Chief State School Officers (2008). Implementer’s guide to growth models: A paper commissioned by the CCSSO accountability systems and reporting state collaborative on assessment and student standards. Washington, DC: The Council of Chief State School Officers. Retrieved August 20, 2009, from http://www.ccsso.org/content/pdfs/IGG%20Final%20AP.pdf.

  • Corcoran, S. P. (2010). Can teachers be evaluated by their students’ test scores? Should they be? The use of value-added measures of teacher effectiveness in policy and practice. New York: Annenberg Institute for School Reform at Brown University.

    Google Scholar 

  • Darling-Hammond, L. (2011, May 24). Too unreliable. The New York Times. Retrieved June 20, 2012, from http://www.nytimes.com/roomfordebate/2010/09/06/assessing-a-teachers-value/value-added-assessment-is-too-unreliable-to-be-useful.

  • De Fraine, B., Van Landeghem, G., Van Damme, J., & Onghena, P. (2005). An analysis of well-being in secondary school with multilevel growth curve models and multilevel multivariate models. Quality and Quantity, 39(3), 297–316.

    Article  Google Scholar 

  • DeVore, D. P. (2011). Evaluation of public school teachers and the role of value-added measures. La Verne, CA: Educational Policy Institute of California. Retrieved May 21, 2012, from http://www.epiculv.org/policy_paper_16.php.

  • Doran, H. C., & Izumi, L. T. (2004). Putting education to the test: A value-added model for California. San Francisco, CA: Pacific Research Institute. Retrieved May 22, 2012, from http://heartland.org/policy-documents/putting-education-test-value-added-model-california.

  • Ferrão, M. E. (2009). Sensibilidad de las especificaciones del modelo de valor-añadido: midiendo el estatus socioeconómico. Revista de Educación, 348(1), 137–152.

    Google Scholar 

  • Florit, E., & Cain, K. (2011). The simple view of reading: Is it valid for different types of alphabetic orthographies? Educational Psychology Review, 23(4), 553–576.

    Article  Google Scholar 

  • Goe, L. (2008). Key issue: Using value-added models to identify and support highly effective teachers. Washington, DC: National Comprehensive Center for Teacher Quality. Retrieved March 04, 2009, from http://www2.tqsource.org/strategies/het/UsingValueAddedModels.pdf.

  • Goldin, R. (2010). Adding it up: How good is value added modeling at measuring teacher performance?. Washington, DC: Statistical Assessment Service. Retrieved March 24, 2012, from http://stats.org/stories/2010/VAM_teacher_performance_sept29_10.html.

  • Gough, P. B., & Tunmer, W. E. (1986). Decoding, reading, and reading disability. Remedial and Special Education, 7(1), 6–10.

    Article  Google Scholar 

  • Harris, D. N. (2009). Teacher value-added: Don’t end the search before it starts. Journal of Policy Analysis and Management, 28(4), 693–699.

    Article  Google Scholar 

  • Hibpshman, T. L. (2004). A review of value-added models. Frankfort, KY: Education Professional Standards Board. Retrieved February 20, 2008, from http://www.kyepsb.net/documents/Stats/Journals/Heterogeneity%20of%20regression.pdf.

  • Hill, H. C. (2009). Evaluating value-added models: A validity argument approach. Journal of Policy Analysis and Management, 28(4), 700–709.

    Article  Google Scholar 

  • Hill, P. W., & Rowe, K. J. (1998). Modelling student progress in studies of educational effectiveness. School Effectiveness and School Improvement, 9(3), 310–333.

    Article  Google Scholar 

  • Hox, J. (2010). Multilevel analysis: Techniques and applications. New York, NY: Routledge.

    Google Scholar 

  • Johnson, S. (2009). How best to add value? Strike a balance between the individual and the organization in school reform (EPI Briefing Paper #249). Washington, DC: Economic Policy Institute.

    Google Scholar 

  • Kirby, J. R., & Savage, R. S. (2008). Can the simple view deal with the complexities of reading? Literacy, 42(2), 75–82.

    Article  Google Scholar 

  • Martínez, R. (2009). Usos, aplicaciones y problemas de los modelos de valor añadido en educación. Revista de Educación, 348(1), 217–250.

    Google Scholar 

  • Martínez, R., Gaviria, J. L., & Castro, M. (2009). Concepto y evolución de los modelos de valor añadido en educación. Revista de Educación, 348(1), 15–45.

    Google Scholar 

  • Maas, C. J. M., & Snijders, T. A. B. (2003). The multilevel approach to repeated measures for complete and incomplete data. Quality & Quantity, 37(1), 71–89.

    Article  Google Scholar 

  • McCaffrey, D. F., Lockwood, J. R., Koretz, D. M., & Hamilton, L. S. (2003). Evaluating value-added models for teacher accountability. Santa Monica, CA: RAND Corporation.

    Google Scholar 

  • McCaffrey, D. F., & Lockwood, J. R. (2011). Missing data in value-added modeling of teacher effects. Annals of Applied Statistics, 6(2A), 773–797.

    Article  Google Scholar 

  • Muthén, B. O. (1991). Analysis of longitudinal data using latent variable models with varying parameters. In L. Collins & J. Horn (Eds.), Best methods for the analysis of change; recent advances, unanswered questions, future directions (pp. 1–17). Washington DC: American Psychological Association.

    Chapter  Google Scholar 

  • National Commission on Excellence in Education. (1983). A nation at risk: The imperative for educational reform. Washington, DC: US Department of Education.

    Google Scholar 

  • No Child Left Behind (NCLB). Act of 2001, Pub. L. No. 107–119, 115, Stat. 1425 (2002).

  • OECD. (2008). Measuring improvements in learning outcomes. Best practices to assess the value-added of schools. Paris: OECD Publishing.

    Google Scholar 

  • OECD. (2011). Establishing a framework for evaluation and teacher incentives: Considerations for Mexico. Paris: OECD Publishing.

    Google Scholar 

  • Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Danish Institute for Educational Research.

    Google Scholar 

  • Raudenbush, S. W., & Xiao-Feng, L. (2001). Effects of study duration, frequency of observation, and sample size on power in studies of group differences in polynomial change. Psychological Methods, 6(4), 387–401.

    Article  Google Scholar 

  • Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis method (2nd ed.). Newbury Park, CA: Sage.

    Google Scholar 

  • Raudenbush, S. (2004). What are value-added models estimating and what does this imply for statistical practice? Journal of Educational and Behavioral Statistics, 29(1), 121–129.

    Article  Google Scholar 

  • Reckase, M. D. (2004). The real world is more complicated than we would like. Journal of Educational and Behavioral Statistics, 29(1), 117–120.

    Article  Google Scholar 

  • Sanders, W. L. (2000). Value-added assessment from student achievement data: Opportunities and hurdles. Journal of Personnel Evaluation in Education, 14(4), 329–339.

    Article  Google Scholar 

  • Sanders, W. L. (2006). Comparisons among various educational assessment value-added models. Paper presented at The Power of Two-National Valued-Added Conference, October 16, 2006, Columbus, Ohio.

  • Sanders, W. L., Saxton, A., & Horn, S. (1997). The Tennessee value-added assessment system: A quantitative, out- comes-based approach to educational assessment. In J. Millman (Ed.), Grading teachers, grading schools: Is student achievement a valid measure? (pp. 137–162). Thousand Oaks, CA: Corwin Press.

    Google Scholar 

  • Seltzer, M., Choi, K., & Thum, Y. M. (2003). Examining relationships between where students start and how rapidly they progress: Implications for conducting analyses that help illuminate the distribution of achievement within schools. Educational Evaluation and Policy Analysis, 25(3), 263–286.

    Article  Google Scholar 

  • Shin, J., Espin, C. A., Deno, S. L., & McConnell, S. (2004). Use of hierarchical linear modeling and curriculum-based measurement for assessing academic growth and instructional factors for students with learning difficulties. Asia Pacific Education Review, 5(2), 136–148.

    Article  Google Scholar 

  • Shin, T. (2007). Comparison of thee growth modeling techniques in the multilevel analysis of longitudinal academic achievement scores: Latent growth modeling, hierarchical linear modeling, and longitudinal profile analysis via multidimensional scaling. Asia Pacific Education Review, 8(2), 262–275.

    Article  Google Scholar 

  • Song, K. J., & Felch, J. (2011, April 12). L.A. unified releases school ratings using ‘value added’ method. Los Angeles Times. Retrieved June 21, 2012, from http://articles.latimes.com/2011/apr/12/local/la-me-0413-value-add-20110414.

  • Steele, J. L., Hamilton, L. S., & Stecher, B. M. (2010). Incorporating student performance measures into teacher evaluation systems. Santa Monica, CA: RAND Corporation.

    Google Scholar 

  • Tekwe, C. D., Carter, R. L., Ma, C. X., Algina, J., Lucas, M. E., Roth, J., et al. (2004). An empirical comparison of statistical models for value-added assessment of school performance. Journal of Educational and Behavioral Statistics, 29(1), 11–35.

    Article  Google Scholar 

  • Thomas, S., Peng, W. J., & Gray, J. (2007). Modelling patterns of improvement over time: Value added trends in English secondary school performance across ten cohorts. Oxford Review of Educacion, 33(3), 261–295.

    Article  Google Scholar 

  • Vista, A. (2013). The role of reading comprehension in maths achievement growth: Investigating the magnitude and mechanism of the mediating effect on maths achievement in Australian classrooms. International Journal of Educational Research, 62(6), 21–34.

    Article  Google Scholar 

  • Wainer, H. (2011). Value-added models to evaluate teachers: A cry for help. CHANCE, 24(1), 11–13.

    Article  Google Scholar 

  • Wallace, S. R., & Gavin, K. A. (2008). Hierarchical Linear Modeling (HLM) Analysis of student growth in secondary education mathematics: A school-based application of growth modeling. Paper presented at the Mid-Western Educational Research Association (MWERA) Annual Meeting, October 15, 2008, Columbus, Ohio.

  • Wang, L., Beckett, G. H., & Brown, L. (2006). Controversies of standardized assessment in school accountability reform: A critical synthesis of multidisciplinary research evidence. Applied Measurement in Education, 19(4), 305–328.

    Article  Google Scholar 

  • Weaver, C. (2002). Reading process and practice (3rd edn). Portsmouth, NH : Heinemann.

  • Webster, W., & Mendro, R. (1997). The dallas value-added accountability system. In J. Millman (Ed.), Grading teachers, grading schools: Is student achievement a valid evaluation measure? (pp. 81–99). Thousand Oaks, CA: Corwin Press, Inc.

    Google Scholar 

  • Willett, J. B., & Sayer, A. G. (1994). Using covariance structure analysis to detect correlates and predictors of individual change over time. Psychological Bulletin, 116(2), 363–381.

    Article  Google Scholar 

  • Wu, M. L., Adams, R. J., Wilson, M. R., & Haldane, S. A. (2007). ACERConQuest Version 2.0: Generalised item response modelling software. Victoria: HACER Press.

    Google Scholar 

  • Zimowski, M. F., Muraki, E., Mislevy, R. J., & Bock, R. D. (1996). BILOG-MG. Multiple-group IRT analysis and test maintenance for binary items. Chicago: Scientific Software International.

    Google Scholar 

  • Zvoch, K., & Stevens, J. J. (2003). A multilevel, longitudinal analysis of middle school math and language achievement. Education Policy Analysis Archives, 11(20). Retrieved 03 August, 2009, from http://epaa.asu.edu/epaa/v11n20/.

  • Zvoch, K., & Stevens, J. J. (2006). Successive student cohorts and longitudinal growth models: An investigation of elementary school mathematics performance. Education Policy Analysis Archives, 14(2). In Retrieved 13 August, 2009, from http://epaa.asu.edu/epaa/v14n2/.

Download references

Acknowledgments

This study is a part of a Research and Development project entitled “Value added in education and the educational production function: a longitudinal study,” sponsored by the Ministry of Science and Technology with reference SEC2003-09742.

Author information

Authors and Affiliations

  1. Department: Research Methods and Diagnosis in Education II, National University of Distance Education, Madrid, Spain

    Esther Lopez-Martin

  2. Department: Quantitative Methods, Aalto University School of Economics, Helsinki, Finland

    Timo Kuosmanen

  3. Department: Research Methods and Diagnosis in Education, Complutense University of Madrid, Madrid, Spain

    Jose Luis Gaviria

  4. Department MIDE II, Universidad Nacional de Educación a Distancia, C/ Juan del Rosal, no. 14, C.P. 28040, Madrid, Spain

    Esther Lopez-Martin

Authors
  1. Esther Lopez-Martin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Timo Kuosmanen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jose Luis Gaviria
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Esther Lopez-Martin.

Appendices

Appendix A

Table 5 Operationalization of variables
Full size table

Appendix B

Table 6 Descriptive statistics for performance
Full size table

Appendix C

Table 7 Correlations among random coefficients in the quadratics models
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lopez-Martin, E., Kuosmanen, T. & Gaviria, J.L. Linear and nonlinear growth models for value-added assessment: an application to Spanish primary and secondary schools’ progress in reading comprehension. Educ Asse Eval Acc 26, 361–391 (2014). https://doi.org/10.1007/s11092-014-9194-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11092-014-9194-1

Keywords

Search

Navigation