Abstract
We use data from the international assessments PISA, TERCE, and ICCS to evaluate the invariance of student socioeconomic background scales among the countries participating in these studies. More specifically, we examine whether measures that were developed regionally exhibit better psychometric properties than other measures that were designed to work in a larger and more diverse number of education systems. First, we test the invariance of socioeconomic status (SES) scales across all the countries participating in each study, and then, we run the same test including only Latin American countries. The results suggest that none of the SES scales are invariant at the scalar level among all the countries participating in each study, and therefore comparisons between nations should be made with caution. As for the second group of analyses, our results show that when the analysis is restricted to Latin American countries, the invariance levels improve considerably. Finally, the levels of invariance achieved by each scale in each study and for each group of countries are discussed, as well as the type of comparisons that can be made given these results.
The support of the Center for Advanced Studies on Educational Justice (CONICYT PIA CIE160007) for the funding for the research and writing of this chapter is appreciated.
This is a preview of subscription content, access via your institution.
Buying options
References
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.
Bentler, P. M. (1982). Confirmatory factor analysis via non-iterative estimation. A fast, inexpensive method. Journal of Marketing Research, 19(4), 417–424.
Billiet, J. (2003). Cross-cultural equivalence with structural equation modeling. In J. Harkness, F. van de Vijver, & P. Mohler (Eds.), Cross-cultural survey methods (pp. 247–264). Wiley.
Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). Guildford Press.
Buchmann, C. (2002). Measuring family background in international studies of education: Conceptual issues and methodological challenges. In A. C. Porter & A. Gamoran (Eds.), Methodological advances in cross-national surveys of educational achievement (pp. 150–197). National Academy Press.
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological test. Psychological Bulletin, 52(4), 281–302. https://doi.org/10.1037/h0040957
Erberber, E., Stephens, M., Mamedova, S., Ferguson, S., & Kroeger, T. (2015). Socioeconomically disadvantaged students who are academically successful: examining academic resilience cross-nationally. IEA Policy Brief Series (No.5). IEA. Retrieved from http://www.iea.nl/policy_briefs.html
Glas, C., & Jehangir, K. (2014). Modeling country-specific differential item functioning. In L. Rutkowski, M. von Davier, & D. Rutkowski (Eds.), Handbook of international large-scale assessment: Background, technical issues, and methods of data analysis. Chapman & Hall/CRC Press.
Jöreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 36(4), 408–426.
Marsh, H. W., Ludtke, O., Muthen, B., Asparouhov, T., Morin, A. J. S., & Trautwein, U. (2010). A new look at the big five factor structure through exploratory structural equation modeling. Psychological Assessment, 22(3), 471–491. https://doi.org/10.1037/a0019227
McGrath, R. E. (2015). Measurement invariance in translations of the VIA inventory of strengths. European Journal of Psychological Assessment, 32, 187–194. https://doi.org/10.1027/1015-5759/a000248
Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in test of measurement invariance. The Journal of Applied Psychology, 93(3), 568–592. https://doi.org/10.1037/0021-9010.93.3.568
Meredith, W. (1993). Measurement invariance, factor analysis and factor invariance. Psychometrika, 58, 525–543.
Messick, S. (1984). The psychology of educational measurement. Journal of Educational Measurement, 21(3), 215–237.
Mullis, I. V. S., Martin, M. O., Foy, P., & Hooper, M. (2016). TIMSS 2015 International results in mathematics. TIMSS & PIRLS International Study Center/IEA
Organization for Economic Cooperation and Development. (2011). Against the odds: Disadvantaged students who succeed in school. OECD Publishing.
Organization for Economic Cooperation and Development. (2014). PISA 2012 Technical report. OECD Publishing.
Organization for Economic Cooperation and Development. (2016a). PISA 2015 background questionnaires. In PISA (pp. 129–196). OECD Publishing. Retrieved from http://www.OCDE-ilibrary.org/content/chapter/9789264255425-8-en
Organization for Economic Cooperation and Development. (2016b). PISA 2015 Assessment and analytical framework. OECD Publishing. Retrieved from http://www.OCDE-ilibrary.org/content/book/9789264255425-en
Oliveri, M. E., & von Davier, M. (2011). Research of model fit and score scale comparability in international assessments. Psychological Test and Assessment Modeling, 53(3), 315–333.
Oliveri, M. E., & von Davier, M. (2014). Toward increasing fairness in score scale calibrations employed in international large-scale assessments. International Journal of Testing, 14(1), 1–21. https://doi.org/10.1080/15305058.2013.825265
Rutkowski, L., & Svetina, D. (2014). Assessing the hypothesis of measurement invariance in the context of large-scale international surveys. Educational and Psychological Measurement, 74(1), 31–57. https://doi.org/10.1177/0013164413498257
Sandoval-Hernandez, A., Rutkowski, D., Matta, T., & Miranda, D. (2019). Back to the drawing board: Can we compare socioeconomic background scales? Revista de Educación, 383, 37–61. https://doi.org/10.4438/1988-592X-RE-2019-383-400
Schulz, W., Ainley, J., & Fraillon, J. (Eds.). (2011). ICCS 2009 Technical report. IEA.
Schulz, W., Carstens, R., Losito, B., & Fraillon, J. (Eds.). (2018). ICCS 2016 Technical report. Amsterdam, Netherlands: IEA.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin Company.
Traynor, A., & Raykov, T. (2013). Household possessions indices as wealth measures: A validity evaluation. Comparative Education Review, 57(4), 662–688. https://doi.org/10.1086/671423
Treviño, E., Fraser, P., Meyer, A., Morawietz, L., Inostroza, P., & Naranjo, E. (2015). TERCE results report: Associated factors. OREAL/UNESCO.
UNESCO-OREALC. (2016). Technical report: Third regional comparative and explanatory study, TERCE. OREAL/UNESCO.
Van Der Linden, W. J. (2005). A comparison of item-Selection methods for adaptive test with content constraints. Journal of Educational Measurement, 42(3), 283–302.
Wilson, M. (2005). Constructing measures: An item response modeling approach. Lawrence Erlbaum Associates.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Treviño, E., Sandoval-Hernández, A., Miranda, D., Rutkowski, D., Matta, T. (2021). Invariance of Socioeconomic Status Scales in International Studies. In: Manzi, J., García, M.R., Taut, S. (eds) Validity of Educational Assessments in Chile and Latin America. Springer, Cham. https://doi.org/10.1007/978-3-030-78390-7_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-78390-7_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-78389-1
Online ISBN: 978-3-030-78390-7
eBook Packages: EducationEducation (R0)