Skip to main content
SpringerLink
Log in

Multicultural Integration Policy as an Explanatory Factor of Immigrant Social Belonging: Multilevel Evidence for a Multilevel Construct

  • Published:
Journal of International Migration and Integration Aims and scope Submit manuscript

Abstract

Large-scale international assessments, like the Program for International Student Assessment (PISA), are useful tools to help track social and educational outcomes. PISA is a test administered to students in the most developed and developing economies in the world such as Australia, Canada, EU countries, the USA as well as those like Brazil or China. PISA explicitly rates/ranks country-level education systems to inform policy development, which implicitly invokes a multilevel framework within an ecological framework. I provide multilevel, validity evidence of the unidimensionality of PISA’s social belonging measure with consideration for one’s immigration background in 36 countries. Then, I demonstrate that the multicultural policy context in host countries defines and effects the social belonging for students with a migration background. Results suggest that students with an immigration background experience decreased levels of social belonging in school when compared to native-born heritage students but that national-level policy can support these students. The novelty of including national-level policy as a part of the ecological framework of construct validation offers the measurement field an example of a more holistic evaluation of immigrant integration using psychological constructs. Much theoretical lip service has been said that policies and country-level characteristics are a part of a larger ecological model of item response and the validation process, but little research has actually been done. This research is the first to make the explicit connection between policy and immigrant outcomes in the context of measurement and validity showing how the policy context, in part, defines the construct of social belonging.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. This is equivalent to Cohen’s f effect size statistic.

  2. For more information on MIPEX methodology, please visit mipex.eu.

  3. The relationship between Korean and Japanese is still contested. However, some scholarships suggest both may belong to a larger “Altaic” group which would subsequently also include Turkish.

  4. Likely an disordinal effect would be with the fb group; however, the model is too complex to assess simple main effects in this way. The ANOVA parallel would resemble a type of two-way interaction in a fourteen-way ANOVA.

References

  • Asparouhov, T., & Muthén, B. (2014). Multiple-group factor analysis alignment. Structural Equation Modeling: A Multidisciplinary Journal, 21(4), 495–508.

    Article  Google Scholar 

  • Baumeister, R. F., & Leary, M. R. (1995). The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 117(3), 497–529.

    Article  Google Scholar 

  • Baumeister, R. F., Twenge, J. M., & Nuss, C. K. (2002). Effects of social exclusion on cognitive processes: Anticipated aloneness reduces intelligent thought. Journal of Personality and Social Psychology, 83(4), 817–827.

    Article  Google Scholar 

  • Berkman, L. F., & Syme, S. L. (1979). Social networks, host resistance, and mortality: A nine-year follow-up study of Alameda County residents. American Journal of Epidemiology, 109(2), 186–204.

    Article  Google Scholar 

  • Blank, R. M. (2005). Tracing the economic impact of cumulative discrimination. The American Economic Review, 95(2), 99–103.

    Article  Google Scholar 

  • Bliese, P. D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel Theory, Research, and Methods in Organizations (pp. 349–381). Jossey-Bass.

  • Bronfenbrenner, Urie. (1981). The ecology of human development: Experiments by nature and design. 1981 edition. Harvard University Press.

  • Chen, G., J. E. Matthieu, and Paul D Bliese. 2004. “A framework for conducting multilevel construct validation.” , Elsevier, 273–303.

  • Cohen, G. L., & Garcia, J. (2005). ‘I am us’: Negative stereotypes as collective threats. Journal of Personality and Social Psychology, 89(4), 566–582.

    Article  Google Scholar 

  • Da Costa, D. P., & Araújo, L. (2012). Differential item functioning (DIF): What function differently for immigrant students in PISA 2009 reading items. In JRC Scientific and Policy Reports. European Commission.

  • Dansereau, F., and F. J. Yammarino. 2000. “Within and between analysis: The variant paradigm as an underlying approach to theory building and testing.” In Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions, eds. Katherine J. Klein and Steve W. J. Kozlowski. , 425–66.

  • Dansereau, F., Alutto, J. A., & Yammarino, F. J. (1984). Theory testing in organizational behavior: The variant approach. Prentice Hall.

  • Davidov, E., Cieciuch, J., Meuleman, B., Schmidt, P., Algesheimer, R., & Hausherr, M. (2015). The comparability of measurements of attitudes toward immigration in the European Social Survey JethwaExact versus approximate measurement equivalence. Public Opinion Quarterly, 79(S1), 244–266.

    Article  Google Scholar 

  • Flake, J. K., & McCoach, D. B. (2018). An investigation of the alignment method with polytomous indicators under conditions of partial measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 25(1), 56–70.

    Article  Google Scholar 

  • Forer, B., & Zumbo, B. D. (2011). Validation of multilevel constructs: Validation methods and empirical findings for the EDI. Social Indicators Research, 103(2), 231–265.

    Article  Google Scholar 

  • Goodman, S. W. (2010). Integration requirements for integration’s sake? Identifying, categorising and comparing civic integration policies. Journal of Ethnic and Migration Studies, 36(5), 753–772.

    Article  Google Scholar 

  • Ham, S.-H., & Yang, K.-E. (2020). The impact of anti-discrimination policies on immigrant youths’ sense of national belonging: An institutionalist account. Policy Studies, 1–17.

  • Helbling, M. (2013). Validating integration and citizenship policy indices. Comparative European Politics, 11(5), 555–576.

    Article  Google Scholar 

  • Hertzman, C., & Boyce, T. (2010). How experience gets under the skin to create gradients in developmental health. Annual Review of Public Health, 31, 329–347.

    Article  Google Scholar 

  • Hu, L.-t., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55.

    Article  Google Scholar 

  • Hubley, A. M., & Zumbo, B. D. (2011). Validity and the consequences of test interpretation and use. Social Indicators Research, 103(2), 219–230.

    Article  Google Scholar 

  • Hummer, R. A., Rogers, R. G., Nam, C. B., & LeClere, F. B. (1999). Race/ethnicity, nativity, and U.S. adult mortality. Social Science Quarterly, 80(1), 136–153.

    Google Scholar 

  • Kane, Michael. (2006). “Validation.” In Educational measurement, ed. Robert Brennan. Westport, CT: American Council on Education and Praeger, 17–64.

  • Klein, K. J., Dansereau, F., & Hall, R. J. (1994). Levels issues in theory development, data collection, and analysis. The Academy of Management Review, 19(2), 195–229.

    Article  Google Scholar 

  • Lyubomirsky, S., Sheldon, K. M., & Schkade, D. (2005). Pursuing happiness: The architecture of sustainable change. Review of General Psychology, 9(2), 111–131.

    Article  Google Scholar 

  • Messick, S. (1998). Test validity: A matter of consequences. Social Indicators Research, 45, 35–44.

    Article  Google Scholar 

  • Muthén, B., & Asparouhov, T. (2012). Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychological Methods, 17(3), 313–335.

    Article  Google Scholar 

  • O’Connor, Brian P. (2019). Within-and-between-analysis for groups and dyads.

  • Oliveri, M. E., & Ercikan, K. (2011). Do different approaches to examining construct comparability in multilanguage assessments lead to similar conclusions? Applied Measurement in Education, 24(4), 349–366.

    Article  Google Scholar 

  • Oliveri, M. E., Olson, B. F., Ercikan, K., & Zumbo, B. D. (2012). Methodologies for investigating item- and test-level measurement equivalence in international large-scale assessments. International Journal of Testing, 12(3), 203–223.

    Article  Google Scholar 

  • Portes, Alejandro, & Rubén G. Rumbaut. (2001). Legacies: The story of the immigrant second generation. First Printing edition. Berkeley: New York: University of California Press.

  • powell, j. a., & Menendian, S. (2006). The problem of othering: Towards inclusiveness and belonging. Othering & Belonging: Expanding the Circle of Human Concern, 1(1), 16–39.

    Google Scholar 

  • Redhead, M. (2002). Charles Taylor: Thinking and living deep diversity. Rowman & Littlefield Publishers.

  • Roberson, N. D., & Zumbo, B. D. (2019). Migration background in PISA’s measure of social belonging: Using a diffractive lens to interpret multi-method DIF studies. International Journal of Testing, 19(4), 363–389.

    Article  Google Scholar 

  • Shepard, L. A. (1997). The centrality of test use and consequences for test validity. Educational Measurement: Issues and Practice, 16(2), 5–24.

    Article  Google Scholar 

  • Slocum-Gori, S. L., & Zumbo, B. D. (2011). Assessing the unidimensionality of psychological scales: Using multiple criteria from factor analysis. Social Indicators Research, 102(3), 443–461.

    Article  Google Scholar 

  • Stone, J., & Zumbo, B. D. (2016). Validity as a pragmatist project: A global concern with local application. In V. Aryadoust & J. Fox (Eds.), Trends in Language Assessment Research and Practice (pp. 555–573). Newcastle: Cambridge Scholars Publishers.

  • Walton, G. M., & Cohen, G. L. (2007). A question of belonging: Race, social fit, and achievement. Journal of Personality and Social Psychology, 92(1), 82–96.

    Article  Google Scholar 

  • Walzer, Michael. (1984). Spheres of justice: A defense of pluralism and equality. Revised ed. edition. Basic Books.

  • Williams, E. J. (1995). Maquiladora industry and environmental degradation in the United States-Mexico Borderlands, The. St. Mary’s Law Journal, 27, 765.

    Google Scholar 

  • Zumbo, B. D. (2015). Consequences, side effects and the ecology of testing: Keys to considering assessment ‘in vivo’. In Presented at the Plenary address, the annual meeting of the Association for Educational Assessment – Europe (AEA-Europe). Glasgow, Scotland http://brunozumbo.com/aea-europe2015/.

  • Zumbo, B. D., Liu, Y., Wu, A. D., Forer, B., & Shear, B. R. (2017). National and international educational achievement testing: A case of multi-level validation framed by the ecological model of item responding. In Social Indicators Research Series. Understanding and Investigating Response Processes in Validation Research (pp. 341–361). https://doi.org/10.1007/978-3-319-56129-5_18.

  • Zumbo, B. D., & Forer, B. (2011). Testing and measurement from a multilevel view: Psychometrics and validation. In J. A. Bovaird et al. (Eds.), High-Stakes Testing in Education: Science and Practice in K–12 Settings (pp. 177–190). American Psychological Association http://ezproxy.library.ubc.ca/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=pzh&AN=2010-21513-011&site=ehost-live&scope=site.

Download references

Acknowledgments

This research is grateful for the support of Paragon Research Corporation. In particular, I owe a special thanks to Dr. Bruno D. Zumbo for his guidance.

Author information

Authors and Affiliations

  1. Centre for Educational Excellence, Simon Fraser University, West Mall Centre 1300, Burnaby, BC, V5A 1S6, Canada

    Nathan D. Roberson

Authors
  1. Nathan D. Roberson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nathan D. Roberson.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

ESM 1

(DOCX 88 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Roberson, N.D. Multicultural Integration Policy as an Explanatory Factor of Immigrant Social Belonging: Multilevel Evidence for a Multilevel Construct. Int. Migration & Integration 23, 267–284 (2022). https://doi.org/10.1007/s12134-021-00827-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12134-021-00827-z

Keywords

Search

Navigation