Measuring Adolescents’ Affective Civic Competence: Validity and Cross-Group Equivalence of a Second-Order Hierarchical Latent Construct

  • Tiziano GerosaEmail author


In the last few decades, educational literature and policy makers have increasingly supported research into affective competencies—attitudes, emotions, values/ethics and motivation—as a key requirement to fostering positive youth development in the area of active and engaged citizenship. Considering that adolescents have limited opportunities to undertake civic and political activities until the age of majority, robust indicators of their affective competencies could represent valid alternatives for the analysis of their educational needs and the prediction of their future engagement. Nonetheless, researchers interested in comparing these self-perceived measures across individuals and groups must pay particularly attention to the assumption of their strong factorial invariance, which could lead to inaccurate inferences if violated. On these grounds, the present study validates and assesses the cross-group equivalence of the affective civic competence (ACC) latent construct on a sample of 3366 Italian 8th grade students, exploring also the individual differences emerging in its latent mean structure. The factorial validity of the construct and its measurement invariance across students’ gender and ethnicity are proved through the statistical techniques of second-order confirmatory factor analysis (CFA) and multi-group mean and covariance structure analysis (MG-MACS). Results of the cross-group comparisons show that students’ gender does not lead to significant differences in their overall ACC mean score after correcting for partial scalar invariance, while members of ethnic minorities prove to be disadvantaged compared to natives. Limitations of the study and recommendations for future research are discussed.


Affective civic competence Measurement invariance Mean and covariance structures analysis Adolescence Italy 


Compliance with Ethical Standards

Conflict of Interest

The author declares that he has no conflict of interest.


  1. Adler, R. P., & Goggin, J. (2005). What do we mean by “civic engagement”? Journal of Transformative Education, 3(3), 236–253. CrossRefGoogle Scholar
  2. Ashkanasy, N., Wilerom, C., & Peterson, M. (2004). Organisational culture and climate. London: Sage.Google Scholar
  3. Bandura, A. (1977). Self-efficacy: toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–215. CrossRefGoogle Scholar
  4. Benson, P. L. (2007). Developmental assets: an overview of theory, research and practice. In R. Silbereisen & R. Lerner (Eds.), Approaches to positive youth development (pp. 33–58). London: Sage. CrossRefGoogle Scholar
  5. Bentler, P. M., & Chou, C. P. (1987). Practical issues in structural modeling. Sociological Methods & Research, 16(1), 78–117. CrossRefGoogle Scholar
  6. Bobek, D., Zaff, J., Li, Y., & Lerner, R. M. (2009). Cognitive, emotional, and behavioral components of civic action: towards an integrated measure of civic engagement. Journal of Applied Developmental Psychology, 30(5), 615–627. CrossRefGoogle Scholar
  7. Brown, T. A. (2015). Confirmatory factor analysis for applied research. New York: Guilford Publications.Google Scholar
  8. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & S. J. Long (Eds.), Testing structural equation models (pp. 136–162). Newbury Park: Sage.Google Scholar
  9. Byrne, B. M. (1988). The self description questionnaire III: testing for equivalent factorial validity across ability. Educational and Psychological Measurement, 48(2), 397–406. CrossRefGoogle Scholar
  10. Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: the issue of partial measurement invariance. Psychological Bulletin, 105(3), 456–466. CrossRefGoogle Scholar
  11. Camerini, A. L., & Schulz, P. J. (2017). Social desirability bias in child-report social well-being: evaluation of the children’s social desirability short scale using item response theory and examination of its impact on self-report family and peer relationships. Child Indicators Research, 1–16.
  12. Celio, C. I., Durlak, J., & Dymnicki, A. (2011). A meta-analysis of the impact of service-learning on students. The Journal of Experimental Education, 34(2), 164–181. CrossRefGoogle Scholar
  13. Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504. CrossRefGoogle Scholar
  14. Chen, F. F. (2008). What happens if we compare chopsticks with forks? The impact of making inappropriate comparisons in cross-cultural research. Journal of Personality and Social Psychology, 95(5), 1005–1018. CrossRefGoogle Scholar
  15. Chen, F. F., Sousa, K. H., & West, S. G. (2005). Teacher’s corner: testing measurement invariance of second-order factor models. Structural Equation Modeling, 12(3), 471–492. CrossRefGoogle Scholar
  16. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255. CrossRefGoogle Scholar
  17. Curran, P. J., West, S. G., & Finch, J. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1(1), 16–29. CrossRefGoogle Scholar
  18. Diemer, M. A., & Li, C. H. (2011). Critical consciousness development and political participation among marginalized youth. Child Development, 82(6), 1815–1833. CrossRefGoogle Scholar
  19. EACEA/Eurydice. (2012). Citizen education in Europe. Bruxelles: EACEA.Google Scholar
  20. Ekman, J., & Amnå, E. (2012). Political participation and civic engagement: towards a new typology. Human Affairs, 22(3), 283–300. Scholar
  21. Ellingson, J. E., Smith, D. B., & Sackett, P. R. (2001). Investigating the influence of social desirability on personality factor structure. Journal of Applied Psychology, 86(1), 122–133. CrossRefGoogle Scholar
  22. Finch, J. F., West, S. G., & MacKinnon, D. P. (1997). Effects of sample size and nonnormality on the estimation of mediated effects in latent variable models. Structural Equation Modeling, 4(2), 87–107. CrossRefGoogle Scholar
  23. Finkel, S. E. (1985). Reciprocal effects of participation and political efficacy: a panel analysis. American Journal of Political Science, 29(4), 891–913. CrossRefGoogle Scholar
  24. Fishbein, M., & Ajzen, I. (2011). Predicting and changing behavior: the reasoned action approach. Taylor & Francis.Google Scholar
  25. Flanagan, C. A., & Faison, N. (2001). Youth civic development: implications of research for social policy and programs. Social Policy Report, 15(1), 3–16.Google Scholar
  26. Freire, P. (1973). Education for critical consciousness. New York: Continuum.Google Scholar
  27. Gastil, J., & Xenos, M. (2010). Of attitudes and engagement: clarifying the reciprocal relationship between civic attitudes and political participation. Journal of Communication, 60(2), 318–343. CrossRefGoogle Scholar
  28. Geboers, E., Geijsel, F., Admiraal, W., & Dam, G. (2014). Typology of student citizenship. European Journal of Education, 49(4), 514–528. CrossRefGoogle Scholar
  29. Geijsel, F., Ledoux, G., Reumerman, R., & Ten Dam, G. (2012). Citizenship in young people’s daily lives: differences in citizenship competences of adolescents in the Netherlands. Journal of Youth Studies, 15(6), 711–729. CrossRefGoogle Scholar
  30. Gustafsson, J. E., & Balke, G. (1993). General and specific abilities as predictors of school achievement. Multivariate Behavioral Research, 28(4), 407–434. CrossRefGoogle Scholar
  31. Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis. London: Pearson.Google Scholar
  32. Hope, E. C., & Jagers, R. J. (2014). The role of sociopolitical attitudes and civic education in the civic engagement of black youth. Journal of Research on Adolescence, 24(3), 460–470. CrossRefGoogle Scholar
  33. Hoskins, B., Jesinghaus, J., Masscherini, M., Munda, G., Nardo, M., Saisana, M., ... & Villalba, E. (2006). Measuring active citizenship in Europe. CRELL Research Paper 4. EUR 22530 EN.Google Scholar
  34. Hoskins, B., Villalba, E., Van Nijlen, D., & Barber, C. (2008). Measuring civic competence in Europe: a composite indicator based on IEA Civic Education Study 1999 for 14 years old in School. JRC Scientific and Technical Reports 23210 EN.Google Scholar
  35. Hox, J. J., De Leeuw, E. D., & Zijlmans, E. A. (2015). Measurement equivalence in mixed mode surveys. Frontiers in Psychology, 6, 87. CrossRefGoogle Scholar
  36. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. CrossRefGoogle Scholar
  37. Inglehart, R., & Welzel, C. (2005). Modernization, cultural change, and democracy: the human development sequence. Cambridge: Cambridge University Press. CrossRefGoogle Scholar
  38. Jennings, M. K., & Stoker, L. (2004). Social trust and civic engagement across time and generations. Acta Politica, 39(4), 342–379. CrossRefGoogle Scholar
  39. Jöreskog, K., & Sörbom, D. (1993). PRELIS 2: user’s reference guide. Chicago: Scientific Software International.Google Scholar
  40. Keating, A., Kerr, D., Benton, T., Mundy, E., & Lopes, J. (2010). Citizenship education in England 2001–2010: young people’s practices and prospects for the future: the eighth and final report from the Citizenship Education Longitudinal Study (CELS). DFE-RR059. London: Department for Education.Google Scholar
  41. Kerr, D., Lopes, J., Nelson, J., White, K., Cleaver, E., & Benton, T. (2007). VISION versus PRAGMATISM: citizenship in the secondary school curriculum in England. Citizenship Education Longitudinal Study: Fifth Annual Report. Research Report RR845. National Foundation for Educational Research. Accessed 12 Jan 2017.
  42. Kim, M. S., & Hunter, J. E. (1993). Relationships among attitudes, behavioral intentions, and behavior: a meta-analysis of past research, part 2. Communication Research, 20(3), 331–364. CrossRefGoogle Scholar
  43. Kirshner, B. (2009). Power in numbers: Youth organizing as a context for exploring civic identity. Journal of Research on Adolescence, 19(3), 414–440. CrossRefGoogle Scholar
  44. Kline, R. B. (2015). Principles and practice of structural equation modeling. New York: Guilford publications.Google Scholar
  45. Knowles, R. T., & McCafferty-Wright, J. (2015). Connecting an open classroom climate to social movement citizenship: a study of 8th graders in Europe using IEA ICCS data. The Journal of Social Studies Research, 39(4), 255–269. CrossRefGoogle Scholar
  46. Kriesi, H., & Westholm, A. (2007). Small-scale democracy: the determinants of action. In J. W. van Deth, J. R. Montero, & A. Westholm (Eds.), Citizenship and involvement in European democracies: a comparative analysis (pp. 255–279). Oxon: Routledge.Google Scholar
  47. Lopez, M. H., Levine, P., Both, D., Kiesa, A., Kirby, E., & Marcelo, K. (2006). The 2006 civic and political health of the nation: a detailed look at how youth participate in politics and communities. Medford: CIRCLE.Google Scholar
  48. Losito, B., & D’Apice, A. (2003). Democracy, citizenship, participation. The results of the second IEA civic education study in Italy. International Journal of Educational Research, 39(6), 609–620. CrossRefGoogle Scholar
  49. Manganelli, S., Lucidi, F., & Alivernini, F. (2014). Adolescents’ expected civic participation: the role of civic knowledge and efficacy beliefs. Journal of Adolescence, 37(5), 632–641. CrossRefGoogle Scholar
  50. Metzger, A., & Smetana, J. G. (2009). Adolescent civic and political engagement: associations between domain-specific judgments and behavior. Child Development, 80(2), 433–441. CrossRefGoogle Scholar
  51. Milfont, T. L., & Fischer, R. (2010). Testing measurement invariance across groups: applications in cross-cultural research. International Journal of Psychological Research, 3(1), 111–130. CrossRefGoogle Scholar
  52. Millsap, R. E., & Kwok, O. (2004). Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods, 9(1), 93–115. CrossRefGoogle Scholar
  53. Mintrop, H. (2003). The old and new face of civic education: expert, teacher, and student views. European Educational Research Journal, 2(3), 446–454. CrossRefGoogle Scholar
  54. Muthén, L. K., & Muthén, B. O. (2012). Mplus: the comprehensive modelling program for applied researchers: user’s guide version 7. Los Angeles: Authors.Google Scholar
  55. Nelson, J., & Kerr, D. (2006). Active citizenship in INCA countries: definitions, policies, practices and outcomes. National Foundation for Educational Research. Accessed 5 Jan 2017.
  56. Niemi, R. G., & Junn, J. (2005). Civic education: what makes students learn. New Haven: Yale University Press.Google Scholar
  57. Pasek, J., Feldman, L., Romer, D., & Jamieson, K. H. (2008). Schools as incubators of democratic participation: building long-term political efficacy with civic education. Applied Development Science, 12(1), 26–37. CrossRefGoogle Scholar
  58. Putnam, R. D. (2000). Bowling alone: the collapse and revival of American community. New York: Simon and Schuster.CrossRefGoogle Scholar
  59. Rokeach, M. (1979). The nature of human values. New York: The Free Press.Google Scholar
  60. Rose, A. J., & Rudolph, K. D. (2006). A review of sex differences in peer relationship processes: potential trade-offs for the emotional and behavioral development of girls and boys. Psychological Bulletin, 132(1), 98–131. CrossRefGoogle Scholar
  61. Rubin, B. C. (2007). There’s still not justice: youth civic identity development amid distinct school and community contexts. Teachers College Record, 109(2), 449–481.Google Scholar
  62. Rychen, D. S., & Salganik, L. (2003). Key competencies for a successful life and a well-functioning society. Göttingen: Hogrefe & Huber.Google Scholar
  63. Sass, D. (2011). Testing measurement invariance and comparing latent factor means within a confirmatory factor analysis framework. Journal of Psychoeducational Assessment, 29(4), 347–363. CrossRefGoogle Scholar
  64. Schulz, W., Ainley, J., Fraillon, J., Kerr, D., & Losito, B. (2010). ICCS 2009 international report: civic knowledge, attitudes, and engagement among lower-secondary school students in 38 countries. Amsterdam: IEA.Google Scholar
  65. Schulz, W., Ainley, J., & Fraillon, J. (2011). ICCS 2009 technical report. Amsterdam: IEA.Google Scholar
  66. Sherrod, L. (2007). Civic engagement as an expression of positive youth development. In R. Silbereisen & R. Lerner (Eds.), Approaches to positive youth development (pp. 59–74). London: Sage. CrossRefGoogle Scholar
  67. Sherrod, L., & Baskir, L. R. (2010). Gender differences in the political interests of U.S. high school students. In A. Ittel, H. Merkens, L. Stecher, & J. Zinnecker (Eds.), Jahrbuch Jugendforschung 8 (2008/2009) (pp. 105–130). Wiesbaden: VS Verlag für Sozialwissenchaften.CrossRefGoogle Scholar
  68. Sherrod, L., & Lauckardt, J. (2008). The development of citizenship. In R. Lerner & L. Steinberg (Eds.), Handbook of adolescent psychology (Vol. 2, 3rd ed., pp. 372–408). Hoboken: John Wiley & Sons.Google Scholar
  69. Ten Dam, G., & Volman, M. (2007). Educating for adulthood or for citizenship: social competence as an educational goal. European Journal of Education, 42(2), 281–298. Scholar
  70. Ten Dam, G., Geijsel, F., Reumerman, R., & Ledoux, G. (2011). Measuring young people’s citizenship competences. European Journal of Education, 46(3), 354–372. Scholar
  71. Teorell, J., Torcal, M., & Montero, J. R. (2007). Political participation: mapping the terrain. In J. W. van Deth, J. R. Montero, & A. Westholm (Eds.), Citizenship and involvement in European democracies: a comparative analysis (pp. 334–357). Oxon: Routledge.Google Scholar
  72. Torney-Purta, J. (2002). The school’s role in developing civic engagement: a study of adolescents in twenty-eight countries. Applied Developmental Science, 6(4), 203–212. CrossRefGoogle Scholar
  73. Torney-Purta, J., Barber, C., & Wilkenfeld, B. (2007). Latino adolescents’ civic development in the United States: research results from the IEA Civic Education Study. Journal of Youth and Adolescence, 36(2), 111–125. CrossRefGoogle Scholar
  74. Torney-Purta, J., Wilkenfeld, B., & Barber, C. (2008). How adolescents in 27 countries understand, support, and practice human rights. Journal of Social Issues, 64(4), 857–880.CrossRefGoogle Scholar
  75. Van de Vijver, F. J. R., & Poortinga, Y. H. (2005). Conceptual and methodological issues in adapting tests. In R. K. Hambleton, P. F. Merenda, & C. D. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment (pp. 39–63). Mahwah: Lawrence Erlbaum.Google Scholar
  76. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70. CrossRefGoogle Scholar
  77. Verba, S., Nie, N., & Kim, J. (1978). Political participation and political equality. A seven-nation comparison. Chicago: Chicago University Press.Google Scholar
  78. Verba, S., Schlozman, K. L., & Brady, H. E. (1995). Voice and equality: civic voluntarism in American politics. Harvard: Harvard University Press.Google Scholar
  79. Wang, J., & Wang, X. (2012). Structural equation modeling: applications using Mplus. Chichester: John Wiley & Sons. CrossRefGoogle Scholar
  80. Westheimer, J., & Kahne, J. (2004). What kind of citizen? The politics of educating for democracy. American Educational Research Journal, 41(2), 237–269. CrossRefGoogle Scholar
  81. Widaman, K. F., & Reise, S. P. (1997). Exploring the measurement invariance of psychological instruments: applications in the substance use domain. In K. J. Bryant, M. E. Windle, & S. G. West (Eds.), The science of prevention: methodological advances from alcohol and substance abuse research (pp. 281–324). Washington: American Psychological Association. CrossRefGoogle Scholar
  82. Youniss, J., & Yates, M. (1997). Community service and social responsibility in youth. Chicago: University of Chicago Press.Google Scholar
  83. Yuan, K. H., & Bentler, P. M. (2000). Three likelihood-based methods for mean and covariance structure analysis with nonnormal missing data. Sociological Methodology, 30(1), 165–200. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V., part of Springer Nature and The International Society for Quality-of-Life Studies (ISQOLS) 2018

Authors and Affiliations

  1. 1.Department of Sociology and Social ResearchUniversity of Milano-BicoccaMilanItaly

Personalised recommendations