Skip to main content
SpringerLink
Log in

Resolving the attitude–achievement paradox based on anchoring vignettes: evidences from the PISA 2012 mathematics study

  • Published:
Asia Pacific Education Review Aims and scope Submit manuscript

Abstract

Secondary data analyses of large-scale international student assessment at the country level often reveal that educational systems which are high in academic achievement are found low in attitudinal outcomes of schooling, and vice versa. This is contrary to the non-negative attitude–achievement relationship frequently found at the student within-country level. Using attitudinal outcome mathematics self-concept, the afore-mentioned attitude–achievement paradox is hypothesized to stem from the incomparable student responses to Likert-type response scale of the attitude items. In this study, anchoring vignettes are used to examine whether it is feasible to tackle the effect of response style so as to improve the comparability of the self-concept measures across cultures and societies. Empirical evidences from PISA 2012 demonstrate that the paradox can be resolved satisfactorily, so that the attitude–achievement relationships at both student within-country and country levels are now in line with that predicted by contemporary self-concept theories in the literature. Using the anchored self-concept measure, Singapore’s and Shanghai-China’s 15-year-olds are found to perform pretty well cognitively and affectively in PISA 2012.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Bertling, J. P., & Kyllonen, P. C. (2013). Using anchoring vignettes to detect and correct for response styles in PISA questionnaires. In M. Prenzel (Chair). The attitudes–achievement paradox: How to interpret correlational patterns in cross-cultural studies, Symposium at the EARLI 2013, Munich, Germany.

  • Bolt, D. M., Lu, Y., & Kim, J. S. (2014). Measurement and control of response styles using anchoring vignettes: A model-based approach. Psychological Methods, 9, 528–541.

    Article  Google Scholar 

  • Bong, M., & Clark, R. E. (1999). Comparison between self-concept and self-efficacy in academic motivation research. Educational Psychologist, 34(3), 139–153. https://doi.org/10.1207/s15326985ep3403_1.

    Article  Google Scholar 

  • Bybee, R., & McCrae, B. (2007). Scientific literacy and student attitudes: Perspectives from PISA 2006 science. International Journal of Science Education, 33, 7–26.

    Article  Google Scholar 

  • Chen, S., Lee, S.-Y., & Stevenson, H. W. (1995). Response styles and cross-cultural comparisons of rating scales among East Asian and North American students. Psychological Science, 6(3), 170–175.

    Article  Google Scholar 

  • Cheung, K. C. (1988). Outcomes of schooling: Mathematics achievement and attitudes towards mathematics learning in Hong Kong. Educational Studies in Mathematics, 19(2), 209–221.

    Article  Google Scholar 

  • Cheung, K. C. (2009a). A decade of school-based MI teaching experiments in Macao. In J. Chen, S. Moran & H. Gardner (Eds.), Multiple intelligences around the world (pp. 66–75). San Francisco: Jossey-Bass.

    Google Scholar 

  • Cheung, K. C. (2009b). Reforming teaching and learning using theory of multiple intelligences: The Macao experiences. In C. H. Ng & P. Renshaw (Eds.), Reforming learning: Concepts, issues and practices in the Asia-pacific region (pp. 159–179). Heidelberg: Springer.

    Chapter  Google Scholar 

  • He, J., Buchholz, J., & Klieme, E. (2017). Effects of anchoring vignettes on comparability and predictive validity of student self-reports in 64 cultures. Journal of Cross-Cultural Psychology, 48(3), 319–334.

    Article  Google Scholar 

  • Ho, S. C. (2006). High-stakes testing and its impact on students and schools in Hong Kong: What we have learned from the PISA studies. KEDI Journal of Educational Policy, 3(1), 69–87.

    Google Scholar 

  • Holton, D., Cheung, K. C., Kesianye, S., De Losada, M., Leikin, R., Makrides, G., Meissner, H., Sheffield, L., & Yeap, B. H. (2009). Teacher development and mathematical challenges. In E. J. Barbeau & P. J. Taylor (Eds.), New ICMI Study Series 12: Challenging mathematics in and beyond the classroom (pp. 205–242). New York: Springer.

    Chapter  Google Scholar 

  • Hopkins, D. J., & King, G. (2010). Improving anchoring vignettes: Designing surveys to correct interpersonal incomparability. Public Opinion Quarterly, 24, 1–22.

    Google Scholar 

  • King, G., Murray, C. J. L., Salomon, J. A., & Tandon, A. (2004). Enhancing the validity and cross-cultural comparability of measurement in survey research. American Political Science Review, 98(1), 191–207.

    Article  Google Scholar 

  • King, G., & Wand, J. (2007). Comparing incomparable survey responses: Evaluating and selecting anchoring vignettes. Political Analysis, 15(1), 46–66.

    Article  Google Scholar 

  • Kyllonen, P. C., & Bertling, J. P. (2013). Innovative questionnaire assessment methods to increase cross-country comparability. In L. Rutkowski, M. von Davier & D. Rutkowski (Eds.), Handbook of international large-scale assessment: Background, technical issues, and methods of data analysis (pp. 277–286). Boca Raton, FL: CRC Press.

    Google Scholar 

  • Lee, J. W., Jones, P. S., Mineyama, Y., & Zhang, X. E. (2002). Cultural differences in response to a Likert scale. Research in Nursing and Health, 25, 295–306.

    Article  Google Scholar 

  • Leung, K. S. F. (2001). In search of an East Asian identity in mathematics education. Educational Studies in Mathematics, 47, 35–51.

    Article  Google Scholar 

  • Leung, K. S. F. (2002). Behind the high achievement of East Asian students. Educational Research and Evaluation: An International Journal on Theory and Practice, 8(1), 87–108.

    Article  Google Scholar 

  • Leung, K. S. F. (2006). Mathematics education in East Asia and the West: Does culture matter? In F. K. S. Leung, K. D. Graf & F. J. Lopez-Real (Eds.), Mathematics education in different cultural traditions: A comparative study of East Asia and the West, New ICMI Study Series Volume 9 (pp. 21–46). New York: Springer.

    Chapter  Google Scholar 

  • Lu, Y., & Bolt, D. M. (2015). Examining the attitude–achievement paradox in PISA using a multilevel multidimensional IRT model for extreme response style. Large-scale Assessments in Education, 3, 2. https://doi.org/10.1186/s40536-015-0012-0.

    Article  Google Scholar 

  • Marsh, H. W., & Yeung, A. S. (1997). Causal effects of academic self-concept on academic achievement: Structural equation models of longitudinal data. Journal of Educational Psychology, 89(1), 41–54.

    Article  Google Scholar 

  • Ministry of Education, China. (2000). National mathematics curriculum standards (compulsory education). Beijing: Beijing Normal University Press.

    Google Scholar 

  • Ministry of Education, Singapore. (2013). O- & N(A)-level mathematics teaching and learning syllabus. Singapore: Curriculum Planning Division.

    Google Scholar 

  • Mullis, I. V. S., Martin, M. O., Foy, P., & Arora, A. (2012). TIMSS 2011 International results in mathematics. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College.

    Google Scholar 

  • National Center on Education and Economy [NCEE]. (2014). Chinese lessons: Shanghai’s rise to the top of the PISA league tables. Washington, DC: Centre on International Education Benchmarking.

    Google Scholar 

  • Organization for Economic Co-operation and Development [OECD]. (2009). PISA data analysis manual: SPSS ® Second Edition. Paris: OECD Publishing.

    Google Scholar 

  • Organization for Economic Co-operation and Development [OECD]. (2013). PISA 2012 results: Ready to learn: Students’ engagement, drive and self-beliefs (Volume III). Paris: OECD Publishing.

    Google Scholar 

  • Organization for Economic Co-operation and Development [OECD]. (2014a). PISA 2012 results: What students know and can do: Student performance in mathematics, reading and science (Volume I, Revised Edition). Paris: OECD Publishing.

    Google Scholar 

  • Organization for Economic Co-operation and Development [OECD]. (2014b). PISA 2012 Technical Report. Paris: OECD Publishing.

    Google Scholar 

  • Organization for Economic Co-operation and Development [OECD]. (2014c). PISA 2012 results: Ready to learn: Students’ engagement, drive and self-beliefs (Volume III, Revised Edition). Paris: OECD Publishing.

    Google Scholar 

  • Pietsch, J., Walker, R., & Chapman, E. (2003). The relationship among self-concept, self-efficacy, and performance in mathematics during secondary school. Journal of Educational Psychology, 95(3), 589–603.

    Article  Google Scholar 

  • Robinson, W. S. (1950). Ecological correlations and the behaviors of the individuals. American Sociological Review, 15(3), 351–357.

    Article  Google Scholar 

  • Stillman, G., Cheung, K. C., Mason, R., Sheffield, L., Sriraman, B., & Ueno, K. (2009). Challenging mathematics classroom practices. In E. J. Barbeau & P. J. Taylor (Eds.), New ICMI Study Series 12: Challenging mathematics in and beyond the classroom (pp. 243–283). New York: Springer.

    Chapter  Google Scholar 

  • von Davier, M., Gonzalez, E., & Mislevy, R. J. (2009). What are plausible values and why they are useful? In M. Von Davier & D. Hastedt (Eds.), IERI monograph series: Issues and methodologies in large scale assessments (Vol. 2, pp. 9–36). Hamburg: IERI Institute.

    Google Scholar 

  • Watkins, D., & Cheung, S. (1995). Culture, gender and response bias: An analysis of responses to the self-description questionnaire. Journal of Cross-Cultural Psychology, 26(5), 490–504.

    Article  Google Scholar 

  • Wigfield, A., & Eccles, J. S. (2000). Expectancy-value theory of achievement motivation. Contemporary Educational Psychology, 25, 68–81.

    Article  Google Scholar 

  • Wilbur, B. B., Shailer, T., & Ann, P. (1964). Self-concept of ability and school achievement. Sociology of Education, 37(3), 271–278.

    Article  Google Scholar 

  • Wu, M. (2005). The roles of plausible values in large scale surveys. Studies in Educational Evaluation, 31, 114–128.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Education, University of Macau, Taipa, China

    Kwok-cheung Cheung & Pou-seong Sit

  2. Educational Testing and Assessment Research Centre, and Macao-PISA Centre, University of Macau, Taipa, China

    Soi-kei Mak

  3. Faculty of Education, University of Macau, Room 2016, E33, Avenida da Universidade, Taipa, Macao, China

    Kwok-cheung Cheung

Authors
  1. Kwok-cheung Cheung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Soi-kei Mak
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pou-seong Sit
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kwok-cheung Cheung.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cheung, Kc., Mak, Sk. & Sit, Ps. Resolving the attitude–achievement paradox based on anchoring vignettes: evidences from the PISA 2012 mathematics study. Asia Pacific Educ. Rev. 19, 389–399 (2018). https://doi.org/10.1007/s12564-018-9526-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12564-018-9526-9

Keywords

Search

Navigation