Abstract
Research on test change often documents high-stakes English test impact on English language learning, whereas evidence for simultaneous impact on affective predictors of learning is still missing. We tested a theoretical model positing that changing high-stake English tests (English Language Requirements for University Entrance, in this study) influences learners’ attitude, perceived test-anxiety, learning stress, and motivation for learning, and that perceived motivation, test-anxiety, and learning stress mediate the link between attitude and achievement (English language achievement). Results of a structural equation modeling (SEM) confirmed the model yielding support for the strong total effects of attitude and motivation on achievement (N = 468 EFL learners). Under such a program, test-anxiety and learning stress negatively correlated with attitude, mediated the effects of attitude on achievement, and their joint effect negatively influenced motivation having the second strongest direct effect on achievement. Motivation was also documented as important mediator of the effects of attitude, text-anxiety, and learning stress on achievement. The model was used as a means to validate the new tests in terms of their perceived consequences and effectiveness and, in effect, helped observe a discrepancy between the idealized policy behind the change and the real practices within the change context. Educational implications are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alderson, J. C., & Banerjee, J. (2001). Language testing and assessment. Language Teaching, 34, 213–236.
Alderson, J. C., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14(2), 115–129.
Andrews, S., Fullilove, J., & Wong, Y. (2002). Targeting washback: a case study. System, 30, 207–223.
Arbuckle, J. L. (2009). IBM SPSS Amos 18 user’s guide. Chicago: IBM.
Arbuckle, J. L., & Wothke, W. (1999). Amos 4.0 user’s guide. Chicago: Small Waters.
Atay, D., & Kurt, G. (2010). The socio-educational model of second language acquisition: the Turkish context. Procedia, 2, 3088–3093.
Author. (2013). Unpublished PhD dissertation. Modares: University of Tarbiat.
Authors (2012). International Journal of Humanities.
Authors (2013). Higher Education.
Aydin, S., Yavuz, F., & Yesilyurt, S. (2006). Test anxiety in foreign language learning. Journal of Social Sciences Institute, 9(16), 145–160.
Bachman, L. F., & Palmer, A. (2010). Language assessment in practice. Oxford: Oxford University Press.
Bailey, K. M. (1996). Working for washback: a review of the washback concept in language testing. Language Testing, 13(3), 257–279.
Balogun, A. G., Balogun, S. K., & Oneyncho, C. V. (2017). Test anxiety and academic performance among undergraduates: the moderating role of achievement motivation. The Spanish Journal of Psychology, 20, E.14. https://doi.org/10.1017/sjp.2017.5.
Black, P. (1998). Testing: friends or foe? The theory and practice of assessment and testing. London: Falmer Press.
Bodas, J. (2003). The mediatory role of anxiety in predicting academic achievement in children. Unpublished M.S. dissertation. Blacksburg, Virginia.
Brown, J. D. (1995). The elements of language curriculum: a systematic approach to program development. New York: Heinle & Heinle.
Burstall, C. (1975). Factors affecting foreign-language learning: a consideration of some recent research findings. Language Teaching, 8, 5–25.
Byrne, B. M. (1994). Structural equation modeling with EQS and EQS/Windows. Thousand Oaks: Sage Publication.
Carless, D. (1997). Managing systematic change: a critical analysis of Hong Kong’s target-oriented curriculum initiative. International Review of Education, 43(4), 349–366.
Cassady, J. C. (2004). The influence of cognitive test anxiety across the learning–testing cycle. Learning and Instruction, 14(6), 569–592.
Cassady, J. C., & Johnson, R. E. (2002). Cognitive test anxiety and academic performance. Contemporary Educational Psychology, 27, 270–295. https://doi.org/10.1006/ceps.2001.1094.
Chapman, D., & Snyder, C. W. (2000). Can high stakes national testing improve instruction: reexamining conventional wisdom. International Journal of Educational Development, 20, 457–474.
Cheng, L. (1998). Impact of public English examination change on students’ perceptions and attitudes toward their English learning. Studies in Educational Evaluation, 24(3), 279–301.
Cheng, L., & Curtis, A. (2004). Washback or backwash: a review of the impact of testing on teaching and learning. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback in language testing: research contexts and methods (pp. 3–17). Mahwah, New Jersey: Lawrence Erlbaum.
Cheng, L., Klinger, D., Fox, J., Doe, C., Jin, Y., & Wu, J. (2014). Motivation and test anxiety in test performance across three testing contexts: the CAEL, CET and GEPT. TESOL Quarterly, 48(2), 300–330.
Choi, I. (2008). The impact of EFL testing on EFL education in Korea. Language Testing, 25(1), 39–62.
Cortés, C. M. (2002). The relationships between attitude, motivation, anxiety, and proficiency in English as a second language of first-year university students in Puerto Rico. Unpublished Ph.D. dissertation, Andrews University.
Couros, A. (2003). Innovation, change theory and the acceptance of new technologies: a literature review. Retrieved November, 2016 from http://www.Educationaltechnology.ca/couros/publication files/unpublishedpapers/change_theory.pdf.
Cronbach, L. (1982). Designing evaluations of educational and social programs. San Francisco: Jossey-Bass.
Davies, A. (1985). Follow my leader: is that what language tests do? In Y. P. Lee, C. Y. Y. Fok, R. Lord, & G. Low (Eds.), New directions in language testing (pp. 1–12). Oxford: Pergamon Press.
Decker, D. M., & Bolt, S. E. (2008). Challenges and opportunities for promoting student achievement through large-scale assessment results: research, reflections, and future directions. Assessment for Effective Intervention, 34(1), 43–51.
Den Brok, P. (2001). Teaching and student outcomes. Utrecht: W. C. C.
Donaldson, S. I. (2007). Program theory-driven evaluation science: strategies and applications. New York: Lawrence Erlbaum Associates.
Dörnyei, Z. (1994). Motivation and motivating in the foreign language classroom. Modern Language Journal, 78, 273–284.
Dörnyei, Z. (2003). Attitudes, orientations, and motivation in language learning: advances in theory, research, and applications. Language Learning, 53, 3–32.
Fakeye, D. (2010). Students’ personal variables as correlates of academic achievement in English as a second language in Nigeria. Journal of Social Sciences, 22(3), 205–211.
Finch, A. E. (2000). A formative evaluation of a task-based EFL programme for Korean university students. Unpublished Ph.D. dissertation, University of Manchester.
Frederiksen, J., & Collins, A. (1989). A system approach to educational testing. Educational Researcher, 18, 27–32.
Gardner, R. C. (1985). Social psychology and second language learning: the role of attitudes and motivation. London: Edward Arnold Publishers.
Gardner, R. C. (2004). Attitude/motivation test battery: international AMTB research project. Canada: The University of Western Ontario.
Gardner, R. C. (2005). Integrative motivation and second language acquisition. Joint plenary talk at Canadian Association of Applied Linguistics/Canadian Linguistics Association, May.
Gardner, R. C. (2007). Motivation and second language acquisition. Porta Linguarum, 8, 9–20.
Gardner, R. C. (2010). Motivation and second language acquisition: the socio-educational model. New York: Peter Lang Publishing Inc..
Gardner, R. C., & MacIntyre, P. D. (1991). An instrumental motivation in language study: who says it isn’t effective? Studies in Second Language Acquisition, 13, 57–72.
Gardner, R. C., Tremley, P. F., & Masgoret, A. M. (1997). Towards a full model of second language learning: an empirical investigation. Modern Language Journal, 81, 344–362.
Hajforoush, A. (2002). Negative consequences of entrance exams on instructional objectives and a proposal for removing them. Evaluating the issues of the entrance exams: Proceedings of the Esfahan University Conference on, Esfahan University, May.
Hamilton, L. S., Stecher, B. M., & Klein, S. P. (2002). Making sense of test-based accountability. Arlington: RAND Publication.
Hancock, D. R. (2001). Effects of test anxiety and evaluative threat on students' achievement and motivation. Journal of Educational Research, 94, 284–290.
Hannon, B. (2016). General and non-general intelligence factors simultaneously influence SAT, SAT-V, and SAT-M performance. Intelligence, 59, 51–63.
Hargreaves, A., Earl, L., & Schmidt, M. (2002). Perspectives on alternative assessment reform. American Educational Research Journal, 39(1), 69–95.
Hembree, R. (1988). Correlates, causes, effects and treatment of test anxiety. Review of Educational Research, 88(1), 47–77.
Hill, K., & McNamara, T. F. (2011). Assessment research in second language curriculum initiatives. In J. P. Keeves & R. Watanabe (Eds.), International handbook of educational research in the Asia-Pacific Region (Vol. 2003, pp. 629–640). Dordrecht: Kluwer Academic.
Horwitz, E. K. (2000). It ain’t over til it’s over: on foreign language anxiety, first language deficits, and the confounding of variables. Modern Language Journal, 84, 256–259.
Horwitz, E. K., Horwitz, M. B., & Cope, J. (1986). Foreign language classroom anxiety. The Modern Language Journal, 70(2), 125–132.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.
Kane, M. (2002). Validating high-stakes testing program. Educational Measurement: Issues and Practice, 21(1), 31–41.
Kane, M. (2006). Validation. In R. Bernnan (Ed.), Educational measurement (4th ed., pp. 17–64). Westport: American Council on Education and Praeger.
Kane, M. (2012). Validating score interpretations and uses. Language Testing, 29(1), 3–17.
Kane, M. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50, 1–73.
Kennedy, C., & Edwards, C. (1998). ELT management. Birmingham: Centre for English Language Studies.
Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York: Guilford Press.
Lane, S. (2014). Validity evidence based on testing consequences. Psicothema, 26(1), 127–135.
Liu, H. (2012). Understanding EFL undergraduate anxiety in relation to motivation, autonomy, and language proficiency. Electronic Journal of Foreign Language Teaching, 9(1), 23–139.
Liu, M., & Huang, W. (2011). An exploration of foreign language anxiety and English learning motivation. Education Research International, 12, 1–8.
Lucas, R. I., Miraflores, E., & Go, D. (2011). English language learning anxiety among foreign language learners in the Philippines. The Philippine ESL Journal, 7, 94–119.
Ma, J., & Cheng, L. (2016). Chinese students’ perceptions of the value of test preparation courses for the TOEFL iBT: merit, worth and significance. TESL Canada Journal, 33(1), 58–79.
MacIntyre, P., & Gregersen, T. (2012). Emotions that facilitate language learning: the positive-broadening power of the imagination. Studies in Second Language Learning and Teaching, 2(2), 193–213.
Markee, N. (1990). The diffusion of communicative innovations and classroom culture: an ethnographic study. University of Illinois at Urbana-Champaign.
Masgoret, A. M., & Gardner, R. C. (2003). Attitudes, motivation and second language learning: a meta-analysis of studies conducted by Gardner and Associates. Language Learning, 53, 123–163.
Matt, R. L., & Knight, P. A. (1994). The impact of perceived group success-failure on motivational beliefs and attitudes: a causal model. Journal of Applied Psychology, 79(5), 555–766.
McDonald, J. (2005). Test anxiety in middle school students. Retrieved Nov, 18, 2017 from http: //voices.yahoo.com/test-anxiety-middle-school-students-11295.html?cat=4.
McLean, L., Sparapani, N., Toste, J. C., & Connor, K. (2016). Classroom quality as a predictor of first graders’ time in non-instructional activities and literacy achievement. Journal of School Psychology, 56, 45–58.
Merki, M., & Oerke, B. (2017). Long-term effects of the implementation of state-wide exit exams: a multilevel regression analysis of mediation effects of teaching practices on students’ motivational orientations. Educational Assessment, Evaluation and Accountability, 29(1), 23–54.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York: American Council on Education and Macmillan.
Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23, 13–23.
Messick, S. (1996). Validity and washback in language testing. Language Testing, 13(3), 241–256.
Murray, J., Riazi, M., & Cross, J. (2012). Test candidates' attitudes and their relationship to demographic and experiential variables: the case of overseas trained teachers in NSW, Australia. Language Testing, 29(4), 577–595.
Pae, T.-L. (2008). Second language orientation and self-determination theory: a structural analysis of the factors affecting second language achievement. Journal of Language and Social Psychology, 27(5), 4–27.
Pan, Y.-C., & Newfields, T. (2012). Tertiary EFL proficiency graduation requirements in Taiwan: a study of washback on learning. Electronic Journal of Foreign Language Teaching, 9(1), 108–122.
Papi, M. (2010). The L2 motivational self-system, L2 anxiety, and motivated behavior: a structural equation modeling approach. System, 38, 467–479.
Park, B. (2014). Cognitive and affective processes in multimedia learning. Learning and Instruction, 29, 125–127.
Pearson, L. (1988). Tests as levers for change. In D. Chamberlain & R. Baumgardner (Eds.), ESP in the classroom: practice and evaluation (Vol. 128, pp. 98–107). London: Modern English Publications.
Pekrun, R. (1992). The impact of emotions on learning and achievement: towards a theory of cognitive/motivational mediators. Applied Psychology, 41, 359–376.
Pekrun, R., Goetz, T., Titz, W., & Perry, R. (2002). Academic emotions in students’ self-regulated learning and achievement: a program of qualitative and quantitative research. Educational Psychologist, 37(2), 91–106.
Prenzel, M. (2007). Studies on the educational quality of schools: the final report on the DFG priority programme. Berlin: Waxmann.
Sarason, I. G. (1984). Stress, anxiety, and cognitive interference: reactions to tests. Journal of Personality and Social Psychology, 46, 929–938.
Shepard, L. A. (1997). The centrality of test use and consequences for test validity. Educational Measurement: Issues and Practice, 16, 5–24.
Shohamy, E. (1992). Beyond proficiency testing: a diagnostic feedback testing model for assessing foreign language learning. Modern Language Journal, 76(4), 513–521.
Shohamy, E. (1993). The power of tests: the impact of language tests on teaching and learning. NFLC occasional paper. College Park, MD: National Foreign Language Center, University of Maryland.
Shrout, P., & Bolger, N. (2002). Mediation in experimental and nonexperimental studies: new procedures and recommendations. Psychological Methods, 7(4), 422–445.
Smyth, E., & Banks, J. (2012). High stakes testing and student perspectives on teaching and learning in the Republic of Ireland. Educational Assessment, Evaluation and Accountability, 24(4), 283–366.
Stiggins, J. R. (2002). Assessment crisis: the absence of assessment for learning. Phi Delta Kappan. Retrieved from: https://www.pdkintl.org/kappan/k0206sti.htm, 83, 758, 765.
Stoller, F. (1994). The diffusion of innovations in intensive ESL programmes. Applied Linguistics, 15(30), 300–327.
Strong, M. (1984). Integrative motivation: cause or result of successful second language acquisition. Language Learning, 34(3), 1–13.
Stufflebeam, D., & Shinkfield, A. J. (2007). Evaluation theory, models, and applications. San Francisco: Jossey-Bass.
Taut, S., & Palacios, D. (2016). Intended and unintended interpretations and uses of PISA results: a consequential validity perspective. Relive, 22(1), 1–14.
The Act of Students Admission to Universities (2007), (In Farsi). The Parliament of the Islamic Republic of Iran. Doc. 132730.
Von der Embse, N. P., & Witmer, S. E. (2014). High-stakes accountability: student anxiety and large-scale testing. Journal of Applied School Psychology, 3(2), 132–156.
Wall, D. (2000). The impact of high-stakes testing on teaching and learning: can this be predicted or controlled. System, 28, 499–509.
Wall, D. & Horak, T. (2008). The impact of changes in the TOFEL examination on teaching and learning in central and eastern Europe: phase 2, coping with change. ETS, TOEFL.
Watanabe, Y. (2001). Does the university entrance examination motivate learners? A case study of learner interviews. In Akita Association of English Studies (Ed.), Trans-equator exchanges: a collection of academic papers in honor of Professor David Ingram (pp. 100–110). Akita: Author.
Winke, P. (2011). Evaluating the validity of a high-stakes ESL test: why teachers' perceptions matter. TESOL Quarterly, 45(4), 628–660.
Zeidner, M. (1998). Test-anxiety: the state of the art. New York: Plenum Press.
Zhao, J., & Cheng, L. (2010). Exploring the relationship between Chinese university students’ attitude towards the college English test and their test performance. In L. Cheng & A. Curtis (Eds.), English language assessment and the Chinese learner (pp. 190–201). New York: Routledge, Taylor & Francis Group.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix 1
Appendix 1
1.1 Results for model testing
1.2 Results for measurement models (QACC)
Parameter estimates of the standardized factor loadings (estimate CFA), standard error (SE), critical ratio (CR), p value (P), and squared multiple correlations (SMC) for the measurement model (QACC, learning objectives scale)
Items | Estimate (CFA) | S.E. | C.R. | P | SMC |
|---|---|---|---|---|---|
Aff.1 ← Affective.OBJ | 0.635 | 0.403 | |||
Aff.2 ← Affective.OBJ | 0.680 | 0.066 | 17.120 | *** | 0.664 |
Aff.3 ← Affective.OBJ | 0.631 | 0.069 | 16.665 | *** | 0.075 |
Aff.4 ←Affective.OBJ | 0.689 | 0.074 | 17.755 | *** | 0.163 |
cog.1 ←Cognitive.OBJ | 0.629 | 0.304 | |||
cog.2 ←Cognitive.OBJ | 0.693 | 0.051 | 21.720 | *** | 0.132 |
cog.3 ← Cognitive.OBJ | 0.638 | 0.062 | 16.130 | *** | 0.408 |
cog.4 ← Cognitive.OBJ | 0.363 | 0.068 | 9.961 | *** | 0.481 |
cog.5 ← Cognitive.OBJ | 0.552 | 0.073 | 14.690 | *** | 0.396 |
cog.6 ← Cognitive.OBJ | 0.404 | 0.091 | 11.378 | *** | 0.474 |
cog.7 ← Cognitive.OBJ | 0.274 | 0.086 | 7.889 | *** | 0.398 |
cog.8 ← Cognitive.OBJ | 0.815 | 0.077 | 17.899 | *** | 0.462 |
cog.9 ← Cognitive.OBJ | 0.635 | 0.065 | 16.866 | *** | 0.403 |
Items | Estimate (CFA) | S.E. | CR | P | SMC |
|---|---|---|---|---|---|
Fair.15 ← Fairness | 0.526 | 0.277 | |||
Fair.16 ← Fairness | 0.424 | 0.053 | 15.504 | *** | 0.180 |
Fair.17 ← Fairness | 0.998 | 0.088 | 20.873 | *** | 0.997 |
Fair.18 ← Fairness | 0.506 | 0.060 | 15.652 | *** | 0.256 |
Fair.19 ← Fairness | 1.001 | 0.088 | 20.851 | *** | 0.998 |
Fair.20 ← Fairness | 0.454 | 0.055 | 15.845 | *** | 0.206 |
1.3 Results for the main model of the study (model 3)
Parameter estimates of the standardized factor loadings (estimate), standard error (SE), critical ratio (CR), and p value (P) for the structural model (SEM analysis of the change program model: model 3). Bootstrapping estimates are also reported.
Path | Maximum-likelihood | Bootstrap | ||||
|---|---|---|---|---|---|---|
Estimate | S.E. | CR. | p | Estimate/M | S.E. | |
TestAnxiety ← Attitude | − 0.236 | 0.153 | 1.544 | 0.048 | − 0.114 | 0.151 |
E.Class.Stress ← Attitude | − 0.341 | 0.059 | − 1.887 | 0.049 | − 0.215 | 0.049 |
E.Class.Stress ← TestAnxiety | 0.363 | 0.020 | 7.022 | *** | 0.490 | 0.022 |
Motivation ← Attitude | 0.309 | 0.164 | 1.881 | 0.046 | 0.185 | 0.121 |
Motivation ← TestAnxiety | − 0.192 | 0.044 | 3.481 | *** | − 0.143 | 0.052 |
Motivation ← E.Class.Stress | − 0.657 | 0.219 | 1.796 | 0.047 | − 0.751 | 0.376 |
Achievement ← Attitude | 0.411 | 0.149 | 3.150 | 0.002 | 0.231 | 0.153 |
Achievement ← Motivation | 0.288 | 0.120 | 1.369 | 0.045 | 0.211 | 0.132 |
Achievement ← TestAnxiety | − 0.221 | 0.057 | − 2.406 | 0.016 | − 0.191 | 0.063 |
Achievement ← E.Class.Stress | − 0.189 | 0.111 | − 1.154 | 0.248 | − 0.107 | 0.533 |
Att.Aff ← Attitude | 0.721 | 0.725 | ||||
Att.Cog ← Attitude | 0.696 | 0.189 | 8.905 | *** | 0.704 | 0.187 |
Att.Fair ← Attitude | 0.573 | 0.136 | 8.707 | *** | 0.569 | 0.135 |
Bod.Sym ← TestAnxiety | 0.841 | 0.842 | ||||
Test.Ir.Th ← TestAnxiety | 0.683 | 0.045 | 18.712 | *** | 0.678 | 0.047 |
Wor ← TestAnxiety | 0.854 | 0.045 | 21.574 | *** | 0.847 | 0.046 |
Ten ← TestAnxiety | 0.892 | 0.048 | 22.659 | *** | 0.891 | 0.049 |
Des.to.le ← Motivation | 0.851 | 0.865 | ||||
Mot.Inte ← Motivation | 0.729 | 0.092 | 9.215 | *** | 0.733 | 0.054 |
Att.t.En ← Motivation | 0.818 | 0.075 | 12.043 | *** | 0.813 | 0.050 |
NEATs2 ← Achievement | 1.000 | 0.972 | ||||
NEATs1 ← Achievement | 0.981 | 0.061 | 15.417 | *** | 0.999 | 0.062 |
Other.P ← E.Class.Stress | 0.554 | 0.551 | ||||
Self.P ← E.Class.Stress | 0.579 | 0.211 | 1.950 | 0.050 | 0.580 | 0.264 |
Rights and permissions
About this article
Cite this article
ShayesteFar, P. A model of interplay between student English achievement and the joint affective factors in a high-stakes test change context: model construction and validity. Educ Asse Eval Acc 32, 335–371 (2020). https://doi.org/10.1007/s11092-020-09326-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11092-020-09326-8