Advertisement

Change in test-taking motivation and its relationship to test performance in low-stakes assessments

  • Christiane Penk
  • Dirk Richter
Article

Abstract

Since the turn of the century, an increasing number of low-stakes assessments (i.e., assessments without direct consequences for the test-takers) are being used to evaluate the quality of educational systems. Internationally, research has shown that low-stakes test results can be biased due to students’ low test-taking motivation and that students’ effort levels can vary throughout a testing session involving both cognitive and noncognitive tests. Thus, it is possible that students’ motivation varies throughout a single cognitive test and in turn affects test performance. This study examines the change in test-taking motivation within a 2-h cognitive low-stakes test and its association with test performance. Based on expectancy-value theory, we assessed three components of test-taking motivation (expectancy for success, value, and effort) and investigated its change. Using data from a large-scale student achievement study of German ninth-graders, we employed second-order latent growth modeling and structural equation modeling to predict test performance in mathematics. On average, students’ effort and perceived value of the test decreased, whereas expectancy for success remained stable. Overall, initial test-taking motivation was a better predictor of test performance than change in motivation. Only the variability of change in the expectancy component was positively related to test performance. The theoretical and practical implications for test practitioners are discussed.

Keywords

Test-taking motivation Low-stakes tests Large-scale assessments Expectancy-value theory Growth modeling 

Notes

Acknowledgments

We thank Sara J. Finney for her enriching comments and methodological support as well as Bo Bashkov for proofreading the manuscript.

References

  1. Asseburg, R. (2011). Motivation zur Testbearbeitung in adaptiven und nicht-adaptiven Leistungstests [Test-taking motivation in adaptive and sequential achievement testing] (Doctoral dissertation). Christian-Albrechts-Universität zu Kiel. Retrieved from the website http://d-nb.info/1013153863/34.
  2. Barry, C. L., & Finney, S. J. (2016). Modeling change in effort across a low-stakes testing session: a latent growth curve modeling approach. Applied Measurement in Education, 29(1), 46–64. doi: 10.1080/08957347.2015.1102914.CrossRefGoogle Scholar
  3. Barry, C. L., Horst, S. J., Finney, S. J., Brown, A. R., & Kopp, J. P. (2010). Do examinees have similar test-taking effort? A high-stakes question for low-stakes testing. International Journal of Testing, 10(4), 342–363. doi: 10.1080/15305058.2010.508569.CrossRefGoogle Scholar
  4. Baumert, J., & Demmrich, A. (2001). Test motivation in the assessment of student skills: the effects of incentives on motivation and performance. European Journal of Psychology of Education, 16(3), 441–462.CrossRefGoogle Scholar
  5. Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.CrossRefGoogle Scholar
  6. Browne, M. W., & Du Toit, S. H. C. D. (1992). Automated fitting of nonstandard models. Multivariate Behavioral Research, 27(2), 269–300. doi: 10.1207/s15327906mbr2702_13.CrossRefGoogle Scholar
  7. Cao, J., & Stokes, S. L. (2008). Bayesian IRT guessing models for partial guessing behaviors. Psychometrika, 73(2), 209–230. doi: 10.1007/s11336-007-9045-9.CrossRefGoogle Scholar
  8. Chen, S.-K., Yeh, Y.-C., Hwang, F.-M., & Lin, S. S. J. (2013). The relationship between academic self-concept and achievement: a multicohort–multioccasion study. Learning and Individual Differences, 23, 172–178. doi: 10.1016/j.lindif.2012.07.021.CrossRefGoogle Scholar
  9. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255. doi: 10.1207/S15328007SEM0902_5.CrossRefGoogle Scholar
  10. Cole, J. S., Bergin, D. A., & Whittaker, T. A. (2008). Predicting student achievement for low stakes tests with effort and task value. Contemporary Educational Psychology, 33(4), 609–624.CrossRefGoogle Scholar
  11. DeMars, C. E., Bashkov, B. M., & Socha, A. B. (2013). The role of gender in test-taking motivation under low-stakes conditions. Research & Practice in Assessment, 8, 69–82.Google Scholar
  12. Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53(1), 109–132. doi: 10.1146/annurev.psych.53.100901.135153.CrossRefGoogle Scholar
  13. Eklöf, H. (2006). Development and validation of scores from an instrument measuring student test-taking motivation. Educational and Psychological Measurement, 66(4), 643–656. doi: 10.1177/0013164405278574.CrossRefGoogle Scholar
  14. Eklöf, H. (2007). Test-taking motivation and mathematics performance in TIMSS 2003. International Journal of Testing, 7(3), 311–326.CrossRefGoogle Scholar
  15. Eklöf, H. (2008). Test-taking motivation on low-stakes tests: A Swedish TIMSS 2003 example. In Issues and methodologies in large-scale assessments, IERI Monograph Series (Vol. 1, pp. 9–21). Hamburg: IEA-ETS Research Institute.Google Scholar
  16. Eklöf, H. (2010a). Skill and will: test‐taking motivation and assessment quality. Assessment in Education: Principles, Policy & Practice, 17(4), 345–356. doi: 10.1080/0969594X.2010.516569.CrossRefGoogle Scholar
  17. Eklöf, H. (2010b). Student motivation and effort in the Swedish TIMSS Advanced field study. Presented at the meeting of the 4th IEA International Research Conference, Gothenburg.Google Scholar
  18. Eklöf, H., & Nyroos, M. (2013). Pupil perceptions of national tests in science: perceived importance, invested effort, and test anxiety. European Journal of Psychology of Education, 28(2), 497–510. doi: 10.1007/s10212-012-0125-6.CrossRefGoogle Scholar
  19. Eklöf, H., Pavešič, B. J., & Grønmo, L. S. (2013). A cross-national comparison of reported effort and mathematics performance in TIMSS Advanced. Applied Measurement in Education, 131127082739006. doi: 10.1080/08957347.2013.853070.
  20. Ferrer, E., Balluerka, N., & Widaman, K. F. (2008). Factorial invariance and the specification of second-order latent growth models. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 4(1), 22–36. doi: 10.1027/1614-2241.4.1.22.CrossRefGoogle Scholar
  21. Freund, P. A., & Holling, H. (2011). Who wants to take an intelligence test? Personality and achievement motivation in the context of ability testing. Personality and Individual Differences, 50(5), 723–728. doi: 10.1016/j.paid.2010.12.025.CrossRefGoogle Scholar
  22. Freund, P. A., Kuhn, J. T., & Holling, H. (2011). Measuring current achievement motivation with the QCM: short form development and investigation of measurement invariance. Personality and Individual Differences, 51(5), 629–634. doi: 10.1016/j.paid.2011.05.033.CrossRefGoogle Scholar
  23. Frey, A., Hartig, J., & Rupp, A. A. (2009). An NCME instructional module on booklet designs in large-scale assessments of student achievement: theory and practice. Educational Measurement: Issues and Practice, 28(3), 39–53. doi: 10.1111/j.1745-3992.2009.00154.x.CrossRefGoogle Scholar
  24. Ganzeboom, H. B. G., De Graaf, P. M., & Treiman, D. J. (1992). A standard international socio-economic index of occupational status. Social Science Research, 21(1), 1–56. doi: 10.1016/0049-089X(92)90017-B.CrossRefGoogle Scholar
  25. Geiser, C., Keller, B. T., & Lockhart, G. (2013). First- versus second-order latent growth curve models: some insights from latent state-trait theory. Structural Equation Modeling: A Multidisciplinary Journal, 20(3), 479–503. doi: 10.1080/10705511.2013.797832.CrossRefGoogle Scholar
  26. Hancock, G. R., Kuo, W.-L., & Lawrence, F. R. (2001). An illustration of second-order latent growth models. Structural Equation Modeling: A Multidisciplinary Journal, 8(3), 470–489. doi: 10.1207/S15328007SEM0803_7.CrossRefGoogle Scholar
  27. Horst, S. J. (2010). A mixture-modeling approach to exploring test-taking motivation in large-scale low-stakes contexts (Unpublished doctoral dissertation). Harrisonburg: James Madison University.Google Scholar
  28. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55.CrossRefGoogle Scholar
  29. Jerrim, J., & Micklewright, J. (2014). Socio-economic gradients in children’s cognitive skills: are cross-country comparisons robust to who reports family background? European Sociological Review, 30(6), 766–781. doi: 10.1093/esr/jcu072.CrossRefGoogle Scholar
  30. Kong, X. J., Wise, S. L., Harmes, J. C., & Yang, S. (2006). Motivational effects of praise in response-time-based feedback: A follow-up study of the effort-monitoring CBT. Presented at the annual meeting of the National Council on Measurement in Education, San Francisco.Google Scholar
  31. Lau, A. R., Swerdzewski, P. J., Jones, A. T., Anderson, R. D., & Markle, R. E. (2009). Proctors matter: strategies for increasing examinee effort on general education program assessments. The Journal of General Education, 58(3), 196–217. doi: 10.1353/jge.0.0045.CrossRefGoogle Scholar
  32. Moneta, G. B., & Csikszentmihalyi, M. (1996). The effect of perceived challenges and skills on the quality of subjective experience. Journal of Personality, 64(2), 275–310. doi: 10.1111/j.1467-6494.1996.tb00512.x.CrossRefGoogle Scholar
  33. Muthén, L. K., & Muthén, B. O. (1998). Mplus user’s guide (7th ed.). Los Angeles: Muthén & Muthén.Google Scholar
  34. Pant, H. A., Stanat, P., Schroeders, U., Roppelt, A., Siegle, T., & Pöhlmann, C. (2013). The IQB National Assessment Study 2012. Competencies in mathematics and the sciences at the end of secondary level I. Summary. Münster: Waxmann. Retrieved from http://www.iqb.hu-berlin.de/laendervergleich/laendervergleich/lv2012/Bericht/IQB_NationalAsse.pdf.
  35. Pekrun, R., Elliot, A. J., & Maier, M. A. (2009). Achievement goals and achievement emotions: testing a model of their joint relations with academic performance. Journal of Educational Psychology, 101(1), 115–135. doi: 10.1037/a0013383.CrossRefGoogle Scholar
  36. Penk, C., & Schipolowski, S. (2015). Is it all about value? Bringing back the expectancy component to the assessment of test-taking motivation. Learning and Individual Differences, 42, 27–35. doi: 10.1016/j.lindif.2015.08.002.CrossRefGoogle Scholar
  37. Penk, C., Pöhlmann, C., & Roppelt, A. (2014). The role of test-taking motivation for students’ performance in low-stakes assessments: an investigation of school-track-specific differences. Large-Scale Assessments in Education, 2(1). doi: 10.1186/s40536-014-0005-4
  38. Preacher, K. J., Wichmann, A. L., MacCallum, R. C., & Briggs, N. E. (2008). Latent growth curve modeling. Los Angeles: Sage.CrossRefGoogle Scholar
  39. Ramm, G., Prenzel, M., Baumert, J., Blum, W., Lehmann, R., Leutner, D., … Schiefele, U. (2006). PISA 2003: Dokumentation der Erhebungsinstrumente [Documentation of the assessment instruments]. Münster: Waxmann.Google Scholar
  40. Rheinberg, F., Vollmeyer, R., & Burns, B. D. (2001). FAM: Ein Fragebogen zur Erfassung aktueller Motivation in Lern-und Leistungssituationen [A questionnaire for the measurement of current achievement motivation in learning and achievement situations]. Diagnostica, 47(2), 57–66.CrossRefGoogle Scholar
  41. Rutkowski, L., & Svetina, D. (2014). Assessing the hypothesis of measurement invariance in the context of large-scale international surveys. Educational and Psychological Measurement, 74(1), 31–57. doi: 10.1177/0013164413498257.CrossRefGoogle Scholar
  42. Sayer, A. G., & Cumsille, P. E. (2001). Second-order latent growth models. In L. M. Collins & A. G. Sayer (Eds.), New methods for the analysis of change (1st ed., pp. 179–200). Washington: American Psychological Association.CrossRefGoogle Scholar
  43. Schunk, D. H., Pintrich, P. R., & Meece, J. L. (2008). Motivation in education: theory, research, and applications (3rd ed.). Upper Saddle River: Pearson Education.Google Scholar
  44. Spinath, B. (2012). Academic achievement. In V. S. Ramachandran (Ed.), Encyclopedia of human behavior (pp. 1–8). London: Elsevier/Academic.CrossRefGoogle Scholar
  45. Stanat, P., & Christensen, G. (2006). Where immigrant students succeed: a comparative review of performance and engagement in PISA 2003. Paris: Organisation for Economic Co-operation and Development.Google Scholar
  46. Stanat, P., & Lüdtke, O. (2013). International large-scale assessment studies of student achievement. In J. Hattie & E. M. Anderman (Eds.), International guide to student achievement (pp. 481–483). New York: Routledge.Google Scholar
  47. Sundre, D. L. (2007). The Student Opinion Scale: a measure of examinee motivation: test manual. Retrieved from the Center for Assessment and Research Studies website: http://www.jmu.edu/assessment/resources/resource_files/sos_manual.pdf.
  48. Sundre, D. L., & Kitsantas, A. (2004). An exploration of the psychology of the examinee: can examinee self-regulation and test-taking motivation predict consequential and non-consequential test performance? Contemporary Educational Psychology, 29(1), 6–26. doi: 10.1016/S0361-476X(02)00063-2.CrossRefGoogle Scholar
  49. Swerdzewski, P. J., Harmes, J. C., & Finney, S. J. (2011). Two approaches for identifying low-motivated students in a low-stakes assessment context. Applied Measurement in Education, 24(2), 162–188. doi: 10.1080/08957347.2011.555217.CrossRefGoogle Scholar
  50. Thelk, A. D., Sundre, D. L., Horst, S. J., & Finney, S. J. (2009). Motivation matters: using the Student Opinion Scale to make valid inferences about student performance. The Journal of General Education, 58(3), 129–151. doi: 10.1353/jge.0.0047.CrossRefGoogle Scholar
  51. Warm, T. A. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54(3), 427–450.CrossRefGoogle Scholar
  52. Wigfield, A. (1994). Expectancy-value theory of achievement motivation: a developmental perspective. Educational Psychology Review, 6(1), 49–78. doi: 10.1007/BF02209024.CrossRefGoogle Scholar
  53. Wigfield, A., & Eccles, J. S. (2000). Expectancy–value theory of achievement motivation. Contemporary Educational Psychology, 25(1), 68–81. doi: 10.1006/ceps.1999.1015.CrossRefGoogle Scholar
  54. Wise, S. L. (2006). An investigation of the differential effort received by items on a low-stakes computer-based test. Applied Measurement in Education, 19(2), 95–114. doi: 10.1207/s15324818ame1902_2.CrossRefGoogle Scholar
  55. Wise, S. L., & DeMars, C. E. (2005). Low examinee effort in low-stakes assessment: problems and potential solutions. Educational Assessment, 10(1), 1–17. doi: 10.1207/s15326977ea1001_1.CrossRefGoogle Scholar
  56. Wise, S. L., & Kong, X. J. (2005). Response time effort: a new measure of examinee motivation in computer-based tests. Applied Measurement in Education, 18(2), 163–183. doi: 10.1207/s15324818ame1802_2.CrossRefGoogle Scholar
  57. Wise, S. L., & Smith, L. F. (2011). A model of examinee test-taking effort. In J. A. Bovaird, K. F. Geisinger, & C. W. Buckendahl (Eds.), High-stakes testing in education: science and practice in K-12 settings (1st ed., pp. 139–153). Washington: American Psychological Association.CrossRefGoogle Scholar
  58. Wise, S. L., Bhola, D. S., & Yang, S. (2006). Taking the time to improve the validity of low-stakes tests: the effort-monitoring CBT. Presented at the annual meeting of the National Council on Measurement in Education, San Francisco.Google Scholar
  59. Wise, S. L., Pastor, D. A., & Kong, X. J. (2009). Correlates of rapid-guessing behavior in low-stakes testing: implications for test development and measurement practice. Applied Measurement in Education, 22(2), 185–205. doi: 10.1080/08957340902754650.CrossRefGoogle Scholar
  60. Wolf, L. F., & Smith, J. K. (1995). The consequence of consequence: motivation, anxiety, and test performance. Applied Measurement in Education, 8(3), 227–242. doi: 10.1207/s15324818ame0803_3.CrossRefGoogle Scholar
  61. Zilberberg, A., Finney, S. J., Marsh, K. R., & Anderson, R. D. (2014). The role of students’ attitudes and test-taking motivation on the validity of college institutional accountability tests: a path analytic model. International Journal of Testing, 14(4), 360–384. doi: 10.1080/15305058.2014.928301.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.German Institute for International Educational Research (DIPF)BerlinGermany
  2. 2.University of Potsdam, Potsdam, Germany & Institute for Educational Quality Improvement (IQB)BerlinGermany

Personalised recommendations