Skip to main content

Ecological rationality in teachers' conceptions of assessment across samples from Cyprus and New Zealand

Abstract

Teacher conceptions of assessment are influential mediators of how assessment policy initiatives are implemented in schools. Four hierarchical, intercorrelated factors (i.e., assessment for improvement, school accountability, and student accountability, and assessment as irrelevant) of how teachers' conceive of assessment have been reported. However, most studies have been conducted only in English in jurisdictions with policies of low-stakes testing. This paper extends the research by surveying 249 Greek-Cypriot teachers with a Greek translation of the Teachers' Conceptions of Assessment inventory. Cyprus has a relatively low-stakes assessment policy during the compulsory school years, suggesting, under the assumption of ecological rationality, that conceptions would be similar to previous English-language studies. Exploratory factor analysis of the Cyprus data led to a five-factor solution with 24 items within two inversely correlated second-order factors (i.e., assessment is positive and negative; r = −0.49). A multigroup nested invariance confirmatory factor analysis found statistical invariance between the Cyprus and the New Zealand data. Mean score differences were small for two improvement-oriented conceptions, moderate for assessment that is bad, and large for school accountability and ignoring assessment factors. Similarities and differences in conceptions appear to reflect commonalities and discrepancies in educational system policies and practices.

This is a preview of subscription content, access via your institution.

Fig. 1

Notes

  1. 1.

    http://www.educationcounts.govt.nz/publications/schooling/teacher_census

References

  1. Adelson, J. L., & McCoach, D. B. (2010). Measuring the mathematical attitudes of elementary students: The effects of a 4-point or 5-point Likert-type scale. Educational and Psychological Measurement, 70(5), 796–807.

    Article  Google Scholar 

  2. Arbuckle, J. L. (2008). AMOS (version 17.0.0) [computer program]. Crawfordville: Amos Development Corporation.

    Google Scholar 

  3. Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7–74.

    Article  Google Scholar 

  4. Brown, G. T. L. (2001–2003). Teachers’ conceptions of assessment (TCoA) inventory (versions 1–3). Unpublished test. Auckland, NZ: University of Auckland.

  5. Brown, G. T. L. (2002). Teachers' conceptions of assessment. Unpublished doctoral dissertation. Auckland: University of Auckland.

    Google Scholar 

  6. Brown, G. T. L. (2004a). Teachers’ conceptions of assessment: Implications for policy and professional development. Assessment in Education: Principles, Policy & Practice, 11(3), 301–318.

    Article  Google Scholar 

  7. Brown, G. T. L. (2004b). Measuring attitude with positively packed self-report ratings: Comparison of agreement and frequency scales. Psychological Reports, 94, 1015–1024.

    Article  Google Scholar 

  8. Brown, G. T. L. (2006). Teachers’ conceptions of assessment: Validation of an abridged instrument. Psychological Reports, 99, 166–170.

    Google Scholar 

  9. Brown, G. T. L. (2007). Teachers’ conceptions of assessment: Comparing measurement models for primary & secondary teachers in New Zealand. Christchurch: Paper presented to the Annual Conference of the New Zealand Association for Research in Education.

    Google Scholar 

  10. Brown, G. T. L. (2008). Conceptions of assessment: Understanding what assessment means to teachers and students. New York: Nova.

    Google Scholar 

  11. Brown, G. T. L. (2009). Teachers’ self-reported assessment practices and conceptions: Using structural equation modelling to examine measurement and structural models. In T. Teo & M. S. Khine (Eds.), Structural equation modeling in educational research: Concepts and applications (pp. 243–266). Rotterdam: Sense Publishers.

    Google Scholar 

  12. Brown, G. T. L. (2011). Teachers’ conceptions of assessment: Comparing primary and secondary teachers in New Zealand. Assessment Matters, 3, in press.

  13. Brown, G. T. L., & Harris, L. R. (2009). Unintended consequences of using tests to improve learning: How improvement-oriented resources engender heightened conceptions of assessment as school accountability. Journal of MultiDisciplinary Evaluation, 6(12), 68–91.

    Google Scholar 

  14. Brown, G. T. L., Kennedy, K. J., Fok, P. K., Chan, J. K. S., & Yu, W. M. (2009). Assessment for improvement: Understanding Hong Kong teachers’ conceptions and practices of assessment. Assessment in Education: Principles, Policy and Practice, 16(3), 347–363.

    Article  Google Scholar 

  15. Brown, G. T. L., Lake, R., & Matters, G. (2011). Queensland teachers’ conceptions of assessment: The impact of policy priorities on teacher attitudes. Teaching and Teacher Education, 27(1), 210–220.

    Article  Google Scholar 

  16. Byrne, B. M., Shavelson, R. J., & Muthen, B. (1989). Testing for the equivalence of factor covariance and mean structures—the issue of partial measurement invariance. Psychological Bulletin, 105(3), 456–466.

    Article  Google Scholar 

  17. Chen, F., Bollen, K. A., Paxton, P., Curran, P. J., & Kirby, J. B. (2001). Improper solutions in structural equation models: Causes, consequences, and strategies. Sociological Methods & Research, 29(4), 468–508.

    Article  Google Scholar 

  18. Cheung, T. K. Y. (2008). An assessment blueprint in curriculum reform. Journal of Quality School Education, 5, 23–37.

    Google Scholar 

  19. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255.

    Article  Google Scholar 

  20. Croft, A. C., Strafford, E., & Mapa, L. (2000). Stocktake/evaluation of existing diagnostic tools in literacy and numeracy in English. Wellington: NZCER.

    Google Scholar 

  21. Crooks, T. J. (2002). Educational assessment in New Zealand schools. Assessment in Education: Principles Policy & Practice, 9(2), 237–253.

    Article  Google Scholar 

  22. Cyprus Ministry of Education and Culture (2002). Elementary education curriculum Programs [Analytika programmata dimotikis ekpedefsis]. Nicosia: Ministry of Education and Culture.

    Google Scholar 

  23. Cyprus Ministry of Education and Culture (2010). Guide to the pancyprian examinations 2010 [Odigos Pagkyprion Exetaseon]. Nicosia: Ministry of Education and Culture.

    Google Scholar 

  24. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood estimation from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society, Series B, 39(1), 1–38.

    Google Scholar 

  25. Fan, X., & Sivo, S. A. (2007). Sensitivity of fit indices to model misspecification and model types. Multivariate Behavioral Research, 42(3), 509–529.

    Article  Google Scholar 

  26. Fiske, E. B., & Ladd, H. F. (2000). When schools compete: A cautionary tale. Washington, DC: Brookings Institution Press.

    Google Scholar 

  27. Hamilton, L. (2003). Assessment as a policy tool. Review of Research in Education, 27(1), 25–68.

    Article  Google Scholar 

  28. Harris, L. R., & Brown, G. T. L. (2009). The complexity of teachers’ conceptions of assessment: Tensions between the needs of schools and students. Assessment in Education: Principles, Policy and Practice, 16(3), 365–381.

    Article  Google Scholar 

  29. Hill, M. F. (2000). Remapping the assessment landscape: primary teachers reconstructing assessment in self-managing schools. Unpublished doctoral dissertation, University of Waikato, Hamilton, NZ.

  30. Himmelfarb, S. (1993). The measurement of attitudes. In A. H. Eagly & S. Chaiken (Eds.), The psychology of attitudes (pp. 23–87). Belmont: Thomson Wadsworth.

    Google Scholar 

  31. Klockars, A. J., & Yamagishi, M. (1988). The influence of labels and positions in rating scales. Journal of Educational Measurement, 25(2), 85–96.

    Article  Google Scholar 

  32. Krosnick, J. A., Judd, C. M., & Wittenbrink, B. (2005). The measurement of attitudes. In D. Albarracín, B. T. Johnson, & M. P. Zanna (Eds.), The handbook of attitudes (pp. 21–76). New York: The Psychology Press.

    Google Scholar 

  33. Lam, T. C. M., & Klockars, A. J. (1982). Anchor point effects on the equivalence of questionnaire items. Journal of Educational Measurement, 19(4), 317–322.

    Article  Google Scholar 

  34. Levin, B. (2001). Reforming education: From origins to outcomes. London: RoutledgeFalmer.

    Google Scholar 

  35. MacCallum, R. C. (1995). Model specification: Procedures, strategies, and related issues. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 16–36). Thousand Oaks: Sage.

    Google Scholar 

  36. Marsh, H. W., Hau, K.-T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11(3), 320–341.

    Article  Google Scholar 

  37. Maruyama, G. M. (1998). Basics of structural equation modeling. Thousand Oaks: Sage.

    Google Scholar 

  38. Michaelides, M. P. (2008). Test-takers perceptions of fairness in high-stakes examinations with score transformations, Paper presented at the biannual meeting of the International Test Commission. Liverpool, UK.

    Google Scholar 

  39. New Zealand Ministry of Education (1994). Assessment: Policy to practice. Wellington: Learning Media.

    Google Scholar 

  40. New Zealand Ministry of Education (2007). The New Zealand curriculum for English-medium teaching and learning in years 1–13. Wellington: Learning Media.

    Google Scholar 

  41. New Zealand Ministry of Education (2010). National Standards. Retrieved February 20 2010, from http://nzcurriculum.tki.org.nz/National-Standards.

  42. Popham, W. J. (2000). Modern educational measurement: Practical guidelines for educational leaders (6th ed.). Boston: Allyn & Bacon.

    Google Scholar 

  43. Rieskamp, J., & Reimer, T. (2007). Ecological rationality. In R. F. Baumeister & K. D. Vohs (Eds.), Encyclopedia of social psychology (pp. 273–275). Thousand Oaks: Sage.

    Google Scholar 

  44. Statistical Service (2009). Statistics of Education 2007/2008 (Series 1, Report no. 40). Nicosia, Cyprus: Printing Office of the Republic of Cyprus.

  45. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(4), 4–70.

    Article  Google Scholar 

  46. Wilson, E. B., & Hilferty, M. M. (1931). The distributions of chi-square. Proceedings of the National Academy of Sciences, 17(12), 684–688.

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Gavin Thomas Lumsden Brown.

Additional information

Gavin T. L. Brown. Department of Psychological Studies, the Hong Kong Institute of Education, 10 Lo Ping Road, Tai Po, New Territories, Hong Kong SAR, China. E-mail: gtlbrown@ied.edu.hk

Current themes of research:

Teacher and student attitudes, values, and beliefs about assessment. Cross-cultural studies.

Most relevant publications in the field of Psychology of Education:

Brown, G. T. L., Lake, R., & Matters, G. (2011). Queensland teachers' conceptions of assessment: The impact of policy priorities on teacher attitudes. Teaching and Teacher Education, 27(1), 210–220. http://dx.doi.org/10.1016/j.tate.2010.08.003.

Peterson, E. R., Brown, G. T. L., & Irving, S. E. (2010). Secondary school students' conceptions of learning and their relationship to achievement. Learning and Individual Differences, 20(3), 167–176.

McInerney, D. M., Brown, G. T. L., & Liem, G. A. D. (Eds.). (2009). Student perspectives on assessment: What students can tell us about assessment for learning. Charlotte, NC: Information Age Publishing.

Brown, G. T. L., Kennedy, K. J., Fok, P. K., Chan, J. K. S., & Yu, W. M. (2009). Assessment for improvement: Understanding Hong Kong teachers' conceptions and practices of assessment. Assessment in Education: Principles, Policy and Practice, 16(3), 347–363.

Brown, G. T. L. (2008). Conceptions of assessment: Understanding what assessment means to teachers and students. New York: Nova Science Publishers.

Michalis P. Michaelides. Educational Research/Measurement and Evaluation, Department of Education Sciences, European University Cyprus, 6, Diogenes Str., Engomi, P.O. Box 22006, 1516 Nicosia, Cyprus. E-mail: M.Michaelides@euc.ac.cy

Current themes of research:

Item analysis within the context of item response theory (IRT). Statistical problems in test equating. Perceptions about assessment and testing by major stakeholders. Test development and adaptation.

Most relevant publications in the field of Psychology of Education:

Michaelides, M. P. (2010). A review of the effects on IRT item parameter estimates with a focus on misbehaving common items in test equating. Frontiers in Quantitative Psychology and Measurement, 1 doi:10.3389/fpsyg.2010.00167.

Michaelides, M. P. (2010). Sensitivity of equated aggregate scores to the treatment of misbehaving common items. Applied Psychological Measurement, 34(5), 365–369.

Michaelides, M. P. (2008). An illustration of a Mantel-Haenszel procedure to flag misbehaving common items in test equating. Practical Assessment Research & Evaluation, 13(7). Available online: http://pareonline.net/getvn.asp?v=13&n=7.

Morgan, D. L., & Michaelides, M. P. (2005). Setting cut scores for college placement. College Board Research Report No. 2005-9.

Appendix A

Appendix A

Table 4 Model D factors and items in English and Greek

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Brown, G.T.L., Michaelides, M.P. Ecological rationality in teachers' conceptions of assessment across samples from Cyprus and New Zealand. Eur J Psychol Educ 26, 319–337 (2011). https://doi.org/10.1007/s10212-010-0052-3

Download citation

Keywords

  • Teacher beliefs
  • Educational assessment
  • Survey research
  • Cross-culture comparison
  • Cyprus
  • New Zealand