Skip to main content
Log in

Accuracy of teachers’ tracking decisions: short- and long-term effects of accountability

  • Published:
European Journal of Psychology of Education Aims and scope Submit manuscript

Abstract

Bias in teachers’ judgment formation and decision making has long been acknowledged. More specifically, studies have repeatedly demonstrated discrepancies between teacher ratings of minority and majority students with similar academic profiles. Studies have also demonstrated that increasing accountability reduced bias. Little is known, however, about the effect of accountability and bias on the accuracy of decisions. This study investigated the short- and long-term effect of accountability priming on the accuracy of transition decisions. It considered both the extent to which teacher decision accuracy differed for minority and majority students with similar academic profiles (accuracy bias) and differences in levels of confidence for accurate versus erroneous decisions (metacognitive judgment bias). In a longitudinal experimental design, we presented 38 primary school teachers with 9 student vignettes at 3 points in time (baseline, post priming, and 6-month follow-up), varying students’ ethnic background, and asked them to make a school tracking decision for each student. We measured decision accuracy as well as teachers’ level of confidence for each decision. Accuracy and confidence levels were combined to provide two indices of metacognitive judgment accuracy. Results confirmed the hypothesis that accuracy of decisions would improve as a result of increased level of accountability. More specifically, we found that teachers made more accurate decisions after priming, whereby ethnic background differences disappeared. In addition, teachers’ metacognitions varied, whereby after priming decision accuracy was better matched with teachers’ confidence levels. Although accuracy levels were still higher at follow-up than at pre-test, the ethnic bias recurred. This study shows that increased levels of accountability are associated with not only increased decision accuracy but also reduced metacognitive judgment bias, especially in regard to minority students. It also demonstrates accountability may be an effective way of reducing systematic errors in decision making. Findings are discussed in terms of theory and current changes in educational practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Baeriswyl, F., Wandeler, C., & Christ, O. (2008). Die Übertrittsempfehlung – zufällig oder zuverlässig? Analyse der Determinanten und Konstanz von Lehrerempfehlungen bei Schulübertritten während sieben Jahren. [School placement recommendation - random or valid? Analysis of the determinants and stability of teachers' school placement recommendations during seven years]. Schweizerische Zeitschrift für Bildungswissenschaften, 30, 549–578.

    Google Scholar 

  • Bamberg, M., Barthelemy, M., Bertemes, J., Besch, E., Boehm, B., Brunner, M., … Ugen, S. (2010). PISA 2009: Nationaler Bericht Luxemburg [PISA 2009: National Report Luxembourg]. Luxembourg: MENFP-SCRIPT et Université du Luxembourg: EMACS.

  • Brehmer, B. (1994). The psychology of linear judgment models. Acta Psychologica, 87, 137–154. doi:10.1016/0001-6918(94)90048-5.

    Article  Google Scholar 

  • Byrnes, J. P., & Miller, D. C. (2007). The relative importance of predictors of math and science achievement: an opportunity–propensity analysis. Contemporary Educational Psychology, 32, 599–629. doi:10.1016/j.cedpsych.2006.09.002.

    Article  Google Scholar 

  • Dawes, R. M., & Corrigan, B. (1974). Linear models in decision making. Psychological Bulletin, 81, 95–106. doi:10.1037/h0037613.

    Article  Google Scholar 

  • De Boer, H., Bosker, R. J., & van der Werf, M. P. C. (2010). Sustainability of teacher expectation bias effects on long-term student performance. Journal of Educational Psychology, 102, 168–179. doi:10.1037/a0017289.

    Article  Google Scholar 

  • Demaray, M. K., & Elliott, S. N. (1998). Teachers’ judgments of students’ academic functioning: a comparison of actual and predicted performances. School Psychology Quarterly, 13, 8–24. doi:10.1037/h0088969.

    Article  Google Scholar 

  • DeMeis, D. K., & Turner, R. R. (1978). Effects of students’ race, physical attractiveness, and dialect on teachers’ evaluations. Contemporary Educational Psychology, 3, 77–86. doi:10.1016/0361-476X(78)90012-7.

    Article  Google Scholar 

  • Dustmann, C. (2004). Parental background, secondary school track choice, and wages. Oxford Economic Papers, 56, 209–230. doi:10.1093/oep/gpf048.

    Article  Google Scholar 

  • Elhoweris, H., Mutua, K., Alsheikh, N., & Holloway, P. (2005). Effect of children’s ethnicity on teachers’ referral and recommendation decisions in gifted and talented programs. Remedial and Special Education, 26, 25–31. doi:10.1177/07419325050260010401.

    Article  Google Scholar 

  • Ferguson, R. F. (2003). Teachers’ perceptions and expectations and the Black-White test score gap. Urban Education, 38, 460–507. doi:10.1177/0042085903254970.

    Article  Google Scholar 

  • Ferreira, M. B., Garcia-Marques, L., Sherman, S. J., & Sherman, J. W. (2006). Automatic and controlled components of judgment and decision making. Journal of Personality and Social Psychology, 91, 797–813. doi:10.1037/0022-3514.91.5.797.

    Article  Google Scholar 

  • Fiske, S. T., & Neuberg, S. L. (1990). A continuum of impression formation from category-based to individuating processes: Influences of information and motivation on attention and interpretation. In M. P. Zanna (Ed.), Advances in experimental social psychology (Vol. 23, pp. 1–74). New York: Academic Press.

    Google Scholar 

  • Fiske, S. T., & Taylor, S. E. (1991). Social cognition (2nd ed.). New York: McGraw-Hill.

    Google Scholar 

  • Fiske, S. T., & Taylor, S. E. (2010). Social cognition: From brains to culture (International ed.). New York: McGraw-Hill.

    Google Scholar 

  • Fiske, S. T., Lin, M., & Neuberg, S. L. (1999). The continuum model. Ten years later. In S. Chaiken & Y. Trope (Eds.), Dual process theories in social psychology (pp. 231–254). New York: Guilford Press.

    Google Scholar 

  • Flavell, J. H. (1979). Metacognition and cognitive monitoring: a new area of cognitive-developmental inquiry. American Psychologist, 34, 906–911. doi:10.1037//0003-066X.34.10.906.

    Article  Google Scholar 

  • Gamoran, A. (1992). The variable effects of high school tracking. American Sociological Review, 57, 812–828.

    Article  Google Scholar 

  • Gigerenzer, G., & Todd, P. M. (1999). Fast and frugal heuristics: The adaptive toolbox. In G. Gigerenzer, P. M. Todd, & ABC Research Group (Eds.), Simple heuristics that make us smart (pp. 3–34). Oxford: Oxford University Press.

    Google Scholar 

  • Gilbert, D. T., & Hixon, J. G. (1991). The trouble of thinking: activation and application of stereotypic beliefs. Journal of Personality and Social Psychology, 60, 509–517. doi:10.1037/0022-3514.60.4.509.

    Article  Google Scholar 

  • Glock, S., & Krolak-Schwerdt, S. (2013). Does nationality matter? The impact of stereotypical expectations on student teachers’ judgments. Social Psychology of Education, 16, 111–127. doi:10.1007/s11218-012-9197-z.

    Article  Google Scholar 

  • Glock, S., Krolak-Schwerdt, S., Klapproth, F., & Böhmer, M. (2012). Improving teachers’ judgments: accountability affects teachers’ tracking decisions. International Journal of Technology and Inclusive Education, 1, 89–98.

    Google Scholar 

  • Glock, S., Krolak-Schwerdt, S., Klapproth, F., & Böhmer, M. (2013). Beyond judgment bias: how students’ ethnicity and academic profile consistency influence teachers’ tracking judgments. Social Psychology of Education, 16, 555–573. doi:10.1007/s11218-013-9227-5.

    Article  Google Scholar 

  • Glock, S., Krolak-Schwerdt, S., & Pit-ten Cate, I. M. (2015). Are school placement recommendations accurate? The effect of students’ race on teachers’ judgments and recognition memory. European Journal of Psychology of Education, 30, 169–188. doi:10.1007/s10212-014-0237-2.

  • Golding, J. M., Fowler, S. B., Long, D. L., & Latta, H. (1990). Instructions to disregard potentially useful information: the effects of pragmatics on evaluative judgments and recall. Journal of Memory and Language, 29, 212–227. doi:10.1016/0749-596X(90)90073-9.

    Article  Google Scholar 

  • Gollwitzer, P. M., & Moskowitz, G. B. (1996). Goal effects on action and cognition. In A. W. Kruglanski & E. T. Higgins (Eds.), Social psychology: Handbook of basic principles (pp. 361–399). New York: Guilford Press.

    Google Scholar 

  • Gräsel, C., Böhmer, I., Hörstermann, T., & Krolak-Schwerdt, S. (2013). Die Erfassung diagnostischer Kompetenz mit Fallvignetten: Welche Rolle spielen Kontextinformationen? (Measuring diagnostic competence using case vignettes: the role of context). In K. Stürmer & T. Seidel (Eds.), Neue Wege zur Erfassung professioneller Lehrerkompetenz im Spiegel ihrer Handlungsrelevanz. (New ways to assess teachers’ professional competence in terms of relevance for praxis). Symposium conducted at the 1st meeting of the Gesellschaft für Empirischsche Bildungsforschung (GEBF). Kiel Germany.

  • Haller, E. J. (1985). Pupil race and elementary school ability grouping: are teachers biased against black children? American Educational Research Journal, 22, 465–483. doi:10.3102/00028312022004465.

    Article  Google Scholar 

  • Harlen, W. (2005). Trusting teachers’ judgement: research evidence of the reliability and validity of teachers’ assessment used for summative purposes. Research Papers in Education, 20, 245–270. doi:10.1080/02671520500193744.

    Article  Google Scholar 

  • Hoffrage, U., & Reimer, T. (2004). Models of bounded rationality: the approach of fast and frugal heuristics. Management Revue, 15, 437–459.

    Google Scholar 

  • Jussim, L. (2005). Accuracy in social perception: criticisms, controversies, criteria, components, and cognitive processes. Advances in Experimental Social Psychology, 37, 1–93. doi:10.1016/S0065-2601(05)37001-8.

    Article  Google Scholar 

  • Jussim, L., & Harber, K. D. (2005). Teacher expectations and self-fulfilling prophecies: knowns and unknowns, resolved and unresolved controversies. Personality and Social Psychology Review, 9, 131–155. doi:10.1207/s15327957pspr0902.

    Article  Google Scholar 

  • Jussim, L., Eccles, J., & Madon, S. J. (1996). Social perception, social stereotypes, and teacher expectations: accuracy and the quest for the powerful self-fulfilling prophecy. Advances in Experimental Social Psychology, 28, 281–388. doi:10.1016/S0065-2601(08)60240-3.

    Article  Google Scholar 

  • Karelaia, N., & Hogarth, R. M. (2008). Determinants of linear judgment: a meta-analysis of lens model studies. Psychological Bulletin, 134, 404–426. doi:10.1037/0033-2909.134.3.404.

    Article  Google Scholar 

  • Kaufman, J. E., & Rosenbaum, J. E. (1992). The education and employment of low-income black youth in white suburbs. Educational Evaluation and Policy Analysis, 14, 229–240. doi:10.3102/01623737014003229.

    Article  Google Scholar 

  • Klapproth, F., Glock, S., Böhmer, M., Krolak-Schwerdt, S., & Martin, R. (2012). School placement decisions in Luxembourg: do teachers meet the Education Ministry’s standards? The Literacy Information and Computer Education Journal, 1, 765–771.

    Google Scholar 

  • Krolak-Schwerdt, S., Böhmer, M., & Gräsel, C. (2013). The impact of accountability on teachers’ assessments of student performance: a social cognitive analysis. Social Psychology of Education, 16, 215–239. doi:10.1007/s11218-013-9215-9.

    Article  Google Scholar 

  • Lee, Y.-T., McCauley, C., & Jussim, L. (2013). Stereotypes as valid categories of knowledge and human perceptions of group differences. Social and Personality Psychology Compass, 7, 470–486. doi:10.1111/spc3.12039.

    Article  Google Scholar 

  • Lerner, J. S., & Tetlock, P. E. (1999). Accounting for effects of accountability. Psychological Bulletin, 125, 255–275. doi:10.1037/0033-2909.125.2.255.

    Article  Google Scholar 

  • Lerner, J. S., & Tetlock, P. E. (2003). Bridging individual, interpersonal, and institutional approaches to judgment and decision making: the impact of accountability on cognitive bias. In S. L. Schneider & J. Shanteau (Eds.), Emerging perspectives on judgment and decision research (pp. 431–457). Cambridge: University Press.

    Chapter  Google Scholar 

  • Macrae, C. N., & Bodenhausen, G. V. (2000). Social cognition: thinking categorically about others. Annual Review of Psychology, 51, 93–120.

    Article  Google Scholar 

  • McCombs, R. C., & Gay, J. (1988). Effects of race, class, and IQ information on judgments of parochial grade school teachers. The Journal of Social Psychology, 128, 647–652. doi:10.1080/00224545.1988.9922918.

    Article  Google Scholar 

  • Oakes, J. (2005). Keeping track: How schools structure inequality. New Haven: Yale University Press.

    Google Scholar 

  • Oakes, J., & Guiton, G. (1995). Matchmaking: the dynamics of high school tracking decisions. American Educational Research Journal, 32, 3–33. doi:10.3102/00028312032001003.

    Article  Google Scholar 

  • OECD. (2010). PISA 2009 results: overcoming social background—equity in learning opportunities and outcomes (Vol II). OECD Publishing. doi:10.1787/9789264091504-en.

  • Parks, F. R., & Kennedy, J. H. (2007). The impact of race, physical attractiveness, and gender on education majors’ and teachers’ perceptions of student competence. Journal of Black Studies, 37, 936–943. doi:10.1177/0021934705285955.

    Article  Google Scholar 

  • Pendry, L. F., & Macrae, C. N. (1996). What the disinterested perceiver overlooks: goal-directed social categorization. Personality and Social Psychology Bulletin, 22, 249–256. doi:10.1177/0146167296223003.

    Article  Google Scholar 

  • Petty, R. E., & Cacioppo, J. T. (1986). The elaboration likelihood model of persuasion. Advances in Experimental Social Psychology, 123–205

  • Pietsch, M., & Stubbe, T. C. (2007). Inequality in the transition from primary to secondary school: school choices and educational disparities in Germany. European Educational Research Journal, 6, 424–445. doi:10.2304/eerj.2007.6.4.424.

    Article  Google Scholar 

  • Pit-ten Cate, I. M., Hörstermann, T. (2012). Towards a criterion to judge the accuracy of transition decisions. Poster presented at the European Conference on Educational Research, The need for Educational research to champion, Freedom. Cadiz, Spain: Education and Development for All.

  • Raudenbush, S. W. (1984). Magnitude of teacher expectancy effects on pupil IQ as a function of the credibility of expectancy induction: a synthesis of findings from 18 experiments. Journal of Educational Psychology, 76, 85–97. doi:10.1037/0022-0663.76.1.85.

    Article  Google Scholar 

  • Schalke, D., Brunner, M., Geiser, C., Preckel, F., Keller, U., Spengler, M., & Martin, R. (2013). Stability and change in intelligence from age 12 to age 52: results from the Luxembourg MAGRIP study. Developmental Psychology, 49, 1529–1543. doi:10.1037/a0030623.

    Article  Google Scholar 

  • Schraw, G. (2009). A conceptual analysis of five measures of metacognitive monitoring. Metacognition and Learning, 4, 33–45. doi:10.1007/s11409-008-9031-3.

    Article  Google Scholar 

  • Schwarz, N., & Bless, H. (1992). Constructing reality and its alternatives: An inclusion/exclusion model of assimilation and contrast effects in social judgment. In L. L. Martin & A. Tesser (Eds.), The construction of social judgments (pp. 217–245). Hillsdale: Erlbaum.

    Google Scholar 

  • Shafir, E., Simonson, I., & Tversky, A. (1993). Reason-based choice. Cognition, 49, 11–36. doi:10.1016/0010-0277(93)90034-S.

    Article  Google Scholar 

  • Stangor, C., & Schaller, M. (1996). Stereotypes as individual and collective representations. In C. N. Macrae, C. Stangor, & M. Hewstone (Eds.), Stereotypes and stereotyping (pp. 3–40). New York: Guilford Press.

    Google Scholar 

  • Südkamp, A., Kaiser, J., & Möller, J. (2012). Accuracy of teachers’ judgments of students’ academic achievement: a meta-analysis. Journal of Educational Psychology, 104, 743–762. doi:10.1037/a0027627.

    Article  Google Scholar 

  • Swets, J. A., Dawes, R. M., & Monahan, J. (2000). Better decisions through science. Scientific American, 2, 82–87.

    Article  Google Scholar 

  • Tetlock, P. E. (1983). Accountability and complexity of thought. Journal of Personality and Social Psychology, 45, 74–83.

    Article  Google Scholar 

  • Tetlock, P. E. (1991). An alternative metaphor in the study of judgment and choice: people as politicians. Theory and Psychology, 1, 451–475. doi:10.1177/0959354391014004.

    Article  Google Scholar 

  • Tetlock, P. E., & Kim, J. I. (1987). Accountability and judgment processes in a personality prediction task. Journal of Personality and Social Psychology, 52, 700–709. doi:10.1037//0022-3514.52.4.700.

    Article  Google Scholar 

  • Tetlock, P. E., & Lerner, J. S. (1999). The social contingency model: Identifying empirical and normative boundary conditions on the error-and-bias portrait of human nature. In S. Chaiken & Y. Trope (Eds.), Dual process theories in social psychology. New York: Guilford Press.

    Google Scholar 

  • Thomson, D. R., & Milliken, B. (2013). Contextual distinctiveness produces long-lasting priming of pop-out. Journal of Experimental Psychology. Human Perception and Performance, 39, 202–215. doi:10.1037/a0028069.

    Article  Google Scholar 

  • Van de Werfhorst, H. G., & Mijs, J. J. B. (2010). Achievement inequality and the institutional structure of educational systems: a comparative perspective. Annual Review of Sociology, 36, 407–428. doi:10.1146/annurev.soc.012809.102538.

    Article  Google Scholar 

Download references

Acknowledgments

The research reported in this paper was funded by Grant C10/LM/784116 from the Fonds National de la Recherche Luxembourg (FNR)

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ineke M. Pit-ten Cate.

Additional information

Dr Ineke M. Pit-ten Cate. University of Luxembourg, Faculty of Language and Literature, Humanities, Arts, and Education, Institute of Teacher Professionalization and Psychology of Education Campus Walferdange, Route de Diekirch, L-7201 Walferdange, Luxembourg. Email: ineke.pit@uni.lu

Current themes of research:

Processes that underlie psychosocial (dys)function of children with chronic illness and physical disability and their families. Children’s developmental outcomes and family adjustment. Teaching and research concerning the diagnostic competence of teachers in regards to judgment of student performance and decisions on educational pathways, especially in consideration of challenges associated with a heterogeneous student population and the identification of special educational needs.

Most relevant publications in the field of Psychology of Education:

Glock, S., Krolak-Schwerdt, S., & Pit-ten Cate, I. M. (2014). Are school placement recommendations accurate? The effect of students’ ethnicity on teachers’ judgments and recognition memory. European Journal of Psychology of Education. doi:10.1007/s10212-014-0237-2.

Markova, M., Pit-ten Cate, I. M., Krolak-Schwerdt, S., & Glock, S. (in press). Preservice teachers' attitudes toward inclusion and toward students with special educational needs from different ethnic backgrounds. Journal of Experimental Education.

Pit-ten Cate, I. M., Krolak-Schwerdt, S., Glock, S., & Markova, M. (2014). Improving teachers’ judgments. Obtaining change through cognitive processes. In S. Krolak-schwerdt, S. Glock, & M. Böhmer (Eds.), Teachers’ professional development: Assessment, training, and learning (pp. 45–61). Rotterdam: Sense.

Prof Dr Sabine Krolak-Schwerdt. University of Luxembourg, Faculty of Language and Literature, Humanities, Arts, and Education, Institute of Teacher Professionalization and Psychology of Education Campus Walferdange, Route de Diekirch, L-7201 Walferdange, Luxembourg. Email: sabine.krolak@uni.lu

Current themes of research:

Educational psychology, social cognition, and language as well as multivariate methods. Professionalization and diagnostic competences of teachers, social cognition and social judgments as well as the application of multivariate approaches to analyse large datasets.

Most relevant publications in the field of Psychology of Education:

Glock, S., & Krolak-Schwerdt, S. (2013). Does nationality matter? The impact of stereotypical expectations on student teachers’ judgments. Social Psychology of Education, 16, 111–127. doi:10.1007/s11218-012-9197-z.

Glock, S., Krolak-Schwerdt, S., & Pit-ten Cate, I. M. (2014). Are school placement recommendations accurate? The effect of students’ ethnicity on teachers’ judgments and recognition memory. European Journal of Psychology of Education. doi:10.1007/s10212-014-0237-2.

Krolak-Schwerdt, S., Böhmer, M., & Gräsel, C. (2013). The impact of accountability on teachers’ assessments of student performance: a social cognitive analysis. Social Psychology of Education, 16, 215–239. doi:10.1007/s11218-013-9215-9.

Markova, M., Pit-ten Cate, I. M., Krolak-Schwerdt, S., & Glock, S. (in press). Preservice teachers' attitudes toward inclusion and toward students with special educational needs from different ethnic backgrounds. Journal of Experimental Education.

Pit-ten Cate, I. M., Krolak-Schwerdt, S., Glock, S., & Markova, M. (2014). Improving teachers’ judgments. Obtaining change through cognitive processes. In S. Krolak-schwerdt, S. Glock, & M. Böhmer (Eds.), Teachers’ professional development: Assessment, training, and learning (pp. 45–61). Rotterdam: Sense.

Dr Sabine Glock. Bergische Universität Wuppertal, School of Education, Institut für Bildungsforschung, Gaußstr. 20, D-42119 Wuppertal. Email: glock@uni-wuppertal.de

Current themes of research:

Educational psychology, particularly, educational decision making and factors influencing educational decisions. Implicit cognition, in particular implicit attitudes toward students with ethnic minority background and how they contribute to teachers’ judgments.

Most relevant publications in the field of Psychology of Education:

Glock, S., & Karbach, J. (2015). Preservice teachers’ implicit attitudes toward racial minority students: Evidence from three implicit measures. Studies in Educational Evaluation, doi:10.1016/j.stueduc.2015.03.006.

Glock, S., & Krolak-Schwerdt, S. (2013). Does nationality matter? The impact of stereotypical expectations on student teachers’ judgments. Social Psychology of Education, 16, 111–127. doi:10.1007/s11218-012-9197-z.

Glock, S., Krolak-Schwerdt, S., & Pit-ten Cate, I. M. (2014). Are school placement recommendations accurate? The effect of students’ ethnicity on teachers’ judgments and recognition memory. European Journal of Psychology of Education. doi:10.1007/s10212-014-0237-2.

Markova, M., Pit-ten Cate, I. M., Krolak-Schwerdt, S., & Glock, S. (in press). Preservice teachers' attitudes toward inclusion and toward students with special educational needs from different ethnic backgrounds. Journal of Experimental Education.

Pit-ten Cate, I. M., Krolak-Schwerdt, S., Glock, S., & Markova, M. (2014). Improving teachers’ judgments. Obtaining change through cognitive processes. In S. Krolak-schwerdt, S. Glock, & M. Böhmer (Eds.), Teachers’ professional development: Assessment, training, and learning (pp. 45–61). Rotterdam: Sense.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1

(PDF 45 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pit-ten Cate, I.M., Krolak-Schwerdt, S. & Glock, S. Accuracy of teachers’ tracking decisions: short- and long-term effects of accountability. Eur J Psychol Educ 31, 225–243 (2016). https://doi.org/10.1007/s10212-015-0259-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10212-015-0259-4

Keywords

Navigation