Abstract
Bias in teachers’ judgment formation and decision making has long been acknowledged. More specifically, studies have repeatedly demonstrated discrepancies between teacher ratings of minority and majority students with similar academic profiles. Studies have also demonstrated that increasing accountability reduced bias. Little is known, however, about the effect of accountability and bias on the accuracy of decisions. This study investigated the short- and long-term effect of accountability priming on the accuracy of transition decisions. It considered both the extent to which teacher decision accuracy differed for minority and majority students with similar academic profiles (accuracy bias) and differences in levels of confidence for accurate versus erroneous decisions (metacognitive judgment bias). In a longitudinal experimental design, we presented 38 primary school teachers with 9 student vignettes at 3 points in time (baseline, post priming, and 6-month follow-up), varying students’ ethnic background, and asked them to make a school tracking decision for each student. We measured decision accuracy as well as teachers’ level of confidence for each decision. Accuracy and confidence levels were combined to provide two indices of metacognitive judgment accuracy. Results confirmed the hypothesis that accuracy of decisions would improve as a result of increased level of accountability. More specifically, we found that teachers made more accurate decisions after priming, whereby ethnic background differences disappeared. In addition, teachers’ metacognitions varied, whereby after priming decision accuracy was better matched with teachers’ confidence levels. Although accuracy levels were still higher at follow-up than at pre-test, the ethnic bias recurred. This study shows that increased levels of accountability are associated with not only increased decision accuracy but also reduced metacognitive judgment bias, especially in regard to minority students. It also demonstrates accountability may be an effective way of reducing systematic errors in decision making. Findings are discussed in terms of theory and current changes in educational practice.
Similar content being viewed by others
References
Baeriswyl, F., Wandeler, C., & Christ, O. (2008). Die Übertrittsempfehlung – zufällig oder zuverlässig? Analyse der Determinanten und Konstanz von Lehrerempfehlungen bei Schulübertritten während sieben Jahren. [School placement recommendation - random or valid? Analysis of the determinants and stability of teachers' school placement recommendations during seven years]. Schweizerische Zeitschrift für Bildungswissenschaften, 30, 549–578.
Bamberg, M., Barthelemy, M., Bertemes, J., Besch, E., Boehm, B., Brunner, M., … Ugen, S. (2010). PISA 2009: Nationaler Bericht Luxemburg [PISA 2009: National Report Luxembourg]. Luxembourg: MENFP-SCRIPT et Université du Luxembourg: EMACS.
Brehmer, B. (1994). The psychology of linear judgment models. Acta Psychologica, 87, 137–154. doi:10.1016/0001-6918(94)90048-5.
Byrnes, J. P., & Miller, D. C. (2007). The relative importance of predictors of math and science achievement: an opportunity–propensity analysis. Contemporary Educational Psychology, 32, 599–629. doi:10.1016/j.cedpsych.2006.09.002.
Dawes, R. M., & Corrigan, B. (1974). Linear models in decision making. Psychological Bulletin, 81, 95–106. doi:10.1037/h0037613.
De Boer, H., Bosker, R. J., & van der Werf, M. P. C. (2010). Sustainability of teacher expectation bias effects on long-term student performance. Journal of Educational Psychology, 102, 168–179. doi:10.1037/a0017289.
Demaray, M. K., & Elliott, S. N. (1998). Teachers’ judgments of students’ academic functioning: a comparison of actual and predicted performances. School Psychology Quarterly, 13, 8–24. doi:10.1037/h0088969.
DeMeis, D. K., & Turner, R. R. (1978). Effects of students’ race, physical attractiveness, and dialect on teachers’ evaluations. Contemporary Educational Psychology, 3, 77–86. doi:10.1016/0361-476X(78)90012-7.
Dustmann, C. (2004). Parental background, secondary school track choice, and wages. Oxford Economic Papers, 56, 209–230. doi:10.1093/oep/gpf048.
Elhoweris, H., Mutua, K., Alsheikh, N., & Holloway, P. (2005). Effect of children’s ethnicity on teachers’ referral and recommendation decisions in gifted and talented programs. Remedial and Special Education, 26, 25–31. doi:10.1177/07419325050260010401.
Ferguson, R. F. (2003). Teachers’ perceptions and expectations and the Black-White test score gap. Urban Education, 38, 460–507. doi:10.1177/0042085903254970.
Ferreira, M. B., Garcia-Marques, L., Sherman, S. J., & Sherman, J. W. (2006). Automatic and controlled components of judgment and decision making. Journal of Personality and Social Psychology, 91, 797–813. doi:10.1037/0022-3514.91.5.797.
Fiske, S. T., & Neuberg, S. L. (1990). A continuum of impression formation from category-based to individuating processes: Influences of information and motivation on attention and interpretation. In M. P. Zanna (Ed.), Advances in experimental social psychology (Vol. 23, pp. 1–74). New York: Academic Press.
Fiske, S. T., & Taylor, S. E. (1991). Social cognition (2nd ed.). New York: McGraw-Hill.
Fiske, S. T., & Taylor, S. E. (2010). Social cognition: From brains to culture (International ed.). New York: McGraw-Hill.
Fiske, S. T., Lin, M., & Neuberg, S. L. (1999). The continuum model. Ten years later. In S. Chaiken & Y. Trope (Eds.), Dual process theories in social psychology (pp. 231–254). New York: Guilford Press.
Flavell, J. H. (1979). Metacognition and cognitive monitoring: a new area of cognitive-developmental inquiry. American Psychologist, 34, 906–911. doi:10.1037//0003-066X.34.10.906.
Gamoran, A. (1992). The variable effects of high school tracking. American Sociological Review, 57, 812–828.
Gigerenzer, G., & Todd, P. M. (1999). Fast and frugal heuristics: The adaptive toolbox. In G. Gigerenzer, P. M. Todd, & ABC Research Group (Eds.), Simple heuristics that make us smart (pp. 3–34). Oxford: Oxford University Press.
Gilbert, D. T., & Hixon, J. G. (1991). The trouble of thinking: activation and application of stereotypic beliefs. Journal of Personality and Social Psychology, 60, 509–517. doi:10.1037/0022-3514.60.4.509.
Glock, S., & Krolak-Schwerdt, S. (2013). Does nationality matter? The impact of stereotypical expectations on student teachers’ judgments. Social Psychology of Education, 16, 111–127. doi:10.1007/s11218-012-9197-z.
Glock, S., Krolak-Schwerdt, S., Klapproth, F., & Böhmer, M. (2012). Improving teachers’ judgments: accountability affects teachers’ tracking decisions. International Journal of Technology and Inclusive Education, 1, 89–98.
Glock, S., Krolak-Schwerdt, S., Klapproth, F., & Böhmer, M. (2013). Beyond judgment bias: how students’ ethnicity and academic profile consistency influence teachers’ tracking judgments. Social Psychology of Education, 16, 555–573. doi:10.1007/s11218-013-9227-5.
Glock, S., Krolak-Schwerdt, S., & Pit-ten Cate, I. M. (2015). Are school placement recommendations accurate? The effect of students’ race on teachers’ judgments and recognition memory. European Journal of Psychology of Education, 30, 169–188. doi:10.1007/s10212-014-0237-2.
Golding, J. M., Fowler, S. B., Long, D. L., & Latta, H. (1990). Instructions to disregard potentially useful information: the effects of pragmatics on evaluative judgments and recall. Journal of Memory and Language, 29, 212–227. doi:10.1016/0749-596X(90)90073-9.
Gollwitzer, P. M., & Moskowitz, G. B. (1996). Goal effects on action and cognition. In A. W. Kruglanski & E. T. Higgins (Eds.), Social psychology: Handbook of basic principles (pp. 361–399). New York: Guilford Press.
Gräsel, C., Böhmer, I., Hörstermann, T., & Krolak-Schwerdt, S. (2013). Die Erfassung diagnostischer Kompetenz mit Fallvignetten: Welche Rolle spielen Kontextinformationen? (Measuring diagnostic competence using case vignettes: the role of context). In K. Stürmer & T. Seidel (Eds.), Neue Wege zur Erfassung professioneller Lehrerkompetenz im Spiegel ihrer Handlungsrelevanz. (New ways to assess teachers’ professional competence in terms of relevance for praxis). Symposium conducted at the 1st meeting of the Gesellschaft für Empirischsche Bildungsforschung (GEBF). Kiel Germany.
Haller, E. J. (1985). Pupil race and elementary school ability grouping: are teachers biased against black children? American Educational Research Journal, 22, 465–483. doi:10.3102/00028312022004465.
Harlen, W. (2005). Trusting teachers’ judgement: research evidence of the reliability and validity of teachers’ assessment used for summative purposes. Research Papers in Education, 20, 245–270. doi:10.1080/02671520500193744.
Hoffrage, U., & Reimer, T. (2004). Models of bounded rationality: the approach of fast and frugal heuristics. Management Revue, 15, 437–459.
Jussim, L. (2005). Accuracy in social perception: criticisms, controversies, criteria, components, and cognitive processes. Advances in Experimental Social Psychology, 37, 1–93. doi:10.1016/S0065-2601(05)37001-8.
Jussim, L., & Harber, K. D. (2005). Teacher expectations and self-fulfilling prophecies: knowns and unknowns, resolved and unresolved controversies. Personality and Social Psychology Review, 9, 131–155. doi:10.1207/s15327957pspr0902.
Jussim, L., Eccles, J., & Madon, S. J. (1996). Social perception, social stereotypes, and teacher expectations: accuracy and the quest for the powerful self-fulfilling prophecy. Advances in Experimental Social Psychology, 28, 281–388. doi:10.1016/S0065-2601(08)60240-3.
Karelaia, N., & Hogarth, R. M. (2008). Determinants of linear judgment: a meta-analysis of lens model studies. Psychological Bulletin, 134, 404–426. doi:10.1037/0033-2909.134.3.404.
Kaufman, J. E., & Rosenbaum, J. E. (1992). The education and employment of low-income black youth in white suburbs. Educational Evaluation and Policy Analysis, 14, 229–240. doi:10.3102/01623737014003229.
Klapproth, F., Glock, S., Böhmer, M., Krolak-Schwerdt, S., & Martin, R. (2012). School placement decisions in Luxembourg: do teachers meet the Education Ministry’s standards? The Literacy Information and Computer Education Journal, 1, 765–771.
Krolak-Schwerdt, S., Böhmer, M., & Gräsel, C. (2013). The impact of accountability on teachers’ assessments of student performance: a social cognitive analysis. Social Psychology of Education, 16, 215–239. doi:10.1007/s11218-013-9215-9.
Lee, Y.-T., McCauley, C., & Jussim, L. (2013). Stereotypes as valid categories of knowledge and human perceptions of group differences. Social and Personality Psychology Compass, 7, 470–486. doi:10.1111/spc3.12039.
Lerner, J. S., & Tetlock, P. E. (1999). Accounting for effects of accountability. Psychological Bulletin, 125, 255–275. doi:10.1037/0033-2909.125.2.255.
Lerner, J. S., & Tetlock, P. E. (2003). Bridging individual, interpersonal, and institutional approaches to judgment and decision making: the impact of accountability on cognitive bias. In S. L. Schneider & J. Shanteau (Eds.), Emerging perspectives on judgment and decision research (pp. 431–457). Cambridge: University Press.
Macrae, C. N., & Bodenhausen, G. V. (2000). Social cognition: thinking categorically about others. Annual Review of Psychology, 51, 93–120.
McCombs, R. C., & Gay, J. (1988). Effects of race, class, and IQ information on judgments of parochial grade school teachers. The Journal of Social Psychology, 128, 647–652. doi:10.1080/00224545.1988.9922918.
Oakes, J. (2005). Keeping track: How schools structure inequality. New Haven: Yale University Press.
Oakes, J., & Guiton, G. (1995). Matchmaking: the dynamics of high school tracking decisions. American Educational Research Journal, 32, 3–33. doi:10.3102/00028312032001003.
OECD. (2010). PISA 2009 results: overcoming social background—equity in learning opportunities and outcomes (Vol II). OECD Publishing. doi:10.1787/9789264091504-en.
Parks, F. R., & Kennedy, J. H. (2007). The impact of race, physical attractiveness, and gender on education majors’ and teachers’ perceptions of student competence. Journal of Black Studies, 37, 936–943. doi:10.1177/0021934705285955.
Pendry, L. F., & Macrae, C. N. (1996). What the disinterested perceiver overlooks: goal-directed social categorization. Personality and Social Psychology Bulletin, 22, 249–256. doi:10.1177/0146167296223003.
Petty, R. E., & Cacioppo, J. T. (1986). The elaboration likelihood model of persuasion. Advances in Experimental Social Psychology, 123–205
Pietsch, M., & Stubbe, T. C. (2007). Inequality in the transition from primary to secondary school: school choices and educational disparities in Germany. European Educational Research Journal, 6, 424–445. doi:10.2304/eerj.2007.6.4.424.
Pit-ten Cate, I. M., Hörstermann, T. (2012). Towards a criterion to judge the accuracy of transition decisions. Poster presented at the European Conference on Educational Research, The need for Educational research to champion, Freedom. Cadiz, Spain: Education and Development for All.
Raudenbush, S. W. (1984). Magnitude of teacher expectancy effects on pupil IQ as a function of the credibility of expectancy induction: a synthesis of findings from 18 experiments. Journal of Educational Psychology, 76, 85–97. doi:10.1037/0022-0663.76.1.85.
Schalke, D., Brunner, M., Geiser, C., Preckel, F., Keller, U., Spengler, M., & Martin, R. (2013). Stability and change in intelligence from age 12 to age 52: results from the Luxembourg MAGRIP study. Developmental Psychology, 49, 1529–1543. doi:10.1037/a0030623.
Schraw, G. (2009). A conceptual analysis of five measures of metacognitive monitoring. Metacognition and Learning, 4, 33–45. doi:10.1007/s11409-008-9031-3.
Schwarz, N., & Bless, H. (1992). Constructing reality and its alternatives: An inclusion/exclusion model of assimilation and contrast effects in social judgment. In L. L. Martin & A. Tesser (Eds.), The construction of social judgments (pp. 217–245). Hillsdale: Erlbaum.
Shafir, E., Simonson, I., & Tversky, A. (1993). Reason-based choice. Cognition, 49, 11–36. doi:10.1016/0010-0277(93)90034-S.
Stangor, C., & Schaller, M. (1996). Stereotypes as individual and collective representations. In C. N. Macrae, C. Stangor, & M. Hewstone (Eds.), Stereotypes and stereotyping (pp. 3–40). New York: Guilford Press.
Südkamp, A., Kaiser, J., & Möller, J. (2012). Accuracy of teachers’ judgments of students’ academic achievement: a meta-analysis. Journal of Educational Psychology, 104, 743–762. doi:10.1037/a0027627.
Swets, J. A., Dawes, R. M., & Monahan, J. (2000). Better decisions through science. Scientific American, 2, 82–87.
Tetlock, P. E. (1983). Accountability and complexity of thought. Journal of Personality and Social Psychology, 45, 74–83.
Tetlock, P. E. (1991). An alternative metaphor in the study of judgment and choice: people as politicians. Theory and Psychology, 1, 451–475. doi:10.1177/0959354391014004.
Tetlock, P. E., & Kim, J. I. (1987). Accountability and judgment processes in a personality prediction task. Journal of Personality and Social Psychology, 52, 700–709. doi:10.1037//0022-3514.52.4.700.
Tetlock, P. E., & Lerner, J. S. (1999). The social contingency model: Identifying empirical and normative boundary conditions on the error-and-bias portrait of human nature. In S. Chaiken & Y. Trope (Eds.), Dual process theories in social psychology. New York: Guilford Press.
Thomson, D. R., & Milliken, B. (2013). Contextual distinctiveness produces long-lasting priming of pop-out. Journal of Experimental Psychology. Human Perception and Performance, 39, 202–215. doi:10.1037/a0028069.
Van de Werfhorst, H. G., & Mijs, J. J. B. (2010). Achievement inequality and the institutional structure of educational systems: a comparative perspective. Annual Review of Sociology, 36, 407–428. doi:10.1146/annurev.soc.012809.102538.
Acknowledgments
The research reported in this paper was funded by Grant C10/LM/784116 from the Fonds National de la Recherche Luxembourg (FNR)
Author information
Authors and Affiliations
Corresponding author
Additional information
Dr Ineke M. Pit-ten Cate. University of Luxembourg, Faculty of Language and Literature, Humanities, Arts, and Education, Institute of Teacher Professionalization and Psychology of Education Campus Walferdange, Route de Diekirch, L-7201 Walferdange, Luxembourg. Email: ineke.pit@uni.lu
Current themes of research:
Processes that underlie psychosocial (dys)function of children with chronic illness and physical disability and their families. Children’s developmental outcomes and family adjustment. Teaching and research concerning the diagnostic competence of teachers in regards to judgment of student performance and decisions on educational pathways, especially in consideration of challenges associated with a heterogeneous student population and the identification of special educational needs.
Most relevant publications in the field of Psychology of Education:
Glock, S., Krolak-Schwerdt, S., & Pit-ten Cate, I. M. (2014). Are school placement recommendations accurate? The effect of students’ ethnicity on teachers’ judgments and recognition memory. European Journal of Psychology of Education. doi:10.1007/s10212-014-0237-2.
Markova, M., Pit-ten Cate, I. M., Krolak-Schwerdt, S., & Glock, S. (in press). Preservice teachers' attitudes toward inclusion and toward students with special educational needs from different ethnic backgrounds. Journal of Experimental Education.
Pit-ten Cate, I. M., Krolak-Schwerdt, S., Glock, S., & Markova, M. (2014). Improving teachers’ judgments. Obtaining change through cognitive processes. In S. Krolak-schwerdt, S. Glock, & M. Böhmer (Eds.), Teachers’ professional development: Assessment, training, and learning (pp. 45–61). Rotterdam: Sense.
Prof Dr Sabine Krolak-Schwerdt. University of Luxembourg, Faculty of Language and Literature, Humanities, Arts, and Education, Institute of Teacher Professionalization and Psychology of Education Campus Walferdange, Route de Diekirch, L-7201 Walferdange, Luxembourg. Email: sabine.krolak@uni.lu
Current themes of research:
Educational psychology, social cognition, and language as well as multivariate methods. Professionalization and diagnostic competences of teachers, social cognition and social judgments as well as the application of multivariate approaches to analyse large datasets.
Most relevant publications in the field of Psychology of Education:
Glock, S., & Krolak-Schwerdt, S. (2013). Does nationality matter? The impact of stereotypical expectations on student teachers’ judgments. Social Psychology of Education, 16, 111–127. doi:10.1007/s11218-012-9197-z.
Glock, S., Krolak-Schwerdt, S., & Pit-ten Cate, I. M. (2014). Are school placement recommendations accurate? The effect of students’ ethnicity on teachers’ judgments and recognition memory. European Journal of Psychology of Education. doi:10.1007/s10212-014-0237-2.
Krolak-Schwerdt, S., Böhmer, M., & Gräsel, C. (2013). The impact of accountability on teachers’ assessments of student performance: a social cognitive analysis. Social Psychology of Education, 16, 215–239. doi:10.1007/s11218-013-9215-9.
Markova, M., Pit-ten Cate, I. M., Krolak-Schwerdt, S., & Glock, S. (in press). Preservice teachers' attitudes toward inclusion and toward students with special educational needs from different ethnic backgrounds. Journal of Experimental Education.
Pit-ten Cate, I. M., Krolak-Schwerdt, S., Glock, S., & Markova, M. (2014). Improving teachers’ judgments. Obtaining change through cognitive processes. In S. Krolak-schwerdt, S. Glock, & M. Böhmer (Eds.), Teachers’ professional development: Assessment, training, and learning (pp. 45–61). Rotterdam: Sense.
Dr Sabine Glock. Bergische Universität Wuppertal, School of Education, Institut für Bildungsforschung, Gaußstr. 20, D-42119 Wuppertal. Email: glock@uni-wuppertal.de
Current themes of research:
Educational psychology, particularly, educational decision making and factors influencing educational decisions. Implicit cognition, in particular implicit attitudes toward students with ethnic minority background and how they contribute to teachers’ judgments.
Most relevant publications in the field of Psychology of Education:
Glock, S., & Karbach, J. (2015). Preservice teachers’ implicit attitudes toward racial minority students: Evidence from three implicit measures. Studies in Educational Evaluation, doi:10.1016/j.stueduc.2015.03.006.
Glock, S., & Krolak-Schwerdt, S. (2013). Does nationality matter? The impact of stereotypical expectations on student teachers’ judgments. Social Psychology of Education, 16, 111–127. doi:10.1007/s11218-012-9197-z.
Glock, S., Krolak-Schwerdt, S., & Pit-ten Cate, I. M. (2014). Are school placement recommendations accurate? The effect of students’ ethnicity on teachers’ judgments and recognition memory. European Journal of Psychology of Education. doi:10.1007/s10212-014-0237-2.
Markova, M., Pit-ten Cate, I. M., Krolak-Schwerdt, S., & Glock, S. (in press). Preservice teachers' attitudes toward inclusion and toward students with special educational needs from different ethnic backgrounds. Journal of Experimental Education.
Pit-ten Cate, I. M., Krolak-Schwerdt, S., Glock, S., & Markova, M. (2014). Improving teachers’ judgments. Obtaining change through cognitive processes. In S. Krolak-schwerdt, S. Glock, & M. Böhmer (Eds.), Teachers’ professional development: Assessment, training, and learning (pp. 45–61). Rotterdam: Sense.
Electronic supplementary material
Below is the link to the electronic supplementary material.
ESM 1
(PDF 45 kb)
Rights and permissions
About this article
Cite this article
Pit-ten Cate, I.M., Krolak-Schwerdt, S. & Glock, S. Accuracy of teachers’ tracking decisions: short- and long-term effects of accountability. Eur J Psychol Educ 31, 225–243 (2016). https://doi.org/10.1007/s10212-015-0259-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10212-015-0259-4