Skip to main content

Advertisement

SpringerLink
Log in

Assessment and development centers: judgment biases and risks of using idiographic and nomothetic approaches to collecting information on people to be evaluated and trained in organizations

  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

Assessment center and development center are two procedures that organizations can use in order to evaluate and train people. They make use of different methods and techniques, some (i.e. interviews) descending from the so called idiographic (or clinical) approach, and some (i.e. standardized instruments) descending from the so called nomothetic (or psychometric) approach. The idea is that different methods and techniques allow assessors and decision makers to collect as much information as possible, in order to come to an integrated judgment of people to be evaluated. Regarding this idea, psychological research has already discovered that it is not the amount of information collected that makes the difference between expert and non-expert assessors and decision makers. Besides, too much information is difficult to manage; and while it increases the confidence of assessors and decision makers about their judgments, it unfortunately does not increase their accuracy as well, since relevant information is mixed with irrelevant one and this makes it difficult to decide which one to consider and which one not. So, the article wants to be a critical review of what psychological science has found, and not so recently, in the field of assessment and development of psychological characteristics, in terms of risks and biases. Finally, it wants to underline the fact that, in spite of risks and biases, nowadays different methods and techniques are actually used to assess one person’s psychological characteristics, which is certainly questionable but also methodologically appropriate if they are appropriately used.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Anastasi A., Urbina S.: Psychologica Testing. Prentice Hall, Upper Saddle River (1997)

    Google Scholar 

  • Anderson N., Herriot P.: Assessment and Selection in Organizations. Wiley, London (1995)

    Google Scholar 

  • Arnott D.: A Taxonomy of Decision Biases. Monash University, School of Information Management and Systems, Caulfield (1998)

    Google Scholar 

  • Barry G.S.: Expert critics: operationalizing the judgement/decisionmaking literature as a theory of “bugs” and repair strategies. Knowl. Acquis. 3, 175–214 (1991)

    Article  Google Scholar 

  • Berant E., Zim S.: Between two worlds: a case study of the integrative process of personality assessment. Rorschachiana 29, 201–232 (2008)

    Article  Google Scholar 

  • Boyle S., Fullerton J., Wood R.: Do assessment/development centres use optimum evaluation procedures? A survey of practice in UK organizations. Int. J. Sel. Assess. 3, 132–140 (1995)

    Article  Google Scholar 

  • Bray D.W., Grant D.L.: The assessment center in the measurement of potential for business management. Psychol. Monogr. Gen. Appl. 80, 1–27 (1966)

    Google Scholar 

  • Burro R., Sartori R., Vidotto G.: The Method of Constant Stimuli with three rating categories and the use of Rasch Models. Qual. Quant. Int. J. Methodol. 45, 43–58 (2011)

    Article  Google Scholar 

  • Carter C.R., Kaufmann L., Michel A.: Behavioral supply management: a taxonomy of judgment and decision-making biases. Int. J. Phys. Distrib. Logist. Manag. 37, 631–669 (2007)

    Article  Google Scholar 

  • Costa P.T., McCrae R.R.: The NEO Personality Inventory Manual. Psychological Assessment Resources, Odessa (1985)

    Google Scholar 

  • Couteur A.L.: Early assessment and diagnosis of children. In: Roth, I., Rezaie, P. (eds) Researching the Autism Apectrum: Contemporary Perspectives, pp. 19–49. Cambridge University Press, New York (2011)

    Chapter  Google Scholar 

  • Craik K.H., Ware A.P., Kamp J., O’Reilly C., Staw B., Zedeck A.: Exploration of construct validity in a combined managerial and personality assessment programme. J. Occup. Organ. Psychol. 75, 171–193 (2002)

    Article  Google Scholar 

  • Creswell J.: Research Design: Qualitative, Quantitative and Mixed Method Approaches. Sage Publications, Thousand Oaks (2003)

    Google Scholar 

  • Creswell J., Clark V.: Designing and Conducting Mixed Methods Research. Sage Publications, Thousand Oaks (2007)

    Google Scholar 

  • Dawes R.M., Corrigan B.: Linear models in decision making. Psychol. Bull. 81, 95–106 (1974)

    Article  Google Scholar 

  • Doherty M.E., Mynatt C.R., Tweney R.D., Schiavo M.B.: Pseudodiagnosticity. Acta Psychol. 43, 111–121 (1979)

    Article  Google Scholar 

  • Doherty M.E., Schiavo M.B., Tweney R.D., Mynatt C.R.: The influence of feedback and diagnostic data on pseudodiagnosticity. Bull. Psychon. Soc. 18, 191–194 (1981)

    Google Scholar 

  • Dunlop P.D., Morrison D.L., Cordery J.L.: Investigating retesting effects in a personnel selection context. Int. J. Sel. Assess. 19, 217–221 (2011)

    Article  Google Scholar 

  • Ebbesen E., Konecni V.: Decision making and information integration in the courts: the setting of bail. J. Pers. Soc. Psychol. 32, 805–821 (1975)

    Article  Google Scholar 

  • Einhorn H.: Expert judgment: some necessary conditions and an example. J. Appl. Psychol. 59, 562–571 (1974)

    Article  Google Scholar 

  • Finn S.E., Tonsager M.E.: Information gathering and therapeutic models of assessment: complementary paradigms: assessment in psychological treatment: a necessary step for effective intervention. Psychol. Assess. 9, 374–385 (1997)

    Article  Google Scholar 

  • Furnham A., Jensen T., Crump J.: Personality, intelligence and assessment center experts ratings. Int. J. Sel. Assess. 16, 356–365 (2008)

    Article  Google Scholar 

  • Gaeth, G.J., Shanteau, J.: A bibliography of research on the effects of irrelevance in psychology. Appl. Psychol. Rep N° 81–13, Kansas State University (1981)

  • Gaeth G.J., Shanteau J.: Reducing the influence of irrelevant information on experienced decision makers. Organ. Behav. Hum. Perform. 33, 263–282 (1984)

    Article  Google Scholar 

  • Gaugler B.B., Rosenthal D.B., Thornton G.C., Bentson C.: Meta-analysis of assessment center validity. J. Appl. Psychol. 72, 493–511 (1987)

    Article  Google Scholar 

  • Goldberg L.R.: Simple models or simple processes? Some research on clinical judgments. Am. Psychol. 23, 483–496 (1968)

    Article  Google Scholar 

  • Goldstein H.W., Yusko K.P., Braverman E.P., Smith D.B., Chung B.: The role of cognitive ability in the subgroup differences and incremental validity of assessment center excersises. Pers. Psychol. 51, 357–374 (1998)

    Article  Google Scholar 

  • Haaland S., Christiansen N.D.: Implications of trait-activation theory for evaluating the construct validity of assessment center ratings. Pers. Psychol. 55, 137–163 (2002)

    Article  Google Scholar 

  • Haefner J.E.: Race, age, sex and competence as factors in employment selection of the disadvantaged. J. Appl. Psychol. 62, 199–202 (1977)

    Article  Google Scholar 

  • Hammond K.R., McClelland G.H., Mumpower J.: Human Judgment and Decision Making: Theories, Methods, and Procedures. Praeger, New York (1980)

    Google Scholar 

  • Hardison C.M.: Construct validity of assessment center overall ratings: an investigation of relationships with and incremental criterion validity over big 5 personality traits and cognitive ability. Diss. Abstr. Int. BL 66, 6959 (2006)

    Google Scholar 

  • Hathaway S.R., McKinley J.C.: A multiphasic personality schedule (Minnesota): I. Construction of the schedule. J. Psychol. 10, 249–254 (1940)

    Google Scholar 

  • Hoffman P., Slovic P., Rorer L.: An analysis of variance model for the assessment of configural cue utilization in clinical judgment. Psychol. Bull. 69, 338–349 (1968)

    Article  Google Scholar 

  • Howard A.: A reassessment of assessment centres. In: Riggio, R., Mayes, B. (eds) Assessment Centres: Research and Applications, pp. 13–52. Select Press, Novato (1997)

    Google Scholar 

  • Hunsley J., Meyer G.J.: The incremental validity of psychological testing and assessment: conceptual, methodological, and statistical issues. Psychol. Assess. 15, 446–455 (2003)

    Article  Google Scholar 

  • International Task Force on Assessment Center Guidelines: Guidelines for assessment center operations. Int. J. Sel. Assess. 17, 243–253 (2009)

    Google Scholar 

  • Kahneman D.: Judgment and decision making: a personal view. Psychol. Sci. 2, 142–145 (1991)

    Article  Google Scholar 

  • Kahneman D., Tversky A.: Subjective probability: a judgment of representativeness. Cogn. Psychol. 3, 430–454 (1972)

    Article  Google Scholar 

  • Kolk N.J, Born M.P., Van Der Flier H.: Three method factors explaining the low correlations between assessment center dimension ratings and scores on personality inventories. Eur. J. Personal. 18, 127–141 (2004)

    Article  Google Scholar 

  • Krause D., Kersting M., Heggestad E., Thornton G.: Incremental validity of assessment center ratings over cognitive ability tests. Int. J. Sel. Assess. 14, 360–371 (2006)

    Article  Google Scholar 

  • Kudisch J., Ladd R., Dobbins R.: New evidence on the construct validity of diagnostic assessment centres. In: Riggio, R., Mayes, B. (eds) Assessment Centres: Research and Applications, pp. 129–144. Select Press, Novato (1997)

    Google Scholar 

  • Lievens F., De Soete B.: Instruments for personnel selection in the 21st century: Research and practice. Gedrag en Organisatie 24, 18–42 (2011)

    Google Scholar 

  • Lievens F., De Fruyt F., Van Dam K.: Assessor’s use of personality traits in descriptions of assessment centre candidates: a five factors model perspective. J. Occup. Organ. Psychol. 74, 623–636 (2001)

    Article  Google Scholar 

  • Lingiardi V., Gazzillo F., Colli A., De Bei F., Tanzilli A., Di Giuseppe M., Nardelli N., Caristo C., Condino V., Gentile D., Dazzi N.: Diagnosis and assessment of personality, therapeutic alliance and clinical exchange in psychotherapy research. Res. Psychother. Psychopathol. Process Outcome 13, 98–125 (2011)

    Google Scholar 

  • Lowry P.: The assessment centre process. In: Riggio, R., Mayes, B. (eds) Assessment Centres: Research and Applications, pp. 53–62. Select Press, Novato (1997)

    Google Scholar 

  • Luthans F., Davis T.R.V.: Idiographic Versus Nomothetic Approaches to Research in Organizations. Defense Technical Information Center, Fort Belvoir (1981)

    Google Scholar 

  • Migliori V., Rolandi A.: Development Center. Etas, Milano (2000)

    Google Scholar 

  • Mlodinow L.: The Drunkard’s Walk. How Randomness Rules Our Lives. Pantheon Books, New York (2008)

    Google Scholar 

  • Moenks F.J.: Idiographic versus nomothetic approach. Eur. J. High Abil. 6, 137–142 (1995)

    Article  Google Scholar 

  • Morgan C., Murray H.A.: A method for investigating phantasies: the thematic apperception test. Arch. Neurol. Psychiatry 34, 289–294 (1935)

    Article  Google Scholar 

  • Mynatt C.R., Doherty M.E., Dragon W.: Information relevance, working memory, and the consideration of alternatives. Q. J. Exp. Psychol. 46, 759–778 (1993)

    Google Scholar 

  • Nisbett R.E., Wilson T.D.: The halo effect: evidence for unconscious alteration of judgments. J. Personal. Soc. Psychol. 35, 250–256 (1977)

    Article  Google Scholar 

  • Oskamp S.: Overconfidence in case study judgments. J. Consult. Psychol. 29, 261–265 (1965)

    Article  Google Scholar 

  • Oswald F.L., Hough L.M.: Personality and its assessment in organizations: theoretical and empirical developments. In: Zedeck, S. (ed) APA Handbook of Industrial and Organizational Psychology, Vol 2: Selecting and Developing Members for the Organization, pp. 153–184. American Psychological Association, Washington (2010)

    Google Scholar 

  • Portman M.E., Starcevic V., Beck A.T.: Challenges in assessment and diagnosis of generalized anxiety disorder. Psychiatr. Ann. 41, 79–85 (2011)

    Article  Google Scholar 

  • Proctor R.W., Lu C.H., Wang H., Dutta A.: Activation of response codes by relevant and irrelevant stimulus information. Acta Psychol. 90, 275–286 (1995)

    Article  Google Scholar 

  • Reilly, B.A., Doherty, M.E.: Self-insight in judgment processes. In: Psychonomic Society Meeting, Chicago, Nov 1988

  • Reilly B.A., Doherty M.E.: A note on the assessment of self-insight in judgment research. Organ. Behav. Hum. Decis. Process. 44, 123–131 (1989)

    Article  Google Scholar 

  • Rice M.F.: Influence of irrelevant biographical information in teacher evaluation. J. Educ. Psychol. 67, 658–662 (1975)

    Article  Google Scholar 

  • Riggio R., Mayes B.: Assessment Centres: Research and Applications. Select Press, Novato (1997)

    Google Scholar 

  • Rorschach H.: Psychodiagnostik: Methodik und Ergebnisse eines wahrnehmungsdiagnostichen Experiments (Deutenlasses von Zufallsformen). Bern: Hüber; trad. it 1981 Psicodiagnostica. Metodologia e risultati di un esperimento diagnostico basato sulla percezione (interpretazione di forme casuali). Kappa, Roma (1921)

  • Rotter J.B., Rafferty J.E.: Manual: The Rotter Incomplete Sentence Blank. Psychological Corporation, New York (1950)

    Google Scholar 

  • Rumiati R., Bonini N.: Le decisioni degli esperti. il Mulino, Bologna (1996)

    Google Scholar 

  • Sartori R.: Tecniche proiettive e strumenti psicometrici per l’indagine di personalità à—Approccio idiografico e approccio nomotetico a confronto. LED, Milano (2010)

    Google Scholar 

  • Sartori R.: Psychological assessment as a psychotherapy phase: a cognitive behavioral example between idiographic (clinical) approach and nomothetic (psychometric) approach. Riv. Psicol. Clin. 2, 67–79 (2010)

    Google Scholar 

  • Sartori R.: Face validity in personality tests: psychometric instruments and projective techniques in comparison. Qual. Quant. Int. J. Methodol. 44, 749–759 (2010)

    Article  Google Scholar 

  • Sartori R.: Metodi e tecniche di indagine e intervento in psicologia—Colloquio, intervista, questionario, test. LED, Milano (2011)

    Google Scholar 

  • Sartori R., Bortolani E.: Approccio idiografico e approccio nomotetico alla persona: il caso dei test psicologici. G. Ital. Psicol. 1, 107–118 (2006)

    Google Scholar 

  • Sartori R., Ceschi A.: Uncertainty and its perception: experimental study of the numeric expression of uncertainty in two decisional contexts. Qual. Quant. Int. J. Methodol. 45, 187–198 (2011)

    Article  Google Scholar 

  • Sartori, R., Rolandi, A.: Validation study of a model for the assessment of potential in Italian Young Professionals working in medium-sized to large companies in different business sectors. Qual. Quant. Int. J. Methodol. 1–9 (2012). doi:10.1007/s11135-012-9683-6

  • Sartori R., Tommasi M.: The effect of prior stimulus information upon category ratings. In: Berglund, B., Borg, E. (eds) Fechner Day 2003—Proceedings of the Nineteenth Annual Meeting of the International Society for Psychophysics, pp. 275–280. The International Society of Psychophysics, Larnaca (2003)

    Google Scholar 

  • Sartori R., Tommasi M., Vidotto G.: Contextual effects on the adjustment of category scales. In: Oliveira, A.M., Teixeira, M., Borges, G.F., Ferro, M.J. (eds) Fechner Day 2004—Proceedings of the Twentieth Annual Meeting of the International Society for Psychophysics, pp. 518–523. The International Society of Psychophysics, Coimbra (2004)

    Google Scholar 

  • Schalock R.L.: Quality of life, quality enhancement, and quality assurance: implications for program planning and evaluation in the field of mental retardation and developmental disabilities. Eval. Program Plan. 17, 121–131 (1994)

    Article  Google Scholar 

  • Shanteau J.: Averaging versus multiplying combination rules of inference judgment. Acta Psychol. 39, 83–89 (1975)

    Article  Google Scholar 

  • Shanteau J.: How much information does an expert use? Is it relevant?. Acta Psychol. 81, 75–86 (1992)

    Article  Google Scholar 

  • Shanteau J., Nagy G.F.: Information integration in person perception. In: Cook, M. (ed) Issues in Person Perception, Methuen, London (1984)

    Google Scholar 

  • Shanteau J., Stewart T.R.: Why study expert decision making? Some historical perspectives and comments. Organ. Behav. Hum. Decis. Process. 53, 95–106 (1992)

    Article  Google Scholar 

  • Shore L.M., Tetrick L.E., Shore T.H.: A comparison of self-, peer, and assessor evaluations of managerial potential. J. Soc. Behav. Personal. 13, 85–101 (1998)

    Google Scholar 

  • Slovic P.: Analyzing the expert judge: a descriptive study of a stockbroker’s decision processes. J. Appl. Psychol. 53, 255–263 (1969)

    Article  Google Scholar 

  • Stanovich K.E., Toplak M.E., West R.F.: The development of rational thought: A taxonomy of heuristics and biases. In: Robert, V.K. (ed) Advances in Child Development and Behavior, pp. 251–285. JAI, Greenwich (2008)

    Google Scholar 

  • Swaffield S.: Getting to the heart of authentic assessment for learning. Assess. Educ. Princ. Policy Pract. 18, 433–449 (2011)

    Article  Google Scholar 

  • Taleb N.N.: The Black Swan. The Impact of the Highly Improbable. Random House, New York (2007)

    Google Scholar 

  • Tett R.P., Christiansen N.D.: Personality assessment in organizations. In: Boyle, G.J., Matthews, G., Saklofske, D.H. (eds) The SAGE Handbook of Personality Theory and Assessment, Vol 1: Personality Theories and Models, pp. 720–742. Sage Publications, Thousand Oaks (2008)

    Chapter  Google Scholar 

  • Thornton G.C., Rupp D.E.: Assessment Center in Human Resource Management. Lawrence Erlbaum Associates Publishers, Mahwah (2006)

    Google Scholar 

  • Thornton G.C., Tziner A., Dahan M., Clevenger J., Meir E.: Construct validity of assessment centre judgments. In: Riggio, R., Mayes, B. (eds) Assessment Centres: Research and Applications, pp. 109–128. Select Press, Novato (1997)

    Google Scholar 

  • Tversky A., Kahneman D.: Availability: a heuristic for judging frequency and probability. Cogn. Psychol. 5, 207–232 (1973)

    Article  Google Scholar 

  • Tversky A., Kahneman D.: Judgment under uncertainty: heuristics and biases. Science 185, 1124–1131 (1974)

    Article  Google Scholar 

  • Vecchione M., Alessandri G., Barbaranelli C.: The five factor model in personnel selection: measurement equivalence between applicant and non-applicant groups. Personal. Individ. Differ. 52, 503–508 (2012)

    Article  Google Scholar 

  • Woodruffe C.: Do Assessment/Development Centers Use Optimum Evaluation Procedures? A Survey of Practice in UK Organizations. CIPD Publishing, London (1995)

    Google Scholar 

  • Woodruffe C.: Development and Assessment Centers. CIPD Publishing, London (2000)

    Google Scholar 

  • Zhang, J., Patel, V.L., Johnson, T.R., Shortliffe, E.H.: A cognitive taxonomy of medical errors. J. Biomed. Inform. 37, 193–204 (2004)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Philosophy, Education, Psychology, University of Verona, Lungadige Porta Vittoria 17, 37129, Verona, Italy

    Riccardo Sartori & Andrea Ceschi

Authors
  1. Riccardo Sartori
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrea Ceschi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Riccardo Sartori.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sartori, R., Ceschi, A. Assessment and development centers: judgment biases and risks of using idiographic and nomothetic approaches to collecting information on people to be evaluated and trained in organizations. Qual Quant 47, 3277–3288 (2013). https://doi.org/10.1007/s11135-012-9718-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-012-9718-z

Keywords

Search

Navigation