Abstract
Competency self-assessment for Performance Appraisal is receiving increasing attention from both researchers and practitioners. Nevertheless, the accuracy and supposed legitimacy of this type of assessment is still an issue. In the context of an industrial use case, we aim to develop and validate a computer-based competency self-assessment technology able to import any type of competency document (for performance appraisal, training need identification, career guidance) and generate semi-automatically self-assessment items. Following the model of Appraisal Effectiveness designed by Levy and Williams, our goal was to build an effective tool meaning that several perspectives must be taken into account: psychometric, cognitive, psychological, political and the reaction’s perspective. In this paper, we will only focus on one specific psychometric property (interrater reliability between an employee and its supervisor). According to a specific rating process and format, our Cross Skill™ technology showed promising results related to interrater reliability in a use case with bank officers and their supervisor.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Employees, supervisors and HR department in charge of building and updating Competency model and deploying related HR processes.
- 2.
We extended this consideration to competency.
- 3.
The privileged criteria by end-users when choosing assessment tool are cost, practicality, legality and not always validity. See [12].
- 4.
Comment about the potential consequences are in Sect. 6.2.
- 5.
The detailed list is under patent filing. https://www.google.com/patents/EP3188103A1?cl=en.
- 6.
The masculine is used in this publication without prejudice for the sake of conciseness.
- 7.
KSA taxonomy, generic proficiency criteria in our item templates instead of specific criteria- for each competency, random sequencing, adaptive test, etc.
References
Aguinis, H.: Performance Management. Pearson Prentice Hall, Upper Saddle River (2013)
Lucia, A.D., Lepsinger, R.: The Art and Science of Competency Models: Pinpointing Critical Success Factors in Organizations. Jossey-Bass, San Francisco (1999)
Ras, E., Baudet, A., Foulonneau, M.: A hybrid engineering process for semi-automatic item generation. In: Joosten-ten Brinke, D., Laanpere, M. (eds.) TEA 2016. CCIS, vol. 653, pp. 105–116. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-57744-9_10
Campion, M.A., Fink, A.A., Ruggeberg, B.J., Carr, L., Phillips, G.M., Odman, R.B.: Doing competencies well: best practices in competency modeling. Pers. Psychol. 64, 225–262 (2011)
Spencer, L.M., Spencer, S.M.: Competence at Work. Models for Superior Performance. Wiley, New York (1993)
Vincent, C., Rainey, R., Faulkner, D., Mascio, C., Zinda, M.: How often should job descriptions be updated? Annual Graduate Conference in Industrial-Organizational Psychology and Organizational Behavior, Indianapolis, IN (2007)
Levy, P.E., Williams, J.R.: The social context of performance appraisal: a review and framework for the future. J. Manag. 30, 881–905 (2004)
Saint-Onge, S., Morin, D., Bellehumeur, M., Dupuis, F.: Manager’s motivation to evaluate subordinate performance. Qual. Res. Organ. Manag.: Int. J. 4, 272–293 (2009)
Darr, W., Borman, W., St-Pierre, L., Kubisiak, C., Grossman, M.: An applied examination of the computerized adaptive rating scale for assessing performance. Int. J. Sel. Assess. 25, 149–153 (2017)
Conway, J.M., Huffcutt, A.I.: Psychometric properties of multi-source performance ratings: a meta-analysis of subordinate, supervisor, peer, and self-ratings. Hum. Perform. 10, 331–360 (1997)
Atwater, L.E., Yammarino, F.J.: Does self-other agreement on leadership perceptions moderate the validity of leadership and performance predictions? Pers. Psychol. 45, 141–164 (1992)
Furnham, A.: HR professionals’ beliefs about, and knowledge of, assessment techniques and psychometric tests. Int. J. Sel. Assess. 16, 300–305 (2008)
Baudet, A., Gronier, G., Latour, T., Martin, R.: L’auto-évaluation des compétences assistée par ordinateur: validation d’un outil de gestion des carrières. In: Bobillier Chaumon, M.E., Dubois, M., Vacherand-Revel, J., Sarnin, P., Kouabenan, R. (eds.) La question de la gestion des parcours professionnels en psychologie du travail. L’Harmattan, Paris (2013)
Winterton, J., Delamare Le Deist, F., Stringfellow, E.: Typology of Knowledge, Skills and Competences: Clarification of the Concept and Prototype. Cedefop Reference Series, vol. 64. Office for Official Publications of the European Communities, Luxembourg (2006)
Luecht, R.M.: An introduction to assessment engineering for automatic item generation. In: Gierl, M.J., Haladyna, T.M. (eds.) Automatic Item Generation. Routledge, New York (2013)
Goffin, R.D., Boyd, A.C.: Faking and personality assessment in personnel selection: advancing models of faking. Can. Psychol. 50, 151–160 (2009)
Alliger, G., Lilienfeld, S., Mitchell, K.: The susceptibility of overt and covert integrity tests to coaching and faking. Psychol. Sci. 7, 32–39 (1996)
Furnahm, A.: Response bias, social desirability and dissimulation. Pers. Individ. Differ. 7, 385–406 (1986)
Tett, R.P., Christiansen, N.D.: Personality tests at the crossroads: a response to Morgeson, Campion, Dipboye, Hollenbeck, Murphy, and Schmitt. Pers. Psychol. 60, 967–993 (2007)
Kline, T.: Psychological testing: a practical approach to design and evaluation. Sage Publications, Thousand Oaks (2005)
Viswesvaran, C., Ones, D.S., Schmidt, F., Le, H., Oh, I.-S.: Measurement error obfuscates scientific knowledge: path to cumulative knowledge requires corrections for unreliability and psychometric meta-analyses. Ind. Organ. Psychol. 7, 505–518 (2014)
Foucher, R., Morin, D., Saint-Onge, S.: Mesurer les compétences déployées en cours d’emploi: un cadre de référence. In: Foucher, R. (ed.) Gérer les talents et les compétences, Tome 2, pp. 151–222. Editions Nouvelles, Montréal (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Baudet, A., Ras, E., Latour, T. (2018). Semi-automatic Generation of Competency Self-assessments for Performance Appraisal. In: Ras, E., Guerrero Roldán, A. (eds) Technology Enhanced Assessment. TEA 2017. Communications in Computer and Information Science, vol 829. Springer, Cham. https://doi.org/10.1007/978-3-319-97807-9_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-97807-9_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97806-2
Online ISBN: 978-3-319-97807-9
eBook Packages: Computer ScienceComputer Science (R0)