Advertisement

Psychology Meets Machine Learning: Interdisciplinary Perspectives on Algorithmic Job Candidate Screening

  • Cynthia C. S. LiemEmail author
  • Markus Langer
  • Andrew Demetriou
  • Annemarie M. F. Hiemstra
  • Achmadnoer Sukma Wicaksana
  • Marise Ph. Born
  • Cornelius J. König
Chapter
Part of the The Springer Series on Challenges in Machine Learning book series (SSCML)

Abstract

In a rapidly digitizing world, machine learning algorithms are increasingly employed in scenarios that directly impact humans. This also is seen in job candidate screening. Data-driven candidate assessment is gaining interest, due to high scalability and more systematic assessment mechanisms. However, it will only be truly accepted and trusted if explainability and transparency can be guaranteed. The current chapter emerged from ongoing discussions between psychologists and computer scientists with machine learning interests, and discusses the job candidate screening problem from an interdisciplinary viewpoint. After introducing the general problem, we present a tutorial on common important methodological focus points in psychological and machine learning research. Following this, we both contrast and combine psychological and machine learning approaches, and present a use case example of a data-driven job candidate assessment system, intended to be explainable towards non-technical hiring specialists. In connection to this, we also give an overview of more traditional job candidate assessment approaches, and discuss considerations for optimizing the acceptability of technology-supported hiring solutions by relevant stakeholders. Finally, we present several recommendations on how interdisciplinary collaboration on the topic may be fostered.

Keywords

Psychology Machine learning Job candidate screening Methodology Explainability Multimodal analysis Interdisciplinarity 

Notes

Acknowledgements

This chapter results from collaborative work in the Erasmus+ Strategic Partnership ‘Big Data in Psychological Assessment’ (2017–2019), ref. 2017-1-DE01-KA203-003569, which is funded by the European Commission.

References

  1. Achmadnoer Sukma Wicaksana (2017) Layered Regression Analysis on Multimodal Approach for Personality and Job Candidacy Prediction and Explanation. URL http://resolver.tudelft.nl/uuid:a527395d-f42c-426d-b80b-29c3b6478802
  2. Achmadnoer Sukma Wicaksana, Liem CCS (2017) Human-Explainable Features for Job Candidate Screening Prediction. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, vol 2017-July,  https://doi.org/10.1109/CVPRW.2017.212
  3. Ambady N, Bernieri FJ, Richeson JA (2000) Toward a histology of social behavior: Judgmental accuracy from thin slices of the behavioral stream. Advances in Experimental Social Psychology 32:201–271, https://doi.org/10.1016/S0065-2601(00)80006-4, URL https://www.sciencedirect.com/science/article/pii/S0065260100800064 CrossRefGoogle Scholar
  4. Anderson J (1983) Lix and Rix: Variations on a Little-known Readability Index. Journal of Reading 26(6):490–496, URL http://www.jstor.org/stable/40031755
  5. Anderson N, Herriot P, Hodgkinson GP (2001) The practitioner-researcher divide in industrial, work and organizational (IWO) psychology: Where are we now, and where do we go from here? Journal of Occupational and Organizational Psychology https://doi.org/10.1348/096317901167451 CrossRefGoogle Scholar
  6. Apers C, Derous E (2017) Are they accurate? Recruiters’ personality judgments in paper versus video resumes. Computers in Human Behavior https://doi.org/10.1016/j.chb.2017.02.063 CrossRefGoogle Scholar
  7. Baltrušaitis T, Mahmoud M, Robinson P (2015) Cross-dataset learning and person-specific normalisation for automatic Action Unit detection. In: FG, vol 06, pp 1–6, https://doi.org/10.1109/FG.2015.7284869, URL http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=7284869
  8. Bar-Yam Y (2016) The limits of phenomenology: From behaviorism to drug testing and engineering design. Complexity 21(S1):181–189,  https://doi.org/10.1002/cplx.21730, URL http://doi.wiley.com/10.1002/cplx.21730 MathSciNetCrossRefGoogle Scholar
  9. Barabas C, Dinakar K, Ito J, Virza M, Zittrain J (2018) Interventions over Predictions: Reframing the Ethical Debate for Actuarial Risk Assessment. In: Proceedings of the Conference on Fairness, Accountability and Transparency, Machine Learning Research, New York, vol 81, pp 1–15, URL http://proceedings.mlr.press/v81/barabas18a/barabas18a.pdf
  10. Barrick MR, Mount MK (1991) The big fice personality dimensions and job performance: A meta-analysis. Personnel Psychology 44:1–26, https://doi.org/10.1111/j.1744-6570.1991.tb00688.x CrossRefGoogle Scholar
  11. Bauer TN, Truxillo DM, Sanchez RJ, Craig JM, Ferrara P, Campion MA (2001) Applicant reactions to selection: Development of the Selection Procedural Justice Scale. Personnel Psychology 54:387–419, https://doi.org/10.1111/j.1744-6570.2001.tb00097.x CrossRefGoogle Scholar
  12. Bauer TN, Truxillo DM, Tucker JS, Weathers V, Bertolino M, Erdogan B, Campion MA (2006) Selection in the information age: The impact of privacy concerns and computer experience on applicant reactions. Journal of Management 32:601–621, https://doi.org/10.1177/0149206306289829 CrossRefGoogle Scholar
  13. Behrend TS, Sharek DJ, Meade AW, Wiebe EN (2011) The viability of crowdsourcing for survey research. Behavior Research Methods 43(3):800–813, https://doi.org/10.3758/s13428-011-0081-0, URL http://www.springerlink.com/index/10.3758/s13428-011-0081-0 CrossRefGoogle Scholar
  14. Bengio Y, Courville A, Vincent P (2013) Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(8):1798–1828,  https://doi.org/10.1109/TPAMI.2013.50 CrossRefGoogle Scholar
  15. Biel JI, Aran O, Gatica-Perez D (2011) You Are Known by How You Vlog: Personality Impressions and Nonverbal Behavior in YouTube. Artificial Intelligence pp 446–449, URL http://www.idiap.ch/~jibiel/pubs/BielAranGaticaICWSM11.pdf
  16. Biran O, Cotton C (2017) Explanation and justification in machine learning: A Survey. In: Proceedings of the 17th international joint conference on artificial intelligence IJCAI, Melbourne, Australia, pp 8–13Google Scholar
  17. Bishop CM (2006) Pattern Recognition and Machine Learning. Springer-VerlagGoogle Scholar
  18. Blacksmith N, Willford JC, Behrend TS (2016) Technology in the employment interview: A meta-analysis. Personnel Assessment and Decisions 2:2,  https://doi.org/10.25035/pad.2016.002 CrossRefGoogle Scholar
  19. Bobick AF, Davis JW (2001) The recognition of human movement using temporal templates. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(3):257–267, https://doi.org/10.1109/34.910878 CrossRefGoogle Scholar
  20. Bolukbasi T, Chang KW, Zou J, Saligrama V, Kalai A (2016) Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. In: Proceedings of the 30th Conference on Neural Information Processing Systems, Barcelona, URL https://papers.nips.cc/paper/6228-man-is-to-computer-programmer-as-woman-is-to-homemaker-debiasing-word-embeddings.pdf
  21. Borkenau P, Brecke S, Möttig C, Paelecke M (2009) Extraversion is accurately perceived after a 50-ms exposure to a face. Journal of Research in Personality 43(4):703–706, https://doi.org/10.1016/j.jrp.2009.03.007 CrossRefGoogle Scholar
  22. Bradley R, Terry M (1952) Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika 39(3/4):324–345, https://doi.org/10.2307/2334029, URL http://www.jstor.org/stable/10.2307/2334029 MathSciNetCrossRefGoogle Scholar
  23. Buolamwini J, Gebru T (2018) Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification *. In: Proceedings of the Conference on Fairness, Accountability and Transparency, Machine Learning Research, vol 81, pp 1–15, URL http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a.pdf
  24. Caliskan A, Bryson JJ, Narayanan A (2017) Semantics derived automatically from language corpora contain human-like biases. Science 356:183–186,  https://doi.org/10.1126/science.aal4230 CrossRefGoogle Scholar
  25. Campion MC, Campion MA, Campion ED, Reider MH (2016) Initial investigation into computer scoring of candidate essays for personnel selection. Journal of Applied Psychology 101:958–975,  https://doi.org/10.1037/apl0000108 CrossRefGoogle Scholar
  26. Caneel R (2005) Social Signaling in Decision Making. PhD thesis, Massachusetts Institute of Technology, URL http://groupmedia.media.mit.edu/datasets/Social_Signaling_in_Decision_Making.pdf
  27. Chamorro-Premuzic T, Winsborough D, Sherman RA, Hogan R (2018) New Talent Signals: Shiny New Objects or a Brave New World? Industrial and Organizational Psychology 9(3):621–640,  https://doi.org/10.1017/iop.2016.6 CrossRefGoogle Scholar
  28. Chapman DS, Webster J (2003) The use of technologies in the recruiting, screening, and selection processes for job candidates. International journal of selection and assessment 11:113–120, https://doi.org/10.1111/1468-2389.00234 CrossRefGoogle Scholar
  29. Chapman DS, Uggerslev KL, Webster J (2003) Applicant reactions to face-to-face and technology-mediated interviews: A field investigation. Journal of Applied Psychology 88:944–953, https://doi.org/10.1037/0021-9010.88.5.944 CrossRefGoogle Scholar
  30. Chapman DS, Uggerslev KL, Carroll SA, Piasentin KA, Jones DA (2005) Applicant attraction to organizations and job choice: A meta-analytic review of the correlates of recruiting outcomes. Journal of Applied Psychology 90:928–944, https://doi.org/10.1037/0021-9010.90.5.928 CrossRefGoogle Scholar
  31. Chen B, Escalera S, Guyon I, Ponce-Lopez V, Shah N, Simon MO (2016) Overcoming calibration problems in pattern labeling with pairwise ratings: Application to personality traits. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol 9915 LNCS, pp 419–432, https://doi.org/10.1007/978-3-319-49409-8_33
  32. Choi BC, Pak AW (2006) Multidisciplinarity, interdisciplinarity and transdisciplinarity in health research, services, education and policy: 1. Definitions, objectives, and evidence of effectiveness. Clinical and Investigative Medicine https://doi.org/10.1016/j.jaac.2010.08.010 Google Scholar
  33. Christiansen ND, Wolcott-Burnam S, Janovics JE, Burns GN, Quirk SW (2005) The good judge revisited: Individual differences in the accuracy of personality judgments. Human Performance 18, https://doi.org/10.1207/s15327043hup1802_2 CrossRefGoogle Scholar
  34. Coleman M, Liau TL (1975) A Computer Readability Formula Designed for Machine Scoring. Journal of Applied Psychology 60(2):283–284, https://doi.org/10.1037/h0076540, URL http://content.apa.org/journals/apl/60/2/283 CrossRefGoogle Scholar
  35. Collobert R, Weston J (2008) A unified architecture for natural language processing: deep neural networks with multitask learning deep neural networks with multitask learning. International Conference on Machine Learning https://doi.org/10.1145/1390156.1390177
  36. Cook M (2016) Personnel selection: adding value through people - a changing picture. Wiley-BlackwellGoogle Scholar
  37. Council of the European Union (2016) General Data Protection Regulation. URL http://data.consilium.europa.eu/doc/document/ST-5419-2016-INIT/en/pdf
  38. Crawford K, Calo R (2016) There is a blind spot in AI research. Nature 538(7625):311–313, https://doi.org/10.1038/538311a, URL http://www.nature.com/doifinder/10.1038/538311a CrossRefGoogle Scholar
  39. Cronbach LJ (1951) Coefficient alpha and the internal structure of tests. Psychometrika https://doi.org/10.1007/BF02310555 CrossRefGoogle Scholar
  40. Davis MH, Scharenborg O (2017) Speech perception by humans and machines. In: Gaskell G, Mirkovi∖’{c} J (eds) Speech Perception and Spoken Word Recognition., Routledge Psychology Press, chap 10, pp 181–204Google Scholar
  41. De Kock FS, Lievens F, Born MP (2015) An In-Depth Look at Dispositional Reasoning and Interviewer Accuracy. Human Performance https://doi.org/10.1080/08959285.2015.1021046 CrossRefGoogle Scholar
  42. De Kock FS, Lievens F, Born MP (2017) A closer look at the measurement of dispositional reasoning: Dimensionality and invariance across assessor groups. International Journal of Selection and Assessment  https://doi.org/10.1111/ijsa.12176 CrossRefGoogle Scholar
  43. Diekmann J, König CJ (2015) Personality testing in personnel selection: Love it? Leave it? Understand it! In: Nikolaou I, Oostrom J (eds) Employee recruitment, selection, and assessment: Contemporary issues for theory and practice, Psychology Press, Hove, UK, pp 117–135Google Scholar
  44. Dwork C, Hardt M, Pitassi T, Reingold O, Zemel R (2012) Fairness through awareness. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference on - ITCS ‘12, ACM Press, New York, New York, USA, pp 214–226, https://doi.org/10.1145/2090236.2090255, URL http://dl.acm.org/citation.cfm?doid=2090236.2090255
  45. Ekman P, Friesen WV (1978) Facial Action Coding System: A Technique for the Measurement of Facial MovementGoogle Scholar
  46. Ekman P, Rosenberg E (2005) What the face reveals. Oxford University PressGoogle Scholar
  47. Escalante HJ, Guyon I, Escalera S, Jacques J, Madadi M, Baro X, Ayache S, Viegas E, Gucluturk Y, Guclu U, Van Gerven MA, Van Lier R (2017) Design of an explainable machine learning challenge for video interviews. In: Proceedings of the International Joint Conference on Neural Networks,  https://doi.org/10.1109/IJCNN.2017.7966320
  48. Escalante HJ, Kaya H, Albert, Salah A, Escalera S, Gmur Güçlütürk Y, Güçlü U, Baró X, Guyon I, Junior JJ, Madadi M, Ayache S, Viegas E, Gürpinar F, Achmadnoer, Wicaksana S, Liem CCS, Van Gerven MAJ, Van Lier R, Salah AA (2018) Explaining First Impressions: Modeling, Recognizing, and Explaining Apparent Personality from Videos. ArXiv e-prints 1802.00745 Google Scholar
  49. Flesch R (1948) A New Readability Yardstick. The Journal of Applied Psychology 32(3):221–233, https://doi.org/10.1037/h0057532 CrossRefGoogle Scholar
  50. Funder DC (1999) Personality judgment: a realistic approach to person perception. Academic PressGoogle Scholar
  51. Funder DC (2012) Accurate Personality Judgment. Current Directions in Psychological Science https://doi.org/10.1177/0963721412445309 CrossRefGoogle Scholar
  52. Furr RM, Bacharach VR (2014) Psychometrics: an introduction, second edition edn. SAGE PublicationsGoogle Scholar
  53. Gilliland SW (1993) The perceived fairness of selection systems: An organizational justice perspective. Academy of Management Review 18:694–734, https://doi.org/10.2307/258595 CrossRefGoogle Scholar
  54. Gilmore DC, Beehr TA, Love KG (1986) Effects of applicant sex, applicant physical attractiveness, type of rater and type of job on interview decisions*. Journal of Occupational Psychology 59:103–109CrossRefGoogle Scholar
  55. Goodfellow I, Bengio Y, Courville A (2016) Deep Learning. MIT PressGoogle Scholar
  56. Goodfellow IJ, Shlens J, Szegedy C (2015) Explaining and Harnessing Adversarial Examples. In: Proceedings of the 3rd International Conference on Learning Representations (ICLR2015), San Diego, URL https://arxiv.org/pdf/1412.6572.pdf
  57. Gorbova J, Lusi I, Litvin A, Anbarjafari G (2017) Automated Screening of Job Candidate Based on Multimodal Video Processing. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE, pp 1679–1685,  https://doi.org/10.1109/CVPRW.2017.214, URL http://ieeexplore.ieee.org/document/8014948/
  58. Graves A, Jaitly N (2014) Towards End-To-End Speech Recognition with Recurrent Neural Networks. In: Proceedings of the 31st International Conference on Machine Learning (ICML-14)Google Scholar
  59. Guion RM (2011) Assessment, measurement, and prediction for personnel decisions. RoutledgeGoogle Scholar
  60. Gunning R (1952) The Technique of Clear Writing. McGraw-Hill, URL https://books.google.nl/books?id=ofI0AAAAMAAJ Google Scholar
  61. Hall GS (1917) Practical relations between psychology and the war. Journal of Applied Psychology https://doi.org/10.1037/h0070238 CrossRefGoogle Scholar
  62. Hamel P, Eck D (2010) Learning Features from Music Audio with Deep Belief Networks. In: International Society for Music Information Retrieval Conference (ISMIR)Google Scholar
  63. Henrich J, Heine SJ, Norenzayan A (2010) The weirdest people in the world? Behavioral and Brain Sciences 33(2–3):61–83, https://doi.org/10.1017/S0140525X0999152X CrossRefGoogle Scholar
  64. Hiemstra AM, Derous E, Serlie AW, Born MP (2012) Fairness Perceptions of Video Resumes among Ethnically Diverse Applicants. International Journal of Selection and Assessment  https://doi.org/10.1111/ijsa.12005 CrossRefGoogle Scholar
  65. Hofstadter D (2018) The Shallowness of Google Translate, The Atlantic. URL https://www.theatlantic.com/technology/archive/2018/01/the-shallowness-of-google-translate/551570/
  66. Hough LM, Oswald FL, Ployhart RE (2001) Determinants, detection and amelioration of adverse impact in personnel selection procedures: Issues, evidence and lessons learned. International Journal of Selection and Assessment 9:152–194, https://doi.org/10.1111/1468-2389.00171 CrossRefGoogle Scholar
  67. Humphries M, Gurney K, Prescott T (2006) The brainstem reticular formation is a small-world, not scale-free, network. Proceedings of the Royal Society B: Biological Sciences 273(1585):503–511,  https://doi.org/10.1098/rspb.2005.3354, URL http://rspb.royalsocietypublishing.org/cgi/doi/10.1098/rspb.2005.3354 CrossRefGoogle Scholar
  68. Kaya H, Gurpinar F, Salah AA (2017) Multi-modal Score Fusion and Decision Trees for Explainable Automatic Job Candidate Screening from Video CVs. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE, pp 1651–1659,  https://doi.org/10.1109/CVPRW.2017.210, URL http://ieeexplore.ieee.org/document/8014944/
  69. Kim J, Urbano J, Liem CCS, Hanjalic A (2018) One Deep Music Representation to Rule Them All? A comparative analysis of different representation learning strategies. ArXiv e-prints 1802.00745 Google Scholar
  70. Kincaid JP, Fishburne RP, Rogers RL, Chissom BS (1975) Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel. Technical Training Research B(February):49, https://doi.org/ERIC:ED108134, URL http://www.dtic.mil/dtic/tr/fulltext/u2/a006655.pdf
  71. Klehe UC (2004) Choosing how to choose: Institutional pressures affecting the adoption of personnel selection procedures. International Journal of Selection and Assessment 12:327–342, https://doi.org/10.1111/j.0965-075x.2004.00288.x CrossRefGoogle Scholar
  72. König CJ, Klehe UC, Berchtold M, Kleinmann M (2010) Reasons for being selective when choosing personnel selection procedures. International Journal of Selection and Assessment 18:17–27, https://doi.org/10.1111/j.1468-2389.2010.00485.x CrossRefGoogle Scholar
  73. König CJ, Steiner Thommen LA, Wittwer AM, Kleinmann M (2017) Are observer ratings of applicants’ personality also faked? yes, but less than self-reports. International Journal of Selection and Assessment 25:183–192,  https://doi.org/10.1111/ijsa.12171 CrossRefGoogle Scholar
  74. Langer M, König CJ, Krause K (2017a) Examining digital interviews for personnel selection: Applicant reactions and interviewer ratings. International Journal of Selection and Assessment 25:371–382,  https://doi.org/10.1111/ijsa.12191 CrossRefGoogle Scholar
  75. Langer M, König CJ, Papathanasiou M (2017b) User reactions to novel technologies in selection and training contexts. In: Annual meeting of the Society for Industrial and Organizational Psychology (SIOP), Orlando, FLGoogle Scholar
  76. Langer M, König CJ, Fitili A (2018) Information as a double-edged sword: The role of computer experience and information on applicant reactions towards novel technologies for personnel selection. Computers in Human Behavior 81:19–30, https://doi.org/10.1016/j.chb.2017.11.036 CrossRefGoogle Scholar
  77. Liem CCS, Müller M, Eck D, Tzanetakis G, Hanjalic A (2011) The need for music information retrieval with user-centered and multimodal strategies. In: MM’11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM’11, pp 1–6, https://doi.org/10.1145/2072529.2072531
  78. Liem CCS, Rauber A, Lidy T, Lewis R, Raphael C, Reiss JD, Crawford T, Hanjalic A (2012) Music Information Technology and Professional Stakeholder Audiences: Mind the Adoption Gap. In: Dagstuhl Follow-Ups, vol 3, Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik,  https://doi.org/10.4230/DFU.VOL3.11041.227, URL http://drops.dagstuhl.de/opus/volltexte/2012/3475/
  79. Long J, Shelhamer E, Darrell T (2015) Fully Convolutional Networks for Semantic Segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),  https://doi.org/10.1109/CVPR.2015.7298965
  80. Mazzocchi F (2016) Could Big Data be the end of theory in science? EMBO reports 16(10),  https://doi.org/10.15252/embr.201541001, URL https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4766450/pdf/EMBR-16-1250.pdf CrossRefGoogle Scholar
  81. McCrae RR, Costa PT Jr (1999) The five-factor theory of personality. In: Handbook of Personality: Theory and Research, Guilford Press, https://doi.org/10.1007/978-1-4615-0763-5_11 CrossRefGoogle Scholar
  82. McLaughlin G (1969) SMOG grading: A new readability formula. Journal of reading 12(8):639–646, https://doi.org/10.1039/b105878a, URL http://www.jstor.org/stable/40011226
  83. Miller T (2017) Explanation in Artificial Intelligence: Insights from the Social Sciences. ArXiv e-prints 1706.07269 Google Scholar
  84. Morgeson FP, Campion MA, Dipboye RL, Hollenbeck JR, Murphy K, Schmitt N (2007) Reconsiderung the use of personality tests in personnel selection contexts. Personnel Psychology 60:683–729, https://doi.org/10.1111/j.1744-6570.2007.00089.x CrossRefGoogle Scholar
  85. Mori M, MacDorman K, Kageki N (2012) The uncanny valley. IEEE Robotics & Automation Magazine 19:98–100,  https://doi.org/10.1109/MRA.2012.2192811 CrossRefGoogle Scholar
  86. Naim I, Tanveer MI, Gildea D, Hoque ME (2015) Automated analysis and prediction of job interview performance: The role of what you say and how you say it. In: 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, Ljubljana, Slovenia, pp 1–14, https://doi.org/10.1109/fg.2015.7163127
  87. Nass C, Brave S (2005) Wired for Speech: How Voice Activates and Advances the Human Computer Relationship. Computational Linguistics 32(3):451–452,  https://doi.org/10.1162/coli.2006.32.3.451, URL https://www.lib.uwo.ca/cgi-bin/ezpauthn.cgi?url=http://search.proquest.com/docview/1037392793?accountid=15115%5Cnhttp://vr2pk9sx9w.search.serialssolutions.com/?ctx%7B_%7Dver=Z39.88-2004%7B&%7Dctx%7B_%7Denc=info:ofi/enc:UTF-8%7B&%7Drfr%7B_%7Did=info:sid/ProQ%7B%25%7D3Aeducation%7B&%7Drft%7B CrossRefGoogle Scholar
  88. Naumann LP, Vazire S, Rentfrow PJ, Gosling SD (2009) Personality judgments based on physical appearance. Personality and social psychology bulletin 35(12):1661–1671, https://doi.org/10.1177/0146167209346309 CrossRefGoogle Scholar
  89. Nguyen LS, Frauendorfer D, Mast MS, Gatica-Perez D (2014) Hire me: Computational inference of hirability in employment interviews based on nonverbal behavior. IEEE Transactions on Multimedia  https://doi.org/10.1109/TMM.2014.2307169 CrossRefGoogle Scholar
  90. Oh IS, Wang G, Mount MK (2011) Validity of Observer Ratings of the Five-Factor Model of Personality Traits: A Meta-Analysis. Journal of Applied Psychology https://doi.org/10.1037/a0021832 CrossRefGoogle Scholar
  91. Peck JA, Levashina J (2017) Impression management and interview and job performance ratings: A meta-analysis of research design with tactics in mind. Frontiers in Psychology  https://doi.org/10.3389/fpsyg.2017.00201
  92. Pentland A (2004) Social Dynamics: Signals and Behavior. Proceedings of the 3rd International Conference on Developmental Learning, Oct 2004 5:263–267, URL http://vismod.media.mit.edu//tech-reports/TR-579.pdf
  93. Ployhart RE, Schmitt N, Tippins NT (2017) Solving the Supreme Problem: 100 Years of selection and recruitment at the Journal of Applied Psychology. Journal of Applied Psychology 102:291,  https://doi.org/10.1037/apl0000081.supp CrossRefGoogle Scholar
  94. Ponce-López V, Chen B, Oliu M, Corneanu C, Clapés A, Guyon I, Baró X, Escalante HJ, Escalera S (2016) ChaLearn LAP 2016: First Round Challenge on First Impressions - Dataset and Results. In: European Conference on Computer Vision (ECCV 2016) Workshops, Springer, Amsterdam, pp 400–418, https://doi.org/10.1007/978-3-319-49409-8_32, URL https://link.springer.com/chapter/10.1007/978-3-319-49409-8_32
  95. Pulakos ED, Schmitt N (1995) Experience-based and situational interview questions: Studies of validity. Personnel Psychology 48:289–308, https://doi.org/10.1111/j.1744-6570.1995.tb01758.x CrossRefGoogle Scholar
  96. Russell SJ, Norvig P (2010) Artificial Intelligence - A Modern Approach (3. internat. ed.). Pearson EducationGoogle Scholar
  97. Ryan AM, McFarland L, Shl HB, Page R (1999) An International Look At Selection Practices: Nation and Culture As Explanations for Variability in Practice. Personnel Psychology https://doi.org/10.1111/j.1744-6570.1999.tb00165.x CrossRefGoogle Scholar
  98. Schmidt FL, Hunter JE (1998) The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological bulletin (1998)Google Scholar
  99. Sitser T (2014) Predicting sales performance: Strengthening the personality – job performance linkage. PhD thesis, Erasmus University Rotterdam, URL https://repub.eur.nl/pub/51139/
  100. Smeulders A, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence https://doi.org/10.1109/34.895972 CrossRefGoogle Scholar
  101. Smith EA, Senter RJ (1967) Automated readability index. AMRL-TR Aerospace Medical Research Laboratories (6570th) pp 1–14Google Scholar
  102. Smith M (1994) A theory of the validity of predictors in selection. Journal of Occupational and Organizational Psychology https://doi.org/10.1111/j.2044-8325.1994.tb00546.x CrossRefGoogle Scholar
  103. van Sonderen E, Sanderman R, Coyne JC (2013) Ineffectiveness of reverse wording of questionnaire items: let’s learn from cows in the rain. PloS one 8(7):e68,967,  https://doi.org/10.1371/journal.pone.0068967, URL http://www.ncbi.nlm.nih.gov/pubmed/23935915 http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC3729568 CrossRefGoogle Scholar
  104. Sturm BL (2014) A simple method to determine if a music information retrieval system is a ‘horse’. IEEE Transactions on Multimedia  https://doi.org/10.1109/TMM.2014.2330697 CrossRefGoogle Scholar
  105. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT PressGoogle Scholar
  106. Urbano J, Schedl M, Serra X (2013) Evaluation in music information retrieval. Journal of Intelligent Information Systems https://doi.org/10.1007/s10844-013-0249-4 CrossRefGoogle Scholar
  107. Watts DJ, Strogatz SH (1998) Collective dynamics of’small-world’networks. Nature 393(6684):440–442, https://doi.org/10.1038/30918, URL http://202.121.182.16/Course/slides2012/NetSci-2012-7.pdf, 0803.0939v1 CrossRefGoogle Scholar
  108. Waung M, Hymes RW, Beatty JE (2014) The Effects of Video and Paper Resumes on Assessments of Personality, Applied Social Skills, Mental Capability, and Resume Outcomes. Basic and Applied Social Psychology 36(3):238–251, https://doi.org/10.1080/01973533.2014.894477, URL http://www.tandfonline.com/doi/abs/10.1080/01973533.2014.894477 CrossRefGoogle Scholar
  109. Youyou W, Kosinski M, Stillwell D (2015) Computer-based personality judgments are more accurate than those made by humans. Proceedings of the National Academy of Sciences of the United States of America 112(4):1036–40,  https://doi.org/10.1073/pnas.1418680112, URL http://www.ncbi.nlm.nih.gov/pubmed/25583507 http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC4313801 CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Cynthia C. S. Liem
    • 1
    Email author
  • Markus Langer
    • 2
  • Andrew Demetriou
    • 1
  • Annemarie M. F. Hiemstra
    • 3
  • Achmadnoer Sukma Wicaksana
    • 4
  • Marise Ph. Born
    • 3
  • Cornelius J. König
    • 2
  1. 1.Multimedia Computing GroupDelft University of TechnologyDelftThe Netherlands
  2. 2.Universität des SaarlandesSaarbrückenGermany
  3. 3.Erasmus School of Social and Behavioral SciencesErasmus UniversityRotterdamThe Netherlands
  4. 4.Datasintesa Teknologi NusantaraJakartaIndonesia

Personalised recommendations