Advertisement

Simulations in the Selection Context: Considerations, Challenges, and Opportunities

  • Anthony S. Boyce
  • Christine E. Corbet
  • Seymour Adler
Chapter

Abstract

This chapter provides an overview of the state of the art and science of using simulations for employee selection. We begin by examining the validity, flexibility, credibility, and downsides of using simulations for selection. Next, we explore key issues to be considered during the design and implementation of simulations (e.g., response and stimuli fidelity, scoring approach, and use of technology) and how these issues can impact validity and utility. Then, we consider the use of simulations in the context of the candidate lifecycle from the perspective of both individuals and organizations: before candidates engage in a simulation (i.e., during the attraction and recruitment stages), while candidates are participating in the simulation (e.g., the simulation experience, test anxiety and administration costs), and after the simulation is complete (e.g., candidate reactions, likelihood of challenge to process or results, and making use of simulation data). We conclude with a brief exploration of how changes in technology are likely to impact the nature of simulations and how candidates interact with them in the future. Throughout the chapter, we offer practical guidance, highlight areas in need of additional research, and share strategies for effective implementation of simulations within organizational selection contexts.

Keywords

Employee selection Job simulations Assessment centers Situational judgment tests Work samples Fidelity Internet testing 

References

  1. Arthur, W., Jr., & Villado, A. J. (2008). The importance of distinguishing between constructs and methods when comparing predictors in personnel selection research and practice. Journal of Applied Psychology, 93, 435–442.PubMedCrossRefGoogle Scholar
  2. Arthur, W., Jr., Day, E. A., McNelly, T. L., & Edens, P. S. (2003). A meta-analysis of the criterion-related validity of assessment center dimensions. Personnel Psychology, 56, 125–154.CrossRefGoogle Scholar
  3. Barr, R., & Coughlin, C. (2013). Software simulations. In M. S. Fetzer & K. A. Tuzinski (Eds.), Simulations for personnel selection. New York: Springer.Google Scholar
  4. Bruk-Lee, V., Drew, E., & Hawkes, B. (2013). Applicant reactions. In M. S. Fetzer & K. A. Tuzinski (Eds.), Simulations for personnel selection. New York: Springer.Google Scholar
  5. Cascio, W. F., Aguinis, H. (2005). Applied psychology in human resource management (6th ed.). Upper Saddle River: Pearson/Prentice Hall.Google Scholar
  6. Chan, D., & Schmitt, N. (1997). Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82, 143–159.PubMedCrossRefGoogle Scholar
  7. Chan, D., & Schmitt, N. (2004). An agenda for future research on applicant reactions to selection procedures: A construct-oriented approach. International Journal of Selection and Assessment, 12, 9–23.CrossRefGoogle Scholar
  8. Chapman, D. S., Uggerslev, K. L., Carroll, S. A., Piasentin, K. A., & Jones, D. A. (2005). Applicant attraction to organizations and job choice: A meta-analytic review of the correlates of recruiting outcomes. Journal of Applied Psychology, 90, 928–944.PubMedCrossRefGoogle Scholar
  9. Christian, M. S., Edwards, B. D., & Bradley, J. C. (2010). Situational judgment tests: Constructs assessed and a meta-analysis of their criterion-related validities. Personnel Psychology, 63, 83–117.CrossRefGoogle Scholar
  10. Downs, S., Farr, R. M., & Colbeck, L. (1978). Self appraisal: A convergence of selection and appraisal. Journal of Occupational Psychology, 51, 271–278.CrossRefGoogle Scholar
  11. Earnest, D. R., Allen, D. G., & Landis, R. S. (2011). Mechanisms linking realistic job previews with turnover: A meta-analytic path analysis. Personnel Psychology, 64, 865–897.CrossRefGoogle Scholar
  12. Felker, D. B., Curtin, P. J., & Rose, A. M. (2007). Tests of Job Performance. In D. L. Whetzel & G. R. Wheaton (Eds.), Applied measurement: Industrial psychology in human resources management (pp. 319–348). Mahwah: Lawrence Erlbaum.Google Scholar
  13. Feltham, R. (1988). Assessment centre decision making: Judgmental vs. mechanical. Journal of Occupational Psychology, 61, 237–241.CrossRefGoogle Scholar
  14. Funke, U., & Schuler, H. (1998). Validity of stimulus and response components in a video test of social competence. International Journal of Selection and Assessment, 6, 115–123.CrossRefGoogle Scholar
  15. Gaugler, B. B., Rosenthal, D. B., Thornton, G. C., & Bentson, C. (1987). Meta-analysis of assessment center validity. Journal of Applied Psychology, 72, 493–511.CrossRefGoogle Scholar
  16. Gilliland, S. W. (1993). The perceived fairness of selection systems: An organizational justice perspective. Academy of Management Review, 18, 694–734.Google Scholar
  17. Gilliland, S. W., & Cherry, B. (2000). Managing customers of selection. In J. F. Kehoe (Ed.), Managing selection in changing organizations (pp. 158–196). San Francisco: Jossey-Bass.Google Scholar
  18. Hausknecht, J. P., Day, D. V., & Thomas, S. C. (2004). Applicant reactions to selection procedures: An updated model and meta-analysis. Personnel Psychology, 57, 639–683.CrossRefGoogle Scholar
  19. Hawkes, B. (2013). Simulation Technologies. In M. S. Fetzer & K. A. Tuzinski (Eds.), Simulations for personnel selection. New York: Springer.Google Scholar
  20. Higgs, A. C., Papper, E. M., & Carr, L. S. (2000). Integrating selection with other organizational processes and systems. In J. F. Kehoe (Ed.), Managing selection in changing organizations (pp. 73–122). San Francisco: Jossey-Bass.Google Scholar
  21. Hoffman, C. C., & Thornton III, G. C. (1997). Examining selection utility where competing predictors differ in adverse impact. Personnel Psychology, 50, 455–470.CrossRefGoogle Scholar
  22. Jansen, P. G. W. (2012). How to apply a dimension-based assessment center. In D. J. R. Jackson, C. E. Lance & B. J. Hoffman (Eds.), The psychology of assessment centers (pp. 121–140). New York: Routledge.Google Scholar
  23. Jones, S., & Fox, S. (2009). Generations online in 2009. Pew Internet & American Life Project. http://pewinternet.org/Reports/2009/Generations-Online-in-2009.aspx. Accessed 2 Jan 2013.
  24. Lievens, F., & Conway, J. M. (2001). Dimension and exercise variance in assessment center scores: A large-scale evaluation of multitrait-multimethod studies. Journal of Applied Psychology, 86, 1202–1222.PubMedCrossRefGoogle Scholar
  25. Lievens, F., & Patterson, F. (2011). The validity and incremental validity of knowledge tests, low-fidelity simulations, and high-fidelity simulations for predicting job performance in advanced-level high-stakes selection. Journal of Applied Psychology, 96, 927.PubMedCrossRefGoogle Scholar
  26. Lievens, F., & Sackett, P. R. (2006). Video-based versus written situational judgment tests: A comparison in terms of predictive validity. Journal of Applied Psychology, 91, 1181–1188.PubMedCrossRefGoogle Scholar
  27. McDaniel, M. A., Morgeson, F. P., Finnegan, E. B., Campion, M. A., & Braverman, E. P. (2001). Use of situational judgment tests to predict job performance: A clarification of the literature. Journal of Applied Psychology, 86, 730–740.Google Scholar
  28. McDaniel, M. A., Whetzel, D. L., Hartman, N. S., Nguyen, N., & Grubb, W. L. (2006). Situational judgment tests: Validity and an integrative model. In R. Ployhart & J. Weekley (Eds.), Situational judgment tests: Theory, measurement, and application (pp. 183–204). San Francisco: Jossey-Bass.Google Scholar
  29. McDaniel, M. A., Hartman, N. S., Whetzel, D. L., & Grubb, W. L. (2007). Situational judgment tests, response instructions, and validity: A meta-analysis. Personnel Psychology, 60, 63–91.CrossRefGoogle Scholar
  30. Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology, 75, 640.CrossRefGoogle Scholar
  31. Muchinsky, P. M. (2004). When the psychometrics of test development meets organizational realities: A conceptual framework for organizational change, examples, and recommendations. Personnel Psychology, 57, 175–209.CrossRefGoogle Scholar
  32. Naglieri, J. A., Drasgow, F., Schmit, M., Handler, L., Prifitera, A., Margolis, A., et al. (2004). Psychological testing on the internet: New problems, old issues. American Psychologist, 59, 150–162.PubMedCrossRefGoogle Scholar
  33. Patterson, F., Baron, H., Carr, V., Plint, S., & Lane, P. (2009). Evaluation of three short-listing methodologies for selection into postgraduate training in general practice. Medical Education, 43, 50–57.PubMedCrossRefGoogle Scholar
  34. Ployhart, R. E. (2006). Staffing in the 21st century: New challenges and strategic opportunities. Journal of Management, 32, 868–897.CrossRefGoogle Scholar
  35. Ployhart, R. E., & Holtz, B. C. (2008). The diversity–validity dilemma: Strategies for reducing racioethnic and sex subgroup differences and adverse impact in selection. Personnel Psychology, 61, 153–172.CrossRefGoogle Scholar
  36. Richman-Hirsch, W. L., Olson-Buchanan, J. B., & Drasgow, F. (2000). Examining the impact of administration medium on examinee perceptions and attitudes. Journal of Applied Psychology, 85, 880–887.PubMedCrossRefGoogle Scholar
  37. Roth, P. L., Bobko, P., & McFarland, L. (2005). A meta-analysis of work sample test validity: Updating and integrating some classic literature. Personnel Psychology, 58, 1009–1037.CrossRefGoogle Scholar
  38. Roth, P., Bobko, P., McFarland, L., & Buster, M. (2008). Work sample tests in personnel selection: A meta-analysis of black-white differences in overall and exercise scores. Personnel Psychology, 61, 637–662.CrossRefGoogle Scholar
  39. Rudner, L. M., Garcia, V., & Welch, C. (2006). An evaluation of IntelliMetric™ essay scoring system. The Journal of Technology, Learning and Assessment, 4. http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1651/1493. Accessed 15 Dec 2012.
  40. Russell, C. J. (1985). Individual decision processes in an assessment center. Journal of Applied Psychology, 70, 737–746.CrossRefGoogle Scholar
  41. Ryan, A. M., & Ployhart, R. E. (2000). Applicants’ perceptions of selection procedures and decisions: A critical review and agenda for the future. Journal of Management, 26, 565–606.CrossRefGoogle Scholar
  42. Ryan, A. M., & Tippins, N. T. (2009). Designing and implementing global selection systems. Malden: Wiley-Blackwell.CrossRefGoogle Scholar
  43. Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274.CrossRefGoogle Scholar
  44. Schmitt, N., Clause, C. S., & Pulakos, E. D. (1996). Subgroup differences associated with different measures of some common job relevant constructs. In C. L. Cooper & I. T. Robertson (Eds.), International review of industrial and organizational psychology (pp. 115–139). New York: Wiley.Google Scholar
  45. Shore, T. H., Thornton III, G. C., & McFarlane Shore, L. (1990). Construct validity of two categories of assessment center dimension ratings. Personnel Psychology, 43, 101–116.CrossRefGoogle Scholar
  46. Sydell, E. (2013). Simulation Scoring. In M. S. Fetzer & K. A. Tuzinski (Eds.), Simulations for personnel selection. New York: Springer.Google Scholar
  47. Terpstra, D. E., Amin Mohamed, A., & Kethley, R. B. (1999). An analysis of federal court cases involving nine selection devices. International Journal of Selection and Assessment, 7, 26–34.CrossRefGoogle Scholar
  48. Thornton III, G. C. & Byham, W. C. (1982). Assessment centers and managerial performance. New York: Academic Press.Google Scholar
  49. Tippins, N. T., Beaty, J., Drasgow, F., Gibson, W. M., Pearlman, K., Segall, D. O., et al. (2006). Unproctored internet testing in employment settings. Personnel Psychology, 59, 189–225.CrossRefGoogle Scholar
  50. United States Merit Systems Protection Board. (2009). Job simulations: Trying out for a federal job. http://www.mspb.gov/netsearch/viewdocs.aspx?docnumber=452039&version=453207&application=ACROBAT. Accessed 25 Nov 2012.
  51. Wernimont, P. F., & Campbell, J. P. (1968). Signs, samples, and criteria. Journal of Applied Psychology, 52, 372–376.PubMedCrossRefGoogle Scholar
  52. Whetzel, D. L., McDaniel, M. A., & Nguyen, N. T. (2008). Subgroup differences in situational judgment test performance: A meta-analysis. Human Performance, 21, 291–309.CrossRefGoogle Scholar
  53. Whetzel, D. L., McDaniel, M. A., & Pollack, J. M. (2012). Work Simulations. In M. A. Wilson, W. Bennett, S. G. Gibson, & G. M. Alliger (Eds.), The handbook of work analysis: Methods, systems, applications and science of work measurement in organizations (pp. 401–418). New York: Routledge.Google Scholar
  54. Yu, K. Y. T., & Cable, D. M. (2012). Recruitment and competitive advantage: A brand equity perspective. In S. W. J. Kozlowski (Ed.), The Oxford handbook of organizational psychology: Vol. 1 (pp. 197–220). New York: Oxford University Press.Google Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Anthony S. Boyce
    • 1
  • Christine E. Corbet
    • 2
  • Seymour Adler
    • 1
  1. 1.Aon HewittNew YorkUSA
  2. 2.Right ManagementNew YorkUSA

Personalised recommendations