Advertisement

The Rise of Immersive Cognitive Assessments: Towards Simulation-Based Assessment for Evaluating Applicants

  • Rebecca Kantar
  • Keith McNulty
  • Erica L. Snow
  • Richard Wainess
  • Sonia D. Doshi
  • Devon B. Walker
  • Matthew A. Emery
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10948)

Abstract

Imbellus is an assessment company that aims to test cognitive processes within the context of immersive simulation-based assessments. This paper explores our work with McKinsey & Company, a best-in-class management consulting firm, to build a simulation-based assessment that gauges applicants’ cognitive skills and abilities. Leveraging a cognitive task analysis grounded in theoretical work and practical observations of on the job activities, we defined key work activities and skills needed to complete them. We then developed scenarios that abstracted and generalized the most crucial skills. To make sense of significant telemetry data from users’ interactions with the assessment, we applied theoretically grounded expert models to guide our scoring algorithms. Our assessment draws inferences across seven major problem-solving constructs. We will present our initial findings and describe implications of our current work for the fields of artificial intelligence and assessment.

Keywords

Simulation-based assessment Cognitive skills Artificial Intelligence 

References

  1. Mayer, R.E.: What problem solvers know cognitive readiness for adaptive problem solving. In: O’Neil, H.F., Perez, R.S., Baker, E.L. (eds.) Teaching and Measuring Cognitive Readiness, pp. 149–160. Springer, Boston (2014).  https://doi.org/10.1007/978-1-4614-7579-8_8CrossRefGoogle Scholar
  2. Mayer, R.E., Wittrock, M.C.: Problem solving transfer. In: Berliner, D.C., Calfee, R.C. (eds.) Handbook of Educational Psychology, pp. 47–62. Simon & Schuster Macmillan, New York (1996)Google Scholar
  3. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. Springer, New York (2009).  https://doi.org/10.1007/978-0-387-84858-7CrossRefzbMATHGoogle Scholar
  4. Schraagen, J.M., Chipman, S.F., Shalin, V.L. (eds.): Cognitive Task Analysis. Psychology Press, New York (2000)Google Scholar
  5. Perkins, D.N., Salomon, G.: Transfer of learning. Int. Encycl. Educ. 2, 6452–6457 (1992)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Rebecca Kantar
    • 1
  • Keith McNulty
    • 2
  • Erica L. Snow
    • 1
  • Richard Wainess
    • 1
  • Sonia D. Doshi
    • 1
  • Devon B. Walker
    • 1
  • Matthew A. Emery
    • 1
  1. 1.ImbellusLos AngelesUSA
  2. 2.McKinsey & CompanyLondonUK

Personalised recommendations