Abstract
The present chapter discusses evaluation in higher education using the example of medical education. Concretely, the chapter has three aims: (1) we illustrate potential limitations of “classic” entrance examinations for medical education in addition to their limited scope in terms of academic competencies, using a very detailed evaluation of the Austrian medical school entrance examination as a sample case; (2) we briefly summarize the changes the concepts and goals of entrance examinations in medical education have undergone in recent years. There has been a shift to also include interpersonal and intrapersonal competencies in the medical school admission process, as these competencies are considered decisively relevant for physicians as members of the future health care workforce; and (3) we propose a comprehensive evaluation model for competence-based teaching that begins by defining the competencies students should acquire through higher education. In a second step, the curriculum has to be developed in a way that addresses all of the goal competencies that have been defined. The final step should then be to define the competencies incoming freshmen should possess in order to be successful in university training and specifically in medical education.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
In Austria school grades vary from 1 (excellent passed) to 5 (failed).
References
Abbiati, M., Baroffio, A., & Gerbase, M. W. (2016). Personal profile of medical students selected through a knowledge-based exam only: Are we missing suitable students? Medical Education Online, 21, 1–10. https://doi.org/10.3402/meo.v21.29705.
Bergsmann, E., Schultes, M. T., Winter, P., Schober, B., & Spiel, C. (2015). Evaluation of competence-based teaching in higher education: From theory to practice. Evaluation and Program Planning, 52, 1–9. https://doi.org/10.1016/j.evalprogplan.2015.03.001.
Blömeke, S., Zlatkin-Troitschanskaia, O., Kuhn, C., & Fege, J. (2013). Modeling and measuring competencies in higher education. Rotterdam: Sense Publishers.
Blömeke, S., Gustafsson, J. E., & Shavelson, R. J. (2015). Beyond dichotomies. Zeitschrift für Psychologie, 223(1), 3–13. https://doi.org/10.1027/2151-2604/a000194.
Cantor, J. C., Cohen, A. B., Barker, D. C., Shuster, A. L., & Reynolds, R. C. (1991). Medical educators’ views on medical education reform. JAMA, 265(8), 1002–1006.
Chomsky, N. (1986). Language and mind. New York: Harcourt, Brace & World.
Cousins, J. B., & Chouinard, J. A. (2012). Participatory evaluation up close: An integration of research-based knowledge. Charlotte: Information Age Publishing.
European Commission. (2014). The Bologna process and the European higher education area. Retrieved from http://ec.europa.eu/education/policy/higher-education/bologna-process_en.htm. Accessed 19 Jan 2017.
Fischer, G. H. (1995). Derivations of the Rasch model. In G. H. Fischer & I. W. Molenaar (Eds.), Rasch models (pp. 15–38). New York: Springer.
Habermas, J. (1981). Theorie der kommunikativen Kompetenz (Vols. 1 & 2). [Theory of communicative competency]. Frankfurt am Main: Suhrkamp.
Hansen, M., Alkin, M. C., & Wallace, T. L. (2013). Depicting the logic of three evaluation theories. Evaluation and Program Planning, 38, 34–43. https://doi.org/10.1016/j.evalprogplan.2012.03.012.
Hänsgen, K.-D., & Spicher, B. (2007). EMS Eignungstest für das Medizinstudium 2007 – Bericht 13 über die Durchführung und Ergebnisse 2007. [EMS entrance exam for medical education 2007 – Report 13 on the implementation and results in 2007]. Freiburg: Center for Test Development and Diagnostics.
Hartig, J., Klieme, E., & Leutner, D. (2008). Assessment of competencies in educational contexts. Göttingen: Hogrefe.
Interprofessional Education Collaborative. (2016). Core competencies for interprofessional collaborative practice: 2016 update. Washington, DC: Interprofessional Education Collaborative. Retrieved from https://ipecollaborative.org/uploads/IPEC-2016-Updated-Core-Competencies-Report__final_release_.PDF
Jones, S., & Myhill, D. (2004). ‘Troublesome boys’ and ‘compliant girls’: Gender identity and perceptions of achievement and underachievement. British Journal of Sociology of Education, 25(5), 547–561.
Kenney-Benson, G. A., Pomerantz, E. M., Ryan, A. M., & Patrick, H. (2006). Sex differences in math performance: The role of children’s approach to schoolwork. Developmental Psychology, 42(1), 11–26.
Kite, M. E., Deaux, K., & Haines, E. L. (2008). Gender stereotypes. In Psychology of women: A handbook of issues and theories (Vol. 2, pp. 205–236). Westport: Praeger.
Klieme, E. (2004). Was sind Kompetenzen und wie lassen sie sich messen? [What are competences and how can they be measured?]. Pädagogik, 56, 10–13.
Klieme, E., Hartig, J., & Rauch, D. (2008). The concept of competence in educational contexts. In J. Hartig, E. Klieme, & D. Leutner (Eds.), Assessment of competencies in educational contexts (pp. 3–22). Göttingen: Hogrefe.
Koeppen, K., Hartig, J., Klieme, E., & Leutner, D. (2008). Current issues in competence modeling and assessment. Journal of Psychology, 216, 61–73. https://doi.org/10.1027/0044-3409.216.2.61.
Kollmayer, M., Schober, B., & Spiel, C. (2016). Gender stereotypes in education: Development, consequences, and interventions. European Journal of Developmental Psychology, 1–17. https://doi.org/10.1080/17405629.2016.1193483.
Kubinger, K. D. (1989). Aktueller Stand und kritische Würdigung der Probabilistischen Testtheorie. [Current state and critical evaluation of probabilistic test theory]. In K. D. Kubinger (Ed.), Moderne Testtheorie – Ein Abriß samt neuesten Beiträgen (pp. 19–83). Weinheim: Beltz.
Leonard, S. N., Fitzgerald, R. N., & Riordan, G. (2016). Using developmental evaluation as a design thinking tool for curriculum innovation in professional higher education. Higher Education Research & Development, 35(2), 309–321. https://doi.org/10.1080/07294360.2015.1087386.
Mahon, K. E., Henderson, M. K., & Kirch, D. G. (2013). Selecting tomorrow’s physicians: The key to the future health care workforce. Academic Medicine, 88(12), 1806–1811.
McClelland, D. C. (1973). Testing for competence rather than for intelligence. American Psychologist, 28, 1–14.
Mitterauer, L., Frischenschlager, O. & Haidinger, G. (2007). Sex differences in study progress at Medical University of Vienna. GMS Zeitschrift für Medizinische Ausbildung, 24(2), Doc111.
Nycander, M. (2006). Pojkars och flickors betyg [Grades received by boys and girls]. Uppsala: Uppsala University, Department of Education.
OECD. (2013a). AHELO feasibility study report: Volume 1 – Design and implementation. Paris: OECD. Retrieved from: www.oecd.org/edu/ahelo
OECD. (2013b). AHELO feasibility study report: Volume 2 – Data analysis and national experience. Paris: OECD. Retrieved from: www.oecd.org/edu/ahelo
Patterson, F., Knight, A., Dowell, J., Nicholson, S., Cousans, F., & Cleland, J. (2016). How effective are selection methods in medical education? A systematic review. Medical Education, 50(1), 36–60. https://doi.org/10.1111/medu.12817.
Perthold-Stoitzner, B. (2016) Universitätsgesetz 2002. [2002 university law]. (4th. Ed.) Vienna: Manz-Verlag.
Pfaffel, A., Kollmayer, M., Schober, B., & Spiel, C. (2016). A missing data approach to correct for direct and indirect range restrictions with a dichotomous criterion: A simulation study. PLoS One, 11(3), e0152330.
Reynolds, W. M., & Miller, G. E. (2013). Educational psychology: Contemporary perspectives. In I. B. Weiner, W. M. Reynolds, & G. E. Miller (Eds.), Handbook of psychology, educational psychology (Vol. 7, pp. 1–22). Hoboken: Wiley.
Roth, H. (1971). Pädagogische Anthropologie (Vol. 2). [Pedagogical anthropology]. Hannover: Schroedel.
Sade, R. M., Stroud, M. R., Levine, J. H., & Fleming, G. A. (1985). Criteria for selection of future physicians. Annals of Surgery, 201(2), 225.
Sawyer, R. (2013). Beyond correlations: Usefulness of high school GPA and test scores in making college admissions decisions. Applied Measurement in Education, 26(2), 89–112.
Schober, B., Spiel, C., & Reimann, R. (2004). Young physicians’ competences from different points of view. Medical Teacher, 26(5), 451–457.
Spiel, C., Schober, B., & Reimann, R. (2006). Evaluation of curricula in higher education: Challenges for evaluators. Evaluation Review, 30, 430–450.
Spiel, C., Litzenberger, M., & Haiden, D. (2007). Bildungswissenschaftliche und psychologische Aspekte von Auswahlverfahren. [Educational science and psychological aspects of selection procedures]. In C. Badelt, W. Wegscheider, & H. Wulz (Eds.), Hochschulzugang in Österreich (pp. 479–552). Graz: Grazer Universitätsverlag.
Spiel, C., Schober, B., & Litzenberger, M. (2008). Evaluation der Eignungstests für das Medizinstudium in Österreich. [Evaluation of the entrance examinations for medical education in Austria]. Vienna: University of Vienna, Faculty of Psychology.
Spiel, C., Schober, B., & Reimann, R. (2013). Modeling and measurement of competencies in higher education: The contribution of scientific evaluation. In O. Zlatkin-Troitschanskaia & S. Blömeke (Eds.), Modeling and measurement of competencies in higher education (pp. 195–206). Rotterdam: Sense Publishers.
Trost, G., Blum, F., Fay, E., Klieme, E., Maichle, U., Meyer, M., et al. (1998). Evaluation des Tests für medizinische Studiengänge (TMS): Synopse der Ergebnisse [Report]. [Evaluation of the test for medical education (TMS): Synopsis of the results]. Bonn: Institute for Test und Giftedness Research.
Weinert, E. F. (1999). Concepts of competence. Contribution within the OECD project definition and selection of competencies: Theoretical and conceptual foundations (DeSeCo). Munich: Max Planck Institute for Psychological Research.
Wesselink, R., Dekker-Groen, A. M., Biemans, H. J., & Mulder, M. (2010). Using an instrument to analyse competence-based study programmes: Experiences of teachers in Dutch vocational education and training. Journal of Curriculum Studies, 42(6), 813–829. https://doi.org/10.1080/00220271003759249.
Zlatkin-Troitschanskaia, O., Pant, H. A., Kuhn, C., Toepper, M., & Lautenbach, C. (2016). Messung akademisch vermittelter Kompetenzen von Studierenden und Hochschulabsolventen. Ein Überblick zum nationalen und internationalen Forschungsstand. Wiesbaden: Springer.
Zlatkin-Troitschanskaia, O., Pant, H. A., Lautenbach, C., Molerov, D., Toepper, M., & Brückner, S. (2017). Modeling and Measuring Competencies in Higher Education - Approaches to Challenges in Higher Education Policy and Practice. Wiesbaden: Springer.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Spiel, C., Schober, B. (2018). Challenges for Evaluation in Higher Education: Entrance Examinations and Beyond: The Sample Case of Medical Education. In: Zlatkin-Troitschanskaia, O., Toepper, M., Pant, H., Lautenbach, C., Kuhn, C. (eds) Assessment of Learning Outcomes in Higher Education. Methodology of Educational Measurement and Assessment. Springer, Cham. https://doi.org/10.1007/978-3-319-74338-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-74338-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-74337-0
Online ISBN: 978-3-319-74338-7
eBook Packages: EducationEducation (R0)