Advances in Health Sciences Education

, Volume 1, Issue 1, pp 41–67 | Cite as

The assessment of professional competence: Developments, research and practical implications

  • C. P. M. Van Der Vleuten
Article

References

  1. 1.
    Ebel RL, Frisbie DA. Essentials of educational measurement. Englewood Cliffs, New Jersey: Prentice-Hall, 1986.Google Scholar
  2. 2.
    Ory JC, Ryan KE. Tips for improving testing and grading. Newbury Park, California: Sage Publications, 1993.Google Scholar
  3. 3.
    Pickering C. Against multiple-choice questions. Medical Teacher 1979; 1: 84–6.Google Scholar
  4. 4.
    Newble DI, Baxter, A, Elmslie G. A comparison of multiple choice and free response tests in examinations of clinical competence. Medical Education 1979; 13: 263–8.Google Scholar
  5. 5.
    McGuire C. Perspectives in assessment. Academic Medicine 1993; 68(Supplement): S3–8.Google Scholar
  6. 6.
    Case SM, Swanson DB. Extended-matching items: A practical alternative to free-response questions. Teaching and Learning in Medicine 1993; 5: 107–15.Google Scholar
  7. 7.
    Schuwirth LWT, van der Vleuten CPM, Donkers HHLM. Open ended questions versus multiple choice questions: An analysis of cuing effects. In: Harden RM, Hart IR, Mulholland H, editors. Approaches to assessment of clinical competence — Part II. Norwich: Page Brothers, 1992: 486–91.Google Scholar
  8. 8.
    Linn RL. Educational assessment: Expanded expectations and challenges. Educational Evaluation and Policy Analysis 1993; 15: 1–16.Google Scholar
  9. 9.
    Van der Vleuten CPM, Newble DI. How can clinical reasoning be tested? The Lancet 1995; 345: 1032–4.Google Scholar
  10. 10.
    McGuire CH, Babnott D. Simulation technique in the measurement of problem solving skills. Journal of Educational Measurement 1967; 4: 1–10.Google Scholar
  11. 11.
    Hodgkin K, Knox JDE. Problem centered learning: The modified essay question in medical education. Edinburgh: Churchill-Livingstone, 1975.Google Scholar
  12. 12.
    Feletti GI, Saunders NA, Smith AJ. Comprehensive assessment of final-year medical student performance based on undergraduate programme objectives. Lancet 1983; 2(8340): 34–7.Google Scholar
  13. 13.
    Barrows HS, Tamblyn RM. The portable patient problem pack (P4). A problem-based learning unit. Journal of Medical Education 1977; 52: 1002–4.Google Scholar
  14. 14.
    Williams RG, Vu NV, Barrows HG, Verhulst S. Profile of the Clinical Reasoning Test (CRT): An objective measure of problem solving skills and proficiency in using medical knowledge. In: Schmidt HG, De Volder ML, editors. Tutorials in problem-based learning. Assen: Van Gorcum, 1984: 81–90.Google Scholar
  15. 15.
    Norcini JJ, Meskauskas JA, Langdon LO, Webster GD. An evaluation of a computer simulation in the assessment of physician competence. Evaluation in the Health Professions 1986; 9: 286–304.Google Scholar
  16. 16.
    Bligh TJ. Written simulation scoring: Comparison of nine systems [dissertation]. Urbana-Champaign (IL): University of Illinois, 1980.Google Scholar
  17. 17.
    Swanson D, Norcini J, Grosso L. Assessment of clinical competence: Written and computer-based simulations. Assessment and Evaluation in Higher Education 1987; 12: 220–46.Google Scholar
  18. 18.
    Norman G, Bordage G, Curry L et al. A review of recent innovations in assessment. In: Wakeford RE, editor. Directions in clinical assessment. Report of the First Cambridge Conference. Cambridge: Cambridge University School of Clinical Medicine, 1985: 9–27.Google Scholar
  19. 19.
    Elstein A, Shulman LS, Sprafka SA. Medical problem solving: An analysis of clinical reasoning. Cambridge Massachusetts: Harvard University Press.Google Scholar
  20. 20.
    Friedman R, Korst D, Schultz J, Beatty E, Entine S. Experience with the simulated patient physician encounter. Journal of Medical Education 1978; 53: 825–30.Google Scholar
  21. 21.
    McLeskey C, Ward R. Validity of written examinations. Anesthesiology 1978; 49: 224.Google Scholar
  22. 22.
    Marshall J. Assessment of problem-solving ability. Medical Education 1977; 11: 329–34.Google Scholar
  23. 23.
    Newble DI, Hoare J, Baxter A. Patient management problems: Issues of validity. Medical Education 1982; 16: 137–42.Google Scholar
  24. 24.
    Norman GR, Feightner JW. A comparison of behaviour on simulated patients and patient management problems. Journal of Medical Education 1981; 55: 529–37.Google Scholar
  25. 25.
    Norcini JJ, Swanson DB, Grosso LJ, Shea JA, Webster Gd. Reliability, validity and efficiency of multiple choice questions and patient management problem item formats in the assessment of physician competence. Medical Education 1985; 19: 238–47.Google Scholar
  26. 26.
    Bordage G, Page G. An alternative approach to PMPs: The “key features” concept. In: Hart IR, Harden RM, editors. Further developments in assessing clinical competence. Montreal: Heal-Publications, 1987: 59–75.Google Scholar
  27. 27.
    De Graaff E, Post G, Drop M. Validation of a new measure of clinical problem-solving. Medical Education 1987; 21: 213–218.Google Scholar
  28. 28.
    Powles ACP, Wintrup N, Neufeld VR, Wakefield JH, Coates G, Burrows J. The triple jump exercise: further studies of an evaluative technique. Proceedings of the 20th Annual Conference on Research in Medical Education, Washington: American Association of Medical Colleges, 1981: 74–9.Google Scholar
  29. 29.
    Fiedman CP, Murphy GC, Smith AC, Mattern WD. Exploratory study of an examination format for problem-based learning. Teaching and Learning in Medicine 1994; 6: 194–8.Google Scholar
  30. 30.
    Boud D. The role of self-assessment in student grading. Assessment and Evaluation in Higher Education 1989; 14: 20–30.Google Scholar
  31. 31.
    De Grave W De Volder M. Peer-evaluation and problem-based learning. In: Schmidt H, De Volder M, editors. Tutorials in problem-based learning. Assen: Van Gorcum, 1984: 116–22.Google Scholar
  32. 32.
    Magzoub M. Studies in community-based education [dissertation]. Masstricht: University of Limburg, 1994.Google Scholar
  33. 33.
    Case SM, Swanson DB, van der Vleuten CPM. Student assessment in problem-based learning curricula. In: Boud D, Feletti G, editors. The challenge of problem-based learning. London: Kogan Page, 1991: 260–73.Google Scholar
  34. 34.
    Gordon MJ. A review of the validity and accuracy of self-assessments in health professions training. Academic Medicine 1991; 66: 762–9.Google Scholar
  35. 35.
    Blake JM, Norman GR, Smith EKM. Report card from McMaster: Student evaluation at a problem-based medical school. Lancet 1995; 345: 899–902.Google Scholar
  36. 36.
    Harden R, Gleeson F. Assessment of clinical competence using an objective structured clinical examination (OSCE). Medical Education 1979; 13: 41–54.Google Scholar
  37. 37.
    Rothman AI, Cohen R. (Editors). Proceedings of the Sixth Ottawa Conference on Medical Education. Toronto: University of Toronto Bookstore Custom Publishing, 1995: 292–321.Google Scholar
  38. 38.
    Reznick R, Blackmore DE, Cohen R et al. An objective structured clinical examination for the licentiate of the Medical Council of Canada: From research to reality. Academic Medicine 1993; 68(Supplement): S4–6.Google Scholar
  39. 39.
    Van der Vleuten CPM, Swanson DB. Assessment of clinical skills with standardized patients: State of the art. Teaching and Learning in Medicine 1990; 2: 58–76.Google Scholar
  40. 40.
    Vu NV, Barrows HS. Use of standardized patients in clinical assessments: recent developments and measurement findings. Educational Researcher 1994; 23: 23–30.Google Scholar
  41. 41.
    Swanson DB, Norman GR, Linn RL. Performance-based assessment: Lessons from the health professions. Educational Researcher 1995; 24: 5–22, 35.Google Scholar
  42. 42.
    Van Thiel J, Kraan HF, Van der Vleuten CPM. Reliability and feasibility of measuring interviewing skills using the revised Maastricht History Taking and Advice Checklist. Medical Education 1991; 25: 224–9.Google Scholar
  43. 43.
    Swanson Db. A measurement framework for performance-based tests. In: Hart IR, Harden RM, editors. Further developments in assessing clinical competence. Montreal: Can-Heal, 1987: 13–45.Google Scholar
  44. 44.
    Stalenhoef-Halling BF, Van der Vleuten CPM, Jaspers TAM, Fiolet JFBM. The feasibility, acceptability and reliability of open-ended questions in a problem-based learning curriculum. In: Bender W, Hiemstra RJ, Scherpbier AJJA, Zwierstra RP, editors. Teaching and assessing clinical competence. Groningen: Boekwerk Publ, 1990: 552–7.Google Scholar
  45. 45.
    Erviti V, Templeton B, Bunce J, Burg F. The relationships of pediatric resident recording behavior across medical conditions. Medical Care 1980; 18: 1020–31.Google Scholar
  46. 46.
    Rethans JJ, Sturmans F, Drop MJ, Van der Vleuten CPM. Assessment of performance in actual practice of general practitioners by use of standardized patients. British Journal of General Practice 1991; 41: 97–9.Google Scholar
  47. 47.
    Shavelson RJ, Baxter GP, Gao X. Sampling variability of performance assessments. Journal of Educational Measurement 1993; 30: 215–32.Google Scholar
  48. 48.
    Klein as cited in: Linn RL. Educational assessment: Expanded expectations and challenges. Educational Evaluation and Policy Analysis 1993; 15: 1–16.Google Scholar
  49. 49.
    Shavelson RJ, Mayberry P, Li W, Webb NM. Generalizability of military performance measurements: Marine Corps rifleman. Military Psychology 1990; 2: 129–44.Google Scholar
  50. 50.
    Frijns PRAM, Van der Vleuten CPM, Verwijnen GM, Van Leeuwen YD. The effect of structure in scoring methods on the reproducibility of tests using open-ended questions. In: Bender W, Hiemstra RJ, Scherpbier AJJA, Zwierstra RP, editors. Teaching and assessing clinical competence. Groningen: Boekwerk Publ, 1990: 466–71Google Scholar
  51. 51.
    Van Ham I, Gerritsma J. The assessment of clinical competence in general practice with chart stimulated recall. In: Bender W, Hiemstra RJ, Scherpbier AJJA, Zwierstra RP, editors. Teaching and assessing clinical competence. Groningen: Boekwerk, 1990: 306–9.Google Scholar
  52. 52.
    Van der Vleuten CPM, Norman GR, De Graaff E. Pitfalls in the pursuit of objectivity: Issues of reliability. Medical Education 1991; 25: 110–8.Google Scholar
  53. 53.
    Streiner DL. Global rating scales. In: Neufeld VR, Norman GR, editors. Assessing clinical competence. New York: Springer, 1985: 119–41.Google Scholar
  54. 54.
    Streiner D. Clinical ratings - ward evaluation. In: Shannon S, Norman G, editors. Evaluation methods: A resource handbook. Hamilton: The Program for Educational Development, McMaster University, 1995: 29–31.Google Scholar
  55. 55.
    Hastorf AH, Schneider DJ, Polefka J. Person perception. Reading, Massachusetts: Addison-Wesley, 1970.Google Scholar
  56. 56.
    Ross M. Relation of implicit theories to the construction of personal histories. Psychological Review 1989; 96: 341–57.Google Scholar
  57. 57.
    Norman GR, Swanson DB, Case SM. Conceptual and methodological issues in studies comparing assessment formats. Teaching and Learning (in press).Google Scholar
  58. 58.
    Norman GR, Smith E, Powles A, Rooney P, Henry N, Dodd P. Factors underlying performance on written tests of knowledge. Medical Education 1987; 21: 297–304.Google Scholar
  59. 59.
    Jean P, Schuwirth L, Van Santen M, Van der Vleuten C. Do problem analysis questions (PAQs) and true/false questions (TFQs) measure different skills? Medical Education (in press).Google Scholar
  60. 60.
    Maatsch J, Huang R. An evaluation of the construct validity of four alternative theories of clinical competence. Proceedings of the Twenty-fifth Annual Conference on Research in Medical Education, American Association of Medical Colleges. Washington, DC, 1986.Google Scholar
  61. 61.
    Maatsch J. Model for a criterion-referenced medical specialty test. Final Report Grant No. HS-02038-02, Office of Medical Education Research and Development Michigan State University, 1980.Google Scholar
  62. 62.
    Van der Vleuten CPM, Van Luijk S, Beckers HJM. A written test as an alternative to performance testing. Medical Education 1989; 23: 97–107.Google Scholar
  63. 63.
    Ward W. A comparison of free-response and multiple choice forms of verbal aptitude tests. Applied Psychological Measurement 1982; 6: 1–11.Google Scholar
  64. 64.
    Thissen D, Wainer H, Wang X. Are tests comprising both multiple-choice and free-response items necessarily more unidimensional than multiple-choice tests? An analysis of two tests. Journal of Educational Measurement 1994; 31: 113–23.Google Scholar
  65. 65.
    Ramsey PG, Carline JD, Inui YS et al. Predictive validity of certification by the American Board of Internal Medicine. Annals of Internal Medicine 1989; 110: 719–26Google Scholar
  66. 66.
    Solomon et al. 1990 as cited in Norman GR. Can an examination predict competence? The role of recertification in maintenance of competence. Annals of the Royal College of Physicians and Surgeons of Canada 1991; 24: 121–4.Google Scholar
  67. 67.
    Norman GR, Davis DA, Painvin A, Rath D, Ragbeer M. Comprehensive assessment of clinical competence of family-general physicians using multiple measures. Proceedings 28th Conference on Research in Medical Education. Washington: American Association of Medical Colleges, 1989.Google Scholar
  68. 68.
    Norman GR. Can an examination predict competence? The role of recertification in maintenance of competence. Annals of the Royal College of Physicians and Surgeons of Canada 1991; 24: 121–4.Google Scholar
  69. 69.
    McGuire C. Written methods for assessing clinical competence. In: Hart IR, Harden RM, editors. Further developments in assessing clinical competence. Montreal: Can-Heal, 1987: 44–58.Google Scholar
  70. 70.
    Hager P, Gonczi A, Athanasou J. General issues about assessment of competence. Assessment & Evaluation in Higher Education 1994; 19: 3–16.Google Scholar
  71. 71.
    Norman GR, Van der Vleuten CPM, De Graaff E. Pitfalls in the pursuit of objectivity: Issues of validity, efficiency and acceptability. Medical Education 1991; 25: 119–26.Google Scholar
  72. 72.
    Van Thiel J, van der Vleuten CPM, Kraan H. Assessment of medical interviewing skills: Generalizability of scores using successive MAAS-versions. In: Harden RM, Hart IR, Mulholland H, editors. Approaches to assessment of clinical competence — Part II. Norwich: Page Brothers, 1992: 536–540.Google Scholar
  73. 73.
    Newble D, Jaeger K. The effect of assessments and examinations on the learning of medical students. Medical Education 1983; 17: 165–71.Google Scholar
  74. 74.
    Popham WJ. Measurement as an instructional catalyst. In: Ekstrom RB, editor. Measurement, technology and individuality in education. San Francisco: Jossey-Bass, 1983: 87–103.Google Scholar
  75. 75.
    Frederiksen N. The real test bias: Influences of testing on teaching and learning. American Psychologist 1984; 39: 193–202.Google Scholar
  76. 76.
    Entwistle N. Styles of learning and teaching. Chichester: John Wiley & Sons, 1981.Google Scholar
  77. 77.
    Stillman P, Swanson D. Ensuring the clinical competence of medical school graduates through standardized patients. Archives of Internal Medicine 1987; 147: 1049–52.Google Scholar
  78. 78.
    Gibbs G. Improving the quality of student learning. Bristol: Technical & Educational Services, 1992.Google Scholar
  79. 79.
    Shepard LA. Psychometrician's beliefs about learning. Educational Researcher 1991; 20: 2–16.Google Scholar
  80. 80.
    Snyder BR. The hidden curriculum. New York: Knopf, 1971.Google Scholar
  81. 81.
    Popham WJ, Cruse KL, Rankin SC, Sandifer PD, Williams PL. Measurement-driven instruction: it's on the road. Phi Delta Kappan 1985; 66: 628–34.Google Scholar
  82. 82.
    Semb GB, Ellis JA. Knowledge taught in school: What is remembered? Review of Educational Research 1994; 64: 253–86.Google Scholar
  83. 83.
    Van Luijk SJ, Van der Vleuten CPM, Schelven RM. The relation between content and psychometric characteristics in performance-based testing. In: Bender W, Hiemstra RJ, Scherpbier AJJA, Zwierstra RP, editors. Teaching and assessing clinical competence. Groningen: Boekwerk Publications, 1990: 202–7.Google Scholar
  84. 84.
    Mischel W. Personality and assessment. New York: John Wiley, 1968.Google Scholar
  85. 85.
    Schmidt H, Norman G, Boshuizen HA. Cognitive perspective on medical expertise: Theory and implications. Academic Medicine 1990; 65: 611–21.Google Scholar
  86. 86.
    Norman G, Allery L, Berkson L et al. Research in the psychology of clinical reasoning: implications for assessment. Paper from the Fourth Cambridge Conference, Cambridge University School of Clinical Medicine, Cambridge, 1989.Google Scholar
  87. 87.
    Higgs J, Jones M (Editors). Clinical reasoning in the health professions. Oxford: Butterworth/Heinemann, 1995.Google Scholar
  88. 88.
    Norman G, Regehr G. Contemporary issues in cognitive psychology: implications for professional education. In: Rothman AI, Cohen R, editors. Proceedings of the Sixth Ottawa Conference on Medical Education. Toronto: University of Toronto Bookstore Custom Publishing, 1995: 17–25.Google Scholar
  89. 89.
    Rethans JJ, Sturmans F, Drop MJ et al. Does competence of general practitioners predict their performance. British Medical Journal 1991; 303: 1377–80.Google Scholar
  90. 90.
    Norman GR. Reliability and construct validity of some cognitive measures of clinical reasoning. Teaching and Learning in Medicine 1989; 1: 194–9.Google Scholar
  91. 91.
    Newble DI, Raymond GA. The Pattern Completion Item (PCI): A potential measure of clinical problem-solving skills. In: Rothman AI, Cohen R, editors. Proceedings of the Sixth Ottawa Conference on Medical Education. Toronto: University of Toronto Bookstore Custom Publishing, 1995: 191–2.Google Scholar
  92. 92.
    Nelson MS, Clayton BL, Moreno R. How medical school faculty regard educational research and make pedagogical decisions 1990; 65: 122–6.Google Scholar
  93. 93.
    Hays RB, Fabb WE, Van der Vleuten CPM. Reliability of the fellowship examination of the Royal Australian College of General Practitioners. Teaching and Learning in Medicine 1995; 7: 43–50.Google Scholar
  94. 94.
    Van Luijk SJ, Van der Vleuten CPM. A comparison of checklists and rating scales in performance-based testing. In: Hart IR, Harden RM, Des Marchais J, editors. Current developments in assessing clinical competence. Montreal: Can-Heal, 1992: 357–62.Google Scholar
  95. 95.
    Newble D, Dawson B, Dauphinee D et al. Guidelines for assessing clinical competence. Teaching and Learning in Medicine 1994; 6: 213–20.Google Scholar
  96. 96.
    Endler NS, Magnusson D, editors. Interfactional psychology and personality. Washington DC: Hemisphere, 1976.Google Scholar
  97. 97.
    Vand er Vleuten CPM, Schuwirth LWT, Ronteltap CFM. A cognitive psychological interpretation of a few remarkable psychometric findings. In: Rothman AI, Cohen R, editors. Proceedings of the Sixth Ottawa Conference on Medical Education. Toronto: University of Toronto Bookstore Custom Publishing, 1995: 506–8.Google Scholar
  98. 98.
    Frijns PHAM. Scoringsmodellen voor open-vraag vormen (Scoring models for free-response formats) [dissertation]. Maastricht: University of Limburg, 1992.Google Scholar
  99. 99.
    Brailovsky C, Bordage G, Carretier H, Page G. Content validity of the key features' approach of the Medical Council of Canada's Exam. In: Harden RM, Hart IR, Mulholland H, editors. Approaches to assessment of clinical competence — Part II. Norwich: Page Brothers, 1992: 476–7.Google Scholar
  100. 100.
    Bordage G, Brailovsky C, Carretier H, Page G. Content validation of key features on a national examination of clinical decision-making skills. Academic Medicine 1995; 70: 276–81.Google Scholar
  101. 101.
    Ebel RL. Must all tests be valid? American Psychologist 1961; 16: 640–7.Google Scholar
  102. 102.
    Ebel R. Measuring Educational Achievement. Englewood Cliffs: Prentice-Hall Inc, 1965.Google Scholar
  103. 103.
    Ebel R. The practical validation of tests of ability. Educational Measurement: Issues and Practice 1983; 2: 7–10.Google Scholar
  104. 104.
    Kane M. The validity of licensure examinations. American Psychologist 1982; 37: 911–8.Google Scholar
  105. 105.
    Arnold L, Willoughby TL. The quarterly profile examination. Academic Medicine 1990; 65: 515–6.Google Scholar
  106. 106.
    Van der Vleuten CPM, Verwijnen GM, Wijnen WHFW Fifteen years of experience with progress-testing. Medical Teacher (in press).Google Scholar
  107. 107.
    Van Berkel HJM, Nuy HJP, Geerligs T. The influence of progress tests and block tests on study behavior. Instructional Science 1995; 22: 315–31.Google Scholar
  108. 108.
    Cohen-Schotanus J. Effecten van curriculumveranderingen [dissertation, with English summary]. Groningen: University of Groningen, 1994.Google Scholar
  109. 109.
    Van der Vleuten CPM, Verwijnen GM. A system for student assessment. In: Van der Vleuten CPM, Wijnen WHFW, editors. Problem-based learning: Perspectives from the Maastricht experience. Amsterdam: Thesis-publ., 1990: 27–49.Google Scholar
  110. 110.
    Klass D, Clauser B, Fletcher E et al. Progress in developing a standardized patient test of clinical skills at the National Board of Medical Examiners: prototype two. In: Rothman AI, Cohen R, editors. Proceedings of the Sixth Ottawa Conference on Medical Education. Toronto: University of Toronto Bookstore Custom Publishing, 1995: 324–6.Google Scholar
  111. 111.
    Reznick RK, Smee S, Baumber JS et al. Guidelines for estimating the real cost of an Objective Structured Clinical Examination. Academic Medicine 1993; 68: 513–7.Google Scholar

Copyright information

© Kluwer Academic Publishers 1996

Authors and Affiliations

  • C. P. M. Van Der Vleuten
    • 1
  1. 1.MaastrichtThe Netherlands

Personalised recommendations