The Relationship between Checklist Scores on a Communication OSCE and Analogue Patients’ Perceptions of Communication
Many efforts to teach and evaluate physician–patient communication are based on two assumptions: first, that communication can be conceptualized as consisting of specific observable behaviors, and second, that physicians who exhibit certain behaviors are more effective in communicating with patients. These assumptions are usually implicit, and are seldom tested. The purpose of this study was to investigate whether specific communication behaviors are positively related to patients’ perceptions of effective communication. Trained raters used a checklist to record the presence or absence of specific communication behaviors in 100 encounters in a communication Objective Structured Clinical Examination (OSCE). Lay volunteers served as analogue patients and rated communication during each encounter. Correlations between checklist scores and analogue patients’ ratings were not significantly different from zero for four of five OSCE cases studied. Within each case, certain communication behaviors did appear to be related to patients’ ratings, but the critical behaviors were not consistent across cases. We conclude that scores from OSCE communication checklists may not predict patients’ perceptions of communication. Determinants of patient perceptions of physician communication may be more subtle, more complex, and more case-specific than we were able to capture with the current checklist.
Keywordsmedical education OSCE physician–patient communication
American Board of Internal Medicine (1989). Final report on the patient satisfaction questionnaire project. Philadelphia: Webster, G.D., MDGoogle Scholar
Bertakis, K.D., Roter, D., Putnam, S.M. 1991The relationship of physician medical interview style to patient satisfactionThe Journal of Family Practice32175181PubMedGoogle Scholar Blue, A.V., Chessman, A.W., Gilbert, G.E., Mainous, AG,III. 2000Responding to patients’ emotions: important for standardized patient satisfactionFamily Medicine32326330PubMedGoogle Scholar Boulet, J.R., McKinley, D.W., Whelan, G.P., Hambleton, R.K. 2003Quality assurance methods for performance-based assessmentsAdvances in Health Sciences Education: Theory and Practice.82747CrossRefGoogle Scholar Brailovsky, C.A., Grand’Maiso, , P., 2000Using evidence to improve evaluation: A Comprehensive Psychometric Assessment of a SP-Based OSCE Licensing ExaminationAdvances in Health Sciences Education: Theory and Practice.5207219CrossRefGoogle Scholar Brazeau, C., Boyd, L., Crosson, J. 2002Changing an existing OSCE to a teaching tool: The making of a teaching OSCEAcademic Medicine.77932CrossRefPubMedGoogle Scholar Brennan, R.L. 2001Generalizability TheorySpringer-VerlagNew YorkGoogle Scholar Duerson, M.C., Romrell, L.J., Stevens, C.B. 2000Impacting faculty teaching and student performance: Nine years’ experience with the Objective Structured Clinical ExaminationTeaching and Learning in Medicine12176182CrossRefPubMedGoogle Scholar Friedlich, M., MacRae, H., Oandasan, I., Tannenbaum, D., Batty, H., Reznick, R., et al. 2001Structured assessment of minor surgical skills (SAMSS) for family medicine residentsAcademic Medicine7612411246PubMedGoogle Scholar Grand’Maison, P., Brailovsky, C.A., Lescop, J. 1996Content validity of the Quebec licensing examination (OSCE) assessed by practising physiciansCanadian Family Physician42254259PubMedGoogle Scholar Hall, J., Roter, D., Katz, N. 1988Meta-analysis of correlates of provider behaviorMedical Care26657672PubMedGoogle Scholar Hodges, B., Regehr, G., McNaughton, N., Tiberius, R., Hanson, M. 1999OSCE checklists do not capture increasing levels of expertiseAcademic Medicine7411291134PubMedGoogle Scholar Hodges, B., Turnbull, J., Cohen, R., Bienenstock, A., Norman, G. 1996Evaluating communication skills in the OSCE format: Reliability and generalizabilityMedical Education303843PubMedGoogle Scholar Hull, A., Hodder, S., Berger, B., Ginsberg, D., Lindheim, N., Quan, J., et al. 1995Validity of three clinical performance assessments of internal medicine clerksAcademic Medicine70517522PubMedGoogle Scholar Kahn, M.J., Merrill, W.W., Anderson, D.S., Szerlip, H.M. 2001Residency program director evaluations do not correlate with performance on a required 4th-year objective structured clinical examinationTeaching and Learning in Medicine13912CrossRefPubMedGoogle Scholar Levinson, W., Roter, D.L., Mullooly, J.P., Dull, V.T., Frankel, R.M. 1997Physician–patient communicationThe relationship with malpractice claims among primary care physicians and surgeons. Journal of the American Medical Association277553559Google Scholar Makoul, G. 2001The SEGUE framework for teaching and assessing communication skillsPatient Education and Counseling452334CrossRefPubMedGoogle Scholar Martin, I.G., Jolly, B. 2002Predictive validity and estimated cut score of an objective structured clinical examination (OSCE) used as an assessment of clinical skills at the end of the first clinical yearMedical Education.36418425CrossRefPubMedGoogle Scholar McKinstry, B. 2002Do patients wish to be involved in decision making in the consultation? A cross sectional survey with video vignettesBritish Medical Journal321867871Google Scholar Ockene, I., Hebert, J., Ockene, J., Saperia, G., Nicolosi, R., Merriam, P., et al. 1999Effect of physician-delivered nutrition counseling training and an office-support program on saturated fat intake, weight, and serum lipid measurements in a hyperlipidemic population: Worcester-area trial for counseling in hyperlipidemia (WATCH)Archives of Internal Medicine159725731CrossRefPubMedGoogle Scholar Ockene, J.K., Kristeller, J., Goldberg, R.J., Amick, T.L., Pekow, P.S., Hosmer, D., et al. 1991Increasing the efficacy of physician-delivered smoking intervention: A randomized clinical trialJournal of General Internal Medicine618PubMedGoogle Scholar Ockene, J.K., Adams, A., Hurley, T.G., Wheeler, E.V., Hebert, J.R. 1999Brief physician- and nurse practitioner-delivered counseling for high-risk drinkers: Does it work?Archives of Internal Medicine15921982205CrossRefPubMedGoogle Scholar Ong, L.M.L., Dehaes, J.C.J.M., Hoos, A.M., Lammes, F.B. 1995Doctor–patient communication – a review of the literatureSocial Science & Medicine4090318Google Scholar Regehr, G., MacRae, H., Reznick, R.K., Szalay, D. 1998Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examinationAcademic Medicine73993997PubMedGoogle Scholar Reznick, R., Smee, S., Rothman, A., Chalmers, A., Swanson, D., Dufresne, L., et al. 1992An objective structured clinical examination for the licentiate: Report of the pilot project of the Medical Council of CanadaAcademic Medicine67487494PubMedGoogle Scholar Roberts, C.A., Aruguette, M.S. 2000Task and socioemotional behaviors of physicians: A test of reciprocity and social interaction theories in analogue physician–patient encountersSocial Science & Medicine50300315Google Scholar Roberts, C., Wass, V., Jones, R., Sarangi, S., Gillett, A. 2003A discourse analysis study of ’good’ and ’poor’ communication in an OSCE: A proposed new framework for teaching studentsMedical Education37192201CrossRefPubMedGoogle Scholar Roter, D.L., Hall, J.A., Katz, N.R. 1987Relation between physicians’ behaviors and analogue patients’ satisfaction, recall, and impressionsMedical Care25437451PubMedGoogle Scholar Silverman, J., Kurtz, S., Draper, J. 1998Skills for Communicating with PatientsOxonUKGoogle Scholar Singer, P.A., Robb, A., Choen, R., Norman, G., Turnbull, J. 1994Evaluation of a mutlicenter ethics objective structured clinical examinationJournal of General Internal Medicine9690692PubMedGoogle Scholar Stewart, M. 1995Effective physician–patient communication and health outcomes: A review of the literatureCanadian Medical Association Journal15214231433PubMedGoogle Scholar Stewart, M., Brown, J., Boon, H., Galajda, J., Meredith, L., Sangster, M. 1999Evidence on doctor – patient communicationCancer Prevention & Control32530Google Scholar Street, R.L. 1992Analyzing communication in medical consultations: Do behavioral measures correspond to patients perceptionsMedical Care30976988PubMedGoogle Scholar Tamblyn, R., Abrahamowicz, M., Schnarch, B., Colliver, JA., Benaroya, A., Snell, L. 1994Can standardized patients predict real-patient satisfaction with the doctor–patient relationship?Teaching and Learning in Medicine.63644Google Scholar Tombleson, P., Fox, R.A., Dacre, J.A. 2000Defining the content for the objective structured clinical examination component of the professional and linguistic assessments board examination: Development of a blueprintMedical Education34566572CrossRefPubMedGoogle Scholar Dalen, J., Kerkhofs, E., Knippenberg-Berg, B.W., Hout, H.A., Scherpbier, A.J., Vleuten, C.P. 2002Longitudinal and concentrated communication skills programmes: Two Dutch medical schools comparedAdvances in Health Sciences Education: Theory and Practice.72940CrossRefGoogle Scholar Wensing, M., Vleuten, C., Grol, R., Felling, A. 1997The reliability of patients judgements of care in general practice: How many questions and patients are needed?Quality Health Care68085Google Scholar Wilkinson, T., Newble, D., Frampton, C. 2001Standard setting in an objective structured clinical examination: Use of global ratings of borderline performance to determine the passing scoreMedical Education3510431049PubMedGoogle Scholar Willson, P., McNamara, J.R. 1982How perceptions of a simulated physician–patient interaction influence intended satisfaction and complianceSocial Science & Medicine.1616991704Google Scholar Woolliscroft, J.O., Howell, J.D., Patel, B.P., Swanson, D.B. 1994Resident–patient interactions: The humanistic qualities of internal medicine residents assessed by patients, attending physicians, program supervisors, and nursesAcademic Medicine.69216224PubMedGoogle Scholar