Skip to main content

Advertisement

SpringerLink
Log in

Multisource Feedback Tool for the Assessment of Medical Student Clerks in Professionalism, Communication, and Collaboration Skills

  • Original Research
  • Published:
Medical Science Educator Aims and scope Submit manuscript

Abstract

Construct

This study investigates the validity of the Bahrain Defence Force instrument to assess professionalism, communication, and collaboration skills (BDF/PCC instrument) in medical graduates during their clerkship/intern years.

Approach

The instrument to assess professionalism, communication, and collaboration skills [BDF/PCC instrument] was developed based on an extensive literature review, other existing valid instruments, and expert opinion. Exploratory and confirmatory factor analysis was conducted to support the construct validity. Twenty-five interns engaged in a multisource feedback evaluation for this study. Each participant was rated by eight individuals from each of the following categories: physicians, nurses, and fellow interns. Cronbach’s α was used to determine the questionnaire’s internal consistency and reliability.

Results

We report response rates (100 %), mean response time to complete each questionnaire (3.7 min), and the number of raters (seven to eight) needed to provide reliable results that support the feasibility of the survey. Reliability analysis using Cronbach’s α of internal consistency indicated that the full scale of the instrument had high internal consistency (Cronbach’s α 0.98). The instrument was found to be suitable for factor analysis (KMO = 0.941; Bartlett test significant, p < 0.001), which found that the data collected from the questionnaire could be grouped into three factors. These three factors represented 77.3 % of the total variance: professionalism, collaboration, and communication. The item-total correlation for this instrument was above 0.40 and showed homogeneity within each composite scale. The generalizability coefficients (Ep 2) were 0.79 for the surveys.

Conclusion

The BDF/PCC instrument to assess professionalism, communication, and collaboration skills is a feasible, reliable, and valid tool to assess physicians in their clerkship year. Testing the instrument on two occasions with a 1-year interval and the confirmatory factor analysis provided some evidence to support the concurrent and the construct validity of the BDF/PCC instrument.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Archer JC, Norcini J, Davies HA. Use of SPRAT for peer review of paediatricians in training. BMJ. 2005;330:1251–3.

    Article  Google Scholar 

  2. Al Ansari A, Donnon T, Al Khalifa K, Darwish A, Violato C. The construct and criterion validity of the multisource feedback process to assess physician and performance: a meta-analysis. Advn in Med Educ & Reser. 2014;5:39–51.

    Google Scholar 

  3. Moonen-van Loon JM, Overeem K, Govaerts MJ, Verhoeven BH, van der Vleuten CP, Driessen EW. The reliability of multisource feedback in competency-based assessment programs: the effects of multiple occasions and assessor groups. Acad Med. 2015;8:1093–9.

    Article  Google Scholar 

  4. Wood L, Hassell A, Whitehouse A, Bullock A, Wall D. A literature review of multi-source feedback systems within and without health services, leading to 10 tips for their successful design. Med Teach. 2006;28:e185–91.

    Article  Google Scholar 

  5. Druskatt V, Wolff S. Effects and timing of developmental peer appraisals in self-managing work groups. J Appl Psychol. 1999;1:58–74.

    Article  Google Scholar 

  6. Edwards M, Ewen A. 360 feedback: the powerful new model for employee assessment and performance improvement. New York: AMACOM; 1996.

    Google Scholar 

  7. Waldman D, Bowen D. The acceptability of 360° appraisals: a customer-supplier relationship perspective. Hum Resour Manag. 1998;2:117–29.

    Article  Google Scholar 

  8. Van der Heijden BI, Nojhof AH. The value of subjectivity: problems and prospects for 360 degree appraisal systems. Int J Hum Resour Manag. 2004;3:493–511.

    Article  Google Scholar 

  9. Brinkman WB, Geraghty SR, Lanphear BP, Khoury JC, del Rey Gonzalez JAG, DeWitt TG, Britto M. Effect of multisource feedback on resident communication skills and professionalism—a randomized controlled trial. Arch Pediatr Adolesc Med. 2007;1:44–9.

    Article  Google Scholar 

  10. Garra G, Wackett A, Thode H. Feasibility and reliability of a multisource feedback tool for emergency medicine residents. J Grad Med Educ. 2011;3:356–60.

    Article  Google Scholar 

  11. Donnon T, Al Ansari A, Al Alawi S, Violato C. The reliability, validity, and feasibility of multisource feedback physician assessment: a systematic review. Acad Med. 2014;3:1–6.

    Google Scholar 

  12. Al Ansari A, Al Khalifa K, Al Azzawi M, Al Amer R, Al Sharqi D, Al-Mansoor A, et al. Cross-cultural challenges for assessing medical professionalism among clerkship physicians in a middle eastern country (Bahrain): feasibility and psychometric properties of multisource feedback. Advanc in Medi Educ and Pract. 2015;6:509–15.

    Article  Google Scholar 

  13. Lockyer JM, Violato C, Fidler H. The assessment of emergency physicians by a regulatory authority. Acad Med. 2006;12:1296–303.

    Article  Google Scholar 

  14. Violato C, Lockyer JM, Fidler H. Assessment of psychiatrists in practice through multisource feedback. Can J Psychiatr. 2008;8:525–33.

    Google Scholar 

  15. Van Thiel J, Van Dalen J, Ram P. MAAS Global Manual 2000. Maastricht: University Press; 2003.

    Google Scholar 

  16. Burt J, Elmore N, Campbell J, Roland M, Benson J, et al. Assessing communication quality of consultations in primary care: initial reliability of the global consultation rating scale, based on the Calgary-Cambridge guide to the medical interview. BMJ Open. 2014;4:e004339.

    Article  Google Scholar 

  17. Archer J, Norcicni J, Davies H. Use of SPRAT for peer review of pediatricians in training. BMJ. 2005;330:1251–3.

    Article  Google Scholar 

  18. Orchard CA, King GA, Khalili H, Bezzina MB. Assessment of interprofessional team collaboration scale (AITCS): development and testing of the instrument. J Contin Educ Heal Prof. 2012;1:58–67.

    Article  Google Scholar 

  19. Violato C, Saberton S. Assessing medical radiation technologists in practice: a multi-source feedback system for quality assurance. Can J Med Radiat Technol. 2006;2:10–7.

    Article  Google Scholar 

  20. Lockyer J, Violato C, Fidler H, Alakija P. The assessment of pathologists/laboratory medicine physicians through a multisource feedback tool. Arch Pathol Lab Med. 2009;8:1301–8.

    Google Scholar 

  21. Renee L, Kiki L, Maas JH. Systematic evaluation of the teaching qualities of obstetrics and gynecology faculty: reliability and validity of the SETQ tool. PLos One. 2011. 6(5).

  22. Streiner DL. Norman GR health measurement scales: a practical guide to their development and use. Oxford: Oxford University Press; 2008.

    Book  Google Scholar 

  23. Brennan RL. Generalizability theory, vol. 79. New York: Springer-Verlag; 2001. p. 441.

    Book  Google Scholar 

  24. Shute VJ. Focus on formative feedback. Rev Educ Res. 2008;78:153–89.

    Article  Google Scholar 

  25. Hawkins RE, Katsufrakis PJ, Holtman MC, Clauser BE. Assessment of medical professionalism: who, what, when, where, how, and … why? Med Teach. 2009;31:348–61.

    Article  Google Scholar 

  26. Pulito AR, Donnelly MB, Plymale M, Mentzer Jr RM. What do faculty observe of medical students’ clinical performance? Teach Learn Med. 2006;18:99–104.

  27. Mazor KM, Holtman MC, Shchukin Y, Mee J, Katsufrakis PJ. The relationship between direct observation, knowledge and feedback: results of a national survey. Acad Med. 2011;86:S63–8.

    Article  Google Scholar 

  28. Dolmans DH, Wolfhagen IH, Heineman E, Scherpbier AJ. Factors adversely affecting student learning in the clinical learning environment: a student perspective. Educ Health (Abingdon). 2008;21:32.

    Google Scholar 

  29. Sargeant J. Reflecting upon multisource feedback as ‘assessment for learning’. Perspect Med Educ. 2015;4:55–6.

    Article  Google Scholar 

  30. Sargeant J, Mann K, van der Vleuten C, Metsemakers J. Reflection: a link between receiving and using assessment feedback. Adv Health Sci Educ Theory Pract. 2009;3:399–410.

    Article  Google Scholar 

  31. Miller A, Archer J. Impact of workplace based assessment on doctors’ education and performance: a systematic review. BMJ. 2010;341:c5064.

    Article  Google Scholar 

  32. Sargeant JM, Mann KV, Ferrier SN, Langille DB, Muirhead PD, Hayes VM, et al. Responses of rural family physicians and their colleague and coworker raters to a multi-source feedback process: a pilot study. Acad Med. 2003;78:S42–4.

    Article  Google Scholar 

  33. Sargeant J, Mann K, Ferrier S. Exploring family physicians’ reactions to multisource feedback: perceptions of credibility and usefulness. Med Educ. 2005;39:497–504.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Training and Education Department, Bahrain Defence Force Hospital, P. O. Box - 28743, West Riffa, Kingdom of Bahrain

    Ahmed Al Ansari

  2. Assistant Professor of General Surgery Arabian Gulf University, Manama, Kingdom of Bahrain, Senior Clinical Lecturers in Medical education, RCSI Bahrain, Adliya, Kingdom of Bahrain

    Ahmed Al Ansari

  3. Ophthalmology Department, Bahrain Defence Force Hospital, Riffa, Bahrain

    Aysha Waheed Agab

  4. Pediatric Department, Bahrain Defence Force Hospital, Riffa, Bahrain

    Sarah Dawood Al Sayed

  5. General Surgery Department, Bahrain Defence Force Hospital, Riffa, Bahrain

    Noora Jassim Al Fudhala

Authors
  1. Ahmed Al Ansari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aysha Waheed Agab
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sarah Dawood Al Sayed
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Noora Jassim Al Fudhala
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmed Al Ansari.

Ethics declarations

Ethical Approval

The research was approved by the research ethics committee in the BDF hospital. Written consent was obtained from the interns, and verbal consent was obtained from raters. The study was conducted between March 2014 and June 2015.

Competing Interests

The authors declare that they have no competing interests.

Financial Competing Interests

All authors declare that there are no financial competing interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Al Ansari, A., Agab, A.W., Al Sayed, S.D. et al. Multisource Feedback Tool for the Assessment of Medical Student Clerks in Professionalism, Communication, and Collaboration Skills. Med.Sci.Educ. 26, 609–616 (2016). https://doi.org/10.1007/s40670-016-0311-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40670-016-0311-2

Keywords

Search

Navigation