Skip to main content

Advertisement

SpringerLink
Log in

The validity of student tutors’ judgments in early detection of struggling in medical school. A prospective cohort study

  • Published:
Advances in Health Sciences Education Aims and scope Submit manuscript

Abstract

Early identification and support of strugglers in medical education is generally recommended in the research literature, though very little evidence of the diagnostic qualities of early teacher judgments in medical education currently exists. The aim of this study was to examine the validity of early diagnosis of struggling in medical school based on informal teacher judgements of in-class behavior. The study design was a prospective cohort study and the outcomes/truth criteria were anatomy failure and medical school drop out. Six weeks into an anatomy course, student tutors attempted to identify medical students, who they reckoned would fail the anatomy course or drop out, based on their everyday experiences with students in a large group educational setting. In addition, they were asked to describe the indicators of struggling they observed. Sixteen student tutors evaluated 429 medical students for signs of struggling. By week six, the student tutors were able to detect approximately 1/4–1/3 of the students who eventually failed or dropped out, and for ¾ of the strugglers they identified, they were correct in their judgments. Informal student tutor’s judgements showed incremental validity for both outcomes when controlling for grades obtained in preceeding exams. Lack of participation, lack of commitment, poor academic performance, poor social interactions and general signs of distress were the main indicators of struggling identified. Teachers’ informal judgements of in-class behavior may be an untapped source of information in the early identification of struggling medical students with added value above and beyond formal testing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Adam, J., Bore, M., McKendree, J., Munro, D., & Powis, D. (2012). Can personal qualities of medical students predict in-course examination success and professional behaviour? An exploratory prospective cohort study. BMC Medical Education, 12, 69.

    Article  Google Scholar 

  • Altman, D. G. (1991). Practical statistics for medical researchers. Boca Raton: Chapman & Hall/CRC.

    Google Scholar 

  • Bierer, B., Dannefer, E. F., & Tetzlaff, J. E. (2015). Time to loosen the apron strings: Cohort-based Evaluation of a learner-driven remediation model at one medical school. Journal of General Internal Medicine, 30(9), 1339–1343.

    Article  Google Scholar 

  • Brenner, A. M., Mathai, S., Satyam, J., & Mohl, P. C. (2010). Can we “predict” problem residents? Academic Medicine, 85(7), 1147–1151.

    Article  Google Scholar 

  • Challis, M. (2000). AMEE medical education guide no. 19: Personal learning plans. Medical Teacher, 22(3), 225–236.

    Article  Google Scholar 

  • Cleland, J., Arnold, R., & Chesser, A. (2005). Failing finals is often a surprise for the student but not the teacher: Identifying difficulties and supporting students with academic difficulties. Medical Teacher, 27(6), 504–508.

    Article  Google Scholar 

  • Cleland, J., Leggett, H., Sandars, J., Costa, M. J., Patel, R., & Moffat, M. (2013). The remediation challenge: theoretical and methodological insights from a systematic review. Medical Education, 47(3), 242–251.

    Article  Google Scholar 

  • Devoe, P., Niles, C., Benjamin, A., Blacklock, L., Brainard, A., Colombo, E., et al. (2007). Lessons learned from a study-group pilot program for medical students perceived to be ‘at risk’. Medical Teacher, 29(2), e37–e40.

    Article  Google Scholar 

  • Dunning, D., Johnson, K., Ehrligner, J., & Kruger, J. (2003). Why people fail to recognize their own incompetence. Current Directions in Psychological Science, 12(3), 83–87.

    Article  Google Scholar 

  • Evans, D. E., Alstead, E. M., & Brown, J. (2010). Applying your clinical skills to students and trainees in academic difficulty. The Clinical Teacher, 7(4), 230–235.

    Article  Google Scholar 

  • Fortin, Y., Kealey, L., Slade, S., & Hanson, M. D. (2015). Investigating Canadian medical school attrition metrics to inform socially accountable admissions planning. Medical Teacher, 1–5 [Epub ahead of print].

  • Georg, W. (2009). Individual and institutional factors in the tendency to drop out of higher education: A multilevel analysis using data from the Konstanz Student Survey. Studies in Higher Education, 34(6), 647–661.

    Article  Google Scholar 

  • Gonsalvez, D. G., Ovens, M., & Ivanusic, J. (2015). Does attendance at anatomy practical classes correlate with assessment outcome? A retrospective study of a large cohort of undergraduate anatomy students. BMC Medical Education, 15, 231.

    Article  Google Scholar 

  • Harackiewicz, J. M., Barron, K. E., Tauer, J. M., & Elliot, A. J. (2002). Predicting success in college: A longitudinal study of achievement goals and ability measures as predictors of interest and performance from freshman year through graduation. Journal of Educational Psychology, 94(3), 562–575.

    Article  Google Scholar 

  • Hauer, K. E., Ciccone, A., Henzel, T. R., Katsufrakis, P., Miller, S. H., Norcross, W. A., et al. (2009). Remediation of the deficiencies of physicians across the continuum from medical school to practice: A thematic review of the literature. Academic Medicine, 84(12), 1822–1832.

    Article  Google Scholar 

  • Hayes, R. B., Lawson, M., & Gray, C. (2011). Problems presented by medical students seeking support: A possible intervention framework. Medical Teacher, 33(2), 161–164.

    Article  Google Scholar 

  • Hodges, B., Regehr, G., & Martin, D. (2001). Difficulties in recognizing one’s own incompetence: Novice physicians who are unskilled and unaware of it. Academic Medicine, 76(10 Suppl), S87–S89.

    Article  Google Scholar 

  • Hojat, M., Gonnella, J. S., Erdmann, J. B., & Veloski, J. J. (1996). The fate of medical students with different levels of knowledge: Are the basic medical sciences relevant to physician competence? Advances in Health Sciences Education, 1(3), 179–196.

    Article  Google Scholar 

  • Iputo, J. E., & Kwizera, E. (2005). Problem-based learning improves the academic performance of medical students in South Africa. Medical Education, 39(4), 388–393.

    Article  Google Scholar 

  • Juul, S. (2004). Epidemiologi og evidens. København: Munksgaard Danmark.

    Google Scholar 

  • Kaufman, D. M., & Hansell, M. M. (1997). Can non-expert PBL tutors predict their students’achievement? An exploratory study. Academic Medicine, 72(Suppl. 1), S16–S18.

    Article  Google Scholar 

  • Kilminster, S., Cotrell, D., Grant, J., & Jolly, B. (2007). AMEE guide no. 27: Effective educational and clinical supervision. Medical Teacher, 29(1), 2–19.

    Article  Google Scholar 

  • Kleinbaum, D. G., & Klein, M. (2002). Logistic regression (2nd ed.). New York: Springer.

    Google Scholar 

  • Loyens, S. M. M., Rikers, R. M. J. P., & Schmidt, H. G. (2007). The impact of students’ conceptions of constructivist assumptions on academic achievement and drop-out. Studies in Higher Education, 32(5), 581–602.

    Article  Google Scholar 

  • Maher, B. M., Hynes, H., Sweeney, C., Khashan, A. S., O’Rourke, M., Doran, K., et al. (2013). Medical school attrition-beyond the statistics. A ten year retrospective study. BMC Medical Education, 13, 13.

    Article  Google Scholar 

  • Mazor, K. M., Fischer, M. A., Haley, H. L., Hatem, D., & Quirk, M. E. (2005). Teaching and medical errors: primary care preceptors’ views. Medical Education, 39(10), 982–990.

    Article  Google Scholar 

  • Menard, S. (2002). Applied logistic regression analysis (2nd ed.). Thousand Oaks: Sage publications Inc.

    Book  Google Scholar 

  • Mørcke, A. M. M., O’Neill, L., Kjeldsen, I. T., & Eika, B. (2012). Selected determinants may account for dropout risks among medical students. Danish Medical Journal, 59(9), A4493.

    Google Scholar 

  • O’Neill, L. D., Wallstedt, B., Eika, B., & Hartvigsen, J. (2011). Factors associated with dropout in medical education: A literature review. Medical Education, 45(5), 440–454.

    Article  Google Scholar 

  • Øiestad, G. (2006). Feedback. Copenhagen: Dansk Psykologisk Forlag.

    Google Scholar 

  • Papadakis, M. A., Hodgson, C. S., Theherani, A., & Kohatsu, N. D. (2004). Unprofessional behavior in medical school is associated with subsequent disciplinary action by a state medical board. Academic Medicine, 79(3), 244–249.

    Article  Google Scholar 

  • Papadakis, M. A., Teherani, A., Banach, M. A., Knettler, T. R., Rattner, S. L., Stern, D. T., et al. (2005). Disciplinary action by medical boards and prior behavior in medical school. New England Journal of Medicine, 353(25), 2673–2682.

    Article  Google Scholar 

  • Patel, R., Tarrant, C., Bonas, S., Yates, J., & Sandars, J. (2015). The struggling student: A thematic analysis from the self-regulated learning perspective. Medical Education, 49(4), 417–426.

    Article  Google Scholar 

  • Roberts, N. K., Williams, R. G., Klingensmith, M., Sullivan, M., Boehler, M., Hickson, G., et al. (2012). The case of the entitled resident: a composite case study of a resident performance problem syndrome with interdisciplinary commentary. Medical Teacher, 34(12), 1024–1032.

    Article  Google Scholar 

  • Schmidt, H. G., Cohen-Schotanus, J., & Arends, L. R. (2009). Impact of problem-based, active learning on graduation rates for 10 generations of Dutch medical students. Medical Education, 43(3), 211–218.

    Article  Google Scholar 

  • Stegers-Jager, K. M., Cohen-Schotanus, J., & Themmen, A. P. (2013). The effect of a short integrated study skills programme for first year medical students at risk of failure: A randomized controlled trial. Medical Teacher, 35(2), 120–126.

    Article  Google Scholar 

  • Steinert, Y. (2013). The “problem learner”: whose problem is it? AMEE Guide No. 76. Medical Teacher, 35(4), e1035–e1045.

    Article  Google Scholar 

  • Stetto, J. E., Gackstetter, G. D., Cruess, D. F., & Hooper, T. I. (2004). Variables associated with attrition from uniformed services university of the health sciences medical school. Military Medicine, 169(2), 102–107.

    Article  Google Scholar 

  • Tabby, D. S., Majeed, M. H., & Schwartzman, R. J. (2011). Problem neurology residents. A national survey. Neorology, 76(24), 2119–2123.

    Article  Google Scholar 

  • Teherani, A., Hodgson, C. S., Banach, M., & Papadakis, M. A. (2005). Domains of unprofessional behavior during medical school associated with future disciplinary action by a state medical board. Academic Medicine, 80(10 Suppl), S17–S20.

    Article  Google Scholar 

  • Tekian, A., & Hruska, L. (2004). A review of medical school records to investigste the effectiveness of enrichment programs for “at risk” students. Teaching and Learning in Medicine, 16(1), 28–33.

    Article  Google Scholar 

  • Tinto, V. (1975). Dropouts from higher education: A theoretical synthesis of recent research. Review of Educational Research, 45(1), 89–125.

    Article  Google Scholar 

  • Van de Watering, G. A., & Claessens, S. (2003). Verschillen tussen de perceptie van tutoren op de leerprestatie en de toetsprestatie van hun studenten [Differences between tutors’perceptions of performandce and students’ actual achievement]. Tijdschrift voor Hoger Onderwijs, 21(3), 199–214.

    Google Scholar 

  • Van den Berg, M. N., & Hofman, W. H. A. (2005). Student success in university education: A multi-measurement study of impact of student and faculty factors on study progress. Higher Education, 50(3), 413–446.

    Article  Google Scholar 

  • Whitfield, C. F., & Xie, S. X. (2002). Correlation of problem-based learning facilitators’ scores with student performance on written exams. Advances in Health Sciences Education, 7(1), 41–51.

    Article  Google Scholar 

  • Wijnia, L., Loyens, S. M. M., Derous, E., Koendjie, N. S., & Schmidt, H. G. (2013). Predicting educational success and attrition in problem-based learning: Do first impressions count? Studies in Higher Education, 38(1), 1–16.

    Article  Google Scholar 

  • Winston, K. A., Van der Veluten, C. P. M., & Scherpbier, A. J. J. A. (2010). An investigation into the design and effectiveness of a mandatory cognitive skills programme for at-risk medical students. Medical Teacher, 32(3), 236–243.

    Article  Google Scholar 

  • Winston, K. A., van der Vleuten, C. P. M., & Scherpbier, A. J. J. A. (2014). Prediction and prevention of failure: An early intervention to assist at-risk medical students. Medical Teacher, 36(1), 25–31.

    Article  Google Scholar 

  • Yao, D. C., & Wright, S. M. (2001). The challenge of problem residents. Journal of General Internal Medicine, 16(7), 486–492.

    Article  Google Scholar 

  • Yates, J. (2011). Development of a ‘toolkit’ to identify medical students at risk of failure to thrive on the course: an explorative retrospective case study. BMC Medical Education, 11, 95.

    Article  Google Scholar 

  • Yates, J., & James, D. (2006). Predicting the “strugglers”: a case-control study of students at Nottingham University Medical School. British Medical Journal, 332(7548), 1009–1013.

    Article  Google Scholar 

  • Yates, J., & James, D. (2010). Risk factors at medical school for subsequent professional misconduct: multicenter retrospective case-control study. British Medical Journal, 340, c2040.

    Article  Google Scholar 

  • Yu, T., Wilson, N. C., Singh, P. P., Lemanu, D. P., Hawken, S. J., & Hill, A. G. (2011). Medical students-as-teachers: a systematic review of peer-assisted teaching during medical school. Advances in Medical Education and Practice, 2, 157–172.

    Google Scholar 

  • Zbieranowski, I., Takahashi, S. G., Verma, S., & Spadafora, S. M. (2013). Remediation of residents in difficulty: A retrospective 10-year review of the experience of a postgraduate board of examiners. Academic Medicine, 88(1), 111–116.

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Associate Professor Peter Holm-Nielsen and the anatomy tutors at the Institute of Anatomy at Aarhus University for their kind help and their participation in this study, as well as Marianne Toftegaard Hansen Marianne from AU Studies Administration for her help with data extraction.

Funding

The study was funded by the Center for Health Sciences Education at Aarhus University.

Author information

Authors and Affiliations

  1. Centre for Health Sciences Education, INCUBA Science Park Skejby, Aarhus University, Palle Juul-Jensens Boulevard 82, building B, 8200, Århus N, Denmark

    Lotte Dyhrberg O’Neill & Anne Mette Morcke

  2. Rector’s Office, Aarhus University, Aarhus, Denmark

    Berit Eika

Authors
  1. Lotte Dyhrberg O’Neill
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anne Mette Morcke
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Berit Eika
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lotte Dyhrberg O’Neill.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

O’Neill, L.D., Morcke, A.M. & Eika, B. The validity of student tutors’ judgments in early detection of struggling in medical school. A prospective cohort study. Adv in Health Sci Educ 21, 1061–1079 (2016). https://doi.org/10.1007/s10459-016-9677-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10459-016-9677-6

Keywords

Search

Navigation