Skip to main content
SpringerLink
Log in

Development and implementation of the Structured Training Trainer Assessment Report (STTAR) in the English National Training Programme for laparoscopic colorectal surgery

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

There is a lack of educational tools available for surgical teaching critique, particularly for advanced laparoscopic surgery. The aim was to develop and implement a tool that assesses training quality and structures feedback for trainers in the English National Training Programme for laparoscopic colorectal surgery.

Methods

Semi-structured interviews were performed and analysed, and items were extracted. Through the Delphi process, essential items pertaining to desirable trainer characteristics, training structure and feedback were determined. An assessment tool (Structured Training Trainer Assessment Report—STTAR) was developed and tested for feasibility, acceptability and educational impact.

Results

Interview transcripts (29 surgical trainers, 10 trainees, four educationalists) were analysed, and item lists created and distributed for consensus opinion (11 trainers and seven trainees). The STTAR consisted of 64 factors, and its web-based version, the mini-STTAR, included 21 factors that were categorised into four groups (training structure, training behaviour, trainer attributes and role modelling) and structured around a training session timeline (beginning, middle and end). The STTAR (six trainers, 48 different assessments) demonstrated good internal consistency (α = 0.88) and inter-rater reliability (ICC = 0.75). The mini-STTAR demonstrated good inter-item reliability (α = 0.79) and intra-observer reliability on comparison of 85 different trainer/trainee combinations (r = 0.701, p = <0.001). Both were found to be feasible and acceptable. The educational report for trainers was found to be useful (4.4 out of 5).

Conclusions

An assessment tool that evaluates training quality was developed and shown to be reliable, acceptable and of educational value. It has been successfully implemented into the English National Training Programme for laparoscopic colorectal surgery.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Birch DW, Bonjer HJ, Crossley C et al (2009) Canadian consensus conference on the development of training and practice standards in advanced minimally invasive surgery: Edmonton, Alta., Jun. 1, 2007. Can J Surg 52(4):321–327

  2. Blue AV, Griffith CH 3rd, Wilson J et al (1999) Surgical teaching quality makes a difference. Am J Surg 177(1):86–89

    Article  CAS  PubMed  Google Scholar 

  3. Hull L, Arora S, Aggarwal R et al (2012) The impact of nontechnical skills on technical performance in surgery: a systematic review. J Am Coll Surg 214(2):214–230

    Article  PubMed  Google Scholar 

  4. Sevdalis N, Davis R, Koutantji M et al (2008) Reliability of a revised NOTECHS scale for use in surgical teams. Am J Surg 196(2):184–190

    Article  PubMed  Google Scholar 

  5. Haynes AB, Weiser TG, Berry WR et al (2009) A surgical safety checklist to reduce morbidity and mortality in a global population. New Engl J Med 360(5):491–499

    Article  CAS  PubMed  Google Scholar 

  6. Peyton J (1998) Teaching and learning in medical practice. Manticore Europe Limited, Rickmansworth

    Google Scholar 

  7. Skeff KM, Stratos GA, Berman J et al (1992) Improving clinical teaching. Evaluation of a national dissemination program. Arch Intern Med 152(6):1156–1161

    Article  CAS  PubMed  Google Scholar 

  8. Copertino N, Blackham R, Hamdorf JM (2010) A short course for surgical supervisors and trainers: effecting behavioural change. ANZ J Surg 80(12):896–901

    Article  PubMed  Google Scholar 

  9. London Deanery ARCP http://www.londondeanery.ac.uk/specialty-schools/surgery. (Accessed 22 Sept 2014)

  10. Copeland HL, Hewson MG (2000) Developing and testing an instrument to measure the effectiveness of clinical teaching in an academic medical center. Acad Med 75(2):161–166

    Article  CAS  PubMed  Google Scholar 

  11. Cohen R, MacRae H, Jamieson C (1996) Teaching effectiveness of surgeons. Am J Surg 171(6):612–614

    Article  CAS  PubMed  Google Scholar 

  12. Beckman TJ, Ghosh AK, Cook DA et al (2004) How reliable are assessments of clinical teaching? A review of the published instruments. J Gen Intern Med 19(9):971–977

    Article  PubMed Central  PubMed  Google Scholar 

  13. Miskovic D, Wyles SM, Ni M et al (2010) Systematic review on mentoring and simulation in laparoscopic colorectal surgery. Ann Surg 252(2):943–951

    Article  PubMed  Google Scholar 

  14. Coleman MG, Hanna GB, Kennedy R (2011) The National Training Programme for laparoscopic colorectal surgery in England: a new training paradigm. Colorectal Dis 13(6):614–616

    Article  CAS  PubMed  Google Scholar 

  15. Wyles SM, Miskovic D, Ni M et al (2012) “Trainee” evaluation of the English National Training Programme for laparoscopic colorectal surgery. Colorectal Dis 14(6):352–357

    Article  Google Scholar 

  16. Lapco. National Training Programme in laparoscopic colorectal surgery. http://www.lapco.nhs.uk. (Accessed 22 Sept 2014)

  17. Kennedy TJ, Lingard LA (2006) Making sense of grounded theory in medical education. Med Educ 40(2):101–108

    Article  PubMed  Google Scholar 

  18. NVIVO http://www.qsrinternational.com/FileResourceHandler.ashx/RelatedDocuments/DocumentFile/456/NVivo_8_brochure.pdf. (Accessed 22 Sept 2014)

  19. Britten N (1995) Qualitative interviews in medical research. BMJ 311(6999):251–253

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  20. Brownpaper-technique http://www.fpxhn.net/tttsubpages/1week_comtemt/process-mapping.pps. (Accessed 17th Mar 2013)

  21. Mackenzie H, Cuming T, Miskovic D et al (2013) Design, delivery, and validation of a trainer curriculum for the national laparoscopic colorectal training program in England. Ann Surg 261(1):149–156

    Article  Google Scholar 

  22. Streiner N, Norman G (2008) Health measurement scales: a practical guide to their development and use, 4th edn. Oxford University Press, Oxford

    Book  Google Scholar 

  23. Whyte WF (1982) Interviewing in field research. George Allen and Unwin, London

    Google Scholar 

  24. Lingard L, Regehr G, Espin S et al (2006) A theory-based instrument to evaluate team communication in the operating room: balancing measurement authenticity and reliability. Qual Saf Health Care 15(6):422–426

    Article  PubMed Central  PubMed  Google Scholar 

  25. Irby DM (1978) Clinical teacher effectiveness in medicine. J Med Educ 53(10):808–815

    CAS  PubMed  Google Scholar 

  26. Nisar PJ, Scott HJ (2011) Key attributes of a modern surgical trainer: perspectives from consultants and trainees in the United kingdom. J Surg Educ 68(3):202–208

    Article  PubMed  Google Scholar 

  27. Rolfe I, McPherson J (1995) Formative assessment: how am I doing? Lancet 345(8953):837–839

    Article  CAS  PubMed  Google Scholar 

  28. Vygotsky L (1978) Mind in society. Harvard University Press, Cambridge

    Google Scholar 

  29. Miskovic D, Ni M, Wyles SM et al (2012) Learning curve and case selection in laparoscopic colorectal surgery: systematic review and international multicenter analysis of 4852 cases. Dis Colon Rectum 55(12):1300–1310

    Article  PubMed  Google Scholar 

  30. Mackenzie H, Miskovic D, Ni M et al (2014) Risk prediction score in laparoscopic colorectal surgery training: experience from the English National Training Program. Ann Surg 261(2):338–344

    Article  Google Scholar 

  31. Arora S, Aggarwal R, Sevdalis N et al (2010) Development and validation of mental practice as a training strategy for laparoscopic surgery. Surg Endosc 24(1):179–187

    Article  PubMed  Google Scholar 

  32. Arora S, Aggarwal R, Sirimanna P et al (2011) Mental practice enhances surgical technical skills: a randomized controlled study. Ann Surg 253(2):265–270

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

This work was funded by the National Cancer Action Team, Department of Health UK. Thanks go to Mrs. Laura Langsford for enabling the form to be piloted within the Lapco TT course and to Mr. Andy McMeeking at the National Cancer Action Team for supporting some of the funding. Also to Dr. John Anderson and Dr. Chris Wells for their invaluable advice and input to both the methodology and the form structure. To the National Training Programme and all trainers and trainees involved in this. Also specifically to the following surgeons and their trainees and theatre staff: Mr. Austin Acheson, Mr. Chris Cunningham, Mr. John Griffiths, Mr. Mark Gudgeon, Mr. Alan Horgan, Mr. Roel Hompes, Mr. Ian Jenkins, Mr. Mark Katory, Mr. Ian Lindsey, Mr. Charles Maxwell-Armstrong, Professor Robin Kennedy, Professor Roger Motson and Professor Timothy Rockall, who all readily welcomed the researchers into their operating theatres in the early piloting phases of the STTAR and mini-STTAR.

Disclosures

Dr. Valori is the director of a consultancy called “Quality Solutions for Healthcare LLP” which specialises in quality improvement and training. Drs. Wyles, Miskovic, Ni, Coleman and Professors Darzi and Hanna have no conflicts of interest or financial ties to disclose.

Author information

Authors and Affiliations

  1. Department of Surgery and Cancer, St Mary’s Hospital, Imperial College London, London, UK

    Susannah M. Wyles & Ara W. Darzi

  2. St James University Hospital, Leeds, UK

    Danilo Miskovic

  3. National Training Programme, Department of Surgery and Cancer, St Mary’s Hospital, Imperial College London, London, UK

    Zhifang Ni

  4. Department of Gastroenterology, Gloucestershire Hospitals NHS Foundation Trust, Gloucester, UK

    Roland M. Valori

  5. National Training Programme, Derriford Hospital, Plymouth, UK

    Mark G. Coleman

  6. National Training Programme, Department of Surgery and Cancer, St Mary’s Hospital, Imperial College London, Praed St, London, W2 1NY, UK

    George B. Hanna

Authors
  1. Susannah M. Wyles
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Danilo Miskovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhifang Ni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ara W. Darzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Roland M. Valori
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mark G. Coleman
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. George B. Hanna
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to George B. Hanna.

Appendix

Appendix

See Table 2.

Table 2 Training technique items with >1 difference in consensus, with all becoming more extreme with a shift to the right of the seven-point scale
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wyles, S.M., Miskovic, D., Ni, Z. et al. Development and implementation of the Structured Training Trainer Assessment Report (STTAR) in the English National Training Programme for laparoscopic colorectal surgery. Surg Endosc 30, 993–1003 (2016). https://doi.org/10.1007/s00464-015-4281-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-015-4281-z

Keywords

Search

Navigation