Skip to main content

Advertisement

SpringerLink
Log in

Product Analysis and Initial Reliability Testing of the Total Mesorectal Excision-Quality Assessment Instrument

  • Colorectal Cancer
  • Published:
Annals of Surgical Oncology Aims and scope Submit manuscript

Abstract

Background

Product analysis of rectal cancer resection specimens before specimen fixation may provide an immediate and relevant evaluation of surgical performance. We tested the interrater reliability (IRR) of a product analysis tool called the Total Mesorectal Excision-Quality Assessment Instrument (TME-QA).

Methods

Participants included two gold standard raters, five pathology assistants, and eight pathologists. Domains of the TME-QA reflect total mesorectal excision principles including: (1) completeness of mesorectal margin; (2) completeness of mesorectum; (3) coning of distal mesorectum; (4) physical defects; and (5) overall specimen quality. Specimens were scored independently. We used the generalizability theory to assess the tool’s internal consistency and IRR.

Results

There were 39 specimens and 120 ratings. Mean overall specimen quality scores for the gold standard raters, pathologists, and assistants were 4.43, 4.43, and 4.50, respectively (p > 0.85). IRR for the first nine items was 0.68 for the full sample, 0.62 for assistants alone, 0.63 for pathologists alone, and 0.74 for gold standard raters alone. IRR for the item overall specimen quality was 0.67 for the full sample, 0.45 for assistants, 0.80 for pathologists, and 0.86 for gold standard raters. IRR increased for all groups when scores were averaged across two raters.

Conclusions

Assessment of surgical specimens using the TME-QA may provide rapid and relevant feedback to surgeons about their technical performance. Our results show good internal consistency and IRR when the TME-QA is used by pathologists. However, for pathology assistants, multiple ratings with the averaging of scores may be needed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Szalay D, MacRae H, Regehr G, et al. Using operative outcome to assess technical skill. Am J Surg. 2000;180:234–7.

    Article  CAS  PubMed  Google Scholar 

  2. Heald RJ. Total mesorectal excision is optimal surgery for rectal cancer: a Scandinavian consensus. Br J Surg. 1995;82:1297–9.

    Article  CAS  PubMed  Google Scholar 

  3. Havenga K, Enker WE, DeRuiter MC, et al. Anatomical basis of total mesorectal excision and preservation of the pelvic autonomic nerves. In: Soreide O, Norstein J, editors. Rectal cancer surgery: optimisation, standardization, documentation. Berlin: Springer; 1997. p. 134–42.

    Chapter  Google Scholar 

  4. MacFarlane JK, Ryall R, Heald RJ. Mesorectal excision for rectal cancer. Lancet. 1993;341:457–60.

    Article  CAS  PubMed  Google Scholar 

  5. Quirke P. Limitations of existing systems of staging for rectal cancer: the forgotten margin. In: Soreide O, Norstein J, editors. Rectal cancer surgery: optimisation, standardization, documentation. Berlin: Springer; 1997. p. 63–81.

    Chapter  Google Scholar 

  6. Nagtegaal ID, van de Velde CJ, van der Worp E, et al. Macroscopic evaluation of rectal cancer resection specimen: clinical significance of the pathologist in quality control. J Clin Oncol. 2002;20:1729–34.

    Article  PubMed  Google Scholar 

  7. Quirke P, Steele R, Monson J, et al. Effect of the plane of surgery achieved on local recurrence in patients with operable rectal cancer: a prospective study using data from the MRCCR07 and NCIC-CTG CO16 randomised clinical trial. Lancet. 2009;373:821–28.

    Article  PubMed Central  PubMed  Google Scholar 

  8. Streiner D, Norman G. Health measurement scales. A practical guide to their development and use, 2nd ed. Oxford: Oxford University Press; 1995.

    Google Scholar 

  9. Lissitz RW, Samuelsen K. A suggested change in terminology and emphasis regarding validity and education. Educ Res. 2007;36:437–48.

    Article  Google Scholar 

  10. Smith AJ, Driman DK, Spithoff K, et al. Guideline for optimization of colorectal cancer surgery and pathology. J Surg Oncol. 2010;101:5–12.

    Article  PubMed  Google Scholar 

  11. Ramsey PG, Carline JD, Blank LL, et al. Feasibility of hospital-based use of peer ratings to evaluate the performances of practicing physicians. Acad Med. 1996;71:364–70.

    Article  CAS  PubMed  Google Scholar 

  12. Donner A, Eliasziw M. Sample size requirements for reliability studies. Stat Med. 1987;6:441–8.

    Article  CAS  PubMed  Google Scholar 

  13. Cronbach LJ, Gleser GC, et al. The dependability of behavioral measurements: theory of generalizability for scores and profiles. New York: Wiley; 1972.

    Google Scholar 

  14. Nenshi R, Baxter N, Kennedy E, et al. Surgery for colorectal cancer. In: Urbach DR, Simunovic M, Schultz SE, editors. Cancer surgery in Ontario: ICES atlas. Toronto: Institute for Clinical Evaluative Sciences; 2008. p. 53–96.

    Google Scholar 

  15. Simunovic M, To T, Baxter NN, et al. Hospital procedure volume and teaching status do not influence treatment and outcome measures of rectal cancer surgery in a large general population. J Gastrointest Surg. 2000;4:324–30.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgment

This study received funding from the McMaster Surgical Associates (MSA), Department of Surgery, McMaster University, Hamilton, Ontario, Canada. The funder played no role in study design or in the collection, analysis, and interpretation of data.

Disclosure

The authors declare no conflict of interest.

ETHICS APPROVAL

The study received ethics approval from the Research Ethics Board of Hamilton Health Sciences/McMaster University.

Author information

Authors and Affiliations

  1. Department of Surgery, McMaster University, Hamilton, ON, Canada

    Marko R. Simunovic MD, Angela J. Coates MEd & David A. Szalay MD

  2. Department of Surgical Oncology, Juravinski Cancer Centre, Hamilton Health Sciences, Hamilton, ON, Canada

    Marko R. Simunovic MD

  3. Department of Pathology, Niagara Health System St. Catharines Site, St. Catharines, ON, Canada

    Franco G. DeNardi MD

  4. Centre for Health Education Scholarship, University of British Columbia, Vancouver, BC, Canada

    Kevin W. Eva PhD

Authors
  1. Marko R. Simunovic MD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Franco G. DeNardi MD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Angela J. Coates MEd
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David A. Szalay MD
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kevin W. Eva PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marko R. Simunovic MD.

Appendix

Appendix

Rights and permissions

Reprints and permissions

About this article

Cite this article

Simunovic, M.R., DeNardi, F.G., Coates, A.J. et al. Product Analysis and Initial Reliability Testing of the Total Mesorectal Excision-Quality Assessment Instrument. Ann Surg Oncol 21, 2274–2279 (2014). https://doi.org/10.1245/s10434-014-3604-y

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1245/s10434-014-3604-y

Keywords

Search

Navigation