Skip to main content
SpringerLink
Log in

Reliability and accuracy assessment of radiation therapy oncology group-endorsed guidelines for brachial plexus contouring

Beurteilung von Zuverlässigkeit und Genauigkeit der von der „Radiation Therapy Oncology Group“ unterstützten Richtlinien für die Plexus-brachialis-Konturierung

  • Original article
  • Published:
Strahlentherapie und Onkologie Aims and scope Submit manuscript

Abstract

Purpose

The goal of this work was to validate the Radiation Therapy Oncology Group (RTOG)-endorsed guidelines for brachial plexus (BP) contouring by determining the intra- and interobserver agreement. Accuracy of the delineation process was determined using anatomically validated imaging datasets as a gold standard.

Materials and methods

Five observers delineated the right BP on three cadaver computed tomography (CT) datasets. To assess intraobserver variation, every observer repeated each delineation three times with a time interval of 2 weeks. The BP contours were divided into four regions for detailed analysis. Inter- and intraobserver variation was verified using the Computerized Environment for Radiation Research (CERR) software. Accuracy was measured using anatomically validated fused CT–magnetic resonance imaging (MRI) datasets by measuring the BP inclusion of the delineations.

Results

The overall kappa (κ) values were rather low (mean interobserver overall κ: 0.29, mean intraobserver overall κ: 0.45), indicating poor inter- and intraobserver reliability. In general, the κ coefficient decreased gradually from the medial to lateral BP regions. The total agreement volume (TAV) was much smaller than the union volume (UV) for all delineations, resulting in a low Jaccard index (JI; interobserver agreement 0–0.124; intraobserver agreement 0.004–0.636). The overall accuracy was poor, with an average total BP inclusion of 38 %. Inclusions were insufficient for the most lateral regions (region 3: 21.5 %; region 4: 12.6 %).

Conclusion

The inter- and intraobserver reliability of the RTOG-endorsed BP contouring guidelines was poor. BP inclusion worsened from the medial to lateral regions. Accuracy assessment of the contours showed an average BP inclusion of 38 %. For the first time, this was assessed using the original anatomically validated BP volume. The RTOG-endorsed BP guidelines have insufficient accuracy and reliability, especially for the lateral head-and-neck regions.

Zusammenfassung

Ziel

Ziel der Studie war es, die von der „Radiation Therapy Oncology Group“ (RTOG) unterstützten Richtlinien für die Plexus-brachialis-(PB-)Konturierung durch Bestimmung von Inter- und Intraobserver-Reliabilität zu validieren. Die Präzision des Abgrenzungsprozesses wurde unter Verwendung anatomisch validierter Bilddatensätze als Goldstandard bestimmt.

Materialien und Methoden

Fünf Beobachter skizzierten im Tomographiedatensatz von drei Leichen den rechten PB. Um die Intraobserver-Abweichung zu bestimmen, wiederholte jeder Beobachter jede Konturierung 3-mal mit einem Zeitintervall von 2 Wochen. Die PB-Konturierungen wurden zur Detailanalyse in 4 Bereiche eingeteilt. Die Inter- und Intraobserver-Abweichungen wurden mit Hilfe der „Computerized-Environment-Radiation-Research“-(CERR-)Software geprüft. Die Präzision wurde anhand anatomisch validierter CT-MRI-Datensätze durch Messung der PB-Inklusion der Konturierung bestimmt.

Ergebnisse

Die gesamten κ-Werte waren eher niedrig (durchschnittliches Gesamt-Interobserver-κ: 0,29; durchschnittliches Gesamt-Intraobserver-κ: 0,45), was auf eine schwache Inter- und Intraobserver-Zuverlässigkeit hinweist. Im Allgemeinen nahm der κ-Koeffizient vom medialen zum seitlichen Bereich schrittweise ab. Das „Total Agreement Volume“ (TAV) war für alle konturierungen viel kleiner als das „Union Volume“ (UV), was einen niedrigen Jaccard-Index (JI) ergab (Interobserver-Übereinstimmung 0–0,124; Intraobserver-Übereinstimmung 0,004–0,636). Die gesamte Präzision war schwach, mit einem gesamten durchschnittlichen PB-Einschluss von 38 %. Die Inklusionen waren unzureichend für die meisten seitlichen Bereiche (Bereich 3: 21,5 %; Bereich 4: 12,6 %).

Schlussfolgerung

Die Inter- und Intraobserver-Reliabilität der von der RTOG unterstützten Richtlinien für die PB-Konturierung waren schwach. Die PB-Inklusion ließ vom mittleren zum seitlichen Bereich nach. Die Bestimmung der Präzision der Konturierung zeigte eine durchschnittliche PB-Inklusion von 38 %. Zum ersten Mal wurde dies anhand des ursprünglichen, anatomisch validierten PB-Volumens bestimmt. Die RTOG-unterstützten PB-Richtlinien zeigen eine unzureichende Präzision und Zuverlässigkeit, vor allem für den seitlichen Kopf-Hals-Bereich.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Hall WH, Guiou M, Lee NY et al (2008) Development and validation of a standardized method for contouring the brachial plexus: preliminary dosimetric analysis among patients treated with IMRT for head-and-neck cancer. Int J Radiat Oncol Biol Phys 72:1362–1367

    Google Scholar 

  2. Truong MT, Nadgir RN, Hirsch AE et al (2010) Brachial plexus contouring with CT and MR imaging in radiation therapy planning for head and neck cancer. Radiographics 30:1095–1103

    Google Scholar 

  3. Amini A, Jang J, Williamson R et al (2012) Dose constraints to prevent radiation-induced brachial plexopathy in patients treated for lung cancer. Int J Radiat Oncol Biol Phys 82:391–398

    Google Scholar 

  4. Lundstedt D, Gustafsson M, Steineck G et al (2012) Long-term symptoms after radiotherapy of supraclavicular lymph nodes in breast cancer patients. Radiother Oncol 103:155–160

    Google Scholar 

  5. Yi SK, Hall WH, Mathai M et al (2012) Validating the RTOG-endorsed brachial plexus contouring atlas: an evaluation of reproducibility among patients treated by intensity-modulated radiotherapy for head-and-neck cancer. Int J Radiat Oncol Biol Phys 82:1060–1064

    Google Scholar 

  6. Atean I, Pointreau Y, Barillot I et al (2012) Volumes de délinéation dans le traitement des cancers du sein: volumes cibleset organes à risque. Cancer/Radiothérapie 16:485–492

    Google Scholar 

  7. Chen AM, Hall WH, Li J et al (2012) Brachial plexus-associated neuropathy after high-dose radiation therapy for head-and-neck cancer. Int J Radiat Oncol Biol Phys 84(1):165–169

    Google Scholar 

  8. Platteaux N, Dirix P, Hermans R et al (2010) Brachial plexopathy after chemoradiotherapy for head and neck squamous ccell carcinoma. Strahlenther Onkol 186:517–520

    Google Scholar 

  9. Kong FM, Ritter T, Quint DJ et al (2011) Consideration of dose limits for organs at risk of thoracic radiotherapy: atlas for lung, proximal bronchial tree, esophagus, spinal cord, ribs, and brachial plexus. Int J Radiat Oncol Biol Phys 81:1442–1457

    Google Scholar 

  10. Van de Velde J, Audenaert EA, Speleers B et al (2013) An anatomically validated brachial plexus contouring method for intensity modulated radiation therapy planning. Int J Radiat Oncol Biol Phys 87:802–808

    Google Scholar 

  11. Kepka L, Bujko K, Garmol D et al (2007) Delineation variation of lymph node stations for treatment planning in lung cancer radiotherapy. Radiother Oncol 85:450–455

    Google Scholar 

  12. Petersen RP et al (2007) Target volume delineation for partial breast radiotherapy planning: clinical characteristics associated with low interobserver concordance. Int J Radiat Oncol Biol Phys 69:41–48

    Google Scholar 

  13. Song WY, Chui B, Bauman GS et al (2006) Prostate contouring uncertainty in megavoltage computed tomography images acquired with a helical tomotherapy unit during image-guided radiation therapy. Int J Radiat Oncol Biol Phys 65:595–607

    Google Scholar 

  14. Breen SL, Publicover J, De Silva S et al (2007) Intraobserver and interobserver variability in GTV delineation on FDG-PET-CT images of head and neck cancers. Int J Radiat Oncol Biol Phys 68:763–770

    Google Scholar 

  15. Simmat I, Georg P, Georg D et al (2012) Assessment of accuracy and efficiency of atlas-based autosegmentation for prostate radiotherapy in a variety of clinical conditions. Strahlenther Onkol 188:807–815

    Google Scholar 

  16. Thiel W (1992) Die Konservierung ganzer leichen in natürlichen farben. Ann Anat 174:185–195

    Google Scholar 

  17. De Crop A, Bacher K, Van Hoof T et al (2012) Correlation of contrast-detail analysis and clinical image quality assessment in chest radiography with a human cadaver study. Radiology 262:298–304

    Google Scholar 

  18. Audenaert EA, Vigneron L, Van Hoof T et al (2011) In vitro validation and reliability study of electromagnetic skin sensors for evaluation of end range of motion positions of the hip. Med Biol Eng Comput 49:1405–1412

    Google Scholar 

  19. Schuenke MD, Vleeming A, Van Hoof T et al (2012) A description of the lumbar interfascial triangle and its relation with the lateral raphe: anatomical constituents of load transfer through the lateral margin of the thoracolumbar fascia. J Anat 221:568–576

    Google Scholar 

  20. Van Hoof T, Gomes GT, Audenaert E et al (2008) 3D computerized model for measuring strain and displacement of the brachial plexus following placement of reverse shoulder prosthesis. Anat Rec 291:1173–1185

    Google Scholar 

  21. Fleiss JL, Levin BA, Paik MC (2003) Statistical methods for rates and proportions. 3rd ed. Wiley, Hoboken

  22. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174

    Google Scholar 

  23. Lawton CAF, Michalski J, El-Naqa I et al (2009) Variation in the definition of clinical target volumes for pelvic nodal conformal radiation therapy for prostate cancer. Int J Radiat Oncol Biol Phys 74:377–382

    Google Scholar 

  24. Ost P, De Meerleer G, Vercauteren T et al (2011) Delineation of the postprostatectomy prostate bed using computed tomography: interobserver variability following the EORTC delineation guidelines. Int J Radiat Oncol Biol Phys 81:143–149

    Google Scholar 

  25. Hoyte L, Ye W, Brubaker L et al (2011) Segmentations of MRI images of the female pelvic floor: a study of inter- and intra-reader reliability. JMRI 33:684–691

    Google Scholar 

  26. Jameson MG, Holloway LC, Vial PJ et al (2010) A review of methods of analysis in contouring studies for radiation oncology. JMIRS 54:401–410

    Google Scholar 

  27. Warfield SK, Zou KH, Wells WM (2008) Validation of image segmentation by estimating rater bias and variance. Phil Trans R Soc A 366:2361–2375

    Google Scholar 

Download references

Compliance with ethical guidelines

Conflict of interest

J. Van de Velde, T. Vercauteren, W. De Gersem, J. Wouters, K. Vandecasteele, P. Vuye, F. Vanpachtenbeke, K. D’Herde, I. Kerckaert, W. De Neve, and T. Van Hoof state that there are no conflicts of interest.

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975 (in its most recently amended version).

Author information

Authors and Affiliations

  1. Department of Anatomy, Ghent University, De Pintelaan 185, 9000, Ghent, Belgium

    Joris Van de Velde Msc, Johan Wouters Ir, Katharina D’Herde MD, PhD, Ingrid Kerckaert MD, PhD & Tom Van Hoof Msc, PhD

  2. Department of Radiotherapy, Ghent University, Ghent, Belgium

    Joris Van de Velde Msc, Tom Vercauteren Msc, Werner De Gersem Ir, PhD, Katrien Vandecasteele MD, PhD, Philippe Vuye Msc, Frank Vanpachtenbeke Msc & Wilfried De Neve MD, PhD

Authors
  1. Joris Van de Velde Msc
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tom Vercauteren Msc
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Werner De Gersem Ir, PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Johan Wouters Ir
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Katrien Vandecasteele MD, PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Philippe Vuye Msc
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Frank Vanpachtenbeke Msc
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Katharina D’Herde MD, PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Ingrid Kerckaert MD, PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Wilfried De Neve MD, PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Tom Van Hoof Msc, PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joris Van de Velde Msc.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Van de Velde, J., Vercauteren, T., De Gersem, W. et al. Reliability and accuracy assessment of radiation therapy oncology group-endorsed guidelines for brachial plexus contouring. Strahlenther Onkol 190, 628–635 (2014). https://doi.org/10.1007/s00066-014-0657-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00066-014-0657-6

Keywords

Schlüsselwörter

Search

Navigation