Skip to main content
SpringerLink
Log in

Development of a Software for Quantitative Evaluation Radiotherapy Target and Organ-at-Risk Segmentation Comparison

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Modern radiotherapy requires accurate region of interest (ROI) inputs for plan optimization and delivery. Target delineation, however, remains operator-dependent and potentially serves as a major source of treatment delivery error. In order to optimize this critical, yet observer-driven process, a flexible web-based platform for individual and cooperative target delineation analysis and instruction was developed in order to meet the following unmet needs: (1) an open-source/open-access platform for automated/semiautomated quantitative interobserver and intraobserver ROI analysis and comparison, (2) a real-time interface for radiation oncology trainee online self-education in ROI definition, and (3) a source for pilot data to develop and validate quality metrics for institutional and cooperative group quality assurance efforts. The resultant software, Target Contour Testing/Instructional Computer Software (TaCTICS), developed using Ruby on Rails, has since been implemented and proven flexible, feasible, and useful in several distinct analytical and research applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. Report 83: prescribing, recording, and reporting photon-beam intensity-modulated radiation therapy (IMRT). J ICRU 10:1–106, 2010

    Google Scholar 

  2. Rosenthal DI, Chambers MS, Fuller CD, et al: Beam path toxicities to non-target structures during intensity-modulated radiation therapy for head and neck cancer. Int J Radiat Oncol Biol Phys 72:747–755, 2008

    Article  PubMed  Google Scholar 

  3. Garden AS, Lewin JS, Chambers MS: How to reduce radiation-related toxicity in patients with cancer of the head and neck. Curr Oncol Rep 8:140–145, 2006

    Article  PubMed  Google Scholar 

  4. Rosenthal DI, Blanco AI: Head and neck squamous cell carcinoma: optimizing the therapeutic index. Expert Rev Anticancer Ther 5:501–514, 2005

    Article  CAS  PubMed  Google Scholar 

  5. Van Kampen M, Levegrun S, Wannenmacher M: Target volume definition in radiation therapy. Br J Radiol 70(Spec No):S25–S31, 1997

    PubMed  Google Scholar 

  6. Fraass BA: The development of conformal radiation therapy. Med Phys 22:1911–1921, 1995

    Article  CAS  PubMed  Google Scholar 

  7. Lichter AS, Ten Haken RK: Three-dimensional treatment planning and conformal radiation dose delivery. Important Adv Oncol 1995:95–109, 1995

  8. Bortfeld T: IMRT: a review and preview. Phys Med Biol 51:R363–R379, 2006

    Article  PubMed  Google Scholar 

  9. Webb S: The physical basis of IMRT and inverse planning. Br J Radiol 76:678–689, 2003

    Article  CAS  PubMed  Google Scholar 

  10. Smith RP, Heron DE, Huq MS, et al: Modern radiation treatment planning and delivery—from Rontgen to real time. Hematol Oncol Clin N Am 20:45–62, 2006

    Article  Google Scholar 

  11. Gregoire V, Jeraj R, Lee JA, et al: Radiotherapy for head and neck tumours in 2012 and beyond: conformal, tailored, and adaptive? Lancet Oncol 13:e292–e300, 2012

    Article  PubMed  Google Scholar 

  12. Schwartz DL, Garden AS, Thomas J, et al: Adaptive radiotherapy for head-and-neck cancer: initial clinical outcomes from a prospective trial. Int J Radiat Oncol Biol Phys 83:986–993, 2012

    Article  PubMed  Google Scholar 

  13. van Herk M: Errors and margins in radiotherapy. Semin Radiat Oncol 14:52–64, 2004

    Article  PubMed  Google Scholar 

  14. Multi-Institutional Target Delineation in Oncology Group: Human–computer interaction in radiotherapy target volume delineation: a prospective, multi-institutional comparison of user input devices. J Digit Imaging 24:794–803, 2011

    Google Scholar 

  15. Njeh CF: Tumor delineation: the weakest link in the search for accuracy in radiotherapy. J Med Phys Assoc Med Physicists India 33:136–140, 2008

    CAS  Google Scholar 

  16. Fuller CD, Nijkamp J, Duppen JC, et al: Prospective randomized double-blind pilot study of site-specific consensus atlas implementation for rectal cancer target volume delineation in the cooperative group setting. Int J Radiat Oncol Biol Phys 79:481–489, 2011

    Article  PubMed Central  PubMed  Google Scholar 

  17. Eskander RN, Scanderbeg D, Saenz CC, et al: Comparison of computed tomography and magnetic resonance imaging in cervical cancer brachytherapy target and normal tissue contouring. Int J Gynecol Cancer: Off J Int Gynecol Cancer Soc 20:47–53, 2010

    Article  Google Scholar 

  18. Peters LJ, O’Sullivan B, Giralt J, et al: Critical impact of radiotherapy protocol compliance and quality in the treatment of advanced head and neck cancer: results from TROG 02.02. J Clin Oncol: Off J Am Soc Clin Oncol 28:2996–3001, 2010

    Article  Google Scholar 

  19. Weber DC, Poortmans PM, Hurkmans CW, et al: Quality assurance for prospective EORTC radiation oncology trials: the challenges of advanced technology in a multicenter international setting. Radiother Oncol: J Eur Soc Ther Radiol Oncol 100:150–156, 2011

    Article  Google Scholar 

  20. Okunieff P, Kachnic LA, Constine LS, et al: Report from the Radiation Therapy Committee of the Southwest Oncology Group (SWOG): Research Objectives Workshop 2008. Clin Cancer Res: Off J Am Assoc Cancer Res 15:5663–5670, 2009

    Article  Google Scholar 

  21. Bekelman JE, Wolden S, Lee N: Head-and-neck target delineation among radiation oncology residents after a teaching intervention: a prospective, blinded pilot study. Int J Radiat Oncol Biol Phys 73:416–423, 2009

    Article  PubMed  Google Scholar 

  22. Li XA, Tai A, Arthur DW, et al: Variability of target and normal structure delineation for breast cancer radiotherapy: an RTOG Multi-Institutional and Multiobserver Study. Int J Radiat Oncol Biol Phys 73:944–951, 2009

    Article  PubMed Central  PubMed  Google Scholar 

  23. Tai P, Van Dyk J, Yu E, et al: Variability of target volume delineation in cervical esophageal cancer. Int J Radiat Oncol Biol Phys 42:277–288, 1998

    Article  CAS  PubMed  Google Scholar 

  24. Goodman KA, Regine WF, Dawson LA, et al: Radiation Therapy Oncology Group consensus panel guidelines for the delineation of the clinical target volume in the postoperative treatment of pancreatic head cancer. Int J Radiat Oncol Biol Phys 83:901–908, 2012

    Article  PubMed  Google Scholar 

  25. Ng M, Leong T, Chander S, et al: Australasian Gastrointestinal Trials Group (AGITG) contouring atlas and planning guidelines for intensity-modulated radiotherapy in anal cancer. Int J Radiat Oncol Biol Phys 83:1455–1462, 2012

    Article  PubMed  Google Scholar 

  26. Nijkamp J, de Haas-Kock DF, Beukema JC, et al: Target volume delineation variation in radiotherapy for early stage rectal cancer in the Netherlands. Radiother Oncol: J Eur Soc Ther Radiol Oncol 102:14–21, 2012

    Article  Google Scholar 

  27. Vorwerk H, Hess CF: Guidelines for delineation of lymphatic clinical target volumes for high conformal radiotherapy: head and neck region. Radiat Oncol 6:97, 2011

    Article  PubMed Central  PubMed  Google Scholar 

  28. Wang D, Bosch W, Roberge D, et al: RTOG sarcoma radiation oncologists reach consensus on gross tumor volume and clinical target volume on computed tomographic images for preoperative radiotherapy of primary soft tissue sarcoma of extremity in Radiation Therapy Oncology Group studies. Int J Radiat Oncol Biol Phys 81:e525–e528, 2011

    Article  PubMed Central  PubMed  Google Scholar 

  29. Kong FM, Ritter T, Quint DJ, et al: Consideration of dose limits for organs at risk of thoracic radiotherapy: atlas for lung, proximal bronchial tree, esophagus, spinal cord, ribs, and brachial plexus. Int J Radiat Oncol Biol Phys 81:1442–1457, 2011

    Article  PubMed  Google Scholar 

  30. Vorwerk H, Beckmann G, Bremer M, et al: The delineation of target volumes for radiotherapy of lung cancer patients. Radiother Oncol: J Eur Soc Ther Radiol Oncol 91:455–460, 2009

    Article  Google Scholar 

  31. Weiss E, Richter S, Krauss T, et al: Conformal radiotherapy planning of cervix carcinoma: differences in the delineation of the clinical target volume. A comparison between gynaecologic and radiation oncologists. Radiother Oncol: J Eur Soc Ther Radiol Oncol 67:87–95, 2003

    Article  Google Scholar 

  32. Allozi R, Li XA, White J, et al: Tools for consensus analysis of experts’ contours for radiotherapy structure definitions. Radiother Oncol: J Eur Soc Ther Radiol Oncol 97:572–578, 2010

    Article  Google Scholar 

  33. Brouwer CL, Steenbakkers RJ, van den Heuvel E, et al: 3D variation in delineation of head and neck organs at risk. Radiat Oncol 7:32, 2012

    Article  PubMed Central  PubMed  Google Scholar 

  34. Rasch CR, Steenbakkers RJ, Fitton I, et al: Decreased 3D observer variation with matched CT-MRI, for target delineation in nasopharynx cancer. Radiat Oncol 5:21, 2010

    Article  PubMed Central  PubMed  Google Scholar 

  35. Steenbakkers RJ, Duppen JC, Fitton I, et al: Observer variation in target volume delineation of lung cancer related to radiation oncologist–computer interaction: a ‘Big Brother’ evaluation. Radiother: J Eur Soc Ther Radiol Oncol 77:182–190, 2005

    Article  Google Scholar 

  36. Steenbakkers RJ, Duppen JC, Fitton I, et al: Reduction of observer variation using matched CT-PET for lung cancer delineation: a three-dimensional analysis. Int J Radiat Oncol Biol Phys 64:435–448, 2006

    Article  PubMed  Google Scholar 

  37. Jansen EP, Nijkamp J, Gubanski M, et al: Interobserver variation of clinical target volume delineation in gastric cancer. Int J Radiat Oncol Biol Phys 77:1166–1170, 2010

    Article  PubMed  Google Scholar 

  38. Roelofs E, Persoon L, Qamhiyeh S, et al: Design of and technical challenges involved in a framework for multicentric radiotherapy treatment planning studies. Radiother Oncol: J Eur Soc Ther Radiol Oncol 97:567–571, 2010

    Article  Google Scholar 

  39. Insight Toolkit: National Library of Medicine. http://www.itk.org/. Accessed 29 August 2013

  40. Jenkinson M: NifTI-1 data format. http://nifti.nimh.nih.gov/nifti-1/. Accessed 29 August 2013

  41. Dice LR: Measures of the amount of ecologic association between species. Ecology 26:297–302, 1945

    Article  Google Scholar 

  42. Jaccard P: The distribution of the flora in the alpine zone. New Phytol 11:37–50, 1912

    Article  Google Scholar 

  43. Crum WR, Camara O, Hill DL: Generalized overlap measures for evaluation and validation in medical image analysis. IEEE Trans Med Imaging 25:1451–1461, 2006

    Article  PubMed  Google Scholar 

  44. Rogers DJ, Tanimoto TT: A computer program for classifying plants. Science 132:1115–1118, 1960

    Article  CAS  PubMed  Google Scholar 

  45. Mitra J, Kato Z, Marti R, et al: A spline-based non-linear diffeomorphism for multimodal prostate registration. Med Image Anal 16:1259–1279, 2012

    Google Scholar 

  46. Mitra J, Marti R, Oliver A, et al: Prostate multimodality image registration based on B-splines and quadrature local energy. Int J CARS 7:445–454, 2012

    Article  Google Scholar 

  47. Oguro S, Tuncali K, Elhawary H, et al: Image registration of pre-procedural MRI and intra-procedural CT images to aid CT-guided percutaneous cryoablation of renal tumors. Int J CARS 6:111–117, 2011

    Article  Google Scholar 

  48. Warfield SK, Zou KH, Wells WM: Simultaneous Truth and Performance Level Estimation (STAPLE): an algorithm for the validation of image segmentation. IEEE Trans Med Imaging 23:903–921, 2004

    Article  PubMed Central  PubMed  Google Scholar 

  49. Gomez D, Komaki R, Yu J, et al: Radiation therapy definitions and reporting guidelines for thymic malignancies. J Thorac Oncol: Off Publ Int Assoc Study Lung Cancer 6:S1743–S1748, 2011

    Article  Google Scholar 

  50. Thomas CR, Kalpathy-Cramer J, Senan S, et al: Development of an expert consensus target delineation atlas for thymic cancers: Initial results of an expert survey. International Thymic Malignancy Interest Group, Amsterdam, 2011

    Google Scholar 

  51. Bongers EM, Dahele MR, Slotman BJ: Proton stereotactic body radiation therapy for clinically challenging cases of centrally and superiorly located stage I non-small-cell lung cancer: in regards to Register et al. Int J Radiat Oncol Biol Phys 82:492, author reply 492, 2012

    Google Scholar 

  52. Kalpathy-Cramer J, Bedrick SD, Boccia K, et al: A pilot prospective feasibility study of organ-at-risk definition using Target Contour Testing/Instructional Computer Software (TaCTICS), a training and evaluation platform for radiotherapy target delineation. AMIA Annu Symp Proc 2011:654–663, 2011

  53. Feng M, Moran JM, Koelling T, et al: Development and validation of a heart atlas to study cardiac exposure to radiation following treatment for breast cancer. Int J Radiat Oncol Biol Phys 79:10–18, 2011

    Article  PubMed Central  PubMed  Google Scholar 

  54. Malone S, Croke J, Roustan-Delatour N, et al: Postoperative radiotherapy for prostate cancer: a comparison of four consensus guidelines and dosimetric evaluation of 3D-CRT versus tomotherapy IMRT. Int J Radiat Oncol Biol Phys 84:725–732, 2012

    Google Scholar 

  55. Lim K, Small Jr, W, Portelance L, et al: Consensus guidelines for delineation of clinical target volume for intensity-modulated pelvic radiotherapy for the definitive treatment of cervix cancer. Int J Radiat Oncol Biol Phys 79:348–355, 2011

    Article  PubMed  Google Scholar 

  56. Matzinger O, Poortmans P, Giraud JY, et al: Quality assurance in the 22991 EORTC ROG trial in localized prostate cancer: dummy run and individual case review. Radiother Oncol: J Eur Soc Ther Radiol Oncol 90:285–290, 2009

    Article  Google Scholar 

  57. Yamazaki H, Nishiyama K, Tanaka E, et al: Dummy run for a phase II multi-institute trial of chemoradiotherapy for unresectable pancreatic cancer: inter-observer variance in contour delineation. Anticancer Res 27:2965–2971, 2007

    PubMed  Google Scholar 

  58. Clark CH, Miles EA, Urbano MT, et al: Pre-trial quality assurance processes for an intensity-modulated radiation therapy (IMRT) trial: PARSPORT, a UK multicentre phase III trial comparing conventional radiotherapy and parotid-sparing IMRT for locally advanced head and neck cancer. Br J Radiol 82:585–594, 2009

    Article  CAS  PubMed  Google Scholar 

  59. Pawlicki T, Yoo S, Court LE, et al: Process control analysis of IMRT QA: implications for clinical trials. Phys Med Biol 53:5193–5205, 2008

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

CDF received support by a training grant from the National Institutes of Health/National Institute of Biomedical Imaging and Bioengineering, “Multidisciplinary Training Program in Human Imaging” (T32EB000817), the National Institutes of Health Clinician Scientist Loan Repayment Program (L30CA136381) the European Society for Therapeutic Radiology and Oncology Technology Transfer Grant, and the Society of Imaging Informatics in Medicine (SIIM) Product Development Grant. JKC is supported in part by a National Institutes of Health/National Library of Medicine Pathway to Independence Award (K99/R00LM009889) and NCI grant 5U01CA154601. These funders played no role in the study design, collection, analysis, and interpretation of data, manuscript writing, or decision to submit the report for publication. Portions of this data were selected for a presentation at the SIIM 2010 Annual Meeting, 3–6 June, Minneapolis, MN, USA.

Author information

Authors and Affiliations

  1. Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Harvard Medical School, Department of Radiology and Neuroscience, Massachusetts General Hospital, Charlestown, MA, USA

    Jayashree Kalpathy-Cramer

  2. Department of Medical Informatics & Clinical Epidemiology, Oregon Health & Science University, Portland, OR, USA

    Steven Bedrick

  3. Head and Neck Section, Division of Radiation Oncology, Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA

    Musaddiq Awan, David I. Rosenthal & Clifton D. Fuller

  4. Department of Radiotherapy, Academic Medical Center, Amsterdam, Netherlands

    Coen R. N. Rasch

Authors
  1. Jayashree Kalpathy-Cramer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Musaddiq Awan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Steven Bedrick
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Coen R. N. Rasch
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. David I. Rosenthal
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Clifton D. Fuller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Clifton D. Fuller.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kalpathy-Cramer, J., Awan, M., Bedrick, S. et al. Development of a Software for Quantitative Evaluation Radiotherapy Target and Organ-at-Risk Segmentation Comparison. J Digit Imaging 27, 108–119 (2014). https://doi.org/10.1007/s10278-013-9633-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-013-9633-4

Keywords

Search

Navigation