European Radiology

, Volume 16, Issue 4, pp 781–790 | Cite as

Inadequacy of manual measurements compared to automated CT volumetry in assessment of treatment response of pulmonary metastases using RECIST criteria

  • Katharina MartenEmail author
  • Florian Auer
  • Stefan Schmidt
  • Gerhard Kohl
  • Ernst J. Rummeny
  • Christoph Engelke


The purpose of this study was to compare relative values of manual unidimensional measurements (MD) and automated volumetry (AV) for longitudinal treatment response assessment in patients with pulmonary metastases. Fifty consecutive patients with pulmonary metastases and repeat chest multidetector-row CT (median interval=2 months) were independently assessed by two radiologists for treatment response using Response Evaluation Criteria In Solid Tumours (RECIST). Statistics included relative measurement errors (RME), intra-/interobserver correlations, limits of agreement (95% LoA), and kappa. A total of 202 metastases (median volume=182.22 mm3; range=3.16–5,195.13 mm3) were evaluated. RMEs were significantly higher for MD than for AV (intraobserver RME=2.34–3.73% and 0.15–0.22% for MD and AV respectively; P<0.05. Interobserver RME=3.53–3.76% and 0.22–0.29% for MD and AV respectively; P<0.05). Overall correlation was significantly better for AV than for MD (P<0.05). Intraobserver 95% LoAs were −1.85 to 1.75 mm for MD and −11.28 to 9.84 mm3 for AV. The interobserver 95% LoA were −1.46 to 1.92 mm for MD and −11.17 to 9.33 mm3 for AV. There was total intra-/interobserver agreement on response using AV (κ=1). MD intra- and interobserver agreements were 0.73–0.84 and 0.77–0.80 respectively. Of the 200 MD response ratings, 28 (14/50 patients) were discordant. Agreement using MD dropped significantly from total remission to progressive disease (P<0.05). We therefore conclude that AV allows for better reproducibility of response evaluation in pulmonary metastases and should be preferred to MD in these patients.


Lung Nodule Lung neoplasms Computed tomography (CT)  Multi-detector row 



We would like to thank Professor John Martin Bland from the Department of Health Sciences, University of York, UK, for invaluable advice on the statistical analysis.


  1. 1.
    Miller AB, Hogestraeten B, Staquet M, Winkler A (1981) Reporting results of cancer treatment. Cancer 47:207–214Google Scholar
  2. 2.
    Warr D, McKinney S, Tannock I (1984) Influence of measurement error on assessment of response to anticancer chemotherapy: proposal for new criteria of tumour response. J Clin Oncol 2:1040–1046Google Scholar
  3. 3.
    Therasse P, Arbuck SG, Eisenhauer EA, Wanders J, Kaplan RS et al. (2000) New guidelines to evaluate the response to treatment in solid tumors. J Natl Cancer Inst 92:205–216Google Scholar
  4. 4.
    Ratain M (2004) Phase II studies of modern drugs directed against new targets: if you are fazed, too, then resist RECIST. J Clin Oncol 22:4442–4444Google Scholar
  5. 5.
    Revel MP, Bissery A, Bienvenu M, Aycard L, Lefort C, Frija G (2004) Are two-dimensional CT measurements of small noncalcified pulmonary nodules reliable? Radiology 231:453–458Google Scholar
  6. 6.
    Kostis WJ, Yankelevitz DF, Reeves AP, Fluture SC, Henschke CI (2004) Small pulmonary nodules: reproducibility of three-dimensional volumetric measurement and estimation of time to follow-up CT. Radiology 231:446–452Google Scholar
  7. 7.
    Wormanns D, Kohl G, Klotz E et al. (2004) Volumetric measurement of pulmonary nodules at multi-row detector CT: in vivo reproducibility. Eur Radiol 14:86–92Google Scholar
  8. 8.
    Fan L, Qian J, Odry BL et al. (2002) Automatic segmentation of pulmonary nodules by using dynamic cross-correlation for interactive CAD systems. In: Sonka M, Fitzpatrick JM (eds) Medical imaging: image processing. Proc SPIE 4684:1362–1396Google Scholar
  9. 9.
    Bland M, Altman DG (1986) Statistical methods for assessing the difference between two methods of measurement. Lancet i:307–310Google Scholar
  10. 10.
    Fleiss JL, Nee CM, Landis JR (1979) Large sample variance of kappa in the case of different sets of raters. Psychol Bull 86:974–977Google Scholar
  11. 11.
    Fleiss JL (1981) Statistical methods for rates and proportions, 2nd edition. Wiley, New YorkGoogle Scholar
  12. 12.
    Altman DG (1991) Practical statistics for medical research. Chapman and Hall, LondonGoogle Scholar
  13. 13.
    Thiesse P, Ollivier L, Di Stefano-Louineau D et al. (1997) Response rate accuracy in oncology trials: reasons for interobserver variability. J Clin Oncol 15:3507–3514Google Scholar
  14. 14.
    Gwyther SJ, Aapro MS, Hatty SR, Postmus PE, Smith IE (1999) Results of an independent oncology review board of pivotal clinical trials of gemcitabine in nonsmall cell lung cancer. Anticancer Drugs 10:693–698Google Scholar
  15. 15.
    Biganzoli L, Lohrisch C, Paridaens R, Therasse P, Piccart M (2002) Analysis of two EORTC trials in metastatic breast cancer (MBC) support the recommendation of external response review (ERR) in trials with response rate (RR) as primary endpoint. Eur J Cancer 36(Suppl 5):S90Google Scholar
  16. 16.
    Marten K, Grillhösl A, Seyfarth T, Obenauer S, Rummeny EJ, Engelke C (2005) Computer-assisted detection of pulmonary nodules: evaluation of diagnostic performance using an expert knowledge-based detection system with variable reconstruction slice thickness settings. Eur Radiol 15:203–212Google Scholar
  17. 17.
    Marten K, Seyfarth T, Auer F, Wiener E, Grillhösl A, Obenauer S, Rummeny EJ, Engelke C (2004) Computer-assisted detection of pulmonary nodules: performance evaluation of an expert knowledge-based detection system in consensus reading with experienced and inexperienced chest radiologists. Eur Radiol 14:1930–1938Google Scholar
  18. 18.
    Sohaib SA, Turner B, Hanson JA, Farquharson M, Oliver RTD, Reznek RH (2000) CT assessment of tumour response to treatment: comparison of linear, cross-sectional and volumetric measures of tumour size. Brit J Radiol 73:1178–1184Google Scholar
  19. 19.
    Prasad SR, Jhaveri KS, Saini S, Hahn PF, Halpern EF, Sumner JE (2002) CT tumour measurement for therapeutic response assessment: comparison of unidimensional, bidimensional and volumetric techniques—initial observations. Radiology 225:416–419Google Scholar
  20. 20.
    Yankelevitz DF, Gupta R, Zhao B, Henschke CI (2000) Small pulmonary nodules: volumetrically determined growth rates based on CT evaluation. Radiology 217:251–256Google Scholar
  21. 21.
    Erasmus JJ, Gladish GW, Broemeling L, Sabloff BS, Truong MT, Herbst RS, Munden RF (2003) Interobserver and intraobserver variability in measurement of nonsmall-cell carcinoma of the lung lesions: implications for assessment of tumour response. J Clin Oncol 21:2574–2582Google Scholar
  22. 22.
    Schwartz M (1961) A biomathematical approach to clinical tumour growth. Cancer 14:1272–1294Google Scholar
  23. 23.
    Tran LN, Brown MS, Goldin JG et al. (2004) Comparison of treatment response classifications between unidimensional, bidimensional, and volumetric measurements of metastatic lung lesions on chest computed tomography. Acad Radiol 11:1355–1360Google Scholar
  24. 24.
    Werner-Wasik M, Xiao Y, Pequignot E, Curran WJ, Hauck W (2001) Assessment of lung cancer response after nonoperative therapy: tumour diameter, bidimensional product, and volume. A serial CT scan-based study. Int J Radiat Oncol Biol Phys 51:56–61Google Scholar

Copyright information

© Springer-Verlag 2005

Authors and Affiliations

  • Katharina Marten
    • 1
    Email author
  • Florian Auer
    • 1
  • Stefan Schmidt
    • 1
  • Gerhard Kohl
    • 2
  • Ernst J. Rummeny
    • 1
  • Christoph Engelke
    • 1
  1. 1.Department of Radiology, Klinikum rechts der IsarTechnical University MunichMunichGermany
  2. 2.Department CTS WSiemens Medical SolutionsForchheimGermany

Personalised recommendations