European Radiology

, Volume 28, Issue 5, pp 2124–2133 | Cite as

Pulmonary subsolid nodules: value of semi-automatic measurement in diagnostic accuracy, diagnostic reproducibility and nodule classification agreement

  • Hyungjin Kim
  • Chang Min ParkEmail author
  • Eui Jin Hwang
  • Su Yeon Ahn
  • Jin Mo Goo



We hypothesized that semi-automatic diameter measurements would improve the accuracy and reproducibility in discriminating preinvasive lesions and minimally invasive adenocarcinomas from invasive pulmonary adenocarcinomas appearing as subsolid nodules (SSNs) and increase the reproducibility in classifying SSNs.


Two readers independently performed semi-automatic and manual measurements of the diameters of 102 SSNs and their solid portions. Diagnostic performance in predicting invasive adenocarcinoma based on diameters was tested using logistic regression analysis with subsequent receiver operating characteristic curves. Inter- and intrareader reproducibilities of diagnosis and SSN classification according to Fleischner’s guidelines were investigated for each measurement method using Cohen’s κ statistics.


Semi-automatic effective diameter measurements were superior to manual average diameters for the diagnosis of invasive adenocarcinoma (AUC, 0.905–0.923 for semi-automatic measurement and 0.833–0.864 for manual measurement; p<0.05). Reproducibility of diagnosis between the readers also improved with semi-automatic measurement (κ=0.924 for semi-automatic measurement and 0.690 for manual measurement, p=0.012). Inter-reader SSN classification reproducibility was significantly higher with semi-automatic measurement (κ=0.861 for semi-automatic measurement and 0.683 for manual measurement, p=0.022).


Semi-automatic effective diameter measurement offers an opportunity to improve diagnostic accuracy and reproducibility as well as the classification reproducibility of SSNs.

Key Points

Semi-automatic effective diameter measurement improves the diagnostic accuracy for pulmonary subsolid nodules.

Semi-automatic measurement increases the inter-reader agreement on the diagnosis for subsolid nodules.

Semi-automatic measurement augments the inter-reader reproducibility for the classification of subsolid nodules.


Carcinoma, non-small-cell lung Multidetector computed tomography Diagnosis, computer-assisted Dimensional measurement accuracy Observer variation 



Adenocarcinoma in-situ


Area under the curve


Confidence interval


Volume CT dose index


Dose-length product


Diameter of solid portion


Diameter of subsolid nodule


Hounsfield unit


Invasive pulmonary adenocarcinoma


Lung CT Screening Reporting and Data System


Minimally invasive adenocarcinomas


Solid proportion within a nodule


Percentage relative difference


Receiver operating characteristic curve


Size-specific dose estimate


Subsolid nodule



This study has received funding by the SNUH Research Fund, Seoul National University Hospital, Seoul, Korea (grant number: 05-2016-0050).

Compliance with ethical standards


The scientific guarantor of this publication is Chang Min Park.

Conflict of interest

The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board approval was obtained.

Study subjects or cohorts overlap

Part of the study population (36/89) had participated in a previous published study [13].


• retrospective

• diagnostic study

• performed at one institution

Supplementary material

330_2017_5171_MOESM1_ESM.docx (15 kb)
ESM 1 (DOCX 14 kb)


  1. 1.
    Austin JH, Garg K, Aberle D et al (2013) Radiologic implications of the 2011 classification of adenocarcinoma of the lung. Radiology 266:62–71CrossRefPubMedGoogle Scholar
  2. 2.
    Naidich DP, Bankier AA, MacMahon H et al (2013) Recommendations for the management of subsolid pulmonary nodules detected at CT: a statement from the Fleischner Society. Radiology 266:304–317CrossRefPubMedGoogle Scholar
  3. 3.
    Lee SM, Park CM, Goo JM, Lee HJ, Wi JY, Kang CH (2013) Invasive pulmonary adenocarcinomas versus preinvasive lesions appearing as ground-glass nodules: differentiation by using CT features. Radiology 268:265–273CrossRefPubMedGoogle Scholar
  4. 4.
    Kim H, Park CM, Koh JM, Lee SM, Goo JM (2014) Pulmonary subsolid nodules: what radiologists need to know about the imaging features and management strategy. Diagn Interv Radiol 20:47–57CrossRefPubMedGoogle Scholar
  5. 5.
    Chae HD, Park CM, Park SJ, Lee SM, Kim KG, Goo JM (2014) Computerized texture analysis of persistent part-solid ground-glass nodules: differentiation of preinvasive lesions from invasive pulmonary adenocarcinomas. Radiology 273:285–293CrossRefPubMedGoogle Scholar
  6. 6.
    Song YS, Park CM, Park SJ, Lee SM, Jeon YK, Goo JM (2014) Volume and mass doubling times of persistent pulmonary subsolid nodules detected in patients without known malignancy. Radiology 273:276–284CrossRefPubMedGoogle Scholar
  7. 7.
    Ko JP, Suh J, Ibidapo O et al (2016) Lung adenocarcinoma: correlation of quantitative CT findings with pathologic findings. Radiology 280:931–939CrossRefPubMedGoogle Scholar
  8. 8.
    Lee HJ, Goo JM, Lee CH et al (2009) Predictive CT findings of malignancy in ground-glass nodules on thin-section chest CT: the effects on radiologist performance. Eur Radiol 19:552–560CrossRefPubMedGoogle Scholar
  9. 9.
    Takahashi S, Tanaka N, Okimoto T et al (2012) Long term follow-up for small pure ground-glass nodules: implications of determining an optimum follow-up period and high-resolution CT findings to predict the growth of nodules. Jpn J Radiol 30:206–217CrossRefPubMedGoogle Scholar
  10. 10.
    Oda S, Awai K, Liu D et al (2008) Ground-glass opacities on thin-section helical CT: differentiation between bronchioloalveolar carcinoma and atypical adenomatous hyperplasia. AJR Am J Roentgenol 190:1363–1368CrossRefPubMedGoogle Scholar
  11. 11.
    Gavrielides MA, Kinnard LM, Myers KJ, Petrick N (2009) Noncalcified lung nodules: volumetric assessment with thoracic CT. Radiology 251:26–37CrossRefPubMedPubMedCentralGoogle Scholar
  12. 12.
    Goo JM (2011) A computer-aided diagnosis for evaluating lung nodules on chest CT: the current status and perspective. Korean J Radiol 12:145–155CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Cohen JG, Goo JM, Yoo RE et al (2016) The effect of late-phase contrast enhancement on semiautomatic software measurements of CT attenuation and volume of part-solid nodules in lung adenocarcinomas. Eur Radiol 85:1174–1180Google Scholar
  14. 14.
    American Association of Physicists in Medicine (2011) Size-specific dose estimates (SSDE) in pediatric and adult body CT examinations. Task Group. American Association of Physicists in Medicine, College Park, p 204Google Scholar
  15. 15.
    Cohen JG, Goo JM, Yoo RE et al (2016) Software performance in segmenting ground-glass and solid components of subsolid nodules in pulmonary adenocarcinomas. Eur Radiol 26:4465–4474CrossRefPubMedGoogle Scholar
  16. 16.
    Scholten ET, Jacobs C, van Ginneken B et al (2015) Detection and quantification of the solid component in pulmonary subsolid nodules by semiautomatic segmentation. Eur Radiol 25:488–496CrossRefPubMedGoogle Scholar
  17. 17.
    Kuhnigk J-M, Dicken V, Bornemann L et al (2006) Morphological segmentation and partial volume analysis for volumetry of solid pulmonary lesions in thoracic CT scans. IEEE Trans Med Imaging 25:417–434CrossRefPubMedGoogle Scholar
  18. 18.
    de Hoop B, Gietema H, van Ginneken B, Zanen P, Groenewegen G, Prokop M (2009) A comparison of six software packages for evaluation of solid lung nodules using semi-automated volumetry: what is the minimum increase in size to detect growth in repeated CT examinations. Eur Radiol 19:800–808CrossRefPubMedGoogle Scholar
  19. 19.
    Lee KH, Goo JM, Park SJ et al (2014) Correlation between the size of the solid component on thin-section CT and the invasive component on pathology in small lung adenocarcinomas manifesting as ground-glass nodules. J Thorac Oncol 9:74–82CrossRefPubMedGoogle Scholar
  20. 20.
    Travis WD, Brambilla E, Noguchi M et al (2011) International association for the study of lung cancer/American thoracic society/European respiratory society international multidisciplinary classification of lung adenocarcinoma. J Thorac Oncol 6:244–285CrossRefPubMedPubMedCentralGoogle Scholar
  21. 21.
    Bland JM, Altman DG (1999) Measuring agreement in method comparison studies. Stat Methods Med Res 8:135–160CrossRefPubMedGoogle Scholar
  22. 22.
    DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845CrossRefPubMedGoogle Scholar
  23. 23.
    Congalton RG, Green K (2008) Assessing the accuracy of remotely sensed data: Principles and practices. CRC press, Boca RatonCrossRefGoogle Scholar
  24. 24.
    de Leeuw J, Jia H, Yang L, Liu X, Schmidt K, Skidmore A (2006) Comparing accuracy assessments to infer superiority of image classification methods. Int J Remote Sens 27:223–232CrossRefGoogle Scholar
  25. 25.
    Scholten ET, de Hoop B, Jacobs C et al (2013) Semi-automatic quantification of subsolid pulmonary nodules: comparison with manual measurements. PLoS One 8:e80249CrossRefPubMedPubMedCentralGoogle Scholar
  26. 26.
    Xie X, Zhao Y, Snijder RA et al (2013) Sensitivity and accuracy of volumetry of pulmonary nodules on low-dose 16- and 64-row multi-detector CT: an anthropomorphic phantom study. Eur Radiol 23:139–147CrossRefPubMedGoogle Scholar
  27. 27.
    Zhao B, James LP, Moskowitz CS et al (2009) Evaluating variability in tumor measurements from same-day repeat CT scans of patients with non-small cell lung cancer. Radiology 252:263–272CrossRefPubMedPubMedCentralGoogle Scholar
  28. 28.
    Ridge CA, Yildirim A, Boiselle PM et al (2016) Differentiating between subsolid and solid pulmonary nodules at CT: inter- and intraobserver agreement between experienced thoracic radiologists. Radiology 278:888–896CrossRefPubMedGoogle Scholar
  29. 29.
    van Riel SJ, Sanchez CI, Bankier AA et al (2015) Observer variability for classification of pulmonary nodules on low-dose CT images and its effect on nodule management. Radiology 277:863–871CrossRefPubMedGoogle Scholar
  30. 30.
    Penn A, Ma M, Chou BB, Tseng JR, Phan P (2015) Inter-reader variability when applying the 2013 Fleischner guidelines for potential solitary subsolid lung nodules. Acta Radiol 56:1180–1186CrossRefPubMedGoogle Scholar
  31. 31.
    Yoo RE, Goo JM, Hwang EJ et al (2017) Retrospective assessment of interobserver agreement and accuracy in classifications and measurements in subsolid nodules with solid components less than 8mm: which window setting is better? Eur Radiol 27:1369–1376CrossRefPubMedGoogle Scholar
  32. 32.
    American College of Radiology (2014) Lung CT Screening Reporting and Data System (Lung-RADS). American College of Radiology. Available via Accessed 15 Oct 2016
  33. 33.
    Yankelevitz DF, Yip R, Smith JP et al (2015) CT screening for lung cancer: nonsolid nodules in baseline and annual repeat rounds. Radiology 277:555–564CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© European Society of Radiology 2017

Authors and Affiliations

  1. 1.Department of RadiologySeoul National University College of MedicineSeoulKorea
  2. 2.Institute of Radiation MedicineSeoul National University Medical Research CenterSeoulKorea
  3. 3.Seoul National University Cancer Research InstituteSeoulKorea

Personalised recommendations