Observer variation in FDG PET-CT for staging of non-small-cell lung carcinoma

  • Michael S. Hofman
  • Nigel C. Smeeton
  • Sheila C. Rankin
  • Tom Nunan
  • Michael J. O’Doherty
Original Article



Error and variation in reporting remains one of the weakest features of clinical imaging despite enormous technological advances in nuclear medicine and radiology. The aim of this study was to evaluate agreement amongst experienced readers in staging non-small-cell lung cancer (NSCLC) with PET-CT.


A series of 18F-FDG PET-CT scans from 100 consecutive patients were reviewed independently by three experienced readers, with two readers reviewing each scan series a second time. Individual mediastinal lymph node stations were assessed as benign/inflammatory, equivocal or malignant, and AJCC N and M stage were also assigned. Kappa (κ) was used to compare ratings from two categories and weighted kappa (κw) for three or more categories, and kappa values were interpreted according to the Landis-Koch benchmarks.


Both intra- and interobserver agreement for N and M staging were high. For M staging there was almost perfect intra- and interobserver agreement (κ = 0.90–0.93). For N staging, agreement was either almost perfect or substantial (intraobserver κw = 0.79, 0.91; interobserver κw = 0.75–0.81). Importantly, there was almost perfect agreement for N0/1 vs N2/3 disease (κ = 0.80–0.97). Agreement for inferior and superior mediastinal nodes (stations 1, 2, 3, 7, 8, 9) was either almost perfect or substantial (κw = 0.71–0.88), but lower for hilar nodes (10; κw = 0.56–0.71). Interreporter variability was greatest for aortopulmonary nodes (5, 6; κw = 0.48–0.55).


Amongst experienced reporters in a single centre, there was a very high level of agreement for both mediastinal nodal stage and detection of distant metastases with PET-CT. This supports the use of PET-CT as a robust imaging modality for staging NSCLC.


Fluorodeoxyglucose FDG Positron emission tomography Intrareporter Interreporter variability Reporter agreement 


  1. 1.
    Robinson PJ. Radiology’s Achilles’ heel: error and variation in the interpretation of the Röntgen image. Br J Radiol 1997;70:1085–98.PubMedGoogle Scholar
  2. 2.
    Shim SS, Lee KS, Kim BT, Chung MJ, Lee EJ, Han J, et al. Non-small cell lung cancer: prospective comparison of integrated FDG PET/CT and CT alone for preoperative staging. Radiology 2005;236:1011–9.PubMedCrossRefGoogle Scholar
  3. 3.
    Lardinois D, Weder W, Hany TF, Kamel EM, Korom S, Seifert B, et al. Staging of non-small-cell lung cancer with integrated positron-emission tomography and computed tomography. N Engl J Med 2003;348:2500–7.PubMedCrossRefGoogle Scholar
  4. 4.
    van Tinteren H, Hoekstra OS, Smit EF, van den Bergh JH, Schreurs AJ, Stallaert RA, et al. Effectiveness of positron emission tomography in the preoperative assessment of patients with suspected non-small-cell lung cancer: the PLUS multicentre randomised trial. Lancet 2002;359:1388–93.PubMedCrossRefGoogle Scholar
  5. 5.
    Mountain CF, Dresler CM. Regional lymph node classification for lung cancer staging. Chest 1997;111:1718–23.PubMedCrossRefGoogle Scholar
  6. 6.
    Greene FL; American Joint Committee on Cancer. AJCC cancer staging manual. 6th ed. American Cancer Society. New York: Springer; 2002. xiv; p. 421.Google Scholar
  7. 7.
    Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas 1960;20:37–46.CrossRefGoogle Scholar
  8. 8.
    Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 1968;70:213–20.CrossRefGoogle Scholar
  9. 9.
    Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74.PubMedCrossRefGoogle Scholar
  10. 10.
    Efron B, Tibshirani R. An introduction to the bootstrap. Monographs on statistics and applied probability; 57. New York: Chapman & Hall; 1993. p. 436. xvi.Google Scholar
  11. 11.
    Stata Corporation. Stata statistical software: release 9.2. College Station: Stata Corporation; 2007.Google Scholar
  12. 12.
    Smulders SA, Gundy CM, van Lingen A, Comans EF, Smeenk FW, Hoekstra OS. Observer variation of 2-deoxy-2-[F-18]fluoro-D-glucose-positron emission tomography in mediastinal staging of non-small cell lung cancer as a function of experience, and its potential clinical impact. Mol Imaging Biol 2007;9:318–22.PubMedCrossRefGoogle Scholar
  13. 13.
    Marom EM, Munden RF, Truong MT, Gladish GW, Podoloff DA, Mawlawi O, et al. Interobserver and intraobserver variability of standardized uptake value measurements in non-small-cell lung cancer. J Thorac Imaging 2006;21:205–12.PubMedCrossRefGoogle Scholar
  14. 14.
    Guyatt GH, Lefcoe M, Walter S, Cook D, Troyan S, Griffith L, et al. Interobserver variation in the computed tomographic evaluation of mediastinal lymph node size in patients with potentially resectable lung cancer. Canadian Lung Oncology Group. Chest 1995;107:116–9.PubMedCrossRefGoogle Scholar
  15. 15.
    Cascade PN, Gross BH, Kazerooni EA, Quint LE, Francis IR, Strawderman M, et al. Variability in the detection of enlarged mediastinal lymph nodes in staging lung cancer: a comparison of contrast-enhanced and unenhanced CT. AJR Am J Roentgenol 1998;170:927–31.PubMedGoogle Scholar
  16. 16.
    Ciatto S, Ambrogetti D, Bonardi R, Catarzi S, Risso G, Rosseli Del Turco M, et al. Second reading of screening mammograms increases cancer detection and recall rates. Results in the Florence screening programme. J Med Screen 2005;12:103–6.PubMedCrossRefGoogle Scholar
  17. 17.
    Okada M, Tsubota N, Yoshimura M, Miyamoto Y, Matsuoka H. Prognosis of completely resected pN2 non-small cell lung carcinomas: what is the significant node that affects survival? J Thorac Cardiovasc Surg 1999;118:270–5.PubMedCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2008

Authors and Affiliations

  • Michael S. Hofman
    • 1
    • 3
  • Nigel C. Smeeton
    • 2
  • Sheila C. Rankin
    • 1
  • Tom Nunan
    • 1
  • Michael J. O’Doherty
    • 1
  1. 1.PET Imaging CentreSt Thomas’ HospitalLondonUK
  2. 2.Division of Health and Social Care ResearchKing’s College LondonLondonUK
  3. 3.Nuclear MedicineSouthern HealthMelbourneAustralia

Personalised recommendations