Development of standardized image interpretation for 68Ga-PSMA PET/CT to detect prostate cancer recurrent lesions

  • Stefano Fanti
  • Silvia Minozzi
  • Joshua James MorigiEmail author
  • Frederik Giesel
  • Francesco Ceci
  • Christian Uprimny
  • Michael S. Hofman
  • Matthias Eiber
  • Sarah Schwarzenbock
  • Paolo Castellucci
  • Cristina Bellisario
  • Stéphane Chauvie
  • Fabrizio Bergesio
  • Louise Emmett
  • Uwe Haberkorn
  • Irene Virgolini
  • Markus Schwaiger
  • Rodney J. Hicks
  • Bernd J. Krause
  • Arturo Chiti
Original Article



After primary treatment, biochemical relapse (BCR) occurs in a substantial number of patients with prostate cancer (PCa). PET/CT imaging with prostate-specific membrane antigen based tracers (68Ga-PSMA) has shown promising results for BCR patients. However, a standardized image interpretation methodology has yet to be properly agreed. The aim of this study, which was promoted and funded by European Association of Nuclear Medicine (EANM), is to define standardized image interpretation criteria for 68Ga-PSMA PET/CT to detect recurrent PCa lesions in patients treated with primary curative intent therapy (radical prostatectomy or radiotherapy) who presented a biochemical recurrence. In the first phase inter-rater agreement between seven readers from seven international centers was calculated on the reading of 68Ga-PSMA PET/CT images of 49 patients with BCR. Each reader evaluated findings in five different sites of recurrence (local, loco-regional lymph nodes, distant lymph nodes, bone, and other). In the second phase the re-analysis was limited to cases with poor, slight, fair, or moderate agreement [Krippendorff’s (K) alpha<0.61]. Finally, on the basis of the consensus readings, we sought to define a list of revised consensus criteria for 68Ga-PSMA PET/CT interpretation.


Between-reader agreement for the presence of anomalous findings in any of the five sites was only moderate (K’s alpha: 0.47). The agreement improved and became substantial when readers had to judge whether the anomalous findings were suggestive for a pathologic, uncertain, or non-pathologic image (K’s alpha: 0.64). K’s alpha calculations for each of the five sites of recurrence were also performed and evaluated. First Delphi round was thus conducted. A more detailed definition of the criteria was proposed by the project coordinator, which was then discussed and finally agreed by the seven readers. After the second Delphi round only four cases of disagreement still remained. These were evaluated for a final round, allowing a final agreement table to be written.


We hope that by developing these consensus guidelines on the interpretation of 68Ga-PSMA PET/CT, clinicians reporting these studies will be able to provide more consistent clinical reports and that within clinical trials, abnormality classifications will be harmonized, allowing more robust assessment of its diagnostic performance.


PSMA Pet/Ct Prostate cancer Biochemical recurrence Consensus guidelines Criteria 



The authors would like to acknowledge the EANM team for its support and help throughout the study design and preliminary work fundamental to the outcome of the study, Sonja Niederkofler and Henrik Silber for their logistical and administrative support.

Also, for his technical support, the authors would like to acknowledge Jason Callahan from Peter MacCallum cancer centre (Melbourne, AUS).

Compliance with ethical standards


The project was promoted and funded by European Association of Nuclear Medicine (EANM).

Conflict of interest

Author SF is Advisory board of BED, Bayer, ANMI and received travel support from Bayer, GE Healthcare, Sanofi.

Author ME received a Research grant from Siemens Medical Solutions.

Author SC is co-founder of University of Torino spin-off Dixit srl, that distributes Widen system used in this investigation.

Author MS received a research grant from Siemens Medical Solution.

Author BJK received Research Grants from Bayer, Schering/Pharma, TauRx, Therapeutics, Piramal, AMGEN and Travel Grants & honoraria from Jannsen-Cilag, Astellas.

All other authors have no conflict of interest to declare.

Ethical approval and informed consent

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.

Supplementary material

259_2017_3725_MOESM1_ESM.docx (76 kb)
ESM 1 (DOCX 75 kb)


  1. 1.
    Siegel RL, Miller KD, Jemal A. Cancer statistics, 2017. CA Cancer J Clin. 2017;67(1):7–30. doi: 10.3322/caac.21387.CrossRefPubMedGoogle Scholar
  2. 2.
    Mohler JL, Kantoff PW, Armstrong AJ, Bahnson RR, Cohen M, D’Amico AV, et al. National Comprehensive Cancer Network. Prostate cancer, version 2.2014. J Natl Compr Cancer Netw. 2014;12(5):686–718.CrossRefGoogle Scholar
  3. 3.
    Pfister D, Bolla M, Briganti A, Carroll P, Cozzarini C, Joniau S, et al. Early salvage radiotherapy following radical prostatectomy. Eur Urol. 2014;65(6):1034–43.CrossRefPubMedGoogle Scholar
  4. 4.
    Stephenson AJ, Scardino PT, Kattan MW, Pisansky TM, Slawin KM, Klein EA et al. Predicting the outcome of salvage radiation therapy for recurrent prostate cancer after radical prostatectomy. J Clin Oncol. 2007;25(15):2035–2041. Erratum in: J Clin Oncol. 2007 Sep 10;25(26):4153.Google Scholar
  5. 5.
    Stephenson AJ, Bolla M, Briganti A, Cozzarini C, Moul JW, Roach M 3rd, et al. Postoperative radiation therapy for pathologically advanced prostate cancer after radical prostatectomy. Eur Urol. 2012;61(3):443–51.CrossRefPubMedGoogle Scholar
  6. 6.
    Martino P, Scattoni V, Galosi AB, Consonni P, Trombetta C, Palazzo S, et al. Role of imaging and biopsy to assess local recurrence after definitive treatment for prostate carcinoma (surgery, radiotherapy, cryotherapy, HIFU). World J Urol. 2011;29(5):595–605.CrossRefPubMedGoogle Scholar
  7. 7.
    Park SY, Oh YT, Jung DC, Cho NH, Choi YD, Rha KH, et al. Prediction of biochemical recurrence after radical prostatectomy with PI-RADS version 2 in prostate cancers: initial results. Eur Radiol. 2016;26(8):2502–9. doi: 10.1007/s00330-015-4077-5.CrossRefPubMedGoogle Scholar
  8. 8.
    Park JJ, Kim CK, Park SY, Park BK, Lee HM, Cho SW. Prostate cancer: role of pretreatment multiparametric 3-T MRI in predicting biochemical recurrence after radical prostatectomy. AJR Am J Roentgenol. 2014;202(5):W459–65. doi: 10.2214/AJR.13.11381.CrossRefPubMedGoogle Scholar
  9. 9.
    Harvey H, de Souza N. The role of imaging in the diagnosis of primary prostate cancer. Journal of Clinical Urology. 2016;9(2_suppl):11–7.CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Choueiri TK, Dreicer R, Paciorek A, Carroll PR, Konety B. A model that predicts the probability of positive imaging in prostate cancer cases with biochemical failure after initial definitive local therapy. J Urol. 2008;179(3):906–10.CrossRefPubMedGoogle Scholar
  11. 11.
    Amzalag G, Rager O, Tabouret-Viaud C, Wissmeyer M, Sfakianaki E, de Perrot T, et al. Target definition in salvage radiotherapy for recurrent prostate cancer: the role of advanced molecular imaging. Front Oncol. 2016;6:73.CrossRefPubMedPubMedCentralGoogle Scholar
  12. 12.
    Fanti S, Minozzi S, Castellucci P, Balduzzi S, Herrmann K, Krause BJ, et al. PET/CT with (11)C-choline for evaluation of prostate cancer patients with biochemical recurrence: meta-analysis and critical review of available data. Eur J Nucl Med Mol Imaging. 2016;43(1):55–69.CrossRefPubMedGoogle Scholar
  13. 13.
    Graziani T, Ceci F, Castellucci P, Polverari G, Lima GM, Lodi F, et al. (11)C-choline PET/CT for restaging prostate cancer. Results from 4,426 scans in a single-centre patient series. Eur J Nucl Med Mol Imaging. 2016;43(11):1971–9. doi: 10.1007/s00259-016-3428-z.CrossRefPubMedGoogle Scholar
  14. 14.
    Afshar-Oromieh A, Avtzi E, Giesel FL, Holland-Letz T, Linhart HG, Eder M, et al. The diagnostic value of PET/CT imaging with the (68)Ga-labelled PSMA ligand HBED-CC in the diagnosis of recurrent prostate cancer. Eur J Nucl Med Mol Imaging. 2015;42(2):197–209.CrossRefPubMedGoogle Scholar
  15. 15.
    Maurer T, Gschwend JE, Rauscher I, Souvatzoglou M, Haller B, Weirich G, et al. Diagnostic efficacy of (68)gallium-PSMA positron emission tomography compared to conventional imaging for lymph node staging of 130 consecutive patients with intermediate to high risk prostate cancer. J Urol. 2016;195(5):1436–43.CrossRefPubMedGoogle Scholar
  16. 16.
    Afshar-Oromieh A, Zechmann CM, Malcher A, Eder M, Eisenhut M, Linhart HG, et al. Comparison of PET imaging with a (68)Ga-labelled PSMA ligand and (18)F-choline-based PET/CT for the diagnosis of recurrent prostate cancer. Eur J Nucl Med Mol Imaging. 2014;41(1):11–20.CrossRefPubMedGoogle Scholar
  17. 17.
    Morigi JJ, Stricker PD, van Leeuwen PJ, Tang R, Ho B, Nguyen Q et al. Prospective comparison of 18F–fluoromethylcholine versus 68 Ga-PSMA PET/CT in prostate cancer patients who have rising PSA after curative treatment and are being considered for targeted therapy. J Nucl Med 56(8):1185–1190.Google Scholar
  18. 18.
    Pfob CH, Ziegler S, Graner FP, Köhner M, Schachoff S, Blechert B, et al. Biodistribution and radiation dosimetry of (68)Ga-PSMA HBED CC-a PSMA specific probe for PET imaging of prostate cancer. Eur J Nucl Med Mol Imaging. 2016;43(11):1962–70. doi: 10.1007/s00259-016-3424-3.CrossRefPubMedGoogle Scholar
  19. 19.
    Fendler WP, Eiber M, Beheshti M, Bomanji J, Ceci F, Cho S, et al. 68Ga-PSMA PET/CT: joint EANM and SNMMI procedure guideline for prostate cancer imaging: version 1.0. Eur J Nucl Med Mol Imaging. 2017; doi: 10.1007/s00259-017-3670-z.
  20. 20.
    Rauscher I, Maurer T, Fendler WP, Sommer WH, Schwaiger M, Eiber M. 68 Ga-PSMA ligand PET/CT in patients with prostate cancer: how we review and report. Cancer Imaging. 2016;16(1):14.CrossRefPubMedPubMedCentralGoogle Scholar
  21. 21.
    Jacene HA, Leboulleux S, Baba S, Chatzifotiadis D, Goudarzi B, Teytelbaum O, et al. Assessment of interobserver reproducibility in quantitative 18F-FDG PET and CT measurements of tumor response to therapy. J Nucl Med. 2009;50(11):1760–9. doi: 10.2967/jnumed.109.063321. CrossRefPubMedGoogle Scholar
  22. 22.
    Nanni C, Zanoni L, Pultrone C, Schiavina R, Brunocilla E, Lodi F, et al. (18)F-FACBC (anti1-amino-3-(18)F-fluorocyclobutane-1-carboxylic acid) versus (11)C-choline PET/CT in prostate cancer relapse: results of a prospective trial. Eur J Nucl Med Mol Imaging. 2016;43(9):1601–10. doi: 10.1007/s00259-016-3329-1.CrossRefPubMedGoogle Scholar
  23. 23.
    Urbaniak, GC, Plous, S. (2013). Research Randomizer (Version 4.0) [Computer software]. Retrieved on June 22, 2013, from
  24. 24.
    Hayes AF, Krippendorff K. Answering the call for a standard reliability measure for coding data. Commun Methods Meas. 2007;1:77–89.CrossRefGoogle Scholar
  25. 25.
    Krippendorff K. Estimating the reliability, systematic error, and random error of interval data. Educ Psychol Meas. 1970;30:61–70.CrossRefGoogle Scholar
  26. 26.
    Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:59–174.Google Scholar
  27. 27.
    Krippendorff K. Content analysis: an introduction to its methodology. Beverly Hills, CA: Sage Publications; 1980.Google Scholar
  28. 28.
    Dalkey NC. The Delphi method: an experimental study of group opinion. Rand Corp public RM-58888-PR. Santa Monica: Rand Corp; 1969.Google Scholar
  29. 29.
    Bennett C, Vakil N, Bergman J, Harrison R, Odze R, Vieth M, et al. Consensus statements for management of Barrett’s dysplasia and early-stage esophageal adenocarcinoma, based on a Delphi process. Gastroenterology. 2012;143(2):336–46.CrossRefPubMedGoogle Scholar
  30. 30.
    Meshkat B, Cowman S, Gethin G, Ryan K, Wiley M, Brick A, et al. Using an e-Delphi technique in achieving consensus across disciplines for developing best practice in day surgery in Ireland. J Hosp Adm. 2014;3(4):1–8.Google Scholar
  31. 31.
    Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CF, Askham J, et al. Consensus development methods, and their use in clinical guideline development. Health Technol Assess. 1998;2(3):i–88.Google Scholar
  32. 32.
    Powell C. The Delphi technique: myths and realities. J Adv Nurs. 2003;41(4):376–82.CrossRefPubMedGoogle Scholar
  33. 33.
    Vakil N, van Zanten SV, Kahrilas P, Dent J, Jones R. The Montreal definition and classification of gastroesophageal reflux disease: a global evidence based consensus. Am J Gastroenterol. 2006;101(8):1900–20.CrossRefPubMedGoogle Scholar
  34. 34.
    Wood L, Black P, Heng D, Kollmannsberger C, Moore R, Soulieres D, et al. Using the Dephi technique to improve clinical outcomes through the development of quality indicators in renal cell carcinoma. J Oncology Practice. 2013;9(5):262–7.CrossRefGoogle Scholar
  35. 35.
    Hsu CC & Sandford BA. The Delphi Technique: Making Sense Of Consensus. Practical Assessment, Research & Evaluation, 2007; 12 (10).Google Scholar
  36. 36.
    Mottet N, Bellmunt J, Bolla M, Briers E, Cumberbatch MG, De Santis M et al. European Association of Urology. EAU-ESTRO-SIOG guidelines on prostate cancer. 2016
  37. 37.
    Krohn T, Verburg FA, Pufe T, Neuhuber W, Vogg A, Heinzel A, et al. [(68)Ga]PSMA-HBED uptake mimicking lymph node metastasis in coeliac ganglia: an important pitfall in clinical practice. Eur J Nucl Med Mol Imaging. 2015;42:210–4.CrossRefPubMedGoogle Scholar
  38. 38.
    Boellaard R, Delgado-Bolton R, Oyen WJ, Giammarile F, Tatsch K, Eschner W, et al. European Association of Nuclear Medicine (EANM). FDG PET/CT: EANM procedure guidelines for tumour imaging: version 2.0. Eur J Nucl Med Mol Imaging. 2015;42(2):328–54. doi: 10.1007/s00259-014-2961-x.CrossRefPubMedGoogle Scholar
  39. 39.
    Hicks RJ. Invited commentary: the customer is always right, even when you are justifiably wrong. The Journal of Nuclear Medicine. 2014;55:1923–4.CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  • Stefano Fanti
    • 1
  • Silvia Minozzi
    • 2
  • Joshua James Morigi
    • 3
    Email author
  • Frederik Giesel
    • 4
  • Francesco Ceci
    • 1
  • Christian Uprimny
    • 5
  • Michael S. Hofman
    • 6
  • Matthias Eiber
    • 7
  • Sarah Schwarzenbock
    • 8
  • Paolo Castellucci
    • 1
  • Cristina Bellisario
    • 9
  • Stéphane Chauvie
    • 10
  • Fabrizio Bergesio
    • 10
  • Louise Emmett
    • 3
  • Uwe Haberkorn
    • 4
  • Irene Virgolini
    • 5
  • Markus Schwaiger
    • 7
  • Rodney J. Hicks
    • 6
  • Bernd J. Krause
    • 8
  • Arturo Chiti
    • 11
  1. 1.Nuclear Medicine UnitUniversity of Bologna, S. Orsola Hospital BolognaBolognaItaly
  2. 2.Department of EpidemiologyLazio Regional Health ServiceRomeItaly
  3. 3.Department of Diagnostic ImagingSt. Vincent’s Public HospitalSydneyAustralia
  4. 4.Department of Nuclear MedicineUniversity Hospital HeidelbergHeidelbergGermany
  5. 5.Department of Nuclear MedicineMedical University InnsbruckInnsbruckAustria
  6. 6.Centre for Molecular Imaging, Department of Cancer ImagingPeter MacCallum Cancer CentreMelbourneAustralia
  7. 7.Department of Nuclear MedicineTechnical University MunichMunichGermany
  8. 8.Department of Nuclear MedicineUniversity Medical CentreRostockGermany
  9. 9.Department of Cancer Screening, Centre for Epidemiology and Prevention in Oncology (CPO)University Hospital “Città della Salute e della Scienza di Torino”TurinItaly
  10. 10.Medical Physics DivisionSanta Croce e Carle HospitalCuneoItaly
  11. 11.Nuclear Medicine, Humanitas Cancer CenterHumanitas Clinical and Research HospitalRozzanoItaly

Personalised recommendations