Inter-rater variability of visual interpretation and comparison with quantitative evaluation of 11C-PiB PET amyloid images of the Japanese Alzheimer’s Disease Neuroimaging Initiative (J-ADNI) multicenter study
- 568 Downloads
The aim of this study was to assess the inter-rater variability of the visual interpretation of 11C-PiB PET images regarding the positivity/negativity of amyloid deposition that were obtained in a multicenter clinical research project, Japanese Alzheimer’s Disease Neuroimaging Initiative (J-ADNI). The results of visual interpretation were also compared with a semi-automatic quantitative analysis using mean cortical standardized uptake value ratio to the cerebellar cortex (mcSUVR).
A total of 162 11C-PiB PET scans, including 45 mild Alzheimer’s disease, 60 mild cognitive impairment, and 57 normal cognitive control cases that had been acquired as J-ADNI baseline scans were analyzed. Based on visual interpretation by three independent raters followed by consensus read, each case was classified into positive, equivocal, and negative deposition (ternary criteria) and further dichotomized by merging the former two (binary criteria).
Complete agreement of visual interpretation by the three raters was observed for 91.3% of the cases (Cohen κ = 0.88 on average) in ternary criteria and for 92.3% (κ = 0.89) in binary criteria. Cases that were interpreted as visually positive in the consensus read showed significantly higher mcSUVR than those visually negative (2.21 ± 0.37 vs. 1.27 ± 0.09, p < 0.001), and positive or negative decision by visual interpretation was dichotomized by a cut-off value of mcSUVR = 1.5. Significant positive/negative associations were observed between mcSUVR and the number of raters who evaluated as positive (ρ = 0.87, p < 0.0001) and negative (ρ = −0.85, p < 0.0001) interpretation. Cases of disagreement among raters showed generally low mcSUVR.
Inter-rater agreement was almost perfect in 11C-PiB PET scans. Positive or negative decision by visual interpretation was dichotomized by a cut-off value of mcSUVR = 1.5. As some cases of disagreement among raters tended to show low mcSUVR, referring to quantitative method may facilitate correct diagnosis when evaluating images of low amyloid deposition.
KeywordsAmyloid imaging Inter-rater variability Multi-center study 11C-PiB PET Mean cortical SUVR
The authors thank the J-ADNI Imaging Pharmaceutical Industry Scientific Advisory Board and other organizations for their support of this work.
Compliance with ethical standards
This work is a part of the Translational Research Promotion Project/Research Project for the Development of a Systematic Method for the Assessment of Alzheimer’s Disease, sponsored by the New Energy and Industrial Technology Development Organization of Japan. The Japanese Alzheimer’s Disease Neuroimaging Initiative is also supported by a Grant-in-Aid for Comprehensive Research on Dementia from the Japanese Ministry of Health, Labour and Welfare, as well as by the grants from J-ADNI Pharmaceutical Industry Scientific Advisory Board companies.
Conflict of interest
TN has been an employee of Eli Lilly Japan since February 2016. However, all of his effort in the present study had been done when he was a member of Institute of Biomedical Research and Innovation. K. Ito has received research grants from Nihon Medi-Physics Co., Ltd. and a speaker honorarium from Eli Lilly Japan. MS is a principal investigator of clinical trials sponsored by AVID/Eli Lilly and GE Healthcare. MS is also an advisor for AVID/Eli Lilly, GE Healthcare and Piramal Imaging for which fee was paid to the institution.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the each institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.
Written informed consent was obtained from every subject prior to participation and also from caregiver of the subject categorized as AD or MCI.
- 1.McKhann GM, Knopman DS, Chertkow H, Hyman BT, Jack Jr CR, Kawas CH, et al. The diagnosis of dementia due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7:263–9.CrossRefPubMedPubMedCentralGoogle Scholar
- 18.Yamane T, Ikari Y, Nishio T, Ishii K, Ishii K, Kato T, et al. Visual-statistical interpretation of (18)F-FDG-PET images for characteristic Alzheimer patterns in a multicenter study: inter-rater concordance and relationship to automated quantitative evaluation. AJNR Am J Neuroradiol. 2014;35:244–9.CrossRefPubMedGoogle Scholar
- 24.Japanese Alzheimer’s disease neuroimaging initiative (J-ADNI) (NBDC Research ID: hum0043.v1). National Bioscience Database Center website. http://humandbs.biosciencedbc.jp/en/hum0043-v1.