Background

The use of 68Ga-radiolabelled somatostatin receptor (SSTR) agonists, such as 68Ga-DOTATATE and 68Ga-DOTATOC, for the positron emission tomography/computed tomography (PET/CT) imaging of neuroendocrine tumours (NETs) is well-established in clinical practice, not only for the localisation and staging of NETs, but also as a theranostic tool or biomarker for assessing the potential response to peptide receptor radionuclide therapy [1]. In the evaluation of new imaging techniques and novel tracers, image quality is usually assessed using a 5-point Likert scale: the exact methodology of which is unique to each clinical trial, but standardly “1” is poor quality and “5” is high quality [2,3,4,5,6,7,8,9]. Despite its widespread use, the Likert scale in biomedical imaging assessment is limited by inter-reader variability, fixed upper and lower limits, and a lack of mathematical and statistical validity [10,11,12].

To the authors’ knowledge, the binary evaluation of image quality in nuclear medicine (wherein an individual reader sees two images simultaneously and designates them as having either optimal or non-optimal image quality) has not been previously described, although binary reading of PET images has been used in the evaluation of the presence or absence of amyloid in Alzheimer's disease [13, 14]. Unlike the Likert assessment, an important advantage of binary visual reading is its simplicity and convenience, as it removes the need for images to be reviewed at different times, particularly in studies with a small number of images. These features suggest that binary contemporaneous visual reading might be used to optimise clinical development of PET/CT imaging agents.

The use of radiolabelled SSTR antagonists rather than agonists has the potential to improve the PET/CT imaging of NETs, because SSTR antagonists bind to significantly more receptor sites than SSTR agonists [15]. 68Ga-satoreotide trizoxetan (also known as 68Ga-IPN01070, 68Ga-NODAGA-JR11, or 68Ga-OPS202) is a novel SSTR antagonist recently evaluated in a prospective phase I/II imaging study conducted in 12 patients with well-differentiated gastroenteropancreatic (GEP) NETs. The authors showed that compared with 68Ga-DOTATOC, 68Ga-satoreotide trizoxetan exhibits substantially higher tumour-to-background ratios and sensitivity for detecting liver metastases [16, 17]. Subsequently, a phase II study was conducted to confirm the optimal administered peptide mass and radioactivity of 68Ga-satoreotide trizoxetan in patients with GEP-NETs [18].

The primary results of this phase II study [18] showed that the ratio of the number of lesions detected by 68Ga-satoreotide trizoxetan imaging to the number of lesions detected by contrast-enhanced CT was overall consistent across different peptide mass and radioactivity range combinations (5–20 or 30–45 µg with one of three gallium-68 radioactivity ranges: 40–80, 100–140, or 160–200 MBq). However, a trend towards a lower ratio in the liver was noted for the radioactivity range of 40–80 MBq compared to higher radioactivity ranges. There were no safety concerns associated with 68Ga-satoreotide trizoxetan.

When comparing different peptide mass and radioactivity combinations of an imaging agent, a key aspect is the evaluation of image quality. We hypothesised that a binary visual reading technique comparing the images contemporaneously (rather than a multi-scoring technique in multiple review sessions) could reduce the readers’ workload and inter-reader variability, and thus improve the key endpoint assessment. Accordingly, here, we report the binary visual reading results in the aforementioned phase II study aimed to determine the optimal peptide mass and radioactivity ranges for 68Ga-satoreotide trizoxetan in patients with GEP-NETs.

Methods

Study design and patient population

This prospective, multinational, multicentre, dose-confirmation phase II study (ClinicalTrials.gov identifier: NCT03220217; EudraCT No.: 2016-004928-39) was conducted between September 2017 and August 2019, using an open-label, reader-blinded, 2 × 3 factorial design. The methodology and primary results have been reported elsewhere [18]. In summary, a total of 24 adult patients with well-differentiated, metastatic, grade 1/2, SSTR-positive GEP-NETs were randomised (1:1:1) to one of three arms (8 patients per arm) with six peptide mass/radioactivity range combinations of 68Ga-satoreotide trizoxetan. Essentially, all patients received two different doses of 68Ga-satoreotide trizoxetan in one of the three arms on two consecutive visits separated by a 2–3-week interval, thereby providing a pair of PET/CT images for each patient (Fig. 1). The administered peptide mass range for all three arms was 5–20 µg on visit 1 (day 1 of the study) and 30–45 µg on visit 2 (day 16–22). The radioactivity range was for:

  • Arm A: 40–80 MBq (visit 1) and 100–140 MBq (visit 2)

  • Arm B: 100–140 MBq (visit 1) and 160–200 MBq (visit 2)

  • Arm C: 160–200 MBq (visit 1) and 40–80 MBq (visit 2)

Fig. 1
figure 1

Study design. Abbreviations: CT, computed tomography; GEP, gastroenteropancreatic; NET, neuroendocrine tumour; PET, positron emission tomography; R, randomised

Full size image

All patients had a SSTR agonist PET/CT scan acquired within the previous six months of study enrolment. The main exclusion criteria were the presence of < 5 lesions in total and > 25 lesions per organ detected by the screening SSTR PET/CT scan in either the liver, lymph nodes, bones, or lungs; treatment with a somatostatin analogue within 28 days before 68Ga-satoreotide trizoxetan administration; and any condition that might preclude the proper performance of a PET and/or CT scan (e.g., patient body weight > 110 kg; inability to raise arms for prolonged imaging purposes or to lie still for the entire imaging time).

The study was performed in four investigational sites in Austria, Denmark and the USA, and was conducted in accordance with the Declaration of Helsinki as well as the Good Clinical Practice (GCP) guidelines. Ethics committee approval was obtained at each participating site, and all patients provided written informed consent.

PET/CT imaging protocol

An image core laboratory (Rad-MD, New York City, New York, USA) provided a comprehensive, manufacturer-specific image acquisition manual, which detailed the requirements for the investigator site personnel to ensure inter-site consistency of image acquisition protocols across scanners and sites.

On both day 1 and day 16–22 study visits, whole-body PET/CT imaging (from skull base to mid-thigh) was performed for all patients 50–70 min after the intravenous injection of 68Ga-satoreotide trizoxetan, using either Siemens Biograph dedicated PET/CT scanners or GE Discovery 690 PET/CT scanners at each of the four study centres. Three-dimensional PET scans were acquired in list mode, with a 4 min per bed position and a 5-slice overlap. CT scan was performed using intravenous iodinated contrast media, with a maximum slice thickness of 3 mm.

The effective administered activity of 68Ga-satoreotide trizoxetan (defined as the activity in the syringe before injection minus the residual activity in the empty syringe after injection) was used for PET image reconstruction. PET/CT data were volume rendered with maximum intensity projection for PET and direct volume rendering for CT. No fasting or dietary restrictions were imposed on study participants prior to PET/CT imaging.

All trial sites qualified their PET scanners using the Clinical Trials Network (CTN) of the Society of Nuclear Medicine and Molecular Imaging (SNMMI) Scanner Validation Program to ensure baseline common quality control metrics for PET scanners used across the sites [19]. Before imaging patients at each study site, the PET/CT scanner was cross-calibrated to a well counter calibrated for gallium-68using a homogeneously filled phantom; the phantom was used to ensure that PET/CT scanner images were comparable and reproducible across the study sites for image noise and texture. PET imaging data were collected and confirmed by the SNMMI CTN, to ensure the standardised uptake values (SUV) used in the trial were reliable.

In addition, quality control by the imaging core laboratory was performed on all acquired 68Ga-satoreotide trizoxetan PET/CT scans. In case of detection of patient movement-related artifacts, a scan repeat would be requested, or a patient would be replaced.

Qualitative imaging analysis

Following anonymisation, 68Ga-satoreotide trizoxetan PET/CT scans were all sent to an imaging core laboratory (Keosys, Nantes, France), and evaluated centrally. A total of 48 PET/CT images were read: two for each of the 24 patients acquired at the two study visits, with 8 images in each peptide mass/radioactivity range combination. Prior to initiating the PET/CT image reads, a formal study training day, including a “calibration” agreement session, was conducted in New York, with all readers present. 68Ga-satoreotide trizoxetan images acquired in the Nicolas et al. (2018) phase I/II study of 12 patients with well-differentiated GEP-NETs [16, 17] were used for training, in a blinded manner by all readers to evaluate inter-reader consistency and to reach consensus on areas of interpretative difference.

The read design schema is shown in Fig. 2, with a total of six read sessions. All readers were blinded to all patient identification, clinical data, and to any information related to the study site, visit, and administered peptide mass and radioactivity. To maintain consistency, the patients’ PET/CT scans were read in batches of approximately six patients. The 68Ga-satoreotide trizoxetan PET images were read either alone (read 1A and 1B) or as a fused PET/CT scan (read 2A and 2B) by two different pairs of nuclear medicine physicians. In parallel, the contrast-enhanced CT scans, which were used for statistical comparisons, were independently read by two radiologists (read 3). The 68Ga-satoreotide trizoxetan PET and PET/CT images were randomised in presentation sequence (visit 1 or visit 2) so as to blind the readers from the timepoint and administered dose, with a minimum interval of 14 days between the two reads in an attempt to decrease reader image memory and possible recall bias. The purpose of the read 1A, read 2A, and read 3 sessions was to count the number of lesions per organ on each patient scan, with the use of an adjudication paradigm during read session 4 in case of a > 10% discrepancy between the two independent readers in the lesion count.

Fig. 2
figure 2

Read design. Abbreviations: CT, computed tomography; PET, positron emission tomography; R, reader

Full size image

Reads 1B and 2B were separate image quality read sessions where the two 68Ga-satoreotide trizoxetan scans acquired from the same patient were presented to the two experienced nuclear medicine physicians for visual assessment, using a split screen technique. Both physicians read the images independently without reference to the other. Each reader directly compared the paired patient images contemporaneously, recording which of the pair provided optimal image quality. Both readers had the ability to use window/level manipulation on each reviewed image. 68Ga-satoreotide trizoxetan PET and PET/CT scans were scored in two different sessions using a binary system (0 for non-optimal image quality and 1 for optimal image quality). For each patient pair of 68Ga-satoreotide trizoxetan scans, one or both images could score 1. Both primary readers’ scores were subsequently combined providing a maximum total image quality score (IQS) of 16 per peptide mass/radioactivity range combination (8 images × 2 readers). No adjudications were held for the image quality reads (reads 1B and 2B).

Quantitative analysis

During a subsequent reading session (read 5), another independent, qualified reader, not involved in the 68Ga-satoreotide trizoxetan or contrast-enhanced CT image reads, reviewed all 48 PET/CT scans, in order to perform the semi-quantitative measurements of maximum standardised uptake values (SUVmax) to support the qualitative imaging analysis. Spherical regions of interest were manually drawn by the reader in up to the five most avid lesions detected in the primary tumour site, liver, lymph nodes, bones, and lungs. As a reference tissue, the SUVmean of non-tumoural liver parenchyma was measured in a 3-cm region of interest in all 68Ga-satoreotide trizoxetan PET/CT scans while avoiding large vessels. The tumour-to-liver ratio was calculated as lesion SUVmax/non-tumoural liver parenchyma SUVmean for each pre-defined anatomic region, and was determined across the different peptide mass and radioactivity ranges.

Statistical analysis

This was a descriptive analysis. As prespecified in the Statistical Analysis Plan, 24 patients were to be included in the per-protocol population (the first eight patients in each arm) to ensure balanced arms and paired reading of images for comparability across doses. Categorical variables were expressed as counts and percentages, and continuous variables as mean ± standard deviation or median (range), depending on the distribution of values. All imaging endpoints were also evaluated based on the administered radioactivity (MBq) per patient’s baseline body weight (kg) in four distinct subgroups: 0.69–0.97, 0.97–1.55, 1.55–2.09, and 2.09–3.72 MBq/kg.

Statistical analyses were performed using SAS version 9.4 (SAS Institute Inc, Cary, NC). Missing values were not replaced.

Results

Patients

The baseline demographic characteristics of the study population are presented in Table 1. Overall, 21/24 (87.5%) patients had intestinal NETs, and 3 (12.5%) pancreatic NETs. The liver (in 22/24 patients; 91.7%) and lymph nodes (19/24; 79.2%) were the most frequent locations of metastases.

Table 1 Baseline demographic characteristics in the per-protocol population
Full size table

All 24 patients had a prior SSTR scan acquired within a median of 1.6 months (range, 0.1–6.0 months) from screening: 14 (58.3%) with 68Ga-DOTATOC, 9 (37.5%) with 64Cu-DOTATATE, and 1 (4.2%) with 68Ga-DOTATATE. The total number of SSTR-positive lesions, expressed as median (range), detected by prior SSTR agonist scans was 14.5 (6.0–94.0), with 1.0 lesions (0–1.0) in the primary tumour site, 9.5 (0–37.0) in the liver, 5.0 (0–36.0) in the lymph nodes, and 0.5 (0–38.0) in the bones. Three patients were enrolled with over 30 lesions identified in the liver or in the lymph nodes.

Image quality

The IQSs are presented in Table 2 for both the 68Ga-satoreotide trizoxetan PET/CT and PET only reads. The IQSs were overall similar across the three radioactivity ranges for the PET/CT reads. However, for the PET only reads, the total IQS was lower in the 40–80 MBq radioactivity range (56.3%) compared to the 100–140 MBq (90.6%) and 160–200 MBq (81.3%) ranges. With regard to the evaluated peptide mass ranges, the total IQSs were 60.4% in the 5–20 µg peptide mass range and 83.3% in the 30–45 µg peptide mass range for the PET/CT readings. For the PET only readings, the total IQSs were 70.8% and 81.3% in the 5–20 and 30–45 µg ranges, respectively.

Table 2 Image quality scores for 68Ga-satoreotide trizoxetan PET/CT and PET, by peptide mass and radioactivity range
Full size table

IQSs based on the administered radioactivity per patient’s body weight are presented in Table 3. While the overall results were similar for the evaluated quartiles of body mass, the IQSs in the 5–20 µg range were in general lower than that in the 30–45 µg peptide mass range, particularly for the 68Ga-satoreotide trizoxetan PET/CT readings.

Table 3 Image quality scores for 68Ga-satoreotide trizoxetan PET/CT and PET, by radioactivity per patient’s body weight
Full size table

During the image quality read sessions, there was only one reading where the readers diverged on scoring; one reader preferred one image because of higher lesion conspicuity, and the other reader preferred the alternative image because of the ability to identify more lesions. All other reads were congruent in that the readers agreed on at least one of the images having an IQS of 1. Compared to 68Ga-satoreotide trizoxetan PET/CT readings, both primary readers reported that it was overall more difficult to localise hepatic and pulmonary lesions in PET only readings.

A pair of 68Ga-satoreotide trizoxetan PET/CT fused images acquired from the same patient at the two study timepoints is illustrated in Fig. 3.

Fig. 3
figure 3

Two 68Ga-satoreotide trizoxetan PET/CT fused images with maximum intensity projection of the same patient acquired at two different timepoints separated by 2–3 weeks. In the top panel, the patient had received 68Ga-satoreotide trizoxetan at 16 µg/165 MBq. In the lower panel, the patient had received 68Ga-satoreotide trizoxetan at 32 µg/72 MBq. Abbreviations: CT, computed tomography; PET, positron emission tomography

Full size image

Semi-quantitative assessment

SUVmax and the tumour-to-liver ratio (SUVmax/SUVmean) were evaluated on all 68Ga-satoreotide trizoxetan scans, and the results were compared across the peptide mass and radioactivity range combinations, with no consistent pattern in the median values identified (Table 4). Similarly, when assessing tumour-to-liver ratio by peptide mass range and radioactivity range (Table 5), no notable differences in the mean and median ratios were observed. There was also no apparent relationship between the radioactivity range per patient’s body weight and SUVmax of the liver and lymph nodes (Table 6).

Table 4 SUVmax and tumour-to-liver ratio for the liver and lymph nodes, by combination
Full size table
Table 5 Tumour-to-liver (SUVmax/SUVmean), by peptide mass and radioactivity range
Full size table
Table 6 SUVmax of the liver and lymph nodes, by radioactivity per patient’s body weight
Full size table

Discussion

In this study, we evaluated a binary visual reading methodology to reduce the readers’ workload and inter-reader variability, and improve study endpoints’ assessment following administration of a 68Ga-labelled radiopharmaceutical. The main finding was that a binary scoring system evaluating image quality in a contemporaneous manner was indeed useful in appraising image quality in a reader-centric manner with a low inter-reader variability, and can consequently be recommended for future evaluation of imaging studies.

The evaluation of image quality in product development and clinical trials is most commonly assessed by using the 5-point Likert scale as a qualitative index [20,21,22] requiring multiple reader sessions. A Likert scoring methodology, while providing the appearance of a robust evaluation, has several practical and methodological challenges. Most importantly, it requires user (reader) calibration to define the exact metric to each unit of score. Without careful definition and calibration, there will be mismatch errors among the readers which might lead to significant statistical noise, as there is potentially more discrepancy in the interpretation of the score values than within the images themselves. Furthermore, the use of a Likert scale requires a relatively large sample size. By comparison, the methodology presented here relies on a small pre-determined balanced approach requiring far fewer patients to be enrolled and a forced direct image comparison creating an IQS. The approach of comparing two doses of 68Ga-satoreotide trizoxetan using a split screen technique simplified the readers’ work and provided more robust responses to the challenge of selecting one of two images. The binary visual reading technique, while described elsewhere, does not, surprisingly, appear to have been widely adopted in the imaging field and in the nuclear medicine arena. The obvious advantage of a reduced number of patients being involved makes a compelling argument for future consideration of binary contemporaneous visual reading for image evaluation. As noted, there was only one discrepant reading between the two primary readers in our study, supporting the hypothesis that a binary scoring system would lead to low inter-reader variability and evaluate image quality in a reader-centric manner.

Besides the adoption of a binary contemporaneous visual reading scoring system, a blinded independent central review, which has been reported to increase the reproducibility of study results [20, 23], was performed to optimise qualitative image evaluation of 68Ga-satoreotide trizoxetan PET/CT scans in this study. In addition, by omitting clinical and dosing data as well as information on the study site and timepoints, the blinded reads provided pure imaging data results. Reading/interpretation bias was further reduced by the adoption of a multiple randomisation scheme. Of note, when evaluating novel radiopharmaceuticals, it is important that the read methodology is carefully described to meet the needs of regulatory agencies such as the United States Food and Drug Administration [24,25,26,27].

A key contributing factor to ensuring successful image quality assessment is carefully designed acquisition protocols, with strict quality control. This was accomplished with the support of the imaging core labs (Rad-MD, Keosys, and SNMMI CTN) and the investigator sites ensuring that patients were imaged according to the study protocol and GCP guidelines. Another key aspect is the reader training/calibration, which was conducted face to face, and included the use of images acquired in the aforementioned Nicolas et al. (2018) phase I/II study [16, 17] to accustom the readers to evaluating 68Ga-satoreotide trizoxetan PET/CT and to provide initial inter-reader variability information.

In terms of radioactivity ranges, the IQSs for the 68Ga-satoreotide trizoxetan PET/CT and PET only scans were lower for the 40–80 MBq range, especially with the PET only images. These results, along with the other published data from the study [18], confirmed that this radioactivity range of 68Ga-satoreotide trizoxetan cannot be recommended for phase III development, particularly given that it has never been evaluated in prior studies with other 68Ga-labelled agents. The combination of 40–80 MBq and 5–20 µg appears to have the poorest quality of reads, as illustrated by the IQSs for 68Ga-satoreotide trizoxetan PET/CT and PET only scans when compared to other peptide mass and radioactivity range combinations, suggesting a potential peptide mass dose effect. However, once the combination of 40–80 MBq and 5–20 µg is ignored, the differences in IQSs across the evaluated peptide mass and radioactivity range combinations are reduced, particularly for the PET only reads. Moreover, the semi-quantitative analysis showed no notable differences in the mean and median values of SUVmax and SUVmax/SUVmean for the liver and lymph nodes across the evaluated peptide mass and radioactivity range combinations. Therefore, the overall results lead to the conclusion that peptide mass does not influence 68Ga-satoreotide trizoxetan imaging. This is in line with the Nicolas et al. (2018) phase I/II imaging study [16, 17] in which two peptide masses (15 and 50 µg) of 68Ga-satoreotide trizoxetan were injected in 12 patients with GEP-NETs. There were no significant differences between the two peptide masses in the numbers of malignant liver or lymph node lesions detected per patient and in tumour uptake [16, 17].

Data regarding the optimal peptide mass range of radiolabelled SSTR antagonists used for functional NET imaging are scarce [28, 29]. Currently, a peptide mass up to 50 µg is used for imaging, as per current administration guidelines for 68Ga-labelled radiopharmaceuticals [30], to avoid the risk of receptor saturation and to allow accurate quantification of the receptor density [28, 29]. However, this recommended peptide mass range is more reflective of common practice than of evidence from controlled studies [28]. Nevertheless, the results of the present study support an optimal peptide mass of 68Ga-satoreotide trizoxetan up to 50 µg for the diagnostic imaging of GEP-NETs.

This present study also supports the administration of a radioactivity range of 68Ga-satoreotide trizoxetan between 100 and 200 MBq, which is in line with the Oncology Committee of the European Association of Nuclear Medicine’s procedural guidelines for PET/CT tumour imaging with 68Ga-conjugated peptides [30]. To ensure a high diagnostic quality of 68Ga-satoreotide trizoxetan PET/CT examination and to avoid negative impacts on image quality or lesion detectability [5], it is not recommended to reduce the administrated radioactivity below 100 MBq. Of note, in a prospective study among 24 patients with SSTR-positive NETs who received a single intravenous injection of 68Ga-DOTATOC administered at a mean dose of 120 MBq, both 68Ga-DOTATOC PET/CT and PET/magnetic resonance imaging were associated with good image quality and detectability of focal PET lesions on both a patient basis and organ system basis [31]. In the present study, the total IQS for 68Ga-satoreotide trizoxetan PET scans was higher in the 100–140 MBq radioactivity range (90.6%) compared to the 160–200 MBq (81.3%) range, suggesting a possibility of narrowing the radioactivity window of 68Ga-satoreotide trizoxetan from 100–200 to 100–140 MBq. However, adopting the wider, guideline-recommended radioactivity range of 100–200 MBq offers increased flexibility and feasibility in routine clinical practice, while maintaining similarity in dosing to other gallium-68-labelled products [30]. The absence of a clear radioactive dose–response relationship in the present study might be related to factors such as heterogeneity in receptor density, hypoxia, interstitial pressure, necrosis, and tumour heterogeneity [32].

This study was mainly limited by a small sample size, the use of a descriptive statistical analysis only, the lack of a direct comparison between binary contemporaneous visual reading and the Likert scale system, and the absence of a comparator arm to evaluate the repeatability of the imaging results. In addition, inter- and intra-reader variabilities were not quantified. However, to minimise inter-reader variability, a full reader training which included the binary visual reading of 68Ga-satoreotide trizoxetan PET/CT scans acquired in the Nicolas et al. (2018) phase I/II study [16, 17] was undertaken prior to the start of the study. Analyses of tumour SUV were also restricted to measurement of SUVmax. Although semi-quantitative assessments using SUVmax may allow a more uniform evaluation of the diagnostic value of 68Ga-satoreotide trizoxetan [33], the number of patients in the present study with lesions in the bones, lungs, and the primary tumour site was too small to draw conclusions for these organs on the differences in the mean and median values of SUVmax and SUVmax/SUVmean across the administered peptide mass and radioactivity ranges. This study nevertheless had several strengths, particularly a robust research design allowing inter- and intra-individual comparisons of different peptide mass and radioactivity range combinations, the adoption of a novel read methodology providing direct image comparisons and consequently removing the subjectivity of inter-timepoint variations, the utilisation of a blinded independent central review, the inclusion of both quantitative and qualitative imaging measures, and a uniform PET/CT imaging protocol across all study sites. Based on the results of the present study, 68Ga-satoreotide trizoxetan administered at 100–200 MBq with a peptide mass up to 50 μg will be further evaluated in larger prospective clinical studies.

Conclusions

This study provided the opportunity to develop a simple and precise image quality scoring system, associated with a low inter-reader variability. This binary contemporaneous scoring system can consequently be recommended for future evaluation of imaging studies. The read quality results in the present study, along with the results reported elsewhere [18], confirm an optimal administered radioactivity of 68Ga-satoreotide trizoxetan of 100–200 MBq with a peptide mass up to 50 μg for future clinical development.