Diagnostic performance of a computer-assisted diagnostic system: sensitivity of BONENAVI for bone scintigraphy in patients with disseminated skeletal metastasis is not so high

Purpose Bone scintigraphy (BS) of disseminated skeletal metastasis is sometimes misinterpreted as normal. The use of computer-assisted diagnosis (CAD) may resolve this problem. We investigated the performance of a CAD system, BONENAVI, in the diagnosis of disseminated skeletal metastasis. Methods Cases of disseminated skeletal metastasis were selected from a BS log. These patients’ BSs were analyzed by BONENAVI to obtain an artificial neural network (ANN) and bone scan index (BSI). Clinical features (type of primary cancer, CT type, and BS type) were compared with the BONENAVI (ANN and BSI) results. The BS findings (diffuse increased axial skeleton uptake, inhomogeneity of uptake, proximal extremity contrast, and degree of renal uptake) and ANN or BSI were evaluated. Then, negative ANN patients were presented. Results Fifty-four patients were diagnosed as having disseminated skeletal metastasis. Regarding the primary cancers, 12 had prostate cancer, 16 gastric cancers, 16 breast cancers, and 10 miscellaneous cancers. Total sensitivity of ANN (≥ 0.5) was 76% (41/54). ANN values correlated with the BS type among clinical features. Diffuse increased axial skeleton uptake was mostly correlated with ANN of the BS findings. Conclusion The BONENAVI CAD system was partially helpful in diagnosing disseminated skeletal metastasis, but the sensitivity of BONENAVI was not sufficient and underestimated the disseminated skeletal metastasis. Further improvement of this CAD system is necessary to improve the detectability of disseminated skeletal metastasis.


Introduction
The diagnosis of disseminated skeletal metastasis by bone scintigraphy (BS) is usually easily established when typical findings are present. However, the BS pattern is easily misinterpreted as normal unless there is an absence of renal uptake, or the presence of diminished activity in the bones of the appendicular skeleton, and when a high ratio of bone to soft tissue activity is recognized [1,2].
Until recently, BS was interpreted visually. However, considering the need for an appropriate quantitative approach for BS, computer-assisted diagnostic (CAD) software for BS was developed and evaluated [3][4][5]. A Japanese version of this software is BONENAVI [6]. We previously published a formal report on the development and clinical evaluation of a revised version, called BONENAVI II [7]. Although the sensitivity and specificity of BONENAVI are reportedly good for skeletal metastasis [8,9], we also reported that skeletal metastasis from prostate cancer with a small tumor burden often shows false-negative results in CAD (BONENAVI II) analysis [10].
Our clinical questions are how the BONENAVI performed in patients with disseminated skeletal metastasis, and what factors, if any, caused the failures (false-negative results) in the diagnosis of the skeletal metastasis by BONE-NAVI in disseminated skeletal metastasis patients. Therefore, we tried to clarify the sensitivity of BONE-NAVI II in patients with disseminated skeletal metastasis. We also tried to determine the clinical features or BS features that cause mismatches in the skeletal involvement and negative CAD analysis.

Patients
We selected patients with disseminated skeletal metastasis from the BS log from January 2013 to August 2019 at the cancer institute hospital, Tokyo, Japan. From the log, 63 patients were listed. These patients were diagnosed on BS to have definite, suspected or not-ruled-out disseminated skeletal metastasis. We reviewed the BS and other clinical data, including computed tomography (CT), magnetic resonance imaging (MRI) and/or fluoro-deoxy glucose positron emission tomography with CT (FDG-PET/CT). Fiftyfour patients were confirmed to have disseminated skeletal metastasis; that is, skeletal metastatic lesions had spread to nearly the entire axial skeleton, equivalent to the extent of the disease (EOD) = IV [11]. These 54 patients were selected for this study. Nine patients were excluded from this study because 8 had skeletal metastasis, but the spread of the disease did not meet the criteria of the term 'disseminated', while the other one patient did not show skeletal metastasis.
The diagnosis of disseminated skeletal metastasis was established clinically; that is, no skeletal metastatic lesion was diagnosed histologically. The diagnosis of disseminated skeletal metastasis was established and confirmed using at least two diagnostic modalities (from BS, CT, MRI, and FDG-PET/CT).
This retrospective study was approved by the local institutional review board.

Bone scintigraphy
BS was performed about 3 h (range 2-4 h) after injection of 740 MBq of 99m Tc-MDP. After obtaining anterior and posterior whole-body scans, local images, the pelvic axial view, were also acquired. Whole-body (anterior and posterior) images were taken using E-CAM (Canon), Infinia (GE) or Intevo (Siemens) using 15-20-cm /min scan speed with a 256 × 1024 matrix size. After taking whole-body images, static studies were conducted using 6-8 million counts/view with a 512 × 512 matrix size.
BS findings were reviewed, and the presence of the characteristics of disseminated skeletal metastasis based on each BS clinical report was recorded. BS findings of disseminated skeletal metastasis were characterized by intense uptake throughout the skeleton; faint or absent renal uptake; patchiness (inhomogeneity) of uptake, particularly in the ribs; and the presence of diminished activity in the appendicular skeleton [1,2]. These findings were changed to: 1. diffuse axial skeleton radionuclide (RN) uptake (positive, weak positive, or negative), 2. inhomogeneity of RN uptake (inhomogenous, partially inhomogenous, or no), 3. decreased appendicular skeleton uptake or relatively increased uptake in a proximal extremity (yes or no), and 4. absent kidney sign (absent kidney uptake, weak renal imaging, or normal renal uptake).
Based on the above findings, BS visual interpretations were classified into (a) definite BS type: definite super bone scan; diffuse increased axial skeleton RN uptake irrespective of other findings, (b) suspected BS type: suspected disseminated bone metastasis (definite skeletal metastasis but not a typical super scan); weak or no diffuse axial skeleton RN uptake and inhomogeneity of RN uptake, irrespective of other findings, (c) not-ruled-out BS type: not-ruled-out disseminated skeletal metastasis; other than (a) or (b) above; weak or no diffuse axial skeleton uptake, partial or no inhomogeneity, and increased proximal extremity uptake or absent or weak renal uptake.

BONENAVI
BONENAVI analysis was performed as previously reported [7]. Briefly, the CAD system, BONENAVI version 2.1.7 (FUJIFILM RI Pharma Co., Ltd., Tokyo, Japan), was used to analyze BS scans using anterior and posterior wholebody images. This BONENAVI system shows two imaging markers: an artificial neural network (ANN) and a bone scan index (BSI). The ANN value indicates the probability of having skeletal metastasis. It has an ANN range of 0-1, where "0" means no possibility of skeletal metastasis, and "1" means a high suspicion of having skeletal metastasis. The BSI indicates the tumor metastatic burden (proportion of bone metastatic area to whole-body skeleton). In the present study, both the ANN and BSI values were used.

CT and classification
CT scans were obtained with a 2-mm thickness at 5-mm intervals.
The CT morphological classification of skeletal metastasis included the categories osteoblastic, osteolytic, mixed osteoblastic and osteolytic, and intertrabecular (invisible on CT). This classification was independently performed by two radiologist-nuclear physicians, and discordant cases were decided through discussion. The diagnostic confirmation of disseminated skeletal metastasis was made by the agreement of two physicians using at least two imaging modalities (BS, CT, MRI, FDG-PET/CT).

Analysis process
1. We observed the relationships among the types of primary cancer, CT types, and BS types. 2. We investigated the relationship between BONENAVI: ANN ≥ 0.5 and the types of primary cancer, CT types or BS types. 3. We analyzed the relationship between BONENAVI (ANN and BSI values) and clinical features; the types of primary cancer, CT types or BS types by one-way analysis of variance (one-way ANOVA).
4. To clarify the BS factors that were related to BONE-NAVI (ANN and BSI values), the relationship of ANN values and BS findings (the presence of diffuse increased skeleton uptake, inhomogeneity, increased proximal extremity uptake, or absent kidney sign) was investigated by one-way ANOVA. Using BONENAVI: ANN ≥ 0.9 as a dependent factor, multivariate logistic analysis was also employed to determine which factor was related to a high ANN value. We initially used ANN ≥ 0.5; however, there was no patient whose diffuse axial skeleton uptake was below 0.5 (ANN), and analysis was not properly performed. Therefore, we selected ANN ≥ 0.9 as the dependent factor instead of ANN ≥ 0.5. 5. We also observed the relationship between BSI values and BS findings by one-way ANOVA. 6. We added the list of patients with various factors who showed negative ANN (ANN < 0.5) despite the presence of disseminated skeletal metastasis. The multivariate logistic regression analysis was carried out using ANN ≥ 0.9 as a dependent factor both by forward and backward stepwise analysis using a likelihood ratio.
Twenty-five CT patterns were osteoblastic type, 19 mixed type, and 10 intertrabecular type. Twenty-eight BS types were of definite type, 16 were of suspected type, and 10 were of not-ruled-out type. The relationships among these factors are summarized in Table 1.
Disseminated skeletal metastasis from prostate cancer tended to have a high percentage of CT osteoblastic type (11/12) and BS definite type (10/12).
Disseminated skeletal metastasis from gastric cancer showed the next highest ratios as CT osteoblastic type (9/16) and BS definite type (10/16).
Breast cancer showed the lowest ratios among the three most frequent cancers as CT osteoblastic type (3/16) and BS definite type (6/16).
In all patients, CT osteoblastic (56%, 14/25) and mixed (63%, 12/19) types showed similar ratios to the definite BS type, and the CT intertrabecular pattern showed a low ratio to the definite BS type (20%, 2/10). There were no patients with osteolytic type who showed disseminated skeletal metastasis.  The sensitivities of the ANN value (ANN ≥ 0.5) are shown in Table 2. Total ANN sensitivity was 76% (41/54).
With BS types, definite BS type showed 100% (28/28), suspected BS type 56% (9/16) and not-ruled-out BS type 40% (4/10). Because we recruited the disseminated skeletal metastasis patients based on visual BS results (all patients were diagnosed to have some abnormal findings from definite to not-ruled-out on visual interpretation), we could not calculate the sensitivity of the visual BS method.
Relation to BONENAVI (ANN and BSI values) and clinical features (primary cancers, CT pattern, and BS pattern).
To clarify the relationships of primary cancer types, CT types, and BS types to ANN value, one-way ANOVA was performed. The results are shown in Table 3.
The whole analysis between ANN values and primary cancer types showed a statistically significant difference (p = 0.019); however, there were no significant differences among ANN values for each type of cancer.
There was no statistical difference in ANN values among CT types.
Statistically significant difference was shown in BS types (p = 0.002), and there was a significant difference between definite type and suspected type (p = 0.01), and between definite type and not-ruled-out type (p = 0.028).
About BSI values, kinds of cancer (p = 0.000) and BS patterns (p = 0.000) were statistically significant; however, CT types (p = 0.247) were not significant. Inter-group subanalysis showed that there was a statistically significant difference between the BSI of prostate cancer and gastric cancer patients, and breast cancer and miscellaneous cancers patients. BSI values of prostate cancer and gastric cancer patients were not significantly different, and neither was there a significant difference in the BSI values between breast cancer and miscellaneous cancers. Fig. 2 This 46-year-old male was diagnosed as having diffuse infiltrating type-4 advanced gastric cancer (poorly differentiated signet ring cell) 2 months prior to this study. Biochemical analysis showed an elevation of alkaline phosphatase (AL p = 4725 U/l); therefore, bone scintigraphy (BS) was performed. Anterior and posterior views of whole-body BS (left) and BONENAVI processed images with artificial neural network (right) a show diffuse increased radionuclide (RN) axial skeleton uptake; inhomogeneity of RN uptake, especially in ribs; increased proximal extremity uptake (or contrast of proximal femur and distal portion); and weak renal uptake. The interpretation of BS was "disseminated skeletal metastasis" (BS definite type). BONENAVI analysis showed ANN = 1.0 and BSI = 11.043. CT images of sagittal (left), and axial (right, upper and lower) slices b. Mainly osteoblastic and partially osteolytic lesions were disseminated in the whole skeleton There was a statistically significant difference between BSI values of definite BS type patients and those of others (suspected and not-ruled-out).

Relationship between ANN value and BS findings
To investigate how BS findings are related to ANN value, one-way ANOVA was performed with ANN value as a dependent factor and BS findings as independent factors. The results are summarized in Table 4. Diffuse increased skeleton uptake (p = 0.001) and absent kidney uptake (p = 0.000) were significant factors. However, inhomogeneity of uptake (p = 0.45) and proximal extremity relatively increased uptake (p = 0.135) were not significant factors.
We further analyzed the relation of ANN and BS findings using a multivariate logistic regression analysis. We initially used ANN ≥ 0.5; however, no patient with diffuse skeleton uptake was below 0.5 (ANN < 0.5), and the analysis was not properly performed. Therefore, we selected ANN ≥ 0.9 as the dependent factor.
This multivariate analysis showed diffuse increased skeleton uptake to be the only significant factor (p = 0.001), as shown in Table 5.
Regarding BSI values (one-way ANOVA), all BS factors showed statistically significant differences, but diffuse increased skeleton uptake and absent kidney sign showed strong correlations with the BSI values. On sub-group analysis, BSI values of diffuse increased skeleton uptake patients showed a significant difference compared with those of others (weak diffuse increased skeleton uptake and normal patients). BSI values of patients showing absent kidney sign were statistically significantly high compared with those of others (patients with weak renal imaging and normal renal imaging). Table 6 shows the list of patients who showed negative (ANN < 0.5). The types of primary cancer were variable but no prostate cancer patients were included in this list. CT types of ANN negative cases were variable: 7 were osteoblastic, 2 were mixed, and 4 were intertrabecular. No patients with definite BS type were included in the ANN negative cases. Regarding the BS findings, patients with diffuse increased skeleton uptake and absent kidney were not included in the ANN negative patient list. Many of the ANN Fig. 3 This 34-year-old female was diagnosed with advanced left breast cancer (invasive ductal carcinoma) with skeletal metastasis. Despite hormone therapy, she had progressive back and right chest pain. Her ALP and CEA values were high (AL p = 501 U/l and CEA = 99.0 ng/ml). Bone scintigraphy (BS) was performed. Anterior and posterior views of whole-body BS (left) and BONENAVI processed images with artificial neural network (right) a show weakly increased radionuclide (RN) axial skeleton uptake; inhomogeneity of RN uptake, especially in vertebrae and ribs; increased proximal extremity uptake (or contrast of proximal femur and distal portion); and normal renal uptake were shown. The interpretation of BS was "definite multiple skeletal metastasis but disseminated spread was not certain" (BS suspected type). However, BONENAVI analysis showed ANN = 0.0 and BSI = 0.0. This is a false-negative BONENAVI case. CT images of sagittal (left), and axial (right, upper and lower) slices (b). CT images revealed a mixed type of skeletal metastasis with partly osteolytic lesion predominance negative patients (10/13) had received therapy for malignancy (chemotherapy and/or hormone therapy). The effect of therapy might have influenced the BONENAVI results. Indeed, 10 in 27 patients with therapy had shown ANN negative, while 3 in 27 patients without therapy showed a negative ANN result (p = 0.027, Fisher exact test). Patients with active cancer therapy (chemotherapy and hormone therapy) statistically significantly frequently showed negative ANN results in the BONENAVI analysis.

Discussion
The diagnosis of disseminated skeletal metastasis by BS is sometimes easily misinterpreted as normal unless the absence of renal uptake, the presence of diminished activity in the bones of the appendicular skeleton, and a high ratio of bone to soft tissue activity are recognized [1,2]. Figures 3and 4 show patients with such a misinterpretation.
We aimed to evaluate the clinical application of the BONENAVI system on BS to resolve this problem. As shown in Table 2, patients with definite BS based on visual interpretation were correctly diagnosed (100%, 28/28) by Fig. 4 This 44-year-old female was recently diagnosed with diffuse infiltrating type-4 advanced gastric cancer (poorly differentiated signet ring cell). Her ALP and CEA values were within the normal limits (AL p = 207 U/l and CEA = 4.1 ng/ml). Bone scintigraphy (BS) was performed as a staging work-up study. Anterior and posterior views of whole-body BS (left) and BONENAVI processed images with artificial neural network (right) a show an almost normal axial skeleton uptake, no inhomogeneity of RN uptake, increased proximal extremity uptake (or contrast of proximal femur and distal portion), and normal renal uptake. The interpretation of BS was "an almost normal BS study; however, proximal extremity increased uptake was not-ruled-out to diagnose disseminated skeletal metastasis" (BS not-rule-out type). BONENAVI analysis showed ANN = 0.0 and BSI = 0.0. CT images of sagittal (left, right upper), and axial (right lower) slices (b). Spotty osteoblastic lesions were disseminated in the whole skeleton. Her skeletal lesions were shown to have progressed gradually in later studies ANN value. Patients with suspected (56%, 9/16) and notruled-out BS (40%, 4/10) were partially diagnosed as positive by ANN value. About half (13/16) of suspected and not-ruled-out BS patients could be identified. Regarding CT interpretation, intertrabecular CT type means undetectable on CT study. Therefore, the sensitivity of CT for disseminated skeletal metastasis was 81% (44/54). Furthermore, in intertrabecular type (CT negative) patients, BONENAVI detected skeletal metastasis in 60% (6/10). These figures are promising that in some patients with BS difficult (not-ruled-out BS type; 4/10) and CT negative (intertrabecular type; 6/10) showed positive ANN values but were not satisfactory. Total sensitivity of ANN in disseminated skeletal metastasis (76%) was lower than the reported sensitivity for all cancers. Nakajima et al. reported that the sensitivity of ANN for all cancers was 91% [8].
According to our classification, our clinical interpretation was that patients with suspected BS could be diagnosed as having bone metastasis, but whether or not disseminated spread was present was not certain. A false-negative example of such a patient is shown in Fig. 3. Our BS clinical interpretation was that inhomogeneity of uptake and increased proximal extremity bone uptake were present, and the interpretation of this patient's BS findings was that definite skeletal metastasis and disseminated spread were highly suspected. However, the BONENAVI result was negative (ANN = 0, BSI = 0). We speculated that the reason for the negative BONENAVI result was the nature of the skeletal metastasis, in that, even though there was a mixed CT pattern, the osteolytic component was predominant in this patient (Fig. 3). Therefore, skeletal RN uptake was not so intense, resulting in a negative ANN value.

3
The ANN value correlated with the BS types rather than the primary cancer or CT types (Tables 2, 3). Among the BS findings, diffuse increased skeleton uptake was best correlated to ANN rather than inhomogeneity of uptake, proximal extremity bone uptake, or absent kidney (Tables 4, 5). The diffuse increased skeleton uptake was considered the most important or impactful factor in the BONENAVI analysis.
To improve the sensitivity of BONENAVI, factors other than diffuse increased skeleton uptake, such as inhomogeneity of uptake, might be considered or included to reach the artificial intelligence of BONENAVI.
Chemotherapy and radiotherapy may influence the bone seeking RN uptake to bone. Radiation usually causes deceased bone seeking RN to bone with well demarcated margins limited to the boundaries of radiation areas [12]. The interpretation is very difficult after chemotherapy: decreased RN uptake usually implies tumor healing but sometimes a rapidly enlarging lytic lesion shows such decreased RN uptake. Conversely, an increased RN uptake usually indicates tumor progression, but in the early phase the increase may indicate a healing process mediated by the flare phenomenon [1]. The interpretation of the chemotherapeutic effect is not easy to understand. Only five of the patients in this study had irradiation therapy to bone (5/54). We abandoned the analysis of irradiation in the present study because of this low number of irradiated patients. The number of patients who underwent chemotherapy or hormone therapy (for breast cancer or prostate cancer) was not small: A serious limitation of this retrospective study was that we started this study from the BS log. We aimed to investigate the performance of BONENAVI in disseminated skeletal metastasis, and we regarded a BS study to be essential in this study. However, there is a possibility that BS was not performed (or not ordered) in all patients with disseminated skeletal metastasis. We recorded the BS log when any suspicious or not-ruled-out findings in BS study were identified. However, there is a possibility that patients with normal BS finding might have had disseminated skeletal metastasis. Therefore, we could not calculate the sensitivity of visual BS interpretation. We established a diagnosis of disseminated skeletal metastasis when at least two imaging modalities were consistent with the diagnosis. However, the sensitivities and specificities of each modality are not 100%, that is, currently there is no gold standard to diagnose disseminated skeletal metastasis.
Another drawback was that the diagnosis of disseminated skeletal metastasis was performed clinically; that is, no histopathological confirmation was carried out.

Conclusion
The BONENAVI CAD system was partially helpful in diagnosing disseminated skeletal metastasis, but the sensitivity of BONENAVI was not sufficient (76%) and underestimated disseminated skeletal metastasis. BONENAVI results may underestimate the disseminated skeletal metastasis by the present version.