Abstract
Objectives
The study aimed to evaluate scoring systems for predicting complicated appendicitis in adults diagnosed with acute appendicitis on computed tomography.
Methods
Three hundred twenty-five consecutive adult patients (mean age 51.9 ± 19.6 years, 212 women) diagnosed with acute appendicitis on computed tomography were retrospectively included. Clinical and imaging findings were compared between patients with and without complicated appendicitis, and independent associations were identified. As C-reactive protein was not available for most patients, 5 out of 8 scoring systems were modified. They, and a newly proposed system, were compared via area under the receiver operating characteristics (ROC) curve (AUC), Additionally, the latter was internally validated. Pairwise comparison was performed, and diagnostic performance of these scoring systems was obtained.
Results
One hundred twenty-seven patients (36.8%) had complicated appendicitis. Significant independent associations were found between complicated appendicitis and duration of symptoms > 12 h, appendicolith, periappendiceal fat stranding, periappendiceal fluid, and extraluminal air (p values < 0.001 to 0.037; AUCs of 0.824–0.829). AUCs of 9 scoring systems ranged from 0.692 to 0.831. Of these, modified Atema, Kim HY, and proposed scores had similarly high and non-significantly different AUCs (0.793–0.831) on pairwise comparison. Their sensitivities, specificities, and accuracies were 73.0–90.6%, 48.5–70.6%, and 64.3–72.3%, respectively. Internal validity test demonstrated high AUCs (0.826–0.844) with one of the proposed scores using odds ratio having 100% sensitivity and 100% negative predictive value.
Conclusions
Few scoring systems, including proposed ones, had high AUCs, sensitivity, and reasonable specificities, which could potentially aid in safely selecting adult patients with acute appendicitis for nonoperative management.
Critical relevance statement
The study suggests few scoring systems for predicting complicated appendicitis with high AUCs and reasonable sensitivities, potentially aiding in selecting patients for nonoperative management.
Key points
• The study evaluated existing and proposed new scoring systems to predict complicated appendicitis in adults with acute appendicitis on computed tomography.
• Several factors were found to be significantly associated with complicated appendicitis, including duration of symptoms, appendicolith, periappendiceal fat stranding, periappendiceal fluid, and extraluminal air.
• The modified Atema, Kim HY, and newly proposed scoring systems performed well, potentially aiding in nonoperative management selection.
Graphical Abstract
![](http://media.springernature.com/lw685/springer-static/image/art%3A10.1186%2Fs13244-023-01540-4/MediaObjects/13244_2023_1540_Figa_HTML.png)
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Introduction
Acute appendicitis is a common emergency condition in adults, which can result in severe complications if not managed appropriately. Complicated appendicitis can lead to perforation, abscess formation, peritonitis, and sepsis and require urgent surgical intervention [1]. Conversely, uncomplicated appendicitis can be treated with either appendectomy or nonoperative management with the use of antibiotics [2]. Nonoperative management is a viable option for selected patients with uncomplicated appendicitis, particularly those who are at increased risk for surgical complications or have a preference for a nonsurgical approach. Patient selection is, therefore, crucial in identifying those with uncomplicated appendicitis and avoiding directing complicated cases to a nonsurgical approach. The guideline issued by the World Society of Emergency Surgery emphasizes the importance of patient selection in the management of acute appendicitis [1].
Clinical scoring systems have been developed to aid in diagnosing appendicitis, such as the Alvarado score, Appendicitis Inflammatory Response score, and Adult Appendicitis Score. However, these scores have limited ability to differentiate between uncomplicated and complicated appendicitis [3, 4]. Several scoring systems have been proposed to aid in identifying complicated appendicitis, with varying methods and success [5,6,7,8,9,10,11]. However, only a few studies [10, 12, 13] have externally validated their performance.
Our investigation aimed to evaluate the performance of existing scoring systems for predicting complicated appendicitis in adults diagnosed with acute appendicitis on computed tomography and compare them to a newly proposed system.
Methods
Study design and patient selection
This investigation was conducted at a 2200-bed urban academic hospital. It was approved by the Institutional Review Board (protocol No. 136/2566(IRB2)). Informed consent was not required due to the retrospective nature and minimal risk involved. Figure 1 provides a flow chart of patient inclusion. The study identified eligible patients by searching the pathological database for a diagnosis of appendicitis among all consecutive adult patients aged 18 years or older from October 2016 to March 2021. Patients who had undergone abdominopelvic CT prior to appendectomy, regardless of the timing of appendectomy relative to the diagnosis of acute appendicitis, were included. Only the first CT examination indicating a clinical suspicion of acute appendicitis was included if there were multiple CT exams. Patients with incomplete clinical data (n = 12) and an appendix not identified at CT (n = 1) were excluded. The investigation ultimately included 325 patients (Table 1). Note that 201 of these patients have been described in our previous investigation [14]. Among the 325 patients, 321 initially underwent a CT scan as their primary imaging modality, while the remaining individuals had an initial ultrasound examination.
Image acquisition, reinterpretation, and definitions
One of the three multidetector CT scanners was used to conduct CT exams. With the exception of one scan, all exams were performed with intravenous administration of nonionic iodinated contrast medium, at a volume of 1.5–2.0 mL/kg (equivalent to approximately 80–100 mL) at a rate of 2–3 mL/s. The exams covered the area from either the upper border of the diaphragms or the upper pole of the kidneys to the ischial tuberosities. For each scan, an unenhanced phase was followed by a portovenous phase (approximately 80 s after contrast administration) with an axial slice thickness of 1.25 mm. All images were then transferred to Picture Archiving and Communication Systems (PACS) for viewing.
Two fellowship-trained radiologists, specialized in abdominal imaging and emergency imaging with 20 years of experience each, independently re-reviewed all CT scans. They were informed of the patient’s age, sex, and diagnosis of acute appendicitis, but remained unaware of other data. The images were analyzed on standard PACS workstations using Synapse (FujiFilm Inc.). Each radiologist provided their own interpretation of the CT findings based on definitions described in Supplementary Material 1 and previously [14]. Discrepancies between the two radiologists were resolved by an abdominal radiologist with 24 years of experience for the 201 previously reported cases, while the rest were resolved by consensus.
Reference standards
The diagnosis of acute appendicitis was confirmed through histopathological analysis. Cases of complicated appendicitis included those with gangrene or perforation. The diagnosis of gangrene was based on histopathology, while perforation was documented either through histopathology or surgical operative findings.
Scoring systems validated
Eight scoring systems were selected for validation due to their inclusion of both clinical features and CT findings in their scores [5,6,7,8,9,10,11]. Details of these scores are provided in Supplementary Material 2. Of these, 5 included serum C-reactive protein in their scores [5, 6, 10], which was documented in only 7 of our patient cohort. Therefore, this laboratory value was removed from the scores. The weighting of included factors remained but the appropriate cutoff values for all scores were reselected during statistical analysis.
Statistical analysis
Qualitative and quantitative information were analyzed using descriptive statistics. Categorical variables were presented in terms of numbers or percentages while continuous data were reported as mean (standard deviation) or median (range) depending on whether the data had normal or skewed data distribution.
To compare the difference between the two groups (uncomplicated vs. complicated appendicitis), inferential statistics were used. The Pearson chi-square or Fisher’s exact test was used for categorical variables, while the independent t-test or the Mann–Whitney U test was used for continuous variables having means or medians, respectively. Binary logistic regression was used for univariable and multivariable analyses to determine the odds ratio (OR) and coefficients for independent predictors of complicated appendicitis. Odds ratio with corresponding 95% confidence interval (95% CI) were used to identify the strength and direction of their association. The selection of factors into the multivariable model was based on a P value of less than 0.1 in the univariable model. In order to prioritize patient safety, we placed a high emphasis on sensitivity to diagnose complicated appendicitis. This approach enables the safe practice of recommending appendectomy for patients with uncomplicated appendicitis, rather than resorting to nonoperative management for those with complicated appendicitis.
The diagnostic performance of the scoring systems in differentiating between uncomplicated and complicated appendicitis was determined using two-by-two tables to calculate metrics such as sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, positive predictive value, negative predictive value, and accuracy. The ROC curves of these scoring systems were compared through pairwise comparison. These analyses were conducted using the Statistical Package for Social Sciences (SPSS, version 23, IBM), with a significance of 0.05.
The discrimination of the scoring systems describes the ability to give different predictions for complicated and uncomplicated appendicitis. The area under the ROC curve (AUC) with the corresponding 95% confidence interval (95% CI) was considered a summary measure for discrimination. The internal validation of the model was carried out by split-sample estimation and validation, in which the entire sample was randomly divided into two subsets, one used exclusively for model estimation ("training") and another used for validation ("testing"). Data were randomly divided with a split-sample approach, with 80% of the data allocated for training the model and 20% for internal validation using the R program (R Core Team (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/).
Results
Baseline characteristics of patients (Table 1)
The mean age of the patients was 51.9 ± 19.6 years. Most of them (60.3%) belonged to the age group of ≥ 45 years, with female predominance (65.2%). They presented to the hospital with a median duration of symptoms of 24 h (range, 2–480) and a median Alvarado score of 7 (range, 1–10). On CT, the mean appendix diameter was 12.0 ± 2.9 mm, and 41.2% of patients had an appendicolith. Periappendiceal fat stranding, periappendiceal fluid, ascites, and extraluminal air were present in 46.8%, 42.5%, 33.8%, and 14.5% of cases, respectively. One hundred twenty-seven patients (39.1%) had complicated appendicitis. Almost all patients (97.2%) had appendectomy at the initial admission of appendicitis. The median length of stay was 3 days (range, 1–44).
Predictive factors of complicated appendicitis (Tables 2 and 3)
Univariable analysis identified multiple clinical, laboratory, and imaging factors that are significantly associated with complicated appendicitis. After multivariable analysis, five factors remained statistically significant: duration of symptoms > 12 h, appendicolith, periappendiceal fat stranding, periappendiceal fluid, and extraluminal air. Their p values ranged from < 0.001 to 0.037. The odds ratios and coefficients were weighted for each factor to identify the risk prediction of complicated appendicitis as shown in Table 3.
Comparison of 9 scoring systems (Table 4)
The Atema, Kim, Imaoka, and Lin (models 1 and 2) scores were modified to exclude C-reactive protein, with their respective cutoff values selected at ≥ 5, ≥ 1, ≥ 1, ≥ 4, and ≥ 4, respectively. The cutoff value of the Avanesov score was reduced from ≥ 4 in the original description to ≥ 2 in our analysis. The Khan and Kim HY scores retained their original cutoff values of ≥ 2 and ≥ 3, respectively. Their AUCs are provided in Fig. 2. The scores based on our multivariable analysis assigned different points to each predictive factor. For both, when a value of ≥ 2 was used as a cutoff, the scores (based on odds ratios or coefficients) demonstrated a sensitivity, specificity, and accuracy of 88.2–89.8%, 48.5–49.0%, and 64.3–64.6%, respectively. The one that utilized the coefficients had slightly better sensitivity and accuracy, but slightly less specificity. Pairwise comparison of these ten scores (Table 5) revealed no significant difference between the modified Atema, Kim HY, and our (identified as “current”) proposed scores (p = 0.110–0.901).
Internal validation of current scores (Fig. 3, Supplementary Material 3)
With a split-model method, 260 cases were allocated for training the model and 65 cases for internal validation of our proposed scores. The scores derived from odds ratios and coefficients both achieved high AUCs (0.826–0.844) with the score using the odds ratio showing a sensitivity and a negative predictive value of 100%, and specificity of 46.4% in predicting complicated appendicitis.
Discussion
Our investigation identified factors independently predictive of complicated appendicitis that are crucial to consider in the era of potential nonoperative management of acute appendicitis. We validated the diagnostic performance of 8 existing scoring systems and proposed a new scoring system to predict complicated appendicitis without the need for serum C-reactive protein. Of these, modified Atema, Kim HY, and our proposed scores showed similarly high AUCs with reasonably high sensitivities and modest specificities in the identification of complicated appendicitis.
Since 2015, multiple scoring systems have been proposed to identify appendicitis with complications, utilizing clinical-only [15,16,17,18], imaging-only [19], or both clinical and imaging data [5,6,7,8,9,10,11]. In this study, we validated eight systems that utilized both clinical features and CT findings as these scores generally performed better than those utilized only clinical or CT features. Previous investigations have validated these models using a traditional statistical methodology [10, 12, 13] and artificial neural network [20]. Fujiwara et al. [13], Lin et al. [10], and Geerdink et al. [12] used 203 to 678 patients (52 to 175 with complicated appendicitis) for validation. In another study by Lin et al. [20], datasets of 592 patients were split for training of and validated by artificial neural network.
The Atema score [5] was introduced in 2015, with an original sensitivity of 97% and specificity of 46% in the differentiation of complicated from uncomplicated appendicitis. The scores demonstrated sensitivities from 64 to 90% and specificities from 51 to 95% in subsequent studies [10, 12, 13, 20]. Our investigation found that even with C-reactive protein excluded from the equation and a cutoff value reduced to ≥ 5, the Atema score still had the best performance with high AUC (0.831; 95% CI 0.787–0.875) and sensitivity (91%; 95% CI 84–95%). However, its specificity was only 61% (95% CI 53–68%).
Another scoring system that demonstrated promising results in our investigation was the Kim HY score [11]. In its original description, this score had an AUC of 0.81, a sensitivity of 93%, and a specificity of 28%. However, subsequent validations reported higher AUCs ranging from 0.84 to 0.92 and specificities between 88 and 100%, but lower sensitivities at 64% [10, 20]. Our study showed a balanced sensitivity and specificity at 73% (95% CI 64–81%), and 71% (95% CI 64–77%), respectively, indicating its potential usefulness. Other validated scoring systems showed varying results, with some demonstrating high specificity (Kim TH, Lin Model 2 scores), and others exhibiting variable performance (Imaoka, Avanesov, Khan, Lin Model 1 scores) [10, 13, 20].
Our proposed scoring system, when validated internally, the score that used odds ratio demonstrated 100% sensitivity and 100% negative predictive value, allowing it to avoid misclassification of complicated appendicitis, albeit at a moderate specificity. It overcame the modified Atema score in terms of less complexity as it consisted of only 5 factors for calculation, did not require C-reactive protein, and accumulated fewer total points.
The performance of other scoring systems in our evaluation was suboptimal. Specifically, the Khan score exhibited a lower AUC of 0.699 (95% CI 0.643–0.756), alongside moderate sensitivity (76%; 95% CI 67–83%) and specificity (48%; 95% CI 41–55%). Similarly, the modified Imaoka score demonstrated a lower AUC of 0.692 (95% CI 0.642–0.741), with moderate sensitivity (80%; 95% CI 72–81%) and specificity (58%; 95% CI 51–65%). Both of these were validated by Lin et al. [10], who reported similar diagnostic performance for predicting complicated appendicitis. Additionally, the Imaoka score had been validated by other studies [13, 20], revealing inconsistent diagnostic performance. For the modified Kim score, it exhibited very high sensitivity (98%; 95% CI 94–100%) but low specificity (23%; 95% CI 17–29%), limiting its utility. Notably, our results diverged significantly from the validation performed by Lin et al. [10, 20], who reported the original score as having much lower sensitivity but higher specificity.
When comparing the elements within the scoring systems that exhibited optimal vs. suboptimal performance, the factors contributing the most to enhanced performance were CT findings. Notably, the presence of extraluminal air, which was found in the modified Atema, Kim HY, and our proposed scores but absent in the modified Imaoka, Kim, or Khan scores, played a significant role. Additionally, the presence of appendicolith, which was included in the modified Atema and our proposed score but excluded from the modified Imaoka and Kim scores, also contributed to improved performance.
While our investigation provided a detailed evaluation of the performance of existing scoring systems, there are several limitations that need to be acknowledged. Firstly, our study was retrospective and conducted in a single center with a small sample size. As appendectomy remained the standard of care for acute appendicitis in our hospital, we were unable to evaluate the success rate of nonoperative management fully. However, our approach allowed us to use pathological results as a standard reference for the diagnosis of complicated appendicitis. Secondly, the absence of C-reactive protein data in most patients prevented us from validating some scores in full. However, this allowed us to test the scores without C-reactive protein and demonstrated that the modified Atema score still performed well. Thirdly, we designed our endpoint to prioritize high sensitivity to detect complicated appendicitis, rather than balancing the sensitivity and specificity. This approach ensured patient safety by avoiding sending complicated appendicitis for nonoperative management. Fourthly, we did not validate scores that utilized only clinical factors [16,17,18] as they were not our target population. Cross-sectional imaging is necessary for safe selection of nonoperative management in this condition even in young individuals [3, 21]. The scores proposed by Mahankali et al. [19] which utilized purely CT findings were not validated in our study due to incomplete data. Additionally, we believe that some data points including grading of periappendicial fat stranding [10] may pose a challenge in terms of real-world applicability as they were subjective.
In conclusion, our study demonstrated that the modified Atema, Kim HY, and our proposed scores were effective in predicting complicated appendicitis with high AUC and reasonable sensitivities. These scores have the potential to aid in the safe selection of patients for nonoperative management. However, further validation is required in larger, multicenter studies with a diverse patient population. Recent publications have shown that artificial neural networks may play a crucial role in this regard [20, 22]. Additionally, it is important to note that a prospective trial [23] focused on this issue is currently ongoing, and its results are eagerly awaited to further guide clinical decision-making.
Availability of data and materials
The data used to support the findings of this study are included in the article.
Abbreviations
- AUC:
-
Area under the ROC curve
- CT:
-
Computed tomography
- PACS:
-
Picture Archiving and Communication System
- ROC:
-
Receiving Operator Characteristics
References
Di Saverio S, Podda M, De Simone B et al (2020) Diagnosis and treatment of acute appendicitis: 2020 update of the WSES Jerusalem guidelines. World J Emerg Surg 15:27
Moris D, Paulson EK, Pappas TN (2021) Diagnosis and management of acute appendicitis in adults: a review. JAMA 326(22):2299–2311
Bom WJ, Scheijmans JCG, Salminen P, Boermeester MA (2021) Diagnosis of uncomplicated and complicated appendicitis in adults. Scand J Surg 110(2):170–179
Song H, Lee S, Park JH et al (2021) Can patient triaging with clinical scoring systems reduce CT use in adolescents and young adults suspected of having appendicitis? Radiology 300(2):350–358
Atema JJ, van Rossem CC, Leeuwenburgh MM, Stoker J, Boermeester MA (2015) Scoring system to distinguish uncomplicated from complicated acute appendicitis. Br J Surg 102(8):979–990
Kim TH, Cho BS, Jung JH, Lee MS, Jang JH, Kim CN (2015) Predictive factors to distinguish between patients with noncomplicated appendicitis and those with complicated appendicitis. Ann Coloproctol 31(5):192
Imaoka Y, Itamoto T, Takakura Y, Suzuki T, Ikeda S, Urushihara T (2016) Validity of predictive factors of acute complicated appendicitis. World J Emerg Surg 11(1):48
Avanesov M, Wiese NJ, Karul M et al (2018) Diagnostic prediction of complicated appendicitis by combined clinical and radiological appendicitis severity index (APSI). Eur Radiol 28(9):3601–3610
Khan MS, Siddiqui MTH, Shahzad N, Haider A, Chaudhry MBH, Alvi R (2019) Factors associated with complicated appendicitis: view from a low-middle income country. Cureus 11(5):e4765
Lin HA, Tsai HW, Chao CC, Lin SF (2021) Periappendiceal fat-stranding models for discriminating between complicated and uncomplicated acute appendicitis: a diagnostic and validation study. World J Emerg Surg 16(1):52
Kim HY, Park JH, Lee SS, Jeon JJ, Yoon CJ, Lee KH (2021) Differentiation between complicated and uncomplicated appendicitis: diagnostic model development and validation study. Abdom Radiol (NY) 46(3):948–959
Geerdink TH, Augustinus S, Atema JJ, Jensch S, Vrouenraets BC, de Castro SMM (2021) Validation of a scoring system to distinguish uncomplicated from complicated appendicitis. J Surg Res 258:231–238
Fujiwara K, Abe A, Masatsugu T, Hirano T, Hiraka K, Sada M (2021) Usefulness of several factors and clinical scoring models in preoperative diagnosis of complicated appendicitis. PLoS One 16(7):e0255253
Iamwat J, Teerasamit W, Apisarnthanarak P, Noppakunsomboon N, Kaewlai R (2021) Predictive ability of CT findings in the differentiation of complicated and uncomplicated appendicitis: a retrospective investigation of 201 patients undergone appendectomy at initial admission. Insights Imaging 12(1):143
Bröker MEE, van Lieshout EMM, van der Elst M, Stassen LPS, Schepers T (2012) Discriminating between simple and perforated appendicitis. J Surg Res 176(1):79–83
Chambers AC, Bismohun SL, Davies H, White P, Patil AV (2015) Predictive value of abnormally raised serum bilirubin in acute appendicitis: a cohort study. Int J Surg 13:207–210
Eddama MMR, Fragkos KC, Renshaw S et al (2019) Logistic regression model to predict acute uncomplicated and complicated appendicitis. Ann R Coll Surg Engl 101:107–118
García-Amador C, Arteaga Peralta V, de la Plaza LR, Torralba M, Medina Velasco A, Ramia JM (2021) Evaluation of preoperative clinical and serological determinantions in complicated acute appendicitis: a score for predicting complicated appendicitis. Cir Esp (Engl Ed) 99(4):282–288
Mahankali SK, Ahamed SA, Gupta GSP, Razek AAKA (2021) CT based acute appendicitis severity index for acute appendicitis and validate its effectiveness in predicting complicated appendicitis. Emerg Radiol 28(5):921–927
Lin HA, Lin LT, Lin SF (2023) Application of artificial neural network models to differentiate between complicated and uncomplicated acute appendicitis. J Med Sys 47(1):38
Podda M, Andersson R, Boermeester M et al (2021) Do young patients with high clinical suspicion of appendicitis really need cross-sectional imaging? Proceedings from a highly controversial debate among the experts’ panel of 2020 WSES Jerusalem guidelines. J Trauma Acute Care Surg 90(5):e101–e107
Liang D, Fan Y, Zeng Y et al (2023) Development and validation of a deep learning and radiomics combined model for differentiating complicated from uncomplicated acute appendicitis. Available from: https://ssm.com/abstract=4297061. [Cited 2023 May 12]
Bom WJ, Scheijmans JCG, Ubels S et al (2022) Optimising diagnostics to discriminate complicated from uncomplicated appendicitis: a prospective cohort study protocol. BMJ Open 12(4):e054304
Funding
No funding was received for this work.
Author information
Authors and Affiliations
Contributions
RK is the guarantor of integrity of the entire manuscript. RK, NN, and PA identified the study concepts and design. RK performed literature search. RK, WT, DW, PK, AC, and PA performed the clinical studies. RK and ST analyzed the data and performed statistical analysis. RK prepared the first draft. All authors read, edited, and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
The retrospective single-center study was approved by our Institutional Review Board (protocol no. SIRB 136/2566 (IRB2) with COA No. Si 163/2023). The informed consent was waived due to a retrospective nature and minimal risk involved.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Additional file 1: Supplementary Material 1.
Definitions of CT findings. Supplementary Material 2. Eight scoring systems under investigation. Supplementary Material 3. Diagnostic performance of current scores to predict complicated appendicitis based on the internal validation (n = 65).
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Kaewlai, R., Tongsai, S., Teerasamit, W. et al. Validation of scoring systems for the prediction of complicated appendicitis in adults using clinical and computed tomographic findings. Insights Imaging 14, 191 (2023). https://doi.org/10.1186/s13244-023-01540-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13244-023-01540-4