Background

Acute appendicitis is the most common cause of an acute abdomen requiring surgery, with a lifetime risk of about 7% [1]. Symptoms of appendicitis overlap with a number of other conditions making diagnosis a challenge, particularly at an early stage of presentation [2]. Patients may be suitably triaged into alternative management strategies: reassurance, pursuit of an alternative diagnosis or observation/admission to hospital. If admitted to hospital, appropriate imaging may be required prior to proceeding to an appendectomy [3].

Clinical prediction rules (CPRs) quantify the diagnosis of a target disorder based on findings of key symptoms, signs and available diagnostic tests, thus having an independent diagnostic or prognostic value [4]. They can also extend into clinical decision making if probability estimates are linked to management recommendations, and are subsequently referred to as clinical decision rules. CPRs have the potential to reduce diagnostic error, increase quality and enhance appropriate patient care [4]. In 1986, Alvarado constructed a 10-point clinical scoring system, also known by the acronym MANTRELS, for the diagnosis of acute appendicitis as based on symptoms, signs and diagnostic tests in patients presenting with suspected acute appendicitis (Figure 1) [5].

Figure 1
figure 1

Probability of appendicitis by the Alvarado score [5]: risk strata and subsequent clinical management strategy.

Full size image

The Alvarado score enables risk stratification in patients presenting with abdominal pain, linking the probability of appendicitis to recommendations regarding discharge, observation or surgical intervention [5]. Further investigations, such as ultrasound and computed tomography (CT) scanning, are recommended when probability of appendicitis is in the intermediate range [6]. However, the time lag, high costs and variable availability of imaging procedures mean that the Alvarado score may be a valuable diagnostic aid when appendicitis is suspected to be the underlying cause of an acute abdomen, particularly in low-resource countries, where imaging is not an option.

A recent clinical policy document from the American College of Emergency Physicians reviews the value of using clinical findings to guide decision making in acute appendicitis [7]. Under the heading of the Alvarado score, they state that 'combining various signs and symptoms into a scoring system may be more useful in predicting the presence or absence of appendicitis'. Although not a strong recommendation, the Alvarado score is the only scoring system presented in the document.

The Alvarado score was originally designed more than two decades ago as a diagnostic score; however, its performance and appropriateness for routine clinical use is still unclear. The aim of this study was to perform a systematic review and meta-analysis of validation studies that assess the Alvarado score in order to determine its performance (diagnostic accuracy or discrimination at two cut-points commonly used for decision making, and calibration of the score). As studies have suggested that the accuracy of the Alvarado is affected by gender and age [8–12], we focused our analysis on three separate groups of patients: men, women and children.

Methods

Data sources and search strategy

An electronic search was performed on PubMed (January 1986 to 4 April 2011), EMBASE (January 1986 to 4 April 2011), Cochrane library, MEDION and DARE databases. The search strategy is presented as a flow diagram in Figure 2. A combination of keywords and MeSH terms were used; 'appendicitis' OR 'alvarado' OR, 'Mantrels', was used in combination with 26 specific terms for CPRs, including 'risk score', 'decision rule', 'predictive value', 'diagnostic score', and 'diagnostic rule' [13]. A citation search of included articles was undertaken using Google Scholar. The references of included studies were also hand searched for relevant papers. Authors of recent papers (2001 onwards) were contacted when included studies did not report sufficient data to enable inclusion. No language restrictions were placed on the searches.

Figure 2
figure 2

Flow diagram for the selection of studies for inclusion in the meta-analysis.

Full size image

Study selection

To be included in this study, participants had to be recruited from an emergency department or a surgical ward and present with symptoms suggestive of acute appendicitis, including abdominal pain, rebound tenderness, nausea, vomiting or elevated temperature. Each included study assessed the performance of the Alvarado score in comparison with the histological examination of the appendix following surgery (reference standard). For those who did not undergo appendectomy and histological examination, outpatient follow-up or no repeat presentation were used as alternative outcome measures. To be included, studies had to report results in a manner that allowed data to be extracted for either the diagnostic test accuracy analysis of the Alvarado score at specific cut points or the calibration analysis. Studies that focused on pregnant patients were excluded.

Two reviewers (RO and FO'R) completed the review process. The inclusion criteria were defined a priori. They reviewed titles and abstracts independently and after discussion decided which articles should be reviewed in full. Full text articles were reviewed independently by the same reviewers and any disagreements were resolved by discussion.

Quality assessment, data extraction and statistical analysis

Quality assessment of included papers was assessed using QUADAS (quality assessment of studies of diagnostic accuracy included in systematic reviews) and the risk of bias table in Review Manager 5 software from the Cochrane collaboration [14, 15]. A summary of the quality of included papers is presented in Figure 3. Quality assessment was performed independently by two investigators (RO an FO'R) and any disagreements were resolved by discussion with a third investigator (KO'B).

Figure 3
figure 3

Summary of quality assessment of included studies.

Full size image

Diagnostic accuracy of the Alvarado score

For the diagnostic accuracy (discrimination performance) of the Alvarado score, data were extracted and 2 × 2 tables constructed for use of the score as a criterion for admission (score 1 to 4 versus score 5 to 10, Figure 1) and as a criterion for surgery (score 7 to 10 versus score 1 to 6, Figure 1). Data extraction was carried out independently by two reviewers (RO and FO'R) and the data compared. A bivariate random-effects model was used to compute summary diagnostic sensitivity and specificity which allowed for heterogeneity beyond chance as a result of clinical and methodological differences between the studies to be taken into account. Heterogeneity was assessed using the variance of logit transformed sensitivity and specificity, where smaller values indicate less heterogeneity across studies. HSROC (hierarchical summary receiver operating characteristic) curves were also constructed with 95% confidence regions illustrating the precision with which pooled values are estimated and a 95% prediction region, illustrating the amount of between-study variation. Analyses were carried out using STATA software (StataCorp LP, College Station, TX, 77845, USA), using the "metandi" command [16, 17].

Calibration analysis of the Alvarado score

The initial derivation study of the Alvarado score was used as the predictive model against which all validation studies were compared [5]. The number of patients diagnosed with appendicitis as estimated by the Alvarado score (predicted events) was compared to the actual number of patients with appendicitis (observed events) in each of the validation studies. The analysis was performed separately across three risk strata of the Alvarado score (low risk, score 1 to 4; intermediate risk, score 5 to 6; and high risk, score 7 to 10) (Figure 1). Within each risk stratum, each of the three main study populations, men, women and children were analysed separately [8, 10–12, 18].

The results from the calibration assessment were presented as risk ratios (RRs with 95% confidence intervals) and are illustrated as forest plots. RR < 1.00 indicates an under-prediction of appendicitis by the score (observed number with appendicitis is greater than the predicted number) and RR > 1.00 indicates an over-prediction of appendicitis by the score (observed number with appendicitis is less than the predicted number). RR = 1 indicates a matched calibration between observed and predicted numbers. Review Manager 5 software from the Cochrane collaboration was used to perform the pooled analysis, determine heterogeneity and produce the forest plots. RRs with their 95% CIs were computed by the Mantel-Haenszel (M-H) method. A random-effects model was used and heterogeneity assessed by I 2 statistic.

Prevalence was investigated as a source of heterogeneity in a subgroup analysis. Studies were dichotomised, based on their prevalence, as being either higher or lower than the Alvarado's derivation study; the effect on heterogeneity and the calibration of the score were also investigated.

Results

The literature search yielded > 3,000 titles and abstracts for screening. The full text of 91 articles met the eligibility criteria, and these articles were retrieved (Figure 2). Thirty-seven articles were included from the search, and a further five articles were retrieved after citation searching, with a total of 42 articles meeting the inclusion criteria. The included studies came from a variety of settings and countries (Table 1). Nine studies took place in a surgical ward; three studies only specified that patients were hospitalised, all remaining studies were performed in an emergency department setting. Detailed characteristics of all included studies are presented in Table 1.

Table 1 Characteristics of included studies
Full size table

Results of the quality assessment are shown in Figure 3. The overall quality of the included studies is considered acceptable for most of the quality items. The assessment of the clinical variables composing the Alvarado score and the reference standard for diagnosis (histological results of the appendectomy) were interpreted independently in most studies. The retrospective studies rarely reported if the scorer was aware of the final diagnosis (blind assessment). The quality item, 'time between tests', is the time between administering the Alvarado score and verifying the diagnosis with pathology or follow-up and was very poorly reported. As part of our inclusion criteria, all studies had to confirm the diagnosis of appendicitis on those undergoing appendectomy; however, follow-up of those discharged was poor in the majority of studies (item 'All verified with reference test').

Diagnostic accuracy of the Alvarado score

The Alvarado score discriminated well as an observation/admission criterion (cut point of 5) by achieving high pooled sensitivity of 99% overall (n = 28 studies, [5, 8, 10, 18–42]) and in studies where data were available, it also performed well in the subgroup analysis for men, woman and children (pooled sensitivities: 0.96 for men, n = 5 [23, 30, 33–35]; 0.99 for women, n = 5 [23, 30, 34, 35, 43] and 0.99 for children, n = 9 [10, 21, 23, 27, 28, 30, 40–42]) (Table 2 and Additional file 1 - Figure S1). In patients presenting with higher Alvarado scores (cut point of 7, the criterion for surgery), pooled diagnostic accuracy results had more limited clinical value (pooled specificity for all studies 0.82, n = 29, [5, 8, 10, 11, 18–25, 27–32, 34–38, 41, 42, 44–47]), with pooled specificities ranging from 0.57 for subgroup analysis of men (n = 6, [9, 23, 30, 34, 35, 45]), 0.73 for subgroup analysis of women (n = 7, [9, 23, 30, 33–35, 45]) and 0.76 for subgroup analysis of children (n = 9, [10, 21, 23, 27, 28, 30, 41, 42, 47]) (Table 2 and Additional file 1 - Figure S1).

Table 2 Summary estimates of sensitivity and specificity calculated by a bivariate random-effects model
Full size table

Overall, heterogeneity was high when all studies were included and was particularly high in the children subgroup as indicated by the variance logit transformed sensitivity and specificity (Table 2) and the prediction ellipses on the SROC curves Additional file 1 - Figure S1).

Calibration of the Alvarado score

The Alvarado score performed well in all three risk strata for men: (low risk RR 1.06, 95% CI 0.87 to 1.28; intermediate risk 1.09, 0.86 to 1.37 and high risk 1.02, 0.97 to 1.08). In women, there was a systematic over-prediction across all risk strata: low risk (RR 5.35, 2.17 to 13.19), intermediate risk (RR 1.82, 1.20 to 2.78) and high risk (RR 1.14, 1.04 to 1.25). In children, there was a non-significant trend towards over-prediction in the low risk strata (5.03, 0.52 to 48.82) and a significant over-prediction in the intermediate risk category (1.81, 1.13 to 2.89) and high risk strata (1.13, 1.01 to 1.27) (Figures 4, 5, 6). Heterogeneity in terms of between-study predicted/observed risk ratio estimates is apparent in children across all risk strata and in women at a high risk (I 2 > 50%), and, therefore, these pooled estimates should be treated with caution.

Figure 4
figure 4

Low risk group (1 to 4): predicted versus observed cases with appendicitis in children, women and men.

Full size image
Figure 5
figure 5

Intermediate risk group (5 to 6): predicted versus observed cases with appendicitis in children, women and men.

Full size image
Figure 6
figure 6

High risk group (7 to 10): predicted versus observed cases with appendicitis in children, women and men.

Full size image

In a subgroup analysis based on prevalence (Additional file 1 - Figure S2), the high prevalence category consisted of six studies [9, 10, 23, 32, 37, 39] - the score predicted well in this group and heterogeneity was below 50% in the high and low risk groups (low risk RR 0.65, 95% CI 0.25 to 1.75, I 2 = 34%; intermediate risk RR 0.99, 95% CI 0.70 to 1.40, I 2 = 72%; high risk RR 0.99, 95% CI 0.96 to 1.02, I 2 = 0%). The low prevalence subgroup consisted of 24 studies, there was a significant overprediction across all risk strata; however, heterogeneity was extremely high (I 2 = 78% to 85%) suggesting that other factors, perhaps age and gender, contributed to the high levels of heterogeneity in this group. Unfortunately, not enough studies had age and gender information to allow us to do further subgroup analysis.

Discussion

Principal findings

This systematic review shows that the Alvarado score at the cut point of 5 performs well as a "rule out" CPR in all patient groups with suspected appendicitis. Pooled diagnostic accuracy in terms of "ruling in" appendicitis at a cut-point of seven points is not sufficiently specific in any patient group to proceed directly to surgery. In terms of calibration, the observed, predicted estimates in men suggest the score is well calibrated across all risk strata. Application of the Alvarado score in women over-predicts the probability of appendicitis across all strata of risk and should be used with caution. The validity of the Alvarado score in children was inconclusive; the calibration analysis showed high levels of heterogeneity across all risk strata. Further validation studies are required before clinical implementation of the Alvarado score for this age group could be recommended.

Clinical implications

A recent clinical policy document from the American College of Emergency Physicians reviewed the value of using clinical findings to guide decision making in acute appendicitis [7]. They state that combining various signs and symptoms, as in the Alvarado score, may be more useful in predicting the presence or absence of appendicitis. This systematic review supports the use of the Alvarado score as a triage CPR that can be applied to 'rule out' appendicitis at a score below five points (sensitivity 94% to 99%), but not as a 'rule in' for appendicitis. Patients with a score less than 5 can be considered for discharge with the proviso that watchful waiting and re-assessment may be required if symptoms change or deteriorate. The advantage of applying the Alvarado score in this way is that resources in terms of admitting a patient to hospital or performing diagnostic imaging can be reserved for higher-risk scoring patients. Such an approach may be particularly useful in low-resource settings where diagnostic testing is limited or not available [38].

Based on the results of this review, the Alvarado score at a cut-off of five points compares favourably with other CPRs used in clinical practice. The Ottawa ankle and knee rules represent "rule out" CPRs of similarly high sensitivity that are used in emergency departments to decide if a patient should be referred for radiography to determine if their ankle or knee is fractured. The application of these CPRs is to identify those patients with a very low risk of fracture, where fracture can be confidently ruled out and the patient can be discharged without unnecessary imaging. For this purpose, it is important that such CPRs have high sensitivity. Meta-analysis of validation studies show these rules achieve high sensitivity that is comparable to the Alvarado score at a cut-off of five points (ankle rule - 97.6% [48], knee rule - 98.5% [49] and Alvarado score at cut-off of five points - 99%).

The use of the Alvarado score as a 'rule in' CPR for surgery at a cut point of 7 is not supported by our diagnostic test accuracy results. Our analysis indicates that the Alvarado score has moderate to high sensitivity (all studies 82%, men 88%, women 86% and children 87%) and a moderate specificity (all studies 81%, men 57%, women 73% and children 76%), suggesting it is not sufficiently accurate to rule in or rule out surgery (Table 2). However, several studies report that the application of Alvarado score as a sole decision criterion for surgery (cut point of 7) produces negative appendectomy rates of 13.3%, 15.6%, 16.2% and 14.3%, respectively, without an increase in perforations [11, 20, 29, 35]. This is comparable with a clinician's judgment in other reports (17.1%, 12%, 12.5% and 11%) [5, 8, 19, 27]. An Alvarado score ≥ 7 is useful at identifying those at high risk of acute appendicitis who require a surgical consultation or further diagnostic imaging, it should not be used as the sole criterion for ruling in surgery in any patient group.

During the last 10 years, the diagnostic imaging by CT scan in the diagnosis of appendicitis has become a common practice. In some centres over 90% of the patients presenting with suspected appendicitis undergo CT imaging. CT has a high sensitivity and specificity for the diagnosis of appendicitis and it considerably reduces the level of negative appendectomy. However, some studies have shown that the use of CT does not necessarily change the clinical management of a patient, especially in those at high risk [33, 50]. CT imaging may also delay the time of operation and, therefore, may increase the subsequent risk of perforation [51]. Assessing the use of the Alvarado score and CT imaging as a series of diagnostic investigations on all these types of outcomes is warranted.

Lastly, the results of this systematic review have important implications in low-resource countries. First, in low-resource settings where the decision to operate may be based on a clinical judgment, the Alvarado score provides an accurate and consistent triage tool for ruling out appendicitis and identifying those at higher risk who would benefit at most from an admission to a hospital. Second, the Alvarado score could serve as a simplified tool for the emergency physician in order to stratify patients for referral for surgical consultation.

Context of other research

Although the Alvarado score was developed in a mixed gender population, the ratio of men to woman was 1.4:1 and the score has subsequently been shown to perform poorly when applied to women of child-bearing age [8–11]. It is also possible that a certain loss of diagnostic information may have occurred due to dichotomisation when the score was originally constructed in the derivation study. Abdominal pain in women is a diagnostically challenging symptom as there are more diagnostic possibilities aside from appendicitis, such as pelvic inflammatory disease and other gynaecological pathologies. Alternative risk scores or CPRs, such as Lindeberg [52], Eskelinen [53] and Fenyo [54] scores for appendicitis, have different numerical values for symptoms depending on whether the patient is male or female [55]. The Van Way, Teicher and Arnbjornssion scores include gender as one of their components [55]. Of note, Ohman et al. [55] reported that the Alvarado score outperformed each of these other scores.

Distinguishing appendicitis from other causes of abdominal pain in children is also challenging, particularly in young children who cannot articulate how they feel or where the pain is. There is also a wide variation in presenting symptoms and it is often hard to elicit the classical presentation [2]. The use of symptoms and signs to identify children who are at risk of acute appendicitis is particularly appealing as diagnostic imaging using a CT scan exposes children to ionizing radiation and the diagnostic accuracy of ultrasound is still uncertain [7]. A recent review found that "fever" in a child is the single most important sign associated with appendicitis, followed by rebound tenderness and migration of pain, suggesting that the Alvarado score may not be the most appropriate scoring system for children as double points are scored for tenderness in the right lower quadrant and leukocytosis, but only one point for each of all other signs (Figure 1) [2]. This review also reported the accuracy of clinical scoring systems, including the Alvarado score, where the likelihood ratio for cut points of 7 and 5 (based on three studies) was similar to our pooled estimates (cut point of 7, 3.1 and 3.5; cut point of 5, 0.05 and 0.02, respectively). Two of the studies in the review by Bundy et al. were included in this review [11, 21].

Strengths and weaknesses of the present study

Our study does have a number of limitations. First, although it is usually related mainly to discrimination, some degree of misclassification may have also occurred when calibration was considered by comparing predicted versus observed patients with appendicitis; however, given the high levels of diagnostic performance, overall (especially, at the cut-off point of 5) this appears unlikely.

Second, a moderate to high level of heterogeneity was shown across the included studies in both the diagnostic test accuracy analysis and the calibration analysis (Table 2, Figures 4, 5, 6 and Additional file 1 - Figure S1). There are a number of possible sources for heterogeneity, including chance; variation in pre-test probability; the case mix of men, woman and children; a threshold effect caused by observer variation in the measurement of signs and symptoms; no active follow-up of patients discharged and other unanticipated factors. We addressed a number of these potential sources of heterogeneity by performing subgroup analysis. The main focus of this paper was an analysis of such subgroups as men, woman and children. The performance of the score has been shown by others [8–12] to be affected by age and gender and, therefore, high heterogeneity in the overall results may be due to the gender and age spectrum of the patients in the included studies (for example, Table 2, all studies, variance logit sensitivity is 3.37). The prevalence of appendicitis among the validation studies was highly variable (range 32% to 91%, Table 1). Although this was investigated in a subgroup analysis a good deal of heterogeneity still existed, suggesting that other factors contributed towards heterogeneity in this analysis (Additional file 1 - Figure S2). Unfortunately it was not possible to do further subgroup analysis based on age, gender and prevalence due to a lack of studies with this information. Finally, a number of the studies used no repeat admission as a negative proxy measure for appendicitis. The lack of active follow-up in these studies may have led to misclassification if patients presented to a different hospital. This may have led to a lower reporting of appendicitis cases, particularly in the low-risk groups, and inflated our estimates of sensitivity and specificity.

Finally, although we used an up-to-date systematic search strategy, we acknowledge that it was not exhaustive and it is possible, as with all systematic reviews, that relevant articles may have been missed. As we did not search the grey literature, there is also the possibility of publication bias, with smaller negative studies being under reported, leading to inflated estimates of sensitivity and specificity in our meta-analysis.

Future research and applications in clinical practice

The criteria for selection of the included articles were broad and reflected the nature of the validation studies themselves, producing a high level of heterogeneity across the studies in some of the risk strata. Further analyses are needed to explore the reasons behind the over-prediction of the Alvarado score in women. Such future analyses may suggest ways to adjust the predicted estimates according to the population prevalence in the various settings and/or a re-calibration or re-modelling of the score itself, mainly in low-prevalence settings and in women. Obtaining individual-level data from the validation studies to perform meta-analysis of the risk ratios can make such approaches possible, particularly for the more detailed exploration of the various sources of heterogeneity.

Conclusions

This study shows that the Alvarado score accurately predicts appendicitis and is well calibrated in men. As a decision rule for observation/admission, the Alvarado score performs well as a 'rule out' criterion (high sensitivity). As a decision rule in relation to surgery the Alvarado score cannot be used to 'rule in' a diagnosis of appendicitis without surgical assessment and further diagnostic testing. Patients presenting in the emergency department and in primary care settings, especially in low-resource countries, could benefit from the implementation of the Alvarado score as a triage decision rule.