Background

Sepsis is defined as the host response to infection and it is an important cause of morbidity and mortality around the world [1, 2]. The surviving sepsis campaign issued a call for global action against sepsis and pointed out diagnosis as a fundamental challenge [3, 4]. In early stages of the process, the source of infection may be unclear and the related systemic response indistinguishable of no-infectious diseases. Consequently, clinicians often miss or delay this diagnosis. This is especially worrying; since there is strong evidence supporting that early treatment is associated with greater clinical success [5]. An ideal “Gold Standard” is not available for sepsis diagnosis, as microbiology is not enough sensitive and laboratory tests are not specific for using as reference standards. The lack of any reference standard has been overcome by using techniques that avoid the need for comparison with a single accurate test. These techniques can be broadly divided into latent class analysis (LCA) and Bayesian analysis [6]. LCA has been used widely in psychiatry as well as other disciplines [7–11] but, it has not been yet applied to the evaluation of sepsis.

On the other hand, sepsis is associated with the simultaneous activation of the inflammatory and coagulation cascades, and most of their components are markers or mediators in the host response [12, 13]. From this close interplay between inflammation and coagulation, which is a recognized way toward organ dysfunction and mortality [14], emerges the rationale to characterize the host response to infection. Three potential biomarkers have shown regular presence in systemic infections: C-reactive protein (CRP), procalcitonin (PCT), and D-dimer (DD); the latter as an unspecific signal of coagulation activation [15–19]. So far, however, no large prospective studies support any of them as a single independent criterion for sepsis. We aimed to estimate the diagnostic accuracy of these three biomarkers as diagnostic tests for sepsis, with the application of the latent class analysis, in patients at the ER admittance with a presumptive infection as main diagnosis.

Methods

Prospective single center study on the diagnostic accuracy of a test. The study protocol and a pre-specified nested analysis were previously published [20, 21].

Setting

Emergency Room (ER) at the “Hospital Universitario San Vicente de Paúl” (Medellín, Colombia). This is a 550-bed, fourth level University Hospital with an admission rate of approximately 1800 patients per month through the ER and is a reference institution for a region including approximately 3 million habitants.

Subjects

Inclusion criteria: 1. Patients hospitalized in the ER within 24 hours before admission to the study. 2. Aged 18 years or older. 3. At least one of the following causes as the main admission diagnosis to the hospital: a) any kind of infectious disease (confirmed or suspected), b) fever of unknown origin, c) delirium or any kind of encephalopathy of unknown origin or d) acute hypotension not explained by hemorrhage, myocardial infarction, stroke or heart failure. We selected these relatively broad criteria according with the last consensus conference on sepsis definitions [22].

Exclusion criteria: 1. Negative of the patients, their families, or the attending physician to be part of the study. 2. Antimicrobial treatment received at another medical institution immediately before admission to the study. 3. Medical decision to treat the patient ambulatory or in a different institution within 24 hours after admission. 4. Homeless or inability of the patient to follow up. 5. Previous participation in the same study.

Recruitment and data collection

We obtained approval for the study from the ethics committee of the Medical Research Centre (University of Antioquia) and the recruited patients provided informed consent. Three physicians (FJ, GDLR, or MLV) and two trained nurses recruited patients by checking admission lists and clinical records and collected data daily from Monday to Saturday of each week. The general protocol for each patient was [20]: collection of baseline clinical data, calculation of entrance Sepsis-related Organ Failure Assessment (SOFA) score [23] and Acute Physiology and Chronic Health Evaluation (APACHE II) score [24] and blood sampling, all of these procedures performed within the first 24 hours of ER admission. During the first 7 days of hospital stay, additionally, the patients were monitored with daily recording of any relevant data registered in medical or nurse records, using a standardized case report form.

Study tests

CRP, PCT and DD were measured in all patients twice: at admission to the study and on the next day morning (i.e., within 24 hours after the first sample). Serum samples for PCT and CRP were collected in a dry tube with gel separator and centrifuged within the first 2 hours. PCT concentrations were measured by an immunoluminometric assay (VIDAS® B•R•A•H•M•S PCT, Biomeriux, France). CRP was measured quantitatively by an immunoturbidimetric assay using an ARCHITECT® c-System (Abbott Laboratories®, USA). Samples for DD were collected in a tube containing citrate as anticoagulant and processed within 2 hours. DD (ng/ml) was measured by a turbidimetric immunoassay in an ACL Elite® coagulometer using a Hemosil™ kit (Instrumentation Laboratory, MA, USA). All previous assays were conducted at the hospital laboratory by trained personnel, under the institution technical standards, and who had no knowledge of the clinical status of the patients, nor the study objectives.

Gold standards

Clinical gold standard: we used an expert consensus based on clinical, microbiologic, laboratory, and radiologic data collected for each patient during the first 7 days of hospitalization. The experts also took into account the definitions stated in 2001 at the International Sepsis Definitions Conference [22] as well as the Centers for Disease Control and Prevention (CDC) definitions for infection [25]. The consensus was formed by a panel of three physicians with certified training and expertise in intensive care (AG), internal medicine (CMA), and infectious diseases (CIG). First, each physician established a diagnosis individually, in which they agreed on 65% of the cases. The remaining 35% of the patients were fully discussed to determine a final diagnosis. All the experts were blinded to the results of CRP, PCT and DD. The consensus classified the admitted patients into no infected, infected without sepsis and sepsis groups.

Likelihood of sepsis in the study population according to a LCA: this analysis postulates the existence of an unobserved categorical variable that divides the population of interest into classes. Members of the population with a set of observed variables will respond differently depending on the latent class (variable) to which they belong. The problem that the outcome of interest cannot be measured directly occurs in many research situations. Examples include constructs such as intelligence, personality traits or, as in our case, the true sepsis diagnosis. These unobservable outcomes, named also latent variables, can only be measured indirectly by eliciting responses that are related to the construct of interest. These measurable responses are called indicators or manifest variables. Latent variable models are a group of methods that use the information from the manifest variables to identify subtypes of cases defined by the latent variable. The classification appears by modeling the relationship between manifest (CRP, PCT and DD) and latent (sepsis/ no sepsis) variables in such a way that the parameters of interest (prevalence, sensitivity, specificity) are estimable from the implied relations between observable variables. In other words, LCA is just a mathematical model that identifies a subtype or a cluster of observations according to certain defined characteristics or variables that are common to those observations. In this case, we know that different expressions of inflammation and coagulation are common responses in the process of infection. Therefore, we provided these observed variables (DD, PCT and CRP) from all the study population to the model and it is able to uncover the hidden group, i.e. the latent variable, to which the patients belong. In summary, the goal of latent class analysis is to use the observed probabilities to estimate the unobserved ones.

Sample size and analysis plan

The number of the patients with the disease (ND) that is needed to give a sensitivity of 95%, with a 95% CI +/− 3%, is calculated with the following formula [26]:

ND = Z 2 α 2 × sensitivity × 1 ‒ sensitivity 0 , 03 × 2 2
ND = 1 , 96 2 × 0 , 95 × 0 , 05 0 , 06 2 = 203

The ND is also determined by the prevalence (P) of the disease. Hence, the total of patients (TP) required is:

TP = ND P

We expected a prevalence of sepsis of 30% [2, 21], and the sample size would be 700 participants.

Clinical gold standard: the cut points for the study tests were determined using receiver operative characteristics (ROC) curves [27], searching for the best sensitivity and specificity. The method based in the Bayes’ theorem was used to determine the operating characteristics of the tests. Additional analyses were done using changes of the values in the first 24 hours (Δ24) for each test and combining pairs of tests (PCT/DD and CRP/DD). For a Δ24 test, it was considered positive in a patient if her values remain without changes or increase. For combining pair of tests, it was considered positive if both biomarkers were above the cut point. Furthermore, as a sensitivity analysis, two alternative reference standards for sepsis patients were considered: only those who had any microbiologically confirmed infection and only those who were diagnosed as sepsis patients independently by one of the experts (65% among the total population). Patients with missing values were excluded for the corresponding analysis, and results are shown with exact 95% CI using STATA SE (Version 10, Stata Corp, College Station, TX).

LCA: it assumes that results from the three tests in the same subject are independent within the real condition of illness [7]. In other words, if the effect to belong to a latent condition of sepsis would be removed, the effects to the CRP, PCT and DD would have a completely random distribution in the study population. Since that both PCT and CRP values are probably a common expression of the same inflammatory process, we controlled this local independence assumption introducing a random effect through a continuous latent variable [20]. The maximum likelihood estimators of prevalence (the “cluster” of sepsis patients), as well as of sensitivity and specificity of each test if requested, are obtained with an integral that uses an EM iteration algorithm. Analyses were carried out with LATENTGOLD 4.0 (Statistical Innovations, Belmont, MA, USA).

Results

Enrollment began on August 2007 and concluded on February 2009. A total of 1,795 patients were eligible and 1,030 were excluded, most of them because of more than 24 hours of hospitalization before recruitment and refusal to participate (Figure 1). Among 765 patients included, 683 (89%) had a suspected infection as admission diagnosis, 56 (7%) fever of unknown cause, 20 (3%) delirium or encephalopathy of unknown origin, and 6 (1%) unexplained hypotension. There were 377 males (49%), the mean age was 51.4 years (SD = 20), and the median time of symptoms before consultation was 72 hours (IQR = 24 to192 hours). There was no comorbidity in 307 (40%) of the participants, and the most frequent previous diseases were diabetes mellitus (n = 146, 19%), chronic obstructive pulmonary disease (n = 94, 12%), chronic renal failure (n = 88, 11%), use of corticosteroids or chemotherapy during the past 3 months (n = 70, 9%) and trauma or surgery in the previous month (n = 53, 7%). As suspected sources of infection, the most frequent were respiratory (n = 179, 23%) and skin and soft tissues (n = 174, 23%), followed by urinary tract (n = 127, 17%), intrabdominal (n = 93, 12%), undetermined (n = 94, 12%) and others (n = 96, 13%). The median SOFA and APACHE II score were 2 (IQR = 1-4) and 9 (IQR = 5-14), respectively, hospital length of stay was 9 days (IQR = 5 to 17 days), ICU admission was required in 66 patients (9%) and the overall 28-day mortality rate was 12% (n = 91). Due to logistic or technical reasons CRP, DD and PCT at admission were measured in748 (98%), 744 (97%) and 747 (98%) patients, respectively. The median and IQR values for these test were CRP = 9.4 mg/dl (3.5-20), DD = 1673 ng/ml (982–1841) and PCT = 0.4 ng/ml (0.1-3.65).

Figure 1
figure 1

Flow chart of recruitment, patients’ classification by expert consensus and biomarker results. The values of biomarkers are presented as number of samples measured, median and interquartile range (IQR). CRP = C-reactive protein (mg/dL), DD = D-dimer (ng/ml), PCT = Procalcitonin (ng/ml), Δ24 = Measurement 2 - Measurement 1.

Full size image

According to the expert committee 505 patients (66%) were classified with sepsis, 112 (15%) with infection but without sepsis and 148 (19%) without infection. The kappa-statistic measure for multi-rater agreement between experts was 0.65 for sepsis-no sepsis and 0.73 for infection with and without sepsis. Figure 1 shows this classification and their respective biomarker values, and Table 1 shows the main characteristics by group. Among infected patients (with and without sepsis, n = 617) a microbiologic diagnosis was confirmed in 104 patients (17%) by blood culture, in 135 (22%) by urinary culture and in 145 (26%) by other samples. Microorganisms isolated in blood were E. coli (n = 29, 28%), S. aureus (n = 17, 16%), coagulase-negative staphylococci (n = 15, 14%) and others (n = 43, 42%); and in urine were E. coli (n = 79, 59%), K. pneumoniae (n = 22, 16%) and others (n = 34, 25%). The main admission diagnosis in the sepsis group (n = 505) was community-acquired pneumonia (n = 115, 23%), followed by urinary tract infection (n = 87, 17%) and soft tissue infection (n = 79, 16%).The main alternative diagnosis in 148 patients without infection were cancer (n = 22,15%), chronic obstructive pulmonary disease (n = 15, 10%), acute pulmonary edema (n = 13, 9%), metabolic diseases (n = 13, 9%), biliary diseases (n = 10, 7%), and others (n = 75, 50%).

Table 1 Clinical characteristics at admission according to the groups defined by the expert consensus
Full size table

According to the ROC curve analysis, the cut-offs points with the best sensitivity and specificity for CRP, DD, and PCT to discriminate at admission between sepsis and not sepsis (infection without sepsis or not infection) were 7.8 mg/dl, 1616 ng/ml, and 0.3 ng/ml, respectively. Their operating characteristics, at both measurement times, are shown in Table 2. Analyses combining pairs of tests or using changes in the first 24 hours (Δ24) did not show any improvement in diagnostic accuracy. Similar results were seen using the alternative reference standards (data no shown).

Table 2 Diagnostic accuracy of CRP, DD and PCT for sepsis diagnosis at admission in the ER according to expert consensus
Full size table

The LCA analysis with the same three tests identified a “cluster” of 187 patients, among those defined as sepsis by the expert committee, with several characteristics suggesting a more severe condition as well as better microbiological confirmation compared to the rest of the study population. According to standard definitions, 70% (n = 131) of these patients had severe sepsis without circulatory failure and 5% (n = 9) had septic shock (Table 3) (Additional file 1). It was not possible to classify 46 patients because of missing values in any of CRP, DD or PCT. Assuming this cluster of 187 patients as the new prevalence of sepsis based on the LCA gold standard, the ROC curve analysis identified new cut-off points for the tests and suggesting a better discriminatory ability for PCT with a value of 2 ng/ml (Table 4).

Table 3 Clinical characteristics at admission according to the LCA classification in clusters
Full size table
Table 4 Diagnostic accuracy of CRP, DD and PCT for sepsis diagnosis according to the LCA gold standard
Full size table

Discussion

Our results suggest that, under a “classical” definition of sepsis, three typical biomarkers (CRP, PCT and DD) are not capable enough to differentiate septic from non-septic patients in the ER. Indeed, the kappa-statistic measure for multi-rater agreement between experts for this definition was 0.65 for sepsis-no sepsis and 0.73 for infection with and without sepsis, which underlines the limitations for clinical diagnosis in this condition. Using another analytic approach, however, a higher cut-off point for PCT (2 ng/ml) is able to identify and to exclude a specific population more severely ill and with better microbiological confirmation. To the best of our knowledge, this is the first research that incorporates the novel concept of a latent class to the process of diagnosis in sepsis.

The performance of a diagnostic test is judged by how accurately the test result can identify a diseased or no diseased person. The true disease status is the “gold standard” against which a test should be compared. However, there are many conditions for which the definitive diagnosis is very difficult or expensive to establish. This is especially true for the diagnosis of a complex clinical condition as sepsis, in which even within the construct of “systemic response to infection” there is not a real “gold standard” against which the diagnostic criteria can be calibrated [22, 28, 29]. Psychological and social sciences have a long tradition in coping with primary study objects that are not directly observable. Constructs such as intelligence, fear or trust can only be measured indirectly. Inference proceeds by modeling the relationship between observable and latent variables in such a way that the parameters of interest are estimable from the implied relations between observable variables. When the unobservable variable is categorical, the term latent class analysis (LCA) applies [6, 7]. In other words, LCA postulates the existence of an unobserved categorical variable that divides the population of interest in to classes. Members of the population with a set of observed variables will respond differently depending on the latent class to which they belong. This technique can be applied to the problems related to diagnostic testing, with the unobserved categorical variable being “disease present” or “disease absent” [20].

Given the established interplay between inflammation and coagulation in sepsis [14, 30–32], it is reasonable to characterize the host response to infection as a potential diagnosis tool on the basis of three recognized markers of these two cascades. The sensible mathematical model of the latent diagnostic classification, using individuals’ values of CRP, DD and PCT, was able to identify a subset of patients attended in the ER with suspicion of infection and with clear differences in clinical status, microbiological profile and 28-day mortality. Although this subset was identified among those patients classified as sepsis by the expert committee, there is not a unique clear cut-off in any variable or test that may define the cluster specifically as severe or bacteremic sepsis (Table 3). Furthermore, among these three potential biomarkers, PCT proved to be the most contributor to the “new” standard of more severe disease but with a higher cut-off point than that usually suggested. Our main result, consequently, is that PCT is useful to identify a subgroup of more severely ill septic patients attending the ER. Such a finding was previously reported by Hausfater P et al. [33], whom studied 243 patients with body temperature of 38.5°C or greater attended in the adult emergency department of an academic tertiary care hospital. They found, using standard statistical methods, that PCT is an independent variable that can predict whether a febrile episode has a bacterial origin, and that at a threshold of 2 μg/l it is independently associated with critical illness. The coincidence with our findings is remarkable, despite the fact that their study population was extremely different: all the patients consulted by a febrile episode, 29% of them were immunocompromised, and only 81% were hospitalized in that consultation.

Needless to say, sepsis is not an illness but a syndrome suspected mainly on clinical criteria, and the misdiagnosis of sepsis is associated with an extremely adverse outcome. Consequently, we are not proposing a new methodological approach for sepsis diagnosis. Instead, we are identifying a new cut-off point for procalcitonin to be able to detect more severely ill patients. This goal was not achieved by the conventional clinical gold standard with expert consensus and, in this way, LCA is just an instrument to show that we can improve the process of sepsis diagnosis in the emergency room. The recent literature is full of studies evaluating PCT for sepsis diagnosis [16, 17, 34, 35], but in the setting of the ER there are less investigations testing its potential usefulness. In a secondary care hospital of Finland, a population of 539 patients admitted to the ER with suspicion of infection and with clinician’s order for blood cultures was studied [16]. In assessing how the parameters differentiated all sepsis patients (n = 358) from patients with no sepsis (n = 181), AUC-ROC for PCT was 0.73 (95% CI 0.69 – 0.78), and PCT emerged as the best marker for severe sepsis with an AUC-ROC = 0.77 (95% CI 0.71 – 0.84). Riedel et al. evaluated the usefulness of PCT as a diagnostic predictive marker of bacteremia and sepsis in 259 patients who had blood cultures obtained in the ER of a tertiary medical center in Baltimore [35]. In16 patients there was evidence of bacteremia and 12(75%) patients had a PCT level of more than 0.1 ng/ml. The PCT cut-off value that maximizes the AUC-ROC (0.79) was 0.1475 ng/ml., but with sensitivity just in 75% and specificity of 79.8%. In a recent meta-analysis [36], Wacker C. et al. analyzed 30 reports, although only two from ER, accounting for 3244 patients. Bivariate analysis yielded a mean sensitivity of 0.77 (95% CI = 0.72–0.81) and specificity of 0.79 (95% CI = 0.74–0.84) and the area under the receiver operating characteristic curve was 0.85 (95% CI = 0.81–0.88). The median cut-off for PCT of the studies included was 1.1 ng/ml (IQR = 0.5–2.0) and the absence of a threshold effect suggests that a cut-off between 1.0 and 2.0 ng/ml, close to our findings, is helpful for discrimination of patients with sepsis from other inflammatory conditions. However, the studies had substantial heterogeneity (I 2 = 96%, 95% CI = 94–99) and none of the subgroups investigated like population, admission category, assay used, severity of disease, and description and masking of the reference standard, could account for that heterogeneity. They concluded that the test may be helpful for diagnosis of sepsis in critically ill patients, but it must be interpreted in context with information from careful medical history, physical examination and microbiological assessment.

Our study has several limitations. First, this is a single center study in a specific geographic location with some particularities from an epidemiological point of view [2, 37], which are obstacles for external validity. Moreover, the pre-test probability of sepsis should be significantly different in patients admitted to the ER and in patients admitted to ICU, even with the same clinical suspicion of bacterial infection, and this is an acknowledged consideration in the use and interpretation of any diagnostic test. As we mentioned before, the clinical diagnostic “gold standard” utilized here performed poorly, as the concordance between experts was 0.65 for sepsis-no sepsis and 0.73 for infection with and without sepsis. This weakness, indeed, underlines the limitations for clinical diagnosis in this condition. On the other hand, LCA also has its limitations as “gold” standard. Under this approach, sepsis is not formally defined but rather is a mathematically defined entity that does not necessarily correspond with a clinically relevant status. Additionally, LCA modeling requires sophisticated analytic techniques and software, and the full model or the hypothetical “true” state of disease cannot be fully tested with the observed data. Finally, although blood sampling was performed immediately after the patient was admitted to the study, he/she could be in the ER at any time within the last 24 hours before recruitment. This is important because biomarker’s kinetic, notably PCT, and their levels may varying considerably during 24 hours.

Conclusions

In summary, the “holy grail” of sepsis diagnosis is an evolving process and the fine exercise of clinical suspicion should be complemented by appropriate laboratory test. In this scenario, PCT emerges as an acceptable choice underscoring both microbiology and prognosis in selected patients. A higher level of PCT seems related more strongly with these two components of the infectious phenomena.

Key messages

  • 1CRP, PCT and DD are not capable enough to differentiate septic from non-septic patients in the ER.

  • A higher level of PCT seems related more strongly with microbiology and prognosis, and discriminates a selected group of patients with severe sepsis.

  • Determination of the PCT level may be useful for screening and prognosis of more-severely ill ER patients.