Background

Mortality from sepsis remains unacceptably high [1], compelling an ongoing search for novel disease mediators which may represent therapeutic targets. Metabolic alterations have represented an under-explored pathophysiologic axis in sepsis [2, 3]. Unbiased, high-dimensional molecular platforms for profiling hundreds of circulating metabolites in concert (metabolomics) [4, 5] have opened opportunities for data-driven, systems biology approaches to clinical risk prediction as well as the search for potential novel therapeutic targets in humans [6].

Although the application of metabolomics in critical care research has increased over recent years [2, 2,7,8,9], optimal statistical analytic approaches to such high-dimensional data remain uncertain, particularly when there is a large imbalance between the number of metabolites profiled relative to the number of clinical events. Conventional statistical methods such as logistic regression may have important limitations when applied to data with high degrees of internal correlation (including the metabolome), missingness, subclass heterogeneity, and imbalance between exposures (metabolites) and outcomes—frequent challenges in high-dimensional human biologic data in critically cohorts. Analytic approaches using machine learning (ML), a subset of artificial intelligence, may overcome some of these challenges [10]. Such approaches have recently been successfully applied to metabolomics data with a focus on building clinical prediction models [11]. However, less focus has been placed on how robustly and consistently ML could enhance biologic discovery through statistical mining of the metabolome to identify metabolites may show the strongest links with clinical outcomes.

In the present study, we hypothesized that parallel and ensemble ML methods could facilitate identification of individual metabolites potentially implicated in sepsis survivorship using gas chromatography–liquid chromatography (GC/LC) mass spectrometry profiling of the human metabolome. Our two complementary aims were: (1) biologically, to uncover metabolite signals associated with mortality in sepsis; and (2) methodologically, to determine to what extent metabolite selection and prioritization through ML provided consistent, robust identification of metabolites despite a small cohort size, and whether these methods enhanced metabolite–outcomes links beyond conventional logistic regression. This study extends prior analyses from this cohort [12, 13] which employed conventional statistical methods including logistic regression, now comparing observations from ML approaches with those from the conventional statistical approaches.

Methods

Study cohort

The study population consisted of 60 adult patients with sepsis—30 of whom also had acute respiratory distress syndrome (ARDS)—admitted to the Medical Intensive Care Unit (ICU) at Brigham and Women’s Hospital in Boston, MA, USA (Registry of Critical Illness; RoCI), as previously described. [12] The study was approved by a local institutional ethics board. All subjects provided informed consent. Patients were enrolled in the larger RoCI cohort study between September, 2008 and May, 2010. In a subset of 225 patients, interleukin (IL)-18 levels were analyzed in a separate study of inflammasome-regulated cytokines in acute lung injury​ [12]. As a follow-up to this study, RoCI patients were selected for metabolomic profiling on the basis of whether ARDS complicated sepsis or not, and whether IL-18 was elevated or not (sepsis patients with low IL-18 levels and ARDS with high IL-18 levels, were included). This initial metabolomics study identified pathway analytes of interest that were independently validated in a separate critical illness cohort (the CAPSOD cohort). [14]. No outcomes enrichment nor selection on the basis of metabolite features was undertaken. Herein, we set out to evaluate differential analytic approaches to this dataset.

Plasma metabolomics

Plasma was obtained from patients within 72 h of ICU admission. Gas and liquid chromatography and mass spectrometry was performed by Metabolon, Inc., as previously described [12]. Briefly, blood samples were collected in EDTA-coated blood collection tubes within 72 h of ICU admission and processed within 4 h after collection. The liquid chromatography/mass spectrometry (LC/MS) portion of the platform was performed on a Waters ACQUITY UPLC and a Thermo-Finnigan LTQ mass spectrometer, consisting of an electrospray ionization (ESI) source and linear ion-trap (LIT) mass analyzer. The gas chromatography/mass spectrometry (GC/MS) portion of the platform was performed on a Thermo-Finnigan Trace DSQ fast-scanning single-quadrupole mass spectrometer using electron impact ionization. Extracts were reconstituted with water and methanol. Data from 411 metabolites were available. Drug metabolites were excluded from this analysis.

Statistical methods

Metabolites were pre-processed, filtered, and imputed where necessary (Additional file 1: Methods). This included a log10 transformation. Metabolites with > 10% of values below the lower detection limit were excluded from the analysis. We fit 7 ML and logistic regression models to predict mortality and then extracted the individual metabolites which contributed to the most accurate prediction models, viewing this as a measure of association with death. We performed multiple diverse ML methods (random forests, support vector machines, random k-nearest neighbors, nearest shrunken centroids, adaptive bagging/boosting, and Lasso regression), conventional bilinear factor models (partial least squares-discriminant analysis [PLS-DA]), as well as penalized logistic regression, predicting mortality using the post-processed metabolomics data. Models were trained under the precision recall (PR) curve using 50 repeats of fivefold cross-validation. Training under the PR curve was undertaken given theoretical advantages in imbalanced datasets, including providing a better assessment of the performance of a classifier on mortality [15]. Model sensitivity was assessed. For comparison, under the PR curve, a no skill classifier (one that performs no better than classifying all patients as positive) has an area under the PR curve equal to the observed event rate in the study population (0.42).

In the primary analysis, innate feature selection and ranking tools of ML models were used to prioritize metabolites (Additional file 1: Methods). The outcome used for prioritization was prediction of 28-day mortality. The top metabolites contributing to successful predictive model generation were compared across each method. An ensemble ranking procedure hybridizing multiple ML methods run in series provided a summary ranking of metabolites [16], and the variables with ensemble importance scores ≥ 0.5 were identified. This method performs iterative and parallel ML steps, and ranks top features (metabolites) based on the strength and consistency with which they are linked with the outcome. For each fitted model, we reported the metabolites that were selected/ranked using each model’s implicit selection/ranking procedure. We assessed the consistency between these different sets of selected (ranked) metabolites. We also assessed the performance of each of these ML models. In view of obtaining a more robust final set of important metabolites, we aimed to utilize the information obtained by each of these models. Thus, we created a score that represented the percentage of times that a metabolite was selected/ranked highly across all ML models.

All analysis was conducted in R v3.3.2 using packages caret (6.0–79), earth (4.6.2), spls (2.2–2), klaR, randomForest (4.6–14), RWeka (0.4–38), fastAdaboost (1.0.0), adabag (4.2), plyr (1.8.4), sparseLDA (0.1–9), glmnet (2.0–16), Matrix (1.2–14), gbm (2.1.3), pamr (1.55), and kernlab (0.9–26).

Results

Patient cohort

Median (IQR) patient age was 58 (47, 62) years and 45% of patients were women (Table 1). Median (IQR) SOFA and APACHE II scores were 9 (6, 12) and 30 (23, 37), respectively. Mortality at 28 days was 42%. Patients who died had higher illness severity scores (SOFA, p = 0.004; APACHE II, p = 0.02), more frequently presented with acute respiratory failure (p = 0.03) and ARDS (p < 0.001, as well as had a prior history of malignancy (p < 0.001).

Table 1 Baseline patient demographics
Full size table

Metabolome in sepsis

A total of 158 metabolites passed quality control and pre-processing filters and were included in the analysis, representing 8 super-pathways (Additional file 1: Table S1). Overall differences in the metabolome were observed among survivors and non-survivors (Fig. 1). Twenty principal components were required to explain 80% of the variance in the metabolome (Additional file 1: Figs. S1 and S2).

Fig. 1
figure 1

Heatmap showing normalized metabolite levels grouped by super-pathway among patients with sepsis. Heatmap showing normalized metabolite levels (rows), grouped by super-pathway, among patients with sepsis following hierarchical clustering (dendrogram, top); survival status is annotated (grey = yes, black = no). Differences in the metabolome among survivors and non-survivors were noted across multiple metabolic super-pathways

Full size image

Model training and performance

Model specificity (for 28-day mortality) ranged from 0.619 to 0.821 (Additional file 1: Table S2). Specificity of survival status (that is, the ability of a model to correctly identify people who did not die) was highest for PLS-DA. Several metabolites were consistently prioritized by multiple ML methods (Fig. 2a, b), which are also separately categorized by metabolic super-pathway (Additional file 1: Fig. S3). PLS-DA and nearest shrunken centroids prioritized the greatest number of metabolites identified by at least one other method. Logistic regression and random forests selected a similar proportion of prioritized metabolites. Sparse linear and flexible discriminant analyses identified the fewest metabolite contributors to mortality prediction, although the metabolites ranked were frequently ranked by multiple other methods. Logistic regression models penalized for multiple hypothesis testing demonstrated consistent results with many ML methods. The metabolites ranked the most frequently by multiple models included lactate, bile acid metabolites (glycolithocholate sulfate and glycochenodeoxycholate) and amino acid metabolism (kynurenine [tryptophan metabolism], 3-hydroxyisobutyrate [valine metabolism], and phenylalanine) (Fig. 2c). Receiver operating characteristic (ROC) curves for the assessment of each ML model’s performance are included in Additional file 1: Fig. S4.

Fig. 2
figure 2

Comparison of top metabolites selected by each analysis method’s innate feature selection algorithm. Comparison of top metabolites selected by each analysis method’s innate feature selection algorithm, identifying metabolites that more meaningfully contribute to successful sepsis mortality prediction models. Such approaches may identify measures of association and individual metabolic links with mortality. Agreement was noted between the lists of top metabolites identified by several machine learning methods, which also overlapped with those identified by conventional panelized logistic regression. FDA flexible discriminant analysis, GBM generalized boosted regression models, LR logistic regression, NSC nearest shrunken centroids, PLS-DA partial least squares-discriminant analysis, sparse LDA sparse discriminant analysis

Full size image

Metabolites linked with survival

Across the metabolome, the 13 metabolites with the strongest linkage to mortality defined through an ensemble importance score (representing the consistency and strength with which a metabolite was highly selected across all the ML models) ≥ 0.5 included lactate, bilirubin, kynurenine, glycolithocholate sulfate, glycochenodeoxycholate, indoleacetate, phenylalanine, 3-hydroxyisobutyrate, beta-hydroxyisovalerate, taurocholenate sulfate, 3-methoxytyrosine, fucose, and hydroxyisovaleroylcarnitine (Table 2, Additional file 1: Fig. S5). These top metabolites are linked with the tryptophan, pyruvate, phenylalanine, pentose phosphate, and bile acid metabolic pathways. Distributions of actual and imputed datasets are provided for each metabolite in Additional file 1: Table S3.

Table 2 Top-ranked metabolites linked with survival ranked by ensemble machine learning-derived summary importance score (defined as those with importance score ≥ 0.5), with corresponding median (interquartile range) normalized levels among septic patients who survived (N = 35) and those who died (N = 25)
Full size table

Discussion

The complementary objectives of this study were: (1) biologically, to uncover metabolites associated with sepsis mortality; and (2) methodologically, to evaluate ML methods as tools to support metabolite selection and prioritization in association studies, comparing their performance to that of conventional statistical methods. Across the broad metabolome profiled, top “hits” identified by ML methods included metabolites with well-established clinical importance (including lactate and bilirubin), as well as metabolites with less established links to sepsis outcomes (including those relating to tryptophan, pyruvate, phenylalanine, pentose phosphate, and bile acid metabolism). An integrated, ML-based ensemble ranking method for prioritizing metabolites based on the strength and consistency of their linkage with survival provided robust metabolite rankings. The ensemble method combined the ranking results of multiple training models in order to obtain a robust set of important metabolites. Overall, these findings support that the application of ML methods metabolomics data may support and possibly enhance biologic discovery, even in small cohorts of critically ill patients; further studies in larger cohorts will be needed to generalize these results.

Sepsis is a major catabolic insult that results in profound changes in carbohydrate, fat, and amino acid metabolism [17, 18]. Several metabolites are routinely used in clinical practice for prognostication and treatment decisions, including lactate [19, 20]. Serum lactate may rise in sepsis due to mitochondrial dysfunction, adrenergic signaling, impaired hepatic metabolism, and tissue hypoxia and resultant anaerobic respiration 21,22,23]. The prioritized identification of lactate as a “top” metabolite by the ensemble consensus ranking procedure herein provides an important marker of external validity (effectively, a positive control for the statistical methods). Similarly, levels of bilirubin and bile salt metabolites are increased in sepsis and have been associated with increased mortality; proposed mechanisms include attenuated bile-acid transporter production and hepatic hypoperfusion, as well as enhanced cytokine and nitric oxide production with subsequent inflammation-mediated cholestasis [24, 25]. Our findings consistently demonstrated strong links between numerous other biliary pathway metabolites and sepsis survivorship. In addition, our study corroborated previously documented associations of several amino acid pathway metabolites with sepsis survivorship, including kynurenine (a downstream tryptophan metabolite) and phenylalanine [12, 13]. Elevated kynurenine levels have been previously identified as a predictor of the development of sepsis in trauma patients, and have also been associated with impaired vascular microreactivity, hypotension, and immune dysregulation in sepsis 26,27,28,29]. Alterations in tryptophan metabolism have also been documented in other inflammatory conditions, such as cardiac arrest, ischemia–reperfusion injury, systemic lupus erythematosus, and COVID-19 30,31,32,33], suggesting that this association may be representative of an broad role for tryptophan metabolism in diseases of dysregulated inflammation. Elevated phenylalanine levels have similarly been shown to both predict the development of sepsis and correlate with increased mortality [12, 13, 23, 34]. Our study provides important clinical corroboration of these findings through data-driven molecular and statistical methods. Future studies are needed to determine to what extent these metabolites may be markers versus causal mediators of outcomes in sepsis.

Moreover, our study identified four metabolites (3-hydroxyisobutyrate, glycolithocholate sulfate, indoleacetate, and fucose) not been previously linked with sepsis survival. 3-hydroxyisobutyrate is a catabolic byproduct of valine metabolism, and elevated levels have been previously linked with insulin resistance and diabetes [35, 36]. Stress hyperglycemia and glycemic variability are associated with sepsis mortality, and accordingly this link may provide further corroboration and insight into this interaction [37, 38]. Fucose is known to play a role in the potentiation of leukocyte adhesion and lymphocyte homing, as well as an inhibitory role in antibody-mediated cellular cytotoxicity [39]. This interaction with both innate and adaptive immunity may play a role in the association of higher fucose levels with poor sepsis survivorship. Lastly, elevations in glycolithocholate sulfate, a secondary bile acid metabolite, and indoleacetate, a tryptophan byproduct, among non-survivors may relate to the complex patterns of bilirubin and tryptophan metabolism in sepsis as described above. ML methods may hence present an important tool for the discovery of new targets to improve sepsis care and better understand pathophysiology.

This current report builds upon a prior analysis from this cohort which had used conventional logistic regression, as well as a network-based approach [12]. One of the key objectives in the current study was to examine whether ML methods could improve the detection of metabolite associations beyond conventional statistical methods. Indeed, 4 of the top 13 metabolites identified by ML prioritization herein had not previously been linked with sepsis survival, while the remainder corroborated this previously work. Conventional statistical approaches may have limitations when applied to large, complex biologic systems such as the metabolome. Many of these limitations are due to the high degree of internal correlation structure, subclass heterogeneity, and missingness inherent in the study of interconnecting biologic pathways and processes [40]. The theoretical advantages of ML strategies include the ability to flexibly integrate multiple forms of data analysis, enhance complex pattern recognition, classify high-order metabolite–metabolite interactions and manage internal correlation, and better address high dimensionality and small sample size of data [41, 42], Each of these advantages may enable ML methods to support deeper interrogation of the metabolome as a complete system. Indeed, ML methods have been used broadly in clinical risk prediction, including in sepsis [12, 13] and other diseases such as cardiovascular disease [11]. However, beyond this role in building accurate risk prediction models, ML methods may also support biologic target discovery and prioritization—as we evaluated herein. This latter application has been demonstrated in functional and regulatory genomics, tumor biomarker discovery, and evolutionary population genetics 43,44,45]. Overall, our results support the use of feature selection tools innate to AI/ML methods for biologic discovery, demonstrating internally consistent and biologically plausible results. Moreover, the ensemble method used herein may be useful as a composite ranking procedure for target prioritization.

These findings should be considered hypothesis-generating in view of several potential limitations. First, patients with sepsis were selected for this exploratory study on the basis of interleukin-18 levels as well as based on having concurrent ARDS or not, which may limit external generalizability, as may the incidentally high proportion of participants with comorbid malignancy in this cohort. Other relevant clinical variables which were not available for assessment include nutritional status and ARDS severity. Given that these clinical features may influence the metabolome of sepsis patients as well as confound the observed metabolic patterns specific to survivors, these results require validation in external cohorts. Additionally, the observed associations of metabolites with survival may not be a specific feature of sepsis; without a control group, these associations may be a broader feature of critical illness or infection. Further studies with appropriate controls will be necessary to delineate this. Similarly, the final list of metabolites linked with survival depends on the selection of metabolites included in the platform; however, the 158 metabolites that passed quality control and pre-processing filters and were included in the primary analysis represented a diversity of biology functions and pathways potentially relevant in sepsis. Furthermore, sepsis 46,47,48,49,50], ARDS [51], and other critical illnesses [52] are increasingly recognized as heterogeneous conditions with potentially important subclasses. Our cohort was too small to stratify by the selection variables or by potential subclasses. Second, numerous, non-physiologic variables contribute to clinical course in the ICU, including patients’ and surrogate decision-makers’ preferences around the provision of life-sustaining care [53, 54], potentially limiting biologic association studies. Third, given that the purpose of our analysis was to identify potential metabolic mediators of sepsis outcomes, and not to build clinical prediction tools [12], we intentionally did not adjust for clinical variables, including illness severity. Worse illness severity may mediate the associations between metabolites and mortality, and adjusting for collinear variables would potentially attenuate associations and limit discovery. Future studies may build upon our exploratory results to establish these metabolic associations in larger cohorts and utilize ML for the purpose of clinical risk prediction. Fourth, plasma was collected within 72 h of ICU admission; this window may be long in view of the rapidity of the disease course in sepsis. Finally, as noted above, we did not undertake external validation of the findings in an independent cohort. While a number of the metabolites linked with sepsis mortality herein are previously known, lending some support to external validity, further investigation is required particularly in the validation of the four new metabolite associations identified. Given the limitations discussed above in the size, selection, and clinical features in this exploratory analysis, further validation in an external cohort is required in order to increase the generalizability and ascertainment of validity of these results.

Conclusions

In this modest-sized cohort of critically ill patients with sepsis, application of multiple artificial/machine learning methods supported identification of metabolites associated with clinical outcomes. While these hypothesis-generating results require validation in external cohorts, such metabolites and metabolic pathways may represent new diagnostic, prognostic, or therapeutic targets. Advancing an understanding of these approaches will be critical in fostering such robust methods of biologic discovery in critical illness.