Findings

The Critical Care Minimum Data Set (CCMDS) was introduced in April 2006 and collates data daily regarding all patients, excluding neonates, in level 2 or 3 critical care beds in England and Wales. The CCMDS was developed by the Critical Care Information Advisory Group and is endorsed by the Intensive Care Society. The data consists of 34 fields which allow calculation of admission duration, basic demographic information, level of care and category of intensive care unit (general, neurosurgical, cardiothoracic etc). Information is also collected regarding the number of organs supported and, in the case of the cardiovascular and respiratory systems, the level of support. Data is currently submitted to the Intensive Care National Audit and Research Centre (ICNARC) as part of the Case Mix Programme. During 2008 data collection became mandatory and will be used to calculate a payment tariff under the Payment by Results (PbR) scheme within all NHS hospitals.

Our aim was to develop a simple organ score derived from the CCMDS to compare with a previously validated score of organ dysfunction. The Sequential Organ Failure Assessment (SOFA) score is a very commonly used scoring system designed to follow morbidity of critically ill patients[1]. It records severity of dysfunction of six organ systems. Improvement in organ dysfunction scores, such as SOFA, monitor improvements in a patient's condition following an intervention or as an end point for clinical trials. More recently, organ dysfunction scores have been validated, not only as an initial indicator of outcome on admission to ICU, but also as a predictor of mortality, dependant on changes in daily scores during a patient's ICU admission [2–4]. Organ dysfunction scores are time consuming and expensive to collect, requiring additional clerical support so a CCMDS-derived score could represent a saving of both time and money to UK intensive care units especially those involved in clinical trials.

The CCMDS collects data regarding the support of seven organ systems. The support of the cardiovascular and respiratory systems is further divided into basic and advanced support, whilst the other five systems are either supported or not. Table 1 defines each method of organ support [5]. SOFA scores were collected retrospectively from each patient in the study. The study ICU routinely documents Bloomsbury sedation scores rather than Glasgow Coma Scores (GCS) as recorded in the SOFA score[6]. Table 2 illustrates the scheme used to convert the sedation score to the neurological component of the SOFA score. In order to create a CCMDS derived score each level of organ support was allocated a numerical value (Table 3) with basic support scoring 1 and advanced support scoring 2. We did not allocate a particular weighting to any of the scores.

Table 1 Summary of definitions of levels of organ support in the critical care minimum data set [5].
Full size table
Table 2 System used for conversion from sedation score to neurological component of the SOFA score.
Full size table
Table 3 The CCMDS-derived score
Full size table

Our aim was to retrospectively collect SOFA scores and the CCMDS-derived score on fifty admissions to a teaching hospital general intensive care unit from January 1st 2007. The local ethics committee was informed and indicated that no formal approval of this retrospective study was required. Data was collected from day 0 to day 5 of each patient's admission. Seven of the original fifty patients had insufficient data to allow calculation of either a SOFA score or CCMDS derived score. In total 147 pairs of data, both SOFA and CCMDS-derived scores were collected with each patient contributing between two and six data points. Data was initially analysed for correlation and agreement by plotting a scatter plot of the scores of one method against those of the other for each day of admission. Subsequently a Spearman's rank test and a Bland and Altman plot were performed on data collected for each day to assess whether these two methods can be used interchangeably[7].

Visual inspection of the graphs comparing the CCMDS-derived score and SOFA score suggests a weak positive association. The graphs for day 0 and day 1 are illustrated in figure 1 and 2. The weak positive correlation is confirmed using Spearman's rank test, applied to data collected on each day. This gives a correlation coefficient between r = 0.350 to r = 0.609 with the p value for each day (shown in table 4), suggesting a significant correlation (range of p values p = 0.002 to p = 0.047), with the exception of day 2 (p = 0.054). This would suggest that the ranking of patients from high to low risk is similar for both scores.

Figure 1
figure 1

Comparison of CCMDS-derived score and the SOFA score on day 0. Figure 1 shows a XY scatter plot comparing the CCMDS derived score with the SOFA score on day 0.

Full size image
Figure 2
figure 2

Comparison of CCMDS-derived score and the SOFA score on day 1. Figure 2 shows a XY scatter plot comparing the CCMDS derived score with the SOFA score on day 1.

Full size image
Table 4 Spearman's rank correlation coefficient for the CCMDS-derived score compared to SOFA scores for day 0 to day 5.
Full size table

Levels of agreement were checked using a Bland and Altman plot for each day of data collected. As the total score for both score was not identical a percentage change in the score was used. An example of the Bland and Altman plot is shown in figure 3 comparing the changes in scores between day 0 and day 1. Both scores were expressed as a percentage change from the baseline admission score. There is a broad spread of results in the graphs produced for each day of data collected, reflecting a poor level of agreement. The average overall bias between the two scores, in the percentage change from baseline was minimal on day 1 to 0 i.e. 1%. However the 95% limits of agreement were -84% to 86%. This wide margin of disagreement could be expected as the SOFA score has a much greater range than the CCMDS-derived score. It is therefore likely to be a more sensitive measure of change. Similar results were obtained for data on all the other days. The limits of agreement for each day as a percentage change from baseline are shown in table 5.

Figure 3
figure 3

Bland and Altman plot comparing the CCMDS-derived score and SOFA day 0 to day 1. Figure 3 shows a Bland and Altman plot comparing the difference in percentage change, from day 0 to day 1, in the CCMDS-derived score and SOFA score with the average of the CCMDS-derived score and SOFA score. the limits of agreement are indicated by the red lines.

Full size image
Table 5 Levels of agreement derived from the Bland and Altman plots for the 5 days of admission.
Full size table

The aim of our CCMDS-derived score was to produce a simple, daily scoring system that would monitor severity of organ dysfunction but require no additional administrative input and therefore no additional cost implications. The results show a statistical correlation between the CCMDS-derived score and SOFA score as indicators of organ dysfunction, confirming the ranking of patients is similar for the two scoring systems. The r2 values suggest that factors not included in the SOFA score affect our CCMDS-derived score. However, the Bland and Altman plots display a poor degree of agreement between the two scores but with minimal overall bias.

Based on this pilot data from 43 patients our CCMDS-derived score cannot be regarded as an indicator of the severity of organ dysfunction when compared with the SOFA score. It is our view, at present in critically ill patients, an organ dysfunction score derived form the CCMDS cannot replace the SOFA score in studies where a daily indicator of the severity of organ dysfunction is required. This may be explained by the different purpose of each score. The SOFA score was developed and validated as a tool to quantify organ dysfunction. In contrast, the CCMDS was developed to calculate the level of dependency of a patient as a way of calculating remuneration for critical care units. Whilst the level of organ dysfunction is related to the dependency of the patient the two scores quantify two distinct entities.

The sample size was set at 50 as this was a pilot study to determine the feasibility of using a CCMDS-derived score as an organ dysfunction score. Although a larger sample size may reduce the chance of a negative result being a type II error we feel that this is unlikely in view of the consistency of the results. The small sample size and original premise of this pilot study makes more detailed statistical analysis beyond the remit of this study. This does, however, limit the conclusions that can be drawn from the data. Based on the results of this study it is unlikely that a large prospective study with detailed statistical analysis would produce a satisfactory new organ dysfunction score based on the CCMDS. Specific weaknesses of the study are the derived neurological component of the SOFA score, its retrospective nature and small sample size. The Glasgow Coma Score is not routinely collected in the study intensive care unit thereby necessitating a conversion of sedation scores to GCS. A prospective study would be able to record GCS. However this would defeat the main intention of the study to record a score that requires no additional administrative costs. This study is to our knowledge, the first attempt to use the critical care minimum data set to predict organ dysfunction.