Immortal time bias and survival in patients who self-monitor blood glucose in the Retrolective Study: Self-monitoring of Blood Glucose and Outcome in Patients with Type 2 Diabetes (ROSSO)
- 719 Downloads
In the February 2006 issue of Diabetologia, the observational Retrolective Study: Self-monitoring of Blood Glucose and Outcome in Patients with Type 2 Diabetes (ROSSO) reported a 51% reduction in the risk of all-cause mortality in patients with type 2 diabetes who performed self-monitoring of blood glucose (SMBG). However, these impressive benefits conflict with results from observational studies and randomised controlled trials. We aimed to show that these findings are caused by a flawed design that introduced immortal time bias.
We illustrate the bias in the ROSSO study and demonstrate that it is large enough to completely explain the apparently protective effect of SMBG on all-cause mortality.
In the ROSSO study, patients were classified as exposed to SMBG for their whole follow-up time if they performed self-monitoring for at least 1 year during the study period. Thus, the time between cohort entry and the date after 1 year self-monitoring was performed is unavoidably ‘immortal’ for patients with SMBG. Patients had to survive at least 1 year to be classified as exposed to this intervention and were artificially ‘protected’ from death. Based on published information, the total amount of misclassified immortal person-time in the SMBG group is at least 5,082 of 9,248 person-years at risk (55%). After re-classification of immortal person-time as unexposed, the unadjusted relative risk changed from 0.59 to 1.95.
The apparently protective effect of SMBG on all-cause mortality observed in the ROSSO study is completely explained by immortal time bias.
KeywordsBias Cohort studies Mortality Self-monitoring of blood glucose Type 2 diabetes
Randomised controlled trial
Retrolective study: self-monitoring of blood glucose and outcome in patients with type 2 diabetes
Self-monitoring of blood glucose
Introduction and methods
In the February 2006 issue of Diabetologia, Martin et al. reported a 51% reduction in the risk of all-cause mortality in patients with type 2 diabetes who perform self-monitoring of blood glucose (SMBG) , which seemed to be exciting news for clinical practice. The Retrolective Study: Self-monitoring of Blood Glucose and Outcome in Patients with Type 2 Diabetes (ROSSO) included 3,268 patients aged 45 years and older who were recruited in 192 primary care practices and were initially diagnosed with type 2 diabetes between 1 January 1995 and 31 December 1999. Baseline characteristics, confounders and information on SMBG were collected based on medical records. Patients were assigned to the SMBG group if self-monitoring was documented for at least 1 year during follow-up. Primary study endpoints were morbidity (defined as diabetes-associated microvascular and macrovascular events) and all-cause mortality. The main target variable was the time from the diabetes diagnosis to a fatal or non-fatal event. Cox proportional hazards models were used to estimate adjusted hazard ratios of all-cause mortality in patients with SMBG (n = 1,479) compared with those without SMBG (n = 1,789). During a mean follow-up of 6.5 years, a total of 120 patients died. Mortality was 2.7% in patients with and 4.6% in patients without SMBG, resulting in an adjusted HR of 0.49 (95% CI 0.31–0.78). The authors controlled for a large number of confounders and the estimates also remained stable after using more sophisticated statistical techniques such as matched-pair or propensity score analyses, as recently published by Kolb et al. .
Based upon the observational ROSSO study, guidelines of the German Diabetes Association (DDG) recommend the use of SMBG as an integral part of the treatment strategy for patients with type 2 diabetes . Furthermore, recommendations regarding SMBG of the International Diabetes Federation (IDF) and the National Institute for Health and Clinical Excellence (NICE) acknowledge the findings of this study [4, 5]. However, the impressive benefits of SMBG in patients with type 2 diabetes that were shown in the ROSSO study could not be replicated in another observational study  and conflict with findings from randomised controlled trials (RCTs). A recent health technology assessment of the National Institute for Health Research reviewed RCTs with a minimum duration of 12 weeks and concluded that SMBG is of limited clinical effectiveness in improving glycaemic control in patients with type 2 diabetes . The German Institute for Quality and Efficiency in Health Care (IQWiG) also found no proof of benefit for SMBG in patients with type 2 diabetes based on randomised trials .
Given all these data, the tremendous benefit observed in the ROSSO study is clearly ‘too good to be true’. Most probably, it was caused by a flawed study design that introduced substantial bias from misclassified immortal person-time. In this report, we illustrate this source of bias in the ROSSO study and try to quantify its magnitude by the approximation given by Suissa . We demonstrate that the bias is large enough to completely explain the apparently protective effect of SMBG on all-cause mortality in this study.
Results and discussion
To understand this kind of bias, we have to look at exposed and unexposed person-times instead of exposed and unexposed patients. Because of the fixed-time analysis, where treatment status (SMBG vs non-SMBG group) is assessed during follow-up, all persons in the SMBG group contribute immortal person-time. For the patient shown in Fig. 1a, this is composed of the time between cohort entry and start of SMBG (2 years) and the 1 year for which SMBG had to be performed. These 3 years are misclassified as SMBG person-time, while in fact it was non-SMBG person-time because the patient was not yet exposed to SMBG. According to the authors’ definition, exposure does not start until 1 year of SMBG was performed. A correct analysis would have to classify the (unexposed) immortal person-time as non-SMBG person-time using a Cox proportional hazard model with a time-dependent exposure variable for SMBG status .
Estimation of the total misclassified immortal person-time in the SMBG group in the ROSSO study
Proportion of SMBG cohort (%)a
Number of patientsb
Immortal person-time per patient (years)
Total immortal person-time (PY)
In the year of diagnosis
1st year after diagnosis
2nd year after diagnosis
3rd year after diagnosis
4th year after diagnosis
5th year after diagnosis
6th year after diagnosis
7th + 8th year after diagnosis
In other words, the strong protective effect of SMBG on all-cause-mortality found in the ROSSO study (i.e. a risk reduction of 51%) is completely explained by ‘guaranteed survival’ due to immortal time bias and is, therefore, introduced by study design. Our corrected estimation even seems to indicate an increased mortality in patients with SMBG. This might reflect a higher morbidity in the SMBG group, for instance because of their poorer metabolic control at baseline and their advanced stage of diabetes as indicated by an approximately tenfold higher proportion of insulin users during follow-up [1, 2]. We have demonstrated the immortal time bias in the ROSSO study for all-cause mortality. However, the analysis for non-fatal events, which shows a reduced risk in the SMBG group (HR 0.68, 95% CI 0.51–0.91) is, of course, also affected by this bias. Our quantification and the correction of the relative risk require several assumptions . An estimation of the ‘true’ influence of immortal time bias on the results would require a re-analysis of the original data using appropriate methods.
Immortal time bias is introduced by the use of a time-fixed analysis, which looks at exposed and unexposed patients instead of exposed and unexposed person-times. This bias cannot be addressed by adjusting for numerous confounders or applying sophisticated statistical techniques such as propensity-score analyses. A substantial number of observational studies have been reported to be subject to immortal time bias . Despite the fact that immortal time bias was already identified in observational studies on survival of patients with heart transplants in the 1970s, studies biased by immortal time seem to have accumulated during the last years. This might be due to the fact that large healthcare or medical-record databases are increasingly used for retrospective observational research. An increased awareness of immortal time bias seems inevitable, particularly if tremendous benefits are shown in observational studies that are not in line with findings of randomised controlled trials .
This study was not supported by any institution.
Duality of interest
The authors declare that there is no duality of interest associated with this manuscript.
- 4.International Diabetes Federation (2009) Guideline on self-monitoring of blood glucose in non-insulin treated type 2 diabetes. Available from www.idf.org/webdata/docs/SMBG_EN2.pdf, accessed 2 September 2010
- 5.National Collaborating Centre for Chronic Conditions (2008) Type 2 diabetes: national clinical guideline for management in primary and secondary care (update). Royal College of Physicians, LondonGoogle Scholar
- 7.Clar C, Barnard K, Cummins E, Royle P, Waugh N, Aberdeen Health Technology Assessment Group (2010) Self-monitoring of blood glucose in type 2 diabetes: systematic review. Health Technol Assess 14:1–140Google Scholar
- 8.Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (2009) Urin- und Blutzuckerselbstmessung bei Diabetes mellitus Typ 2. IQWiG-Berichte – Jahr: 2009 Nr. 65. Available from www.iqwig.de/a05-08-urin-und-blutzuckerselbstmessung-bei.986.html?tid=1152, accessed 2 September 2010