Introduction

Cardiovascular morbidity and mortality is the major threat to patients with diabetes. In the majority of patients this risk starts even before the diagnosis of diabetes, hence, before an overt pathological increase of fasting glycaemia. Interventions aiming at the reduction of fasting glycaemia had an important effect on the risk of microvascular complications in Type 1 and Type 2 diabetes [1, 2], but their ability to prevent macrovascular disease was limited [3, 4, 5, 6]. However, most of the people eat several times a day and therefore fasting glycaemic conditions are of shorter duration than post-prandial conditions. Hence, theoretically, post-prandial glycaemia could have a more important effect on the risk of cardiovascular events than fasting blood glucose. This association was clearly and consistently present in several epidemiological studies [7, 8]. However, an epidemiological association can be the result of different relationships. Either post-prandial glycaemia causes macrovascular disease, or vice versa, or both are caused by a yet unidentified factor [9]. The gold standard for the evaluation of a causal relationship between two factors is a controlled randomised intervention study aiming at the modification of one of the factors. Recently such an intervention study—STOP-NIDDM (Study to Prevent Non-Insulin Dependent Diabetes Mellitus)—has been published [10, 11]. In this study Acarbose was given to patients with impaired glucose tolerance aiming at the reduction of postload and post-prandial glycaemia. Indeed, following an Acarbose-induced reduction of post-prandial glycaemia a significant reduction in the risk of developing diabetes, hypertension and cardiovascular events was reported [10, 11]. If the results of this study were valid, they would be of paramount importance for the care of diabetic and non-diabetic patients and could change treatment and monitoring strategies. Therefore we have undertaken a critical review of the available information about the published methods and results of the STOP-NIDDM study.

Methods

We searched Medline, EMBASE, Clinical Evidence, the Cochrane Database of Systematic Reviews (CDSR), the Cochrane Controlled Trials Register (CENTRAL) and we hand-searched all available articles and Internet sites for information about the methods and results of the STOP-NIDDM study without any language exclusion criteria. The published material we used comprised a report of a press conference of the study aligned on behalf of Bayer Vital GmbH & Co. KG in Bad Neuenahr, published in 1997 [12]; rationale, study design and preliminary results of the study published in Diabetes Care in 1998 [13]; publication of the primary results in Lancet in 2002 [10]; an abstract by Chiasson et al. [14] and the corresponding poster presentation [15] at the 38th EASD Annual Meeting in Budapest, September 2002; an abstract by Hanefeld et al. presented at the same meeting [16]; publication of the results of selected secondary endpoints in JAMA in 2003 [11]; the study homepage on behalf of the main study sponsor, Bayer AG, Leverkusen, Germany [17]; a presentation (PowerPoint) of study results on the aforementioned homepage of the study [18] and the Glucobay homepage [19].

We evaluated this available information with regard to the validity and consistency of information regarding baseline characteristics of the study population and the drop-outs, blinding, comparison of the baseline risk between the groups, follow-up time and followed sample size, predefinition, determination and evaluation of study end-points and the attempts to minimise sponsor bias.

Results

Sources of selection bias

Although 1429 participants were enrolled in the STOP-NIDDM study and randomised to treatment with either Acarbose or placebo (“safety population” [15]), only 1368 patients were included in the analysis regarding the primary and secondary endpoints [10, 11]. This group is named “modified intention-to-treat population” [10, 11]. However, by combining the data reported in different sources, the baseline characteristics of those randomised but not analysed (“drop-outs”) can be compared. In the safety population 49.7% of the participants were female and 50.3% were male [14, 15, 16]. In the modified ITT population it was 50.8% and 49.2%, respectively [10, 11]. Hence, it can be calculated that about 25% of the drop-outs were female and 75% were male. According to data given in the Lancet publication, 57 patients of the modified ITT population in the Acarbose group and 78 patients in the placebo group, respectively, already had diabetes at baseline (FPG≥7.0 mmol/l) [10]. For the safety population, these numbers were 85 and 103, respectively. Consequently, 28 (Acarbose) and 25 (placebo) of the drop-outs already had diabetes at baseline. In the JAMA article some baseline characteristics of the 44 participants without “valid post randomisation data” are given (age, BMI, smoking history, cardiovascular medication) [11]. In Table 1 the baseline characteristics of the drop-outs are compared with those of the modified intention-to-treat population, showing an increased cardiovascular risk profile in the dropout population.

Table 1 Baseline characteristics of the “modified intention-to-treat population” (modified ITT population) and drop-outs in the STOP-NIDDM study as calculated by combining information from different sources [10, 11, 14, 15]
Full size table

The numbers of patients with “no valid post randomisation data” are conflicting. In the first publication of the results the numbers are 24 patients in the Acarbose group and 20 in the placebo group [10]. In the recent report the numbers changed to 23 and 21, respectively [11]. This could be due to lacking diligence, but the number of those without impaired glucose tolerance (IGT) changed accordingly [10, 11].

Sources of inadequate blinding

Because of its well-known gastrointestinal side effects Acarbose was initially titrated. However, conflicting data have been reported for the dose and method of this titration in the verum and the placebo group (Table 2). According to a diagram published in 1997 [12] both Acarbose and placebo were titrated; however, according to other sources only Acarbose was titrated [17, 19], or a titration of placebo is not mentioned [10]. It is not clear if titration was done in both study groups in the same way and, in consequence, if participants of the intervention group were seen more often by the investigators. Such information is of major importance to the efficacy of intervention and blinding of the study medication. In the Acarbose group, 79% of the patients correctly guessed their treatment, which supports the theory of inadequate blinding [11].

Table 2 Method of titration of Acarbose as stated in the different sources of information
Full size table

Sources of bias in data analysis and interpretation

Duration of follow-up

In cumulative analyses of events the risk to develop an end point increases with the duration of follow-up. The mean follow-up time was 3.3 years for the whole population [10, 11]. It can be assumed that the mean follow-up time was shorter for Acarbose because patients in this group discontinued taking the study medication more often and earlier during the trial [10, 11]. The incidence of diabetes per 1000 person-years was 101 in the Acarbose group and 121 in the placebo group [10]. Therefore, the mean follow-up time in both groups can be calculated (Table 3). The mean follow-up time was 0.22 years longer in the placebo group than in the Acarbose group, which equals about 3 months. The washout-phase also lasted for 3 months, and in this period 15.4% of observed patients in the Acarbose group developed diabetes, which is more than the absolute difference in the incidence of diabetes (9.1%) seen at the end of the study [10].

Table 3 Mean follow-up time for the intervention and the control group in STOP-NIDDM [10]
Full size table

Primary endpoint—risk to develop diabetes

Baseline risk

A total of 57 patients in the Acarbose group and 78 patients in the placebo group already had diabetes at baseline [10]. However, these patients remained part of the modified ITT population and were assumed to be at risk for the development of diabetes [10]. Therefore, about one third of the reported difference in the incidence of diabetes (as determined by fasting plasma glucose), can be solely attributed to differences present in baseline characteristics. Surprisingly, the relative difference in diabetes prevalence at baseline is exactly the same as the apparent difference in diabetes incidence during follow-up: 36% [10].

Missing data—exclusion of the washout phase

During the active treatment phase 32% of patients randomised to Acarbose and 42% of patients randomised to placebo developed diabetes as measured by OGTT [10]. As prospectively planned, a 3 month washout treatment with placebo was done at the end of the study [10, 13]. During this period, additional cases of diabetes occurred in another 47 patients in the Acarbose group (15.4% of those still followed) and in 21 in the placebo group (10.6%) [10]. The Kaplan-Meier analysis in the Lancet article does not include these 3 months [10]. It is therefore difficult to estimate the statistical relevance of this observation, but the incidence per 1000 person-years can be calculated (Table 4). At the end of the washout phase, the relative difference in the incidence of diabetes per 1000 person-years is only about 7%. Therefore, Acarbose could have only masked and not prevented diabetes [20].

Table 4 Incidence of diabetes in STOP-NIDDM per 1000 patient-years including 3 months Acarbose washout phase (single blind treatment with Placebo in both groups)
Full size table

In addition, the risk of diabetes depends largely on changes in body weight. Gastrointestinal side effects of Acarbose could be responsible for a decrease in food intake and consecutive loss of body weight. In the Lancet article in 2002 [10] the mean decrease in body weight in the Acarbose group is reported to be 0.5 kg. However, in the JAMA publication in 2003 [11] this mean weight reduction is described as 1.15 kg—an effect, which may well explain the lower incidence of diabetes in this group. The dietary intervention to reduce body weight in the placebo group was ineffective.

Missing data—additional parameters

Most regretfully, post-prandial glycaemia and glycosylated haemoglobin values are not reported, despite the fact that such an evaluation had been planned [13]. Such information is of major importance for the assessment of any clinical relevance of the intervention. In STOP-NIDDM, patients with impaired glucose tolerance were investigated at the brink of manifesting diabetes; in such patients very small changes in glycaemia and in body weight can relate to impressive reductions in the risk of diabetes [21].

Secondary endpoint—risk to develop hypertension

Baseline risk

The prevalence of hypertension at baseline is not clear. Whereas in the 2002 report 629 patients had hypertension [10], in the 2003 article this number is 702, of which 357 were treated with Acarbose and 345 with placebo [11]. All patients including those with hypertension at baseline were included in the evaluation of the effect of Acarbose on the risk of development of hypertension (Fig. 4 in [11]).

Validity of diagnosis

As mentioned in the publication of the study design, hypertension was diagnosed if the mean of three readings was above 140/90 mmHg and increased blood pressure was confirmed during the next 2 weeks [13]. In the publication of the results, hypertension was also diagnosed if the mean blood pressure equalled 140/90 mmHg or more, or if antihypertensive treatment was started by a family physician [11]. The authors did not state how often hypertension was diagnosed by the family physician, i.e. without quality control by the study personnel.

Secondary endpoint—risk of cardiovascular events

Predefined endpoints—multiple hypothesis testing

Administration of Acarbose was associated with a combined cardiovascular event risk reduction of 49% [11]. However, the kind and the number of assessed cardiovascular events changed during the course of the study. Prospectively predefined end-points comprised myocardial infarction, cerebrovascular event and heart failure [13]. In the final report angina, revascularisation procedures, cardiovascular death and peripheral vascular disease were also included [11]. The report of the safety population also includes coronary disorder, hypertension and arrhythmia [18]. In total, ten different cardiovascular end-points were assessed. This results in 1023 possible statistical comparisons of different combinations of these endpoints. The reason for choosing a non-predefined combination of events for assessment of the cumulative risk of cardiovascular end-points is not stated. At least, a separate analysis of the prospectively defined combination of endpoints is missing.

Missing data—the safety population

Differences are also present concerning the population for which details of cardiovascular events are given. Whereas in 2003 events for the modified ITT population are stated [11], in the slide show on the study homepage this is done for all patients randomised to Acarbose or placebo (“safety population”, n=1429) [18]. In addition, the term “no valid post-randomisation data”, which was the reason for the exclusion of 44 patients from endpoint evaluation [11], is also explained in this slide show: these patients did not have an oral glucose tolerance test or fasting plasma glucose measured after randomisation [18]. The exclusion of these patients from an evaluation of cardiovascular endpoints seems inappropriate. In Table 5 cardiovascular events are given as published in JAMA [11] and in the study home page [17, 18]. Obviously, in the safety population no significant difference between the study groups was present, (Fishers exact test p=0.55). Moreover, it is striking that in all cases the transmission of end-point data from the safety population to the data of the modified ITT population favours Acarbose (Table 5). However, for the safety population, there is no description of the nature of almost half of the cardiovascular events in the Acarbose group (16 of 33). In contrast to what is said in the JAMA article, before exclusion of patients angina was more often present in the Acarbose group than in the placebo group (Table 5). These differences could be attributed solely to the exclusion of these 61 participants (drop-outs). It seems unlikely that the independent blinded end-point review committee would exclude cardiovascular events only in the Acarbose group.

Table 5 Numbers of patients with cardiovascular events in the modified intention-to-treat population and in all randomised patients (safety population) in the STOP-NIDDM trial [11, 17, 18]
Full size table

Flawed reporting

In addition, despite the fact that patients who discontinued Acarbose treatment were followed for outcome variables [11], for unknown reasons more patients in the Acarbose group were excluded from the at risk population (Fig. 2 in [11]). After the first 300 days of follow-up 658 patients in the placebo group but only 622 patients in the Acarbose group remained in the study. This selective non-follow-up continues during the entire duration of the study and could have further decreased the number of high risk patients in the Acarbose group.

Sponsor bias

In the publication of the study design, it was explicitly mentioned “to guarantee quality and credibility of the data, randomisation and analysis of the data are done by an independent group” [13]. In contrast, as indicated in the publications of the results, “enrolment and randomisation was handled at the sites” and “Bayer AG provided the personnel and the infrastructure for site monitoring and data collection” [10, 11]. As reliable blinding of the treatment is questionable and randomisation was done in blocks of four and six [10], allocation concealment could have been influenced.

The publication of the study design mentions two employees of the main sponsor as members of the STOP-NIDDM trial research group [13]. This group had “invested much effort, thought, and discussion in developing the study design, deciding on the sample size, and choosing an appropriate treatment for the prevention of Type 2 diabetes” [13]. Five from 11 members of the steering committee were Bayer AG representatives [13]. These conflicts of interests are not mentioned in the publications of the results [10, 11]. In contrast, it was explicitly mentioned that “the main sponsor had no role in study design” [10], which is obviously contradictory. In 2002 no conflict of interest is mentioned by any of the authors [10]. In 2003, three authors confirm that they have received research support and/or honoraria for lectures from Bayer AG [11].

Conclusions

The data reported from the STOP-NIDDM trial at different publication sites are in many cases conflicting, contradictory and not supporting the conclusions. Acarbose has no proven specific effect on the risk of diabetes, hypertension and cardiovascular events. The effects of reduction of post-prandial glycaemia on morbidity in patients with impaired glucose tolerance and in patients with diabetes remain unknown.

Recently Bayer AG removed the data from the references no. 15, 17 and 18. Part of these data can be accessed at http://www.di-em.de.