Introduction

Healthcare professionals are increasingly asked to communicate research results to patients and consumers [1, 2]. This requires translation of study results into clear and understandable information [3]. Framing of data is a well-recognised cause of misconceptions about the efficacy of health interventions by physicians [4–6], healthcare decision-makers [7], and patients [8]. This is particularly relevant for preventive medicine [9].

Recently, several randomised controlled trials on the primary prevention of diabetes have been published [10–13]. Since then a reduction of diabetes by more than 50% has been publicly claimed and large-scale population interventions are being promoted [14, 15].

However, publications of the prevention studies do not always report important outcome data, and framing of results may be a major problem, possibly leading to overestimation of the clinical significance of the intervention effect on diabetes risk [16].

The aim of the present study is to survey the understanding of different formats of data presentation of results of preventive diabetes studies among diabetes healthcare professionals.

Subjects, materials and methods

Participants

The survey sample comprised the participants of three annual diabetes meetings: (1 and 2) the Federation of European Nurses in Diabetes (n=162) and Primary Care Diabetes Europe (primarily physicians with a special interest in diabetes, n=49), both held in September 2005 in Athens on the occasion of the 41st Annual Meeting of the European Association for the Study of Diabetes; (3) the Postgraduate Course in Clinical Diabetology (n=101), primarily physicians specialising in diabetes, held in November 2005 at the University of Jena, Germany.

Questionnaire

Results of four publications formed the basis of the questionnaires [10, 12, 17, 18]. Articles were selected according to their clinical and public impact and their appropriateness for data extraction for different presentations of study results. Two studies explicitly targeted primary prevention of diabetes. Results of the lifestyle intervention arms were used [10, 12]. The third study, the Women’s Health Initiative (WHI), described the effects of combined oestrogen plus progestin or placebo on metabolic parameters and the self-reported incidence of diabetes over more than 5 years of follow-up [17]. The fourth article was the main publication of the United Kingdom Prospective Diabetes Study (UKPDS), a landmark study on secondary prevention in persons with newly diagnosed type 2 diabetes [18]. Ten items were developed. The questionnaire with an explanation on how alternative presentations were derived is available as electronic supplementary material (ESM). Corresponding study results were used in items 1 and 4 [10], items 2, 3, 5 and 9 [17], and items 7 and 8 [12]; item 6 was not based on real data, but displayed relative risk reductions that were comparable to item 1 but at lower absolute risks. In item 10, the main results of the UKPDS on the primary endpoint were used to evaluate understanding of natural frequencies [18]. In some items, there were minor changes of the original data to avoid immediate recognition of related study results, e.g. study duration 3.3 years instead of 3 years. Almost identical wording was used in all items: ‘Looking at this result (these results, this graph), what is your impression about the effect of this (preventive) intervention on diabetes risk?’ Participants were asked to mark their ratings as either ‘very important (++)’, ‘important (+)’, ‘not very important (−)’ or ‘not important at all (− −)’. The first draft of the questionnaire was pilot tested with 20 diabetes healthcare professionals from various European countries.

Assumptions

Based upon a vast literature on framing of data [3–8] and the results of the pilot study, the following assumptions were made. Effect sizes are rated highest when presented as relative risk reduction (RRR) with higher numerical RRRs being more important than lower values (item 1 vs item 3). Effect sizes are perceived to be of decreasing importance when results are presented as proportions of subjects with a diagnosis of diabetes (item 8) or as graphs with truncated scales (item 5), and of lowest importance when (1) a complete scale from 0.0 to 1.0 is used (item 9), (2) results are presented as proportions of subjects remaining free of disease (item 2) or (3) when the actual blood glucose (item 4) or HbA1c values are presented (item 7). Identical relative differences are rated of lower importance when absolute risks are low (item 6).

Data collection and analysis

The English version of the questionnaire was distributed to all conference participants by one or two of the authors. An identical standard introduction was used. Participants were asked to rate the ten items and to fill in an anonymous data sheet on personal characteristics within 15 min. They were informed that there were no correct or wrong answers except for item 10.

Since the three survey samples differed in important characteristics such as participants’ sex, profession or country of work, only descriptive data are presented. No statistical comparisons between subgroups were made.

The risk reduction for the primary endpoint in the UKPDS was 12% (95% CI 1–21%, p=0.029) with a number needed to treat for 10 years of 20 (95% CI 10–500) [18]. In the conventional treatment group with a median HbA1c of 7.9% over 10 years, 46 out of 100 patients had a primary endpoint, and in the intensified treatment group with a median HbA1c of 7.0%, 41 out of 100 patients suffered some kind of diabetes-related endpoint [18–20]. Responses for item 10 were rated as correct if estimates for natural frequencies did not deviate by more than ±10% (41–51 out of 100 for conventional, 37–45 out of 100 for intensified therapy groups), figures were higher for the conventional treatment group than for the intensified treatment group, and the difference between the two groups was >0≤10.

Results

Participation rate was 96% (299/312). A table with the characteristics of the three survey samples is available as ESM (ESM Table 1). In short, participants came from 20 different countries, although the JENA sample comprised almost solely participants from Germany. About half of all participants were nurses in diabetes (diabetes nurse specialists or educators), almost 40% were physicians. Two-thirds had a university degree. Place of work was evenly distributed between primary, secondary and tertiary care levels, though nurses were more likely to work at the secondary and tertiary care levels. On average, participants had worked for more than 10 years in clinical diabetology. The JENA group tended to have fewer years of practice in diabetes care.

Predefined assumptions were supported by the survey results (Table 1). Effects presented as changes in diabetes risk were considered more important than corresponding changes in glucose or glycosylated haemoglobin values (Table 1). More than two-thirds of respondents (188/277) considered a diabetes risk reduction of 57% (item 1) more important than corresponding changes in fasting plasma glucose values (mean difference 0.3 mmol/l) (item 4). About 85% (234/274) viewed the difference in diabetes risk between 14% in the intervention group and 29% in the control group (item 8) to be more important than related changes in glycosylated haemoglobin values (6.0 versus 6.1%; item 7). The impact of graphical framing becomes evident by the extent of variation between the ratings of the four presentation formats of the WHI study results (items 2, 3, 5, 9; Table 1). About 73% (202/278) rated the study effect on diabetes risk as more important when the slightly modified graph of the original publication was displayed (item 5) rather than a graph with a scale ranging from 0.0 to 1.0 (item 9).

Table 1 Ratings of study effects as important or very important in relation to format of presentation
Full size table

Table 2 shows that only between 13 and 23% of subjects gave identical ratings for corresponding study results for the three diabetes prevention studies (items 1 and 4, items 7 and 8), and for items 2, 3, 5 and 9 if at least three items were rated identically.

Table 2 Participants with identical ratings for corresponding study results
Full size table

Item 10 (UKPDS) was filled in by only 53% (158/299) of participants, and was answered correctly by only two people. Only 6% (19/299) gave correct estimates for the event rate of the conventional treatment group, and 4% (13/299) for the intensified treatment group. The difference between groups was within the range of correct estimates for 39% (61/158) of those with valid responses, 15% (23/158) reported a zero difference, whereas the rest substantially overestimated the intervention effect: in 34% (54/158) estimates ranged from 10 to 30% points, and 13% (20/158) thought the difference was more than 30% points. Twenty-eight per cent (45/158) of respondents erroneously gave higher estimates for the conventional treatment group than for the intensified treatment group. Even if a wider range of responses had been considered as correct (30–55% for the conventional treatment group, and 30–45% for the intensified treatment group) only six participants would have answered correctly.

Discussion

The present study uncovers a lack of understanding of the results of diabetes prevention studies by diabetes healthcare providers, who overestimate the benefit of preventive interventions when outcomes are presented as changes in diabetes risk rather than real metabolic changes.

The diabetes prevention studies included individuals with elevated fasting and post-load glucose concentrations who were already at the brink of diabetes [10–13]. Therefore, minimal differences in fasting plasma glucose of 0.3 mmol/l or HbA1c values of 0.1% may relate to pronounced differences in the proportions of persons with a diagnosis of diabetes of 15% and diabetes risk reductions of more than 50% [16]. Small metabolic differences are magnified by transformation of continuous data into categorical data [16].

In some of the publications of the primary prevention studies, crucial metabolic data are not communicated or are difficult to extract [16]. The Finnish Diabetes Prevention Study included HbA1c as a secondary outcome measure [21] but did not report results in the main publication [10]. Neither blood glucose nor HbA1c values were reported in the core publication of the STOP–NIDDM Acarbose prevention study [13]. A recent systematic analysis by Chan and Altman has found that incomplete reporting of outcomes within published articles of randomised trials is common and is associated with statistical non-significance [22]. Biased reporting and framing of data by researchers and authors may enhance misconceptions about treatment effects among users of study results.

The survey points to other possible sources of misunderstanding when reporting study results. Framing of graphical scales in publications is of concern. Identical study results of the WHI were rated as more important when the original scale was used rather than a scale ranging from 0.0 to 1.0. A truncated scale that is narrowed to the range of results visually magnifies the effects. In accordance with previous reports [23], participants considered graphically displayed effects more important when areas under the curves were large (item 2).

The impact of framing on understanding of clinical research data by diabetes healthcare professionals may be substantial. Only two to six of the 299 participants were able to correctly state the natural frequencies for the primary outcome measure of the UKPDS, a most popular study among diabetes healthcare professionals since its publication in 1998. About half of the participants did not answer at all, and of those who provided valid data, the effects of intensified therapy were substantially overestimated by 40% of participants. The UKPDS has been used as an example to demonstrate the significance of framing of data previously [19, 20]. Therefore, results of reporting UKPDS data as natural frequencies could have been familiar to diabetes experts. Our findings may be interpreted as an indication that diabetes healthcare providers do not use or communicate natural frequencies of studies such as the UKPDS in their daily work with patients, students or colleagues.

The present study has limitations. The study population is a convenience sample and may not be representative of the whole community of healthcare workers in the field of diabetes. It consisted of a group of professionals particularly dedicated to continuous medical education, with long-standing experience in diabetes care and high academic levels on average—about half had participated in a postgraduate course in evidence-based medicine/nursing/healthcare. Language barriers might have interfered with their understanding of the questionnaire.

From the results of this survey, we would urgently recommend that study results be presented in a format that can easily be understood by healthcare professionals. Several reviews have summarised criteria for the communication of risk information to the public [3, 8, 24]. Research publication journals should insist on adequate data presentation by authors. This includes the use of scales with complete ranges of proportions from 0.0 to 1.0 and the presentation of results as natural frequencies and absolute risk reductions rather than relative risk reductions, particularly in the abstract. All outcome measures should be reported [22]. In addition, the research findings should be discussed with respect to their clinical relevance [16]. Unbiased communication of research data is a prerequisite for participation of patients and the public in medical decision-making.