In recent years about 18,000 clinicians and scientists have attended the EASD Annual Meeting. The advantage of participating in such a conference is that original research is presented long before the results are published. However, because the review process for submitted abstracts is less stringent than that for full-length articles, some doubts remain as to the validity of the abstracts’ methodology and conclusions [1]. Therefore, it is mandatory to assess the quality of presentations selected for medical conferences. The basis of this quality assessment is the publication rate of accepted and rejected abstracts and the impact of the publications resulting from presentations. Whereas some comparable academic societies recently published evaluations of their conference abstracts [2–5], for the EASD and ADA the publication rate of abstracts as full-length papers has only been evaluated for a small sample of abstracts presented at meetings held in 1992 [6]. The aim of our study was to evaluate the efficacy of the EASD abstract review system using subsequent publication success as the outcome and to compare these results with evaluations of other conferences.

There were 2,008 abstracts submitted to the 2004 EASD Annual Meeting. Following anonymous review by seven groups of specialists, each one consisting of five members, 1,306 abstracts were selected to be presented (264 oral and 1,042 poster presentations). A random sample of 493 submitted abstracts was evaluated. Each of the 2,008 abstracts was assigned a random number and the abstracts numbered 1–493 were selected. The abstracts in the random sample were representative in terms of country of origin and field of research. For all abstracts a systematic search was undertaken using MEDLINE. In addition, all first authors received an e-mail asking for information about publications. The period of time evaluated was until 21 August 2008 in order to study a time frame of 48 months. The impact factor of the journals for the year 2007 was used, because a review found that the mean time until publication in a major congress is about 1.5 years and since the meeting was in September 2004, 2007 was the closest full year evaluation [7].

Statistical analyses were performed using SPSS Statistics (available from www-01.ibm.com/software/analytics/spss/). Metric values were assessed for distribution using the Kolmogorov–Smirnov test. Comparing two groups the Mann–Whitney U test was used; comparing more than two groups the Kruskal–Wallis test was used. Proportional differences derived from categorical data were compared using Fisher’s exact test or the χ 2 test when appropriate and the results presented as ORs and 95% CIs. The p values were two sided and values <0.001 considered statistically significant.

Of the submitted 493 abstracts selected at random, 42.4% were published. The publication rate was 51.1% for accepted abstracts and 26.7% for rejected abstracts. Of the 493 first authors contacted via e-mail, 201 (40.8%) replied. More responses were received from authors whose abstract was accepted for presentation (46.1%) compared with authors of rejected abstracts (31.3%).

A MEDLINE search identified 194 studies resulting from the abstracts. Another 15 publications could only be identified using e-mails received from the authors. A total of 3.4% of the publications of the rejected abstracts occurred in journals not listed in MEDLINE; this percentage amounted to 0.9% for the accepted abstracts.

Accepted abstracts were published (mean ± SD) 16.5 ± 12.7 months following the conference; rejected abstracts, 19.3 ± 14 months (p < 0.001). In the e-mail questionnaire, 74.2% of the authors who had not (yet) published the results at the time of the evaluation, reported that they were still intending to do so.

The mean scores given by the review committee were compared with the publication rate following the conference and the impact factor of the journals. Fig. 1 shows the publication rate of abstracts in four different groups of mean scores, with 1 being the highest score and 5 the worst. In the top group, i.e. those with the best mean scores (from 1 to 2), 84.2% of the abstracts were published, whereas in the group with mean scores between 4.1 and 5.0, 12.9% were published, none of which in a journal providing an impact factor in 2007 and 3.2% in journals not included in MEDLINE. The mean (±SD) impact factor of the journals was 6.94 ± 3.4 for the abstracts scored between 1 and 2, 4.96 ± 2.47 for the abstracts scored between 2.1 and 3.0 and 3.47 ± 2.12 for the abstracts scored 3.1–4.0.

Fig. 1
figure 1

Publication rate (%) of the abstracts submitted to the EASD Annual Meeting 2004, held in Munich, divided into four groups according to mean reviewer score. Groups: mean score 1.0–2.0, n = 19; mean score 2.1–3.0, n = 200; mean score 3.1–4.0, n = 243; mean score 4.1–5.0, n = 31. Dark grey fill, published, listed in MEDLINE; medium grey fill, published, not in MEDLINE; light grey fill, no publication identified

Full size image

Accepted and rejected abstracts were compared in relation to impact factors of the journals. For accepted abstracts, publications were found in journals with a mean impact factor of 4.98 ± 2.65, whereas the value for rejected abstracts was 2.91 ± 1.78 (p < 0.001) (Fig. 2).

Fig. 2
figure 2

Box plot of the impact factor of journals that published full-length papers for abstracts rejected or accepted for presentation at the EASD Annual Meeting 2004

Full size image

In terms of evaluating the quality of the review process of abstracts submitted to the EASD Annual Meeting our results revealed that the reviewer score correlated closely to the future publication rate. Rejected abstracts were not only published less frequently but those that were published appeared in journals with a lower average impact factor.

The other aim of our study was to compare the results with those of evaluations of other major medical conferences. The percentage of abstracts presented in the 2004 EASD Annual Meeting that were published in the following 4 years was 51.1%. This percentage is similar to data evaluating other major meetings, as summarised in the Cochrane review by Scherer et al [7]. Societies evaluating the fate of abstracts presented at meetings of a comparable size in Europe have reported similar results. In an evaluation of the European Society of Cardiology meeting held in 2006, Winnik et al [3] found that 38% of accepted abstracts were subsequently published. For the 2008 European League Against Rheumatism, 34.7% abstracts were later published as full-length papers [2], and Miguel-Dasit et al [5] reported a publication rate of 45% for abstracts presented at the 2001 meeting of the European Society of Radiology. It can be concluded that the anonymous peer review process used to select abstracts for the EASD meeting results in a publication rate that is similar, or may even be superior, to other conferences in Europe.

The impact factor of journals where presented work is published is another indicator for the quality of conferences. Fosbøl et al [4] found a median impact factor of publications resulting from the American Heart Association meeting of 4.8, the American College of Cardiology meeting of 4.0 and the European Society of Cardiology meeting of 3.9. Comparisons between such data need to take into consideration that impact factors vary between the different scientific fields, and therefore future research should consider a field-normalised impact factor. Such a factor could be used to compare the impact factors between different areas not only in one single meeting, but also between different conferences.