Background

Evidence-based medicine [1] and clinical research are positively valued by health care professionals [2–5], many of whom engage in research [6–9] alongside their clinical activity, in many countries [7, 8] and in Switzerland [6]. Research is necessary to improve medical knowledge, and medical (or clinical) researchers pursuing an academic career have the incentive to publish their results so as to obtain credit from their peers and institutions. However, time devoted to research is generally scarce [9, 10] because clinical activity has priority. Furthermore, medical researchers report moderate levels of research skills [9, 11, 12]. As a result, the numbers of medical researchers [13, 14] and the number of applications for scientific career awards and for research funding have been decreasing over the past three decades, notably in the United States, Finland and Sweden [15–17]. This gap between the necessity of medical research and the limited ability of doctors to actually conduct medical research suggests the need to better understand the difficulties encountered by medical researchers.

Several difficulties in conducting clinical research have been already reported. Challenges include designing the study, calculating the adequate sample size, obtaining funding [18, 19], getting approval from the Research Ethics Committee (REC) [20–22], enrolling enough patients [23], performing appropriate statistical analyses [11] and writing up and publishing the findings [19, 24, 25]. Among discontinued clinical studies, four out of ten were not started because of lack of funding [26] and three out of ten were abandoned because of insufficient patient enrollment [26]. For randomized clinical trials, the ability to recruit patients during the first months after their initiation is a strong indicator for study completion and publication [25, 27]. Finally, getting the paper published in a peer-reviewed journal is also difficult: a quarter of submitted paper send up never being published [28].

While a number of aspects regarding study processes and outcomes have been investigated [25, 27–34], such as patient recruitment, completion rates, publication or number of citations, less is known about possible difficulties experienced during the course of a clinical study. Previous studies have explored the general impression of medical researchers about difficulties in research [19, 26, 35], but not the recall of actual difficulties experienced in the course of a specific study. Surveying medical researchers on a specific and completed study can provide a more accurate recall of the aspects of the research process that have actually been troublesome. We conducted therefore a survey among medical researchers of study protocols approved by a Swiss REC to assess the frequencies and types of difficulties experienced, and to examine factors associated with the occurrence of difficulties.

Methods

Setting

We defined medical researchers as principal investigators (PIs) of study protocols approved by the REC of the Geneva University Hospitals, Switzerland between January 1, 2001 and December 31, 2005. The Geneva University Hospitals is a single 1800-bed public teaching hospital, covering the entire range of medical disciplines, with eight sites across the Geneva region. About 1200 research protocols are currently signaled as ongoing by the REC.

Study design and questionnaire

We conducted a cross-sectional survey among PIs of the selected protocols. We sent them a self-administered online questionnaire between November and December 2010, followed by two reminders. We asked PIs to answer one questionnaire for each separate protocol approved by the REC, identified by its title. The questionnaire had been pre-tested previously among of 12 medical researchers who were not part of the study. We asked PIs to report if the study was completed (defined as recruitment, data collection and analysis completed, whatever the study is published or not) or not (ongoing or abandoned) and whether they had received training in quantitative methods (none, continuing education at the University level, or bachelor/master/PhD in quantitative methods) at the time of the submission to the REC. If the study was completed, we asked PIs to state if they encountered any of 14 specific difficulties in the course of research (Fig. 1) on the following scale of difficulty: none, minor, medium, great, very great, or not applicable. Based on a review of the literature, a group of experts including methodologists (AGA, GH, SC, TA, TVP) and biostatisticians (DSC and Dr. Christophe Combescure, see acknowledgements) identified these 14 difficulties in conducting clinical research. The results from the pre-test with medical researchers suggested the difficulties were clearly described. “Other difficulties” could be reported in a fifteenth item as free text. We did not invite PIs to rate their difficulties with ongoing studies, as we wanted to include only hindsight evaluations of experience. We also asked PIs whether the results of the study were published or “in press”, to provide the reference in an open field, and also whether they had papers in submission or in preparation; for the latter studies, we updated the database by identifying relevant papers published up to November 2013 (PubMed search).

Fig. 1
figure 1

Self-reported difficulties* (rated as medium or great or very great) in clinical research among principal investigators (N = 386)

Full size image

We identified characteristics of study protocols using information recorded in the archives of the REC. We abstracted data on study design (interventional, observational), first decision of the REC (positive, positive with recommendations, positive with written modifications required, positive with conditions and reassessment, negative, and not considered), public vs. private sponsorship of the study (i.e. hospital or university vs. industry or private foundation), link with the industry (none, industry with indication of support, industry without indication of support), planned sample size (later dichotomized as up to 100 vs. larger), reporting of sample size calculation (yes, no), number of centers (single versus multi center), clinical specialty (oncology, neurology, infectious disease, rheumatology, psychiatry, cardiology, general, other specialties, combination of two specialties), and presence of a statistician on study team (yes, no). The category “Other specialties” included all other specialties distributed across the medical services of the Geneva University Hospitals (65 medical services), or studies with no clear clinical specialty. We classified the type of research conducted into the following seven groups: study of health or illness (i.e., organs, tissues, cells, receptors, etc.), interventional or treatment, diagnosis, prognosis, public health or health economics or opinions or perceptions, study methods, and other. We also dichotomized this variable in study protocols informing clinical questions (i.e., interventional/treatment, diagnostic and prognostic studies) versus other types of research (i.e., studies of health/illness, public health, study methods, and other).

Dependent variables

In the survey among PIs, each of the 14 items of self-reported difficulties was dichotomized into medium, great or very great difficulty (1) versus no or minor difficulty (0). When PIs answered “not applicable”, the value “0” (no difficulty) was assigned, because we considered not applicable difficulties as an absence of difficulty. The fifteenth item for “other difficulty(ies)” was followed by an open field for description. Some free-text responses from this item were similar to the other 14 difficulties already defined. One co-author (SC) recoded the text responses into the pre-defined difficulties and another co-author (TP) checked the recoding. Eight free-text responses were re-classified and 41 remained in the “other” category.

Dichotomized self-reported difficulties were used to calculate sums of difficulties for each study and then to compare mean numbers of difficulties across study characteristics.

We defined three dependent variables. The first was a score summing all difficulties across the whole research process. We then classified the 14 difficulties into dimensions using factor analysis with varimax rotation (for a complete description, see Additional file 1: Appendix 1). The analysis suggested two dimensions of seven difficulties each: a dimension related to “scientific difficulties” (literature review, research question, study design, data management, data analysis, manuscript writing, publication in a peer-reviewed journal) and a dimension related to “logistical difficulties” (approval by the REC, getting funding, getting approval by safety agencies, patient enrollment, data collection, collaboration with caregivers for patient enrollment, recruiting competent staff). Dimension scores based on the sum of corresponding items were the second and third dependent variables. Both had adequate internal consistency, with a Cronbach alpha coefficient of 0.83 for scientific difficulties and 0.73 for logistical difficulties.

Sample size calculation

The analysis of difficulties encountered by researchers was a secondary objective of this study. The primary objective of this study was concerned with publication of clinical studies (manuscript in preparation) and sample size was determined with regard to the primary objective. However, a sample size of 400 protocols yielded the following precision (exact 95 % confidence interval) in the estimation of prevalence for a given difficulty: 5.0 % (3.1 to 7.6 %), 10.0% (7.2 to 13.4 %), 20 % (16.2 to 24.3 %), 30% (25.5 to 34.8 %), and 40 % (35.2 to 45.0 %). A comparison of means of 2 groups of 200 observations each would allow for the detection of a small effect size (0.28) with an alpha of 0.05 and a power of 80 %.

Statistical analysis

The unit of analysis was the study protocol. We treated the three dependent variables (sum of all difficulties, sum of scientific difficulties and sum of logistical difficulties) as continuous. We first compared the means across the study characteristics using one-way ANOVA. Then a linear regression model was constructed to predict the mean scores of difficulties using predictors that were statistically significant in univariable analysis (at p ≤ 0.05). We conducted all analyses with SPSS version 22.0 (IBM Corporation, Armonk, United States).

Results

Sample

Among the 996 study protocols, 588 questionnaires were returned by the PIs (participation rate 59.0 %). Among these 588 studies, 391 (66.5 %) were completed at the time of the survey, 96 (16.3 %) were still ongoing and 101 (17.2 %) had been abandoned.

More than three of four completed protocols were sponsored by a public institution, about two thirds were published, a majority was single center studies, and about half were observational studies (Table 1). The median of the sample sizes was 100 (mean 591, interquartile range 260, range 1 to 25,000) and a majority of study protocols did not report sample size calculation. Study protocols reporting a sample size calculation had more often a sample size above the median compared to study protocols that did not (60.9 % versus 30.4 %, p-value <0.001). Almost three protocols out of four did not have a statistician in the study team. Most PIs reported training in quantitative methods – courses or continuing education or a University degree –at the time of the protocol submission to the REC.

Table 1 Characteristics of the 391 completed studies with follow-up questionnaire returned
Full size table

For 408 study protocols the questionnaire was not returned by the PI. Information on these 408 studies was collected using their study protocol stored in the archives of the REC. More than seven of ten studies had no statistician in the study team, around two third had public funding, and the majority had no link with the industry, were single center studies, were unpublished and had an interventional design (Additional file 1: Appendix 2). The median sample size was 100 (mean 973, interquartile range 321, range 3 to 136,000) and a majority of study protocols did not justify sample size. Compared with included studies, studies with no questionnaire had higher proportions of interventional studies, private sponsorship, link with the industry and were more frequently unpublished.

Difficulties in clinical research and associated study characteristics

Levels of difficulties are reported in Fig. 1: patient enrollment and data collection were the most frequently reported. The mean number of difficulties was 2.8 (standard deviation (SD) 2.8, min 0, max 12); 26.4 % of PIs reported no difficulty and 18.7 % reported one difficulty only (Additional file 1: Appendix 3). The mean number of scientific difficulties was 1.1 (SD 1.7, min 0, max 7) and the mean number of logistical difficulties was 1.6 (SD 1.6, min 0, max 7).

In univariable analysis, study protocols sponsored by a public institution (hospital or University) and single center studies encountered more difficulties overall and more scientific difficulties in particular (Table 2). Compared to interventional designs, observational designs encountered more scientific difficulties, but fewer logistical difficulties. The presence of a statistician in the study team was associated with fewer scientific difficulties. Study protocols informing clinical questions (i.e., interventional, diagnostic or prognostic studies) were associated with more logistical difficulties compared to other types of research (studies of health/illness, public health, study methods, etc.). The same applied to the first REC decision: study protocols whose first REC decision either required an additional assessment or was negative (including non-consideration) reported more logistical difficulties compared to study protocols that received a positive first decision.

Table 2 Univariable associations (one-way ANOVA) with mean number of difficulties in clinical research
Full size table

In multivariable analysis, study design, first decision of the REC and the presence of a statistician on the study team were not associated with a higher number of difficulties (Additional file 1: Appendix 5) while several risk factors remained significantly associated. In a parsimonious model (Table 3), publicly sponsored and single center studies were associated with more difficulties overall and more scientific difficulties. Studies informing clinical questions were associated with higher number of difficulties overall and with more logistical difficulties.

Table 3 Multivariable linear regression of sum of difficulties in clinical research
Full size table

Specific difficulties by risk factors

The three study characteristics from the above multivariate model (private vs. public, multi vs. single center and “clinical” vs. other) were cross-tabulated with specific difficulties (Table 4). Compared to publicly sponsored studies, studies initiated by a private sponsor experienced fewer difficulties with the research question, study design, obtaining funding, data management, data analysis and interpretation, manuscript writing, and publication in a peer-reviewed journal. Similar results were observed when multi center studies were compared to single-center studies. Publicly sponsored studies and single center studies experienced fewer difficulties concerning the approval by safety agencies, however more difficulty with getting approval from the REC.

Table 4 Difficulties in clinical research by study characteristicsa
Full size table

Furthermore, studies informing clinical questions reported more difficulties with approval by safety agencies or with patient enrollment. Non-clinical studies reported more difficulties with the formulation of the research question.

Discussion

The present study describes difficulties experienced in the course of a specific study by a broad sample of medical researchers at a Swiss university hospital. The main finding is that patient enrollment posed a significant difficulty to 44 % of study protocols, and data collection posed difficulty to 27 % of them. In other words, logistical aspects of the research process seem to be the major sources of problems in medical research. In contrast, formulating the research question and performing the literature review were less likely to be an issue for medical researchers. Across the whole research process, medical researchers experienced on average three (2.8) of 14 difficulties and only one out of four reported no difficulty at all in conducting their study.

In a previous study investigating perceived barriers in general, a majority of medical researchers reported numerous obstacles in conducting research [19]. We found lower prevalence rates, probably because we assessed difficulties in a given study, and not over the career of a researcher. Furthermore, we only included completed studies, which did therefore not encounter a “fatal” problem.

The correlations between 14 types of difficulties suggested two distinct dimensions: first, problems with the logistics of a study, from funding to patient enrollment and research personnel issues; second, scientific problems. Thus, different kinds of interventions are probably needed to facilitate the conduct of clinical research. Logistical problems could be alleviated by a core facility that would provide administrative support, fund-raising assistance, and qualified research associates. Scientific problems could be solved by the provision of statistical assistance and methodological training of clinical researchers. Such interventions have been implemented with success in other settings [36]. It should be noted that such structures were set up only in 2007 at the hospital under study, most likely too late to have any major influence on the course of research protocols approved between 2001 and 2005. The provision of statistical assistance may also have various impacts on the number of patients to enroll [37]: one the one hand, statistical assistance could increase the sample size at the stage of designing the study; on the other hand, statistical assistance could help medical researchers determine the smallest necessary sample size. Thus, it is uncertain whether the provision of statistical assistance would reduce or increase logistical problems. In our results (multivariable model), a statistician in the team was not associated with logistical difficulties.

Studies initiated by a public sponsor (hospital or university) as well as single center studies were associated with more difficulties compared to those with private sponsor or those multi center: these associations were significant for the overall and the scientific scores, but not for the logistical score. When the study protocol came from a public sponsor and when it was a single center study, it is more likely that the PI was in charge of most steps, from conception to publication. In contrast, many of these tasks are typically not conducted by hospital based PIs when study protocols are initiated by the industry.

Investigators of studies addressing clinical questions also encountered more logistical difficulties, probably because they need to enroll and collect data from patients. Most often, patient recruitment is conducted in parallel to the delivery of healthcare, thus requiring complex and numerous interactions with healthcare providers. Although as many as 72 % of patients invited to take part in a clinical study in our institution accept to do so [38], a better coordination of projects enrolling similar patients in parallel might facilitate recruitment. Other challenges relate to the patients’ willingness to participate, which will depend on personal factors (such as level of education), but also on the attributes of the study. As shown in a previous work, patients are more willing to participate in studies that explicitly value safety, convenience, appropriate oversight, and open communication [39].

In contrast with other surveys [18, 19], we found that getting funding did not represent a significant barrier in conducting clinical research. Three explanations can be considered: first, we surveyed medical researchers about completed studies (which were all funded), while other surveys assessed the general opinion of medical researchers. Second, Switzerland is a relatively favorable country for conducting research: between 2000 and 2008, the funding for research and development increased from 2.5 % of the gross domestic product to 3.0 % [40]. Third, in some cases, clinical studies can be conducted without external funding when benefiting from non-financial support of the hospital.

Another result worth noting is the similarity in the average number of difficulties encountered by PI trained in quantitative methods compared to those not trained in these methods. Training in quantitative methods probably has limited impact on logistical issues; however we expected an impact on scientific issues. It is possible that trained investigators design and conduct more complex or ambitious studies, thus facing a similar absolute number of challenges, yet at a more advanced level.

More than one fourth (26.4 %) of study protocols were conducted without difficulties. More research on these study protocols, using qualitative methods, could add evidence on the facilitating factors for conducting clinical research.

Limitations

This study has several limitations. First, it was carried out at a single Swiss university hospital and we do not know to what extent the findings apply to other settings. Second, we only surveyed PIs of completed clinical studies, which may have decreased the apparent prevalence of major difficulties, because abandoned clinical studies [25, 29] may have experienced more difficulties. Third, PIs’ answers may have been affected by various recall biases [38]. Social desirability may also have influenced responses, considering that the investigators in charge of the present study belong to the same institution as the surveyed PIs. Fourth, the scale of difficulties (and the two subscales, logistic and scientific difficulties) was built for the purpose of this study, based on literature review and expert consensus. We have no evidence of their validity beyond face validity, though their reliability was acceptable (Cronbach alpha between 0.73 and 0.83). Moreover, among the 14 items composing the scale, four items (getting funding, data collection, recruitment of competent staff to conduct the study, and data management) shared variance on both subscales (see Additional file 1: Appendix 1). Fifth, the analysis of studies with no questionnaire suggested some evidence for non-response bias: undocumented studies were more frequently interventional studies, with industry sponsorship, and unpublished.

Conclusions

Patient enrollment and logistical problems topped the list of difficulties. Clinical studies and single-center studies were at higher risk of experiencing logistical difficulties. Providing both logistical and methodological support to researchers may facilitate the conduct of research projects.

Ethical approval

This study was approved by the REC of the Geneva University Hospitals.

Availability of data and materials

The data are not available online but can be requested to the corresponding author.