Background

Eligibility criteria for clinical trials are designed to ensure the safety of research participants [1] and reflect the scientific objectives of a study [2]. Automated eligibility determination systems help strengthen the enrollment process by standardizing decisions so that study personnel do not influence the selection process [3, 4]. However, in most clinical trials, investigators or their designees determine whether individual participants meet selection criteria, a process vulnerable to selection bias because of subjective interpretations [5] and the personal views of study personnel [6]. Thus, a challenge for clinical trial researchers is to render eligibility determination with procedures that facilitate consistent decision-making and avoid enrollment error, thereby reducing the opportunity for participant risk [1, 7, 8]. Determinations that lack a consistent and transparent process may lead to an unrepresentative sample [9] and influence the generalizability of study results [10, 11].

Although inclusion and exclusion criteria are reported routinely in scientific publications [12], the specific processes by which participant eligibility is determined and selection bias mitigated are often unclear. A 2010 systematic review studying interventions for improving adherence to eligibility criteria in randomized controlled trials did not find any primary publications describing such procedures [5]. Thus, there is a need to report eligibility determination procedures specifically designed to address potential limitations and to reduce the need for clinical trial managers to continually re-invent this process, which is necessary when established methods are not yet documented or disseminated broadly [13].

This article describes a multidisciplinary eligibility determination process developed and in use at a research center engaged in community-based recruitment for clinical trials of chiropractic care for non-surgical spinal pain. Our research teams developed this process to determine eligibility requiring clinical decision-making for three federally-funded randomized clinical trials in the United States: Trial 1 was a study of spinal manipulation with physiological and kinetic measurements for 221 adults aged 21 to 65 years with sub-acute or chronic low back pain [14]; Trial 2 was a study of collaborative care involving chiropractic and medical care for 131 participants with low back pain in adults aged over 64 years [15]; and Trial 3 was a study utilizing spinal traction mobilization for 48 adults aged 18 to 70 years with neck pain.

Methods

Ethics approval

Trials 1 and 3 were overseen by and received ethical approval from the Palmer College of Chiropractic Institutional Review Board (assurance numbers 2007M093 and 2012G151). Trial 2 was overseen and approved by both the Palmer College of Chiropractic and Genesis Health Systems Institutional Review Boards (assurance numbers 2011G138, 11-005). The institutional review boards approved the study protocols that included the case review process discussed in this manuscript. All study participants provided written informed consent.

Clinical trials overview

We developed our case review process for eligibility determination during three separate clinical trials conducted with overlapping enrollment periods over a 5 year timespan. Trial 1 [14] was conducted to determine pre- to post-treatment changes in postural sway following spinal manipulation. Participants with acute, sub-acute, or chronic low back pain received 13 treatments over 6 weeks of either i) high velocity, low amplitude spinal manipulation; ii) low velocity, variable amplitude spinal manipulation; or iii) sham treatment. Sensorimotor measures included postural sway, repositioning accuracy, and response to a sudden load. Table 1 includes the specific eligibility criteria for Trial 1.

Table 1 Eligibility criteria determined during case review for Trial 1
Full size table

Trial 2 [15] was a 12-week study conducted to evaluate the feasibility of a collaborative care model for chronic low back pain treatment in adults 65 years or older. Participants were allocated to one of three treatment groups: i) conventional care by a family medicine physician; ii) medical and chiropractic care separately but concurrently provided by a family medicine physician and a doctor of chiropractic; and iii) collaborative care by both a family medicine physician and a doctor of chiropractic. Table 2 lists the eligibility criteria for Trial 2 as listed in the study protocol.

Table 2 Eligibility criteria determined during case review for Trial 2
Full size table

Trial 3 was conducted to study a manual cervical distraction procedure for adults with radiating neck pain. Trial 3 represents the most recent iteration of the case review process and was conducted to i) measure the ability of chiropractic research clinicians to deliver a cervical traction mobilization procedure within specified force ranges and ii) provide preliminary outcomes data for a future full-scale clinical trial. There were five study visits and all included manual cervical distraction delivered within three treatment group-specific force ranges. Biomechanical procedures included repeated neck range of motion testing in multiple positions and performing repeated isometric muscular contractions in the cervical region. Table 3 presents eligibility criteria as listed in the study protocol and assessed during case review for Trial 3.

Table 3 Eligibility criteria assessed during case review for Trial 3
Full size table

Automated eligibility screening process

Eligibility determination is a multi-staged process that begins when an interested community-based volunteer contacts our research center via phone, direct mail post card, e-mail, or in person to initiate the screening process [16]. Research personnel provide the volunteer with basic information about the trial and conducts a computer-assisted telephone interview (phone screen) to assess preliminary eligibility. A phone screen-eligible participant is scheduled for an in-person baseline visit that includes an informed consent process conducted by a study coordinator. A consented participant then completes research questionnaires and an interview with the study coordinator to review select eligibility criteria. The study coordinator enters objective eligibility data that do not require clinical decision making (e.g., pain rating, body mass index, depression index) into a web-based participant tracking system programmed to generate automated decisions for specific eligibility criteria [17]. Participants who do not meet eligibility criteria at this point are excluded from the study and thanked for their interest and participation.

If eligible after the interview with the study coordinator, the participant receives a clinical evaluation from a research clinician who is a licensed doctor of chiropractic. The clinical evaluation includes a detailed health history review and focused examination of the spinal region under investigation. Additional clinical evaluation may occur when the health history, symptoms or examination findings merit further assessment to obtain information necessary to render clinically-related eligibility decisions or to determine whether referral to another provider is warranted. Clinicians may also use standard radiography when clinically appropriate [18]. Following the clinical evaluation, the research clinician completes an eligibility questionnaire on the web-based tracking system, entering only objective inclusion or exclusion criteria that require clinical evaluation, but which are not related to the diagnosis under study such as surgical history or the presence of a fracture. Participants may be excluded at this point by the research clinician. Participants who remain eligible following the baseline clinical evaluation move forward for formal eligibility determination at the case review meeting.

Case review eligibility determination

The case review panel consists of 6 to 12 study personnel representing key roles and includes 2 to 4 research clinicians, 2 to 4 study coordinators, 1 clinical project manager, and the senior research clinician who leads the panel and is a trial investigator. Other personnel or study investigators may also be included in the panel depending on trial requirements. At least half of the study coordinators and research clinicians participating on the panel are graduate students enrolled in a master’s degree program in clinical research. All graduate students are degreed healthcare professionals (doctors of chiropractic); case review serves as a foundational training experience in the conduct of clinical trials for these students.

The case review panel typically meets twice per week for 1.5 hours per session. A study coordinator prepares paper and electronic records to assure their availability for case review meetings. Secure, password-protected network file servers hold documents containing personal health information. A web-based manual of operating procedures is also available to reference study protocols, eligibility criteria, and operational definitions for individual studies. The goals for the case review meeting are to: i) ensure complete and consistent case presentation; ii) encourage contributions by all panel members; iii) establish the clinical diagnosis [19]; and iv) reach consistent and scientifically-valid eligibility decisions.Case review is initiated by a study coordinator who also serves as moderator and leads the panel through the eligibility determination process using a stepwise flowchart. Figure 1 depicts the case review flowchart for Trial 3. Flowcharts depicting the case review process for Trial 1 and Trial 2 are illustrated in Figures 2 and 3, respectively. The moderator manages the case review discussion, ensures that each step of the process is addressed before subsequent discussion topics are introduced, and tallies consensus votes. Thus, the moderator allows the research clinician to focus on the presentation of the case, panel members to listen to the presentation and formulate discussion questions, and the project manager and senior clinician to oversee that the case review process addresses the eligibility criteria as outlined in the study protocol.

Figure 1
figure 1

Case review flowchart for the manual cervical distraction trial (Trial 3). Yellow boxes, case review moderator process; Light grey boxes, study coordinator process; Dark grey boxes, research clinician process; Green boxes, senior clinician process; Purple boxes, case review panel process.

Full size image
Figure 2
figure 2

Case review flowchart for the spinal manipulation and sensorimotor function trial (Trial 1). Light grey boxes, study coordinator process; Dark grey boxes, research clinician process; Green boxes, senior clinician process; Purple boxes, case review panel process.

Full size image
Figure 3
figure 3

Case review flowchart for the collaborative care trial (Trial 2). Light grey boxes, study coordinator process; Dark grey boxes, research clinician process; Green boxes, senior clinician process; Purple boxes, case review panel process.

Full size image

The moderator first asks the study coordinator who completed the informed consent process to introduce the participant and disclose information that could affect eligibility such as factors that could influence the ability to comply with study procedures (e.g., family responsibilities, work schedule, transportation issues). Next, the moderator directs the research clinician to present imaging findings and health records obtained elsewhere, when available. Following the health records review, the research clinician provides a formal case presentation including a detailed health history, focused history of the chief complaint, review of systems, and findings from the clinical evaluation.

Following the case presentation, the moderator designates each panel member an opportunity to ask questions. The research clinician answers panel members’ questions as they are posed. When all panel questions are addressed, the moderator directs the research clinician to review the study-specific eligibility criteria and their operational definitions, which are displayed electronically for the panel. The moderator then verbally polls each panel member individually regarding whether any particular exclusion criterion is met or requires discussion or clarification and tallies the poll results. If no exclusion concern exists, the moderator addresses the next item on the case review flowchart (e.g., diagnosis determination).

When an exclusion reason is posed by a panel member, the moderator reviews the corresponding study-specific eligibility definition. The moderator again verbally polls each panel member individually to determine the degree of consensus. Members provide verbal answers on their agreement with the exclusion criterion, along with justification for their opinion. When an eligibility decision can only be rendered after the diagnosis is determined, exclusion discussion is pended until after diagnosis confirmation by the panel. Each proposed exclusion decision occurs separately by a voting process. Consensus occurs when at least 80% of the panel members present agree with the eligibility determination decision. If consensus is not reached, the senior research clinician, or other designated trial investigator, renders final eligibility determination.

Once eligibility consensus is reached, the research clinician completes the participant-specific case review form located on a secure, password-protected web module programmed with study-specific exclusion criteria. The eligibility decision, along with the reasons for exclusion, comments regarding the case review process including whether or not consensus was reached, and the names of the research personnel who participated in the case review discussion are documented. The research clinician who performed the examination contacts participants excluded at case review to inform them of their eligibility status, report their findings, and provide clinical recommendations when appropriate. Referral recommendations for conditions such as abdominal aortic aneurysm, peripheral arterial disease, severe hypertension (urgency/crisis), evaluation for other suspected or unmanaged health conditions, and counselling for alcohol or drug dependence may also occur at this time. Participants included at case review are scheduled for the next study visit, which includes concealed allocation.

Results

Combined trial results

Our case review panel rendered eligibility determination decisions for three separate clinical trials from January 2009 through December 2013. The case review panel considered a total of 697 cases, excluded 225 participants, and allocated 400 participants across the three trials. Of the 225 exclusions at case review, 156 participants were excluded from Trial 1, 33 participants from Trial 2, and 36 participants from Trial 3. Table 4 outlines the eligibility determination dispositions across the three clinical trials. The number of participants excluded for each eligibility criterion are presented by study in Table 5 (Trial 1), Table 6 (Trial 2), and Table 7 (Trial 3).

Table 4 Eligibility determinations for three clinical trials using the case review process
Full size table
Table 5 Exclusion decisions rendered during case review for Trial 1
Full size table
Table 6 Exclusion decisions rendered during case review for Trial 2
Full size table
Table 7 Exclusion decisions rendered during case review for Trial 3
Full size table

Across the three trials, 72 participants who were deemed eligible for random allocation at case review were not randomized into a study, including 51 participants in Trial 1, 9 participants in Trial 2, and 12 participants in Trial 3. Of these, 28 participants (Trial 1 = 16 participants, Trial 2 = 7 participants, and Trial 3 = 3 participants), decided not to continue in their respective trial, usually citing time commitment conflicts with work or family as their reasons for non-participation. Another 20 participants in Trial 1 and 2 participants in Trial 3 did not return to the clinic after being eligible at case review, and thus were never randomly allocated to treatment. Finally, 24 participants (Trial 1 = 15 participants, Trial 2 = 2 participants, and Trial 3 = 7 participants) were excluded from their respective trials after being eligible at case review due to an exclusionary response at the pre-allocation interview, with the majority of these exclusions (n = 18) categorized as no longer meeting the established pain rating threshold for the study.

The most common reasons for exclusion across the three trials included neck or back pain not meeting the stipulated Quebec Task Force for Spinal Disorders classification [20]. Other common exclusions occurred for safety concerns related to treatment or testing procedures, referral for the neck or back pain condition or other health concern, participant-reported alcohol or drug use, and concerns related to a participant’s ability to comply with the study protocol. No participants were reported to the principal investigators as inappropriately enrolled in the respective trials. However, several participants withdrew from each trial and may, in some cases, represent persons who were judged incorrectly by the panel to be those who would be compliant with study protocols. In total, 34 participants withdrew or were lost to follow-up across the 3 trials: 22 participants from Trial 1; 9 participants in Trial 2; and 3 participants from Trial 3. We were unable to obtain reasons for withdrawal for the majority of these enrolled participants due to loss of contact.

Trial 3 results

Figure 4 presents the eligibility decisions from Trial 3 as an exemplar of our multi-staged process for eligibility determination. Study personnel initiated 291 phone screens that 9 participants opted not to complete and through which 72 participants were excluded per the automated eligibility determination process, with the majority of these exclusions (n =48) being related to neck pain ratings outside those established for this study (3 to 7 on a 0 to 10 numerical rating scale). We scheduled 210 baseline 1 evaluations in which 95 participants opted not to continue in the study and 19 participants were excluded through the automated process, again, with most exclusions (n =14) related to self-reported neck pain rating. A total of 96 participants were brought forward to the case review meeting for eligibility determination, during which 36 were excluded from further participation. A total of 48 participants were allocated to treatment in Trial 3. Of the remaining 12 who were not allocated, 5 participants chose not to continue with the study and 7 rated their pain level outside the established inclusion criteria for the trial.

Figure 4
figure 4

Eligibility decisions from the multi-staged determination process for the manual cervical distraction trial (Trial 3).

Full size image

Table 7 presents the reasons why participants were excluded from Trial 3 at case review. All exclusions were due to a specific eligibility criterion. Twenty-five participants were excluded for a single exclusion criterion while 11 were excluded for multiple criteria (varying from 2 to 4 exclusions). Participants were most often excluded for neck pain not matching the modified Quebec Task Force for Spinal Pain classification ratings 2 to 4 [20, 21] (n =13), an inability to tolerate study procedures during the baseline examination (n =9), referral needed for neck pain or other health condition (n =8), self-reported high levels of alcohol or drug use on standardized substance use screening instruments (n =7) [22–24], and the inability to safely receive study treatments or perform biomechanical tests (n = 7).

We did not systematically collect the number of eligibility determinations where case review panel consensus was not reached until Trial 3. The study investigator made 3 of 96 (3%) decisions under circumstances where the case review panel did not achieve at least 80% consensus on the eligibility determination. The investigator excluded one participant on the suspicion of alcohol dependence or abuse, one for referral to evaluate another health condition, and one for the inability to safely tolerate the study treatment and biomechanical tests.

Resource considerations

Eligibility discussions during case review varied from 15 minutes to 1.5 hours (averaging 30 minutes) per participant, meaning a substantial amount of combined personnel time is dedicated to the process. Lengthy decisions tended to occur early in the clinical trial, when participants were excluded for multiple reasons, or when new panel members, typically graduate students in the clinical research program, were present and more time was dedicated to reviewing study aims, definitions, protocols, and discussing rationale for decisions. At our facility, each panel member dedicated 7.5% of their working hours to case review meetings (3 hours/week).

Discussion

The novel case review process presented here is a systematic method to mitigate selection bias in clinical trials of spinal conditions for subjectively determined eligibility criteria such as clinical diagnosis, tolerance of study procedures, participant safety, need for referral, substance abuse, and compliance concerns. Our structured case review process creates an environment for collaborative discussion, which includes vital contributions from clinicians, coordinators, and investigators who fill fundamentally different yet synergistic roles in the eligibility determination process for participants enrolling in a clinical trial. Each panel member is solicited to voice their perspective so as to capitalize on their unique expertise and experience to: i) facilitate clinical and scientific interpretation of eligibility criteria; ii) ensure participant safety regarding the ability to perform and receive trial procedures; and iii) mitigate selection bias by providing an environment to challenge, and therefore encourage, justifiable decision making, consistent with established eligibility definitions.

Unfortunately, we cannot compare the effectiveness or efficiency of our case review process with other methods for eligibility determination as the biomedical literature is silent regarding standard or systematic procedures designed to accurately enroll participants into clinical trials, particularly where clinical decision making is required [5]. This article represents a first step in describing systematic methods that specifically incorporate steps to mitigate selection bias where it is known to occur.

Our case review process is designed with the awareness that clinicians and other healthcare professionals do not always make eligibility determinations consistent with study aims or established inclusion and exclusion criteria. For example, clinicians may avoid enrolling eligible patients into a clinical trial because of the perceived effect such participation could have on the current doctor-patient relationship and to avoid feelings of responsibility if treatments were found to be unequal [25]. Clinicians also may prefer one trial therapy over another [6, 25–27] or avoid enrolling participants due to the concerns that the patient might receive a placebo [28]. Clinicians have reported reluctance to enroll study participants due to the complexity of clinical trials [26], the practical difficulty in following procedures, dislike of discussions with patients regarding uncertainty [25], and excessive time commitment [28]. Interviews with hospital personnel involved in one clinical trial found personal judgments about patients possibly explains why more than 50% of eligible patients did not receive an explanation of a study they could have been involved with [27].

Healthcare professionals may experience conflict between what they understand as their clinical responsibility to the patient and the variables by which trial eligibility decisions should be made. Clinicians may not fully understand key components of clinical research and may lack time, motivation, or opportunity to become knowledgeable about study procedures. However, skilled clinical assessment is usually essential for eligibility determination. To address this issue, our process includes clinicians and other study personnel to confirm interpretation of clinical information and to serve as a quality control mechanism. Our case review procedures and the inclusion of at least one study investigator [29] in the eligibility determination process are intended to prevent selection drift [30] and may be beneficial during trials in which participant recruitment lasts several years, occurs in multiple sites, or when implementation occurs with changing staff.

Eligibility assessment for clinical trials involving non-surgical spinal pain is challenging, in part because of the difficulties in establishing a definitive diagnosis [19, 31]. Other limitations include incongruent terminology among investigators, clinicians, and other study personnel with divergent areas of expertise involved in eligibility determination [32]. Without the use of consistent terminology it is easy to conceive of eligibility decisions that are inconsistent with criteria as envisioned by investigators. One example of inconsistent medical terminology is represented by the term lumbar spinal stenosis. Lumbar spinal stenosis is an imaging finding (a narrowed vertebral canal) whereas the clinical diagnosis of lumbar stenosis syndrome, with subtypes, is characterized by both imaging findings and distinctive symptoms. However, a variety of terms are used to describe the clinical condition [33, 34]. Without clear definitions of the clinical presentation, detailed examination procedures, and the specific factors indicating inclusion or exclusion, determining eligibility in a consistent manner can be difficult.

Because eligibility criteria can be unclearly defined [35] or interpreted inconsistently [27], we provide electronic access to study protocols and definitions during case review meetings. Real-time access to study protocols and definitions is designed to help prevent errant decisions due to complex study designs or protocols, and inadvertent misunderstanding [13]. We believe it is especially helpful when conducting simultaneous trials. We incorporate mandatory review of all eligibility criteria into our case presentations. To further prevent definition drift or systematic misunderstanding we include at least one investigator in case review meetings. The investigator also serves in the role of clarifying definitions, answering questions when they arise, and rendering final determination when consensus is not reached.

While our research center has automated the eligibility determination process for objective criteria, there remain eligibility decisions requiring clinical expertise that are more subjective in nature. The safety of participants when a manual therapy is delivered per the allocation scheme instead of tailored to an individual’s condition [36], the ability to tolerate and safely undergo challenging biomechanical tests without condition aggravation, the need to refer for further evaluation or treatment of a health condition, and the ability of a participant to comply with the study protocol [37] are eligibility considerations that might be swayed by misunderstanding or the personal biases of study personnel. The consensus-based case review process described here uses expertise from across the research team and a stepwise procedure to systematically render and support these decisions.

This eligibility determination process is not without its limitations. The diagnosis of spinal disorders is challenging [31, 38, 39]; participants often present with multiple overlapping conditions and a diagnosis is often based on the clinical interpretation of a wide range of information. We include multiple clinicians to aid this process. However, participants are not present during panel discussions of their case to provide clarifying information about their health history due to the challenges and costs of scheduling this additional visit. When the case review panel determines that additional information is needed, the eligibility decision is pended and the participant is re-contacted by a research clinician. Additional information is then presented to the panel at a future case review meeting for final eligibility determination. In addition, the act of verbally polling the opinions of panel members, necessary to confirm if reasons for individual eligibility decisions are consistent with study aims and operational definitions, can potentially hinder the free expression of thoughts and influence panel members.

This case review process is limited by personnel schedules and the amount of time and resources study teams allot to the process of eligibility determination. Our research center and personnel dedicate more time to the case review process than may be available elsewhere. We estimate that this case review process may work sufficiently with as few as three to four panel members. Further, key components of the case review process are transferrable to other research environments, including those that train students and clinicians to conduct clinical research [40]. We allocate extra time to case review at our research center to clarify procedures and to help educate graduate students serving as panel members on relevant topics such as clinical trial management [13], diagnostic reasoning, research ethics, and other critical aspects of clinical research. With experienced personnel and a developed process, the efficiency of case presentation and eligibility determination can be improved.

Exclusions during baseline visits conducted by a single study coordinator do not involve clinical decision making; rather, they are determined from objective measurements (e.g., weight, pain level). However, it is possible that enrollment error could occur due to incorrect data entry. While individual research clinicians also make eligibility determinations requiring clinical evaluation during the baseline process, these decisions are focused on facets of the participant’s health history that are similarly objective (e.g., surgical status in past year). Though procedures are designed to mitigate selection bias, subjective decisions are still made by personnel subject to unintentional bias and some decisions are made by a single individual when consensus is not reached.

Conclusions

Case review is based on the concept of providing a formally structured decision-making process that includes at least one person from each of four roles: the consenting coordinator, examining clinician, project manager, and an investigator. It also includes designated steps to facilitate thorough case presentations, incorporate user-friendly and secure access to operational definitions for eligibility across multiple trials conducted simultaneously, moderate structured member discussions, and provide a formalized decision-making process to render eligibility determinations that reflect the scientific objectives of a clinical trial.