Promoting Recruitment using Information Management Efficiently (PRIME): statistical analysis plan for a stepped wedge cluster randomised trial within the REstart or STop Antithrombotics Randomised Trial (RESTART)
- 375 Downloads
Promoting Recruitment using Information Management Efficiently (PRIME) is a stepped wedge, cluster randomised trial-within-a-trial of a complex intervention to help sites in the United Kingdom to attain their own target number of participants to recruit to the REstart or STop Antithrombotics Randomised Trial (RESTART, ISRCTN71907627).
Seventy-two hospital sites had opted into PRIME and were randomly allocated (using a computer-generated block randomisation algorithm, stratified by hospital location) to one of 12 months in which a complex intervention would be delivered. All sites began in the control state. The primary outcome is the total number of patients randomised into RESTART per month per site, which will be analysed in a negative binomial generalised linear mixed model. Secondary outcomes include the proportion of sites using stroke databases to identify potentially eligible patients before PRIME, frequency of using bespoke stroke audit data exports during PRIME, barriers to recruitment in PRIME, barriers to using the bespoke stroke audit data exports, and disadvantages of the bespoke stroke audit exports identified by PRIME sites. PRIME began in September 2015. The last intervention will be delivered in August 2016. Six-month follow-up will be complete in February 2017. This statistical analysis plan was written and submitted for publication before all sites received the PRIME intervention and before outcome data were known.
Final results of PRIME will be analysed and disseminated in 2017.
Northern Ireland Hub for Trials Methodology Research SWAT repository (SWAT22).
KeywordsStatistical analysis plan Complex intervention Stepped wedge trial Cluster randomised trial Recruitment Study-within-a-trial Trial-within-a-trial
Generalised linear mixed model
Promoting Recruitment using Information Management Efficiently
REstart or STop Antithrombotics Randomised Trial
The REstart or STop Antithrombotics Randomised Trial (RESTART, ISRCTN71907627) is an on-going randomised controlled trial (RCT) of secondary prevention after stroke due to intracerebral haemorrhage (ICH). Promoting Recruitment using Information Management Efficiently (PRIME) is a stepped wedge, cluster randomised trial of a complex intervention to help RESTART sites increase their recruitment and attain their own target number of participants. Seventy-two hospital sites that were active in RESTART in 2015 opted into PRIME, so the trial can be considered as a study-within-a-trial (SWAT).
Discussion on how staff at the site have been finding RESTART to date
Review of the recruitment commitment made at the site initiation visit and how many patients they have recruited
Review of the information in the pre-review questionnaire about recruitment completed prior to the recruitment review
Explanation of the availability of bespoke stroke audit data exports and examples provided of how the pilot sites effectively used them
Review of the use of a template invitation letter for approaching prevalent patients and discussion about how they can be used effectively in conjunction with the bespoke stroke audit data exports
Review of the opportunities to recruit inpatients and outpatients at the hospital site
Sharing of what other methods top recruiting sites have been using to identify, consent and randomise patients
After the review, an email is sent to all the collaborators at the site summarising what had been discussed at the review, providing attendance certificates, and giving instructions for running the relevant bespoke stroke audit data exports
A stepped wedge design was chosen because it would have been very difficult to apply the intervention across all intervention clusters simultaneously in a standard, parallel-group, cluster randomised trial design. By allowing a phased introduction of the intervention in a stepped wedge design, this meant that it was only necessary to have a single PRIME coordinator to implement a tailored intervention at each individual site; and so it was logistically and practically easier to implement the intervention. Furthermore, all sites will have been given a potentially efficacious intervention by the end of the trial, whilst the expected burden on these sites was perceived to be low. There was also no foreseeable negative impact on patients in terms of harm or burden due to the intervention. In addition, we would expect that the intracluster correlation coefficient might be relatively high in this trial as a result of high variability in average recruitment rates between sites; leading to possibly greater statistical power for the stepped wedge design compared to a standard cluster trial design .
PRIME has a closed-cohort, stepped wedge design whereby all participating sites are included from the start through to the end of the trial, and we are interested in assessing the recruitment rate within fixed time periods . The timing of the recruitment review was randomised, stratified by hospital location (Scotland versus England/Wales). This involved using computer-generated block randomisation to randomly allocate the 72 sites to groups of six with 12 different months for the recruitment review in one calendar year, stratified by hospital location. The investigators, recruitment coordinator, and sites were all blinded to the timing of the recruitment review until this had to be revealed in order to organise the recruitment review. Members of staff at each site were asked to complete questionnaires before the recruitment review and 6 months afterwards enabling us to characterise sites, identify barriers to recruitment, and determine any perceived disadvantages to applying the intervention. Further details about the PRIME trial design, including the sample size calculation, can be found in the trial protocol paper . In this article we describe the statistical analysis plan which outlines how data analysis will be conducted at the end of the trial. The statistical analysis plan was finalised and approved by the co-authors on 4 April 2016.
The primary outcome is the total number of patients randomised into the RESTART trial per month per site.
Number of sites in PRIME that routinely used stroke databases to identify potentially eligible patients before receiving the recruitment review
Number of sites in PRIME that used the bespoke stroke audit data exports, and the frequency of their use in the 6 months after the recruitment review
Barriers to recruitment in PRIME, identified by sites in the questionnaires completed before, at, and 6 months after the recruitment review
Barriers to using the bespoke stroke audit data exports identified by PRIME sites at the 6-month follow-up review
Disadvantages of the bespoke stroke audit exports identified by PRIME sites at the 6-month follow-up review
General analysis principles
Analyses will include all sites as randomised unless otherwise stated below. We will include all 72 sites that agreed to take part in PRIME regardless of the circumstances of implementation of the recruitment review or subsequent withdrawal. Any sites for which the recruitment review was not implemented at the correct time or failed to be implemented will still be included in an intention-to-treat analysis provided that outcome data are available.
There will be no imputation of missing data: missing values will be left as missing for all statistical analyses.
In general, categorical data will be presented using counts and percentages, whilst continuous variables will be presented using the mean, median, standard deviation (SD), minimum, maximum, first quartile, third quartile, and number of sites with a response (n).
Outliers will be identified by viewing boxplots or histograms and will be queried at the data checking stage if an error is suspected. All analyses will include outliers as standard.
All statistical tests and confidence intervals will be two-sided. Ninety-five percent confidence intervals will be presented with the significance of p values assessed based on a 5% significance level. No adjustment for multiplicity will be made in any of the analyses. There is a single primary outcome. The secondary outcomes are exploratory, and interpretation of their analyses will be suitably cautious.
There will be no interim analysis of the trial data and the final analysis will be performed after all follow-up data have been collected.
List of analyses
Recruitment of sites and retention
Descriptive analysis of baseline characteristics
Site location (Scotland versus England/Wales)
Average number of patients randomised per month in the first 6 months of the trial, before any of the sites were scheduled to implement the intervention (March 2015 to August 2015)
Proportion of stroke inpatients, who are suitable for follow-up, seen in clinic after hospital discharge
Number and percentage of sites that had approached patients looked after by their stroke unit in the past to invite them back to clinic with a view to recruit them to RESTART
Number and percentage of sites with complete and accurate stroke audit data
Number and percentage of sites already routinely using the stroke audit data to recruit to RESTART
Number and percentage of sites using screening logs as a source of information to identify eligible RESTART patients
Number and percentage of sites using a database other than the stroke audit as a source of information to identify eligible RESTART patients
Number and percentage of sites using another source of information (other than databases or screen logs) to identify eligible RESTART patients
Number and percentage of sites already using other methods to boost recruitment
Number and percentage of sites already identifying barriers to finding suitable patients to recruit to RESTART
Primary analysis of the primary outcome
Informed by a published method to model recruitment , we will fit a negative binomial generalised linear mixed model (GLMM) to the primary outcome, adjusting for the fixed effects of time since start of study (in months), season (December/January (when recruitment rates in RESTART have been low) versus all other months), site location (Scotland versus England/Wales), and an indicator variable for whether the PRIME complex intervention has been implemented or not (according to the planned randomisation schedule). Site will also be included as a random effect in the model. For the primary analysis, the secular change in recruitment rate over time will be adjusted for in the models using a single continuous linear term, unless the graphs of recruitment rate over time strongly indicate otherwise.
The results will be expressed as rate ratios with 95% confidence intervals. The number of decimal places will be determined by the ‘rule of four’ . The estimated time effect will also be reported from the fitted model.
The primary analysis will follow an ‘as-randomised’ principle, which means that the data will be analysed according to the randomised timing of the recruitment review rather than the actual time that the recruitment review occurred. In addition, for the primary analysis, all sites will be included regardless of any site withdrawals and/or compliance with the PRIME trial procedures.
Secondary analyses of the primary outcome
The negative binomial GLMM will additionally include model terms representing the length of time after the randomised introduction of the intervention to assess whether there is a time trend in the effect of the complex intervention on recruitment rate. This will consist of including categorical dummy variables in the model representing the categories ‘3-6 months after recruitment review’ and ‘more than 6 months after review’; with ‘first 3 months after review’ as the reference category
A per-protocol ‘actual times’ analysis will be performed: based on the actual times of the recruitment review, and only including those sites that continued to be active in both the RESTART and PRIME trials until the end of the PRIME study period
The analysis will be restricted to use only data collected within the rollout period (i.e. the period of time when there were sites in both intervention and control conditions). This is to reduce the risk of confounding due to secular changes over time [2, 4], at the expense of a reduced sample size. An additional site-level explanatory variable will be included in the GLMM: the average number of people randomised per month before the PRIME rollout phase (i.e. before the first site received the intervention)
We will assess the sensitivity of the model results with respect to different ways of specifying the adjustment for secular trend in the models. For example, we may model time using a categorical factor variable or by using spline functions that may more appropriately reflect the change in recruitment rates observed in the graphical analysis of primary outcome (i.e. in plots such as those depicted in Figs. 2 and 3)
Primary outcome analysis assumptions
All analyses of the primary outcome assume that the evaluation will not be confounded by other simultaneous methods to boost recruitment, and that the recruitment coordinator’s effectiveness will remain constant over time (i.e. it will not be diluted by the increasing number of sites that she will support over time, or influenced by seasonal changes in trial recruitment).
Secondary outcome analyses
The number and percentage of sites generating and using the stroke audit data exports
The number and percentage of sites experiencing any problems running or using the audit reports out of all those answering ‘yes’ to the question of whether they extracted and used the stroke audit data exports
We will also report the number and percentage of sites that had used a template invitation letter to invite potential RESTART patients to clinic before the recruitment review took place, as recorded in the pre-recruitment review questionnaire. (The template invitation letter was originally sent out to RESTART sites in a substantial amendment notification (Ref. 12/SS/0138, amendment REC REF AM18/1) in May 2015.)
How far back the site ran the reports to
How many times the site ran the reports
The number of patients identified by the audit reports
The number of eligible patients identified by the audit reports
The percentage of patients who were actually eligible out of all those identified by the audit reports
The number of eligible patients who the site contacted
The percentage of patients who were actually contacted out of all those eligible
The number of eligible patients responding
The percentage of patients responding out of all those contacted
The number of eligible patients who came back to clinic
The percentage of patients coming back to clinic out of all those responding
The number of eligible patients declining to come back to clinic
The percentage of patients declining to come back to clinic out of all those responding
The number of patients who were randomised as a result of being identified by the stroke audit data exports
The percentage of patients who were randomised out of all patients who came back to clinic
Question 10 of the questionnaire asks ‘Do you think that the reports were useful in identifying potentially eligible patients?’ and answers are on a 5-point Likert scale ranging from ‘Strongly Agree’ to ‘Strongly Disagree’. For this outcome, the number and percentage of sites in each category will be reported; and a bar chart will be used to display the data graphically.
The final three questions require mainly qualitative responses and so, if appropriate, frequency tables will be produced to show the most frequent responses.
The number of complaints from PIs and site co-ordinators about the interventions will be reported, along with the number of sites making at least one complaint. The qualitative reasons for these complaints will also be listed and categorised in a frequency table if appropriate.
We will record any costs of implementing the interventions (for the stroke audits, local RESTART investigators, and participants in RESTART) if available, and we will report corresponding summary statistics (e.g. mean and SD).
We will report PRIME in a manner consistent with the adaptation of the CONSORT reporting guidelines for cluster randomised trials , the guidelines for reporting embedded recruitment trials , and the recommendations for reporting stepped wedge trials proposed by Hemming et al.  and Davey et al. .
We thank the Scottish Stroke Care Audit and the Sentinel Stroke National Audit Programme for providing the data extraction tools and supporting their implementation. We thank the RESTART collaborators who took part in PRIME. We thank Jonathan Drever and Ronnie Harkness for their help with PRIME. RAP and CJW are supported in this work by NHS Lothian via the Edinburgh Clinical Trials Unit and RA-SS was supported by a MRC senior clinical fellowship.
The British Heart Foundation funds PRIME (PG/14/50/30891) and RESTART (SP/12/2/29422). RA-SS was funded by a MRC senior clinical fellowship.
Availability of data and materials
The datasets acquired and/or analysed during PRIME will be available from the corresponding author on reasonable request after the publication of the PRIME results.
RAP and CJW drafted and finalised the statistical analysis plan. CJW wrote the statistical analysis section in the trial protocol. RA-SS is the chief investigator for PRIME and RESTART and approved the statistical analysis plan. AEM is the recruitment coordinator and also approved the statistical analysis plan. All authors read and approved the manuscript for publication.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
We obtained approval from the Scotland A research ethics committee on 24 April 2015 (Ref. 12/SS/0138, amendment REC REF AM18/1) for two template invitation letters for RESTART investigators to use in PRIME and RESTART to approach potentially eligible prevalent patients, identified using the bespoke data exports, and seen by their stroke service. RESTART investigators opted in to participate in recruitment reviews.
- 3.Maxwell AE, Dennis M, Rudd A, Weir C, Parker RA, Al-Shahi Salman R. Promoting Recruitment using Information Management Efficiently (PRIME): study protocol for a stepped wedge cluster randomised trial within the REstart or STop Antithrombotics Randomised Trial (RESTART). Trials. 2017.Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.