Background

The requirement to randomly allocate cases to treatment and control conditions is a crucial aspect of experimental designs, with transparency and integrity of assignment being central to the delivery of experiments. These considerations have been set out in detail (Ariel et al. 2012). This note serves to build on that position rather than repeat it.

TheRandomiser.co.uk

This note describes a new online portal that allows researchers to deliver RCTs. TheRandomiser builds on the learning from the original Cambridge Randomiser (Ariel et al. 2012) and delivers a number of extra features that make the tool more useful. TheRandomiser allows secure and user-friendly randomisation with researchers able to specify questions to users that ensure only eligible cases are randomised. The major improvement this new version of the tool, now just called “TheRandomiser” (TR), offers over most other methods is the ability of researchers to set up experiments themselves, without requiring the input of an IT specialist. The rest of this note serves to document the functionality and randomisation process built into TR. This will be done by means of a fictional, worked example.

Procedure

Let us consider a fictional researcher who has designed an RCT to evaluate the impact of referring victims of burglary to a charity that offers target hardening advice. The treatment is for the local neighbourhood officer to send the victim a postcard with their details and the details of the charity. Only those who live in privately owned accommodation are eligible for the advice, and the charity will only accept referrals for those owners aged over 50. The trial will be conducted across 20 neighbourhood policing teams, each of which has five officers. The charity is able to offer advice to 200 home owners.

The researcher goes to www.therandomiser.co.uk and signs up for a free account—and is automatically emailed a password. On receipt of the password, the researcher follows the link in the email to login to the administrative section of the site, at www.therandomiser.co.uk/admin. The researcher creates a new experiment. “TheRandomiser” (TR) requires entry of the name of the experiment (researchers can have multiple experiments). TR asks for the unique reference number for each case to be randomised—in this case, the crime reference number. TR then asks for:

  • The email addresses of those to be notified when a new randomisation occurs,

  • The number of treatment conditions,

  • The expected number of cases in each treatment.

Researchers can select up to four treatments and a control, and request up to 15,000 cases for randomisation. The researcher in our fictional study would select one treatment and request 400 cases—200 treatment and 200 control. There is no disadvantage to requesting slightly more, as the systems randomises in blocks.

Once the experiment is created, the researcher can use TR to customise further features of the experiment. TR can be set to request further reference numbers (such as officer’s shoulder number). The researcher can also make TR require users to specify the text to be shown when the randomisation is completed. In our study, for example, the options would be “Send a postcard” versus “Don’t send a postcard”. Finally, the researcher can set the qualification (case eligibility) questions that must be completed correctly for the randomisation to be conducted. In our experiment, these would be “Is the property privately owned?” and “Is the victim aged over 50?”

At this stage, the experiment is ready to be deployed, and the researcher can add the email addresses of all the neighbourhood officers who will work on the trial. As their email addresses are entered into TR, the officers will have a password generated which will be sent automatically to them. This password can be changed by the user when they login.

The neighbourhood officers can then go through their local burglary reports, entering the details into TheRandomiser, and send postcards to those selected for the treatment. Should an officer enter a crime number that has already been randomised, TheRandomiser will identify that and alert the officer.

Key Features

Randomisation Algorithm

The algorithm used by TheRandomiser is a MySQL function “ORDER BY RAND()”. This selects a result in a manner which can in no way be influenced by the researcher nor the treatment provider and gives assurance that there is true random assignment.

Block Randomisation

In order to facilitate effective evaluation during trickle-flow treatment assignment experiments (Sherman 2010), TheRandomiser performs block randomisation. The block size is the number of treatment conditions multiplied by five. For one treatment and one control (two conditions), the block size is ten, so in every ten randomisations performed sequentially, five will be treatment and five control. For two treatments and one control (three conditions), the block size is fifteen, and so on.

Pre-qualification (Case Eligibility) Questions

Researchers can create as many pre-qualification questions as they require, and select whether “yes” or “no” is the required answer. This prevents the situation where treatment providers simply check “yes” to all questions without care. Researchers can also set TheRandomiser to give a “are you sure all the answers are correct” popup on completion of these questions.

Rejection of Previously Assigned Cases

TheRandomiser checks the unique reference number entered by the treatment provider against previously randomised cases. If there is a match, TR identifies that fact to the user, reminding them of the previous assignment.

Multiple Treatment Groups

TheRandomiser supports up to 4 treatment groups and a control group meaning that, for example, our fictional researcher could design four different postcards and evaluate the difference between each of them and the control group.

Error Rate

No instances of software error have been identified to date.

Zero Cost

The online portal is provided at no cost to researchers.

One Big Drawback

Personal Data

TheRandomiser is stored on a secure server with SSL encryption. This gives good assurance of the security of data uploaded. However, the system should not be used to store personal data as this may breach data sharing protocols, especially when considering suspect data. TheRandomiser does not sit within any individual police force, so for any force to store personal data on TheRandomiser may breach data sharing laws—researchers would effectively be sharing personal data outside their agency. Researchers are required to acknowledge and accept this fact when they sign up and it is important that this is recognised. It does lead to a number of issues, which are believed to be unavoidable at the moment. For instance, tracking offenders between multiple custody suites is difficult. Since the name, date of birth, or any other personal data of the suspect cannot be entered into TheRandomiser, identifying whether that person has been previously randomised is challenging. The solution is less than ideal, but has been shown to work in a number of experiments—a secure spreadsheet is stored in force, and treatment providers enter the details of the subject into that spreadsheet, checking whether they have previously been entered. The spreadsheet assigns a unique reference number to that subject which can then be entered into TheRandomiser to be linked to the spreadsheet by the researcher later. This necessary, but frustrating hurdle seems unavoidable, but TheRandomiser still facilitates the randomisation process and importantly, the pre-qualification questions.

Conclusion

TheRandomiser is a useful tool for researchers and the evidence seems to suggest that many researchers are benefitting from it. The authors welcome feedback, requests for further features, and ideas to improve TR.