Investigators and Patients
This was a multicenter usability study, performed in France in compliance with Good Clinical Practice according to European directives and French laws (EudraCT reference number: 2014-A0141245). The investigators were three rheumatologists with mixed private/hospital activity. All study assessments were performed during a single consultation.
The trial was carried out with 42 subjects allocated to three user groups. The first two groups were composed of RA patients split by their hand disability measured by Cochin scale [7, 8] (Table 1). Patients with high hand disability (Cochin score ≥20) and low hand disability (Cochin score <20) were enrolled respectively in groups 1 and 2. The third group was composed of professional nurses or caregivers already involved in RA patients’ treatment. For RA patients, any previous treatment by any auto-injector device was an exclusion criterion.
Study Product and Simulated Injections
The study product was an autoinjector from SHL Group aimed to MTX subcutaneous (SC) self-administration. It is a single-use, disposable, fixed-dose needle-based injection system with automatic functionality (NIS-AUTO) device (Fig. 1).
After the removal of the cap, the button-free device is activated simply by pressing the needle shield directly against the injection site and holding it during the MTX distribution (maximum duration of 10 s). The beginning and the end of the injections are signaled to the subject by sounds, called “clicks” and by slight vibrations.
For this study, the autoinjector has been assembled with a pre-filled glass syringe containing 0.6 ml water for injection. No active MTX was used in the study and only simulated-use of the autoinjector was performed on foam pads mimicking skin behavior (Fig. 2).
All the subjects included in the study were ≥18 years old and were able to understand the study procedures, which was materialized by signing the informed consent. Additional inclusion criteria for the first two groups were: RA, according to 2010 ACR/EULAR classification ; Cochin score ≥20 for the first group and Cochin score <20 for the second group; absence of previous treatment by any autoinjector device.
The study consisted of two phases. During the first phase, the instructor (a trained person from the investigator’s staff or the investigator himself/herself) gave the autoinjector and its instructions for use (IFU) to the subject without further explanations. Immediately after reading the IFU, the subject performed the first simulated injection, his/her performances were evaluated by an independent observer using a detailed questionnaire, called “score card” (Table 2).
At the end of the simulated injection, the instructor interviewed the subject and completed the usability questionnaire: part 1 (understanding the IFU and general impression) and part 2 (usability questionnaire) (Table 3).
At the beginning of the second phase, the instructor could provide explanations concerning the IFU upon request of the subject. Afterwards, the subject performed two simulated injections: in an optimal environment (full light, no noise) and in a “worst-case” home environment (dimmed light, TV or radio on, phone ringing). During both simulations, the independent observer assessed the performance in compliance with the IFU using the same “score card” as during the first phase. At the end of the second phase, the instructor interviewed the subject and completed part 2 of the usability questionnaire and part 3 (satisfaction) (Table 4).
The protocol, the informed consent form and all other documents related to the study were submitted for review to Independent Ethics Committee and to ANSM (Agence Nationale de Sécurité du Médicament et des produits de santé). These two regulatory authorities confirmed that this study didn’t meet the biomedical research criteria as defined in Article R.L1121-1 of the French Public Health Code. Nevertheless, the Ethics Committee stated that there was no ethical obstacle to do this study (notification number: MD-092014).
The primary objective of this study was to evaluate the usability of the new MTX autoinjector based on the rate of correct handling during the second phase “worst-case” injection according to the “score card” and on the rate of positive responses of the usability questionnaire.
The secondary objectives were to assess the understanding of IFU (usability questionnaire part 1), the usability (usability questionnaire part 2), the subject satisfaction (usability questionnaire part 3), and the correct device handling during the first phase and the optimal environment simulation during the second phase.
Determination of the Sample Size and Statistics
According to the recommendations by the FDA, usability studies should include 15 subjects from each major user group . Therefore, initially, three groups of 15 subjects were scheduled to be enrolled in the study. As the investigators had recruitment difficulties in completing the first user group (RA patients with Cochin score ≥20), they decided to stop the recruitment process when 12 subjects were enrolled in this group.
Statistical analyses were performed using SAS® version 9.1. Considering the sample sizes, only descriptive analyses were performed.
The handling steps according to the “score card” were considered correct if the observer filled “Yes, immediately” or “Yes, but after some hesitations or problems”, and incorrect if “Not at all” was filled (Table 2). The rate of correct handling steps for each subject was the number of correct steps out of total number of answers. A rate of 75% was considered as a “correct handling of the device”.
For the usability questionnaires were considered as positive answers “Yes”, when the choice was “Yes/No opinion/No” and the two first points of the adjusted 5-point Likert scale when applicable (Table 3). The neutral answers were “No opinion” and the 3rd point of the Likert scale; the neutral answers have not been taken into account in the total number of answers. Answers “No” and the two last points of the Likert scale were considered as negative. The positive response rate was calculated for each subject as follows: number of positive answers out of the sum of positive and negative answers. A rate of 75% was considered as “positive usability”. Safety analysis was carried out on the full analysis set.