Background

Loss to follow-up (LTFU) in clinical trials is an important source of bias that can affect statistical power and generalisability of findings, and trials with large LTFU can be downgraded in systematic reviews. Systemic differences in LTFU between groups can result in attrition bias [1].

Successful follow-up requires participants to be found and also willing to participate with the data collection procedure [2]. LTFU can be classified as missing at random (MAR) or not missing at random (NMAR), such as increased attrition in younger people. In this situation a complete case analysis, restricted to participants with available follow-up data, may produce a biased estimation of the true effect. A common approach to dealing with missing outcome data is to impute outcomes and treat them as if they were real measurements, known as multiple imputation [3]. The aim of this paper is to assess factors associated with LTFU in the MObile Technology for Improved Family Planning (MOTIF) trial in Cambodia and compare how the result might have varied using different analysis methods.

Main text

For the MOTIF trial, the primary outcome—self-reported use of an effective contraceptive—was assessed by attempting to contact participants by phone at 4 and 12 months. We used evidence-based methods to try to reduce the chance of LTFU, such as collecting as many possible phone numbers on recruitment, including those of friends or family members, with consent of the participant [4, 5]. At least three attempts were made to contact participants by phone. If it was not a convenient time for the participant to talk, another time would be arranged. At the time of the trial, few participants used email, there was not a functioning postal system in rural areas in Cambodia, and we had limited resources with which to visit participants’ homes. At 4 months we obtained primary outcome data from 431 participants (86%). Amongst the 69 clients LTFU, 6 withdrew, and we were unable to contact 63 participants. At 12 months we obtained primary outcome data from 328 participants (66%). Amongst the 172 clients LTFU, 8 withdrew, and research assistants were unable to contact 164 participants.

For the primary analysis we undertook a complete case analysis, similar to the approach used in similar trials of interventions delivered by mobile phone to increase contraception use [6, 7]. Using a complete case analysis, we found that the intervention increased self-reported use of an effective contraception method at 4 months post-abortion (64% vs. 46%; risk ratio [RR] 1.39, 95% CI 1.17–1.66; p < 0.001) but not after 12 months (50% vs. 43%; RR 1.16, 95% CI 0.92–1.47; p = 0.208).

Investigating missing data

Reasons, as documented by the research assistants, for LTFU at 12 months amongst these 164 participants were as follows: 76 (46%) times the phone was switched off; 57 (35%) times there was an automated response that the number was not in use and not yet re-assigned; 16 (10%) times someone else answered the phone; in 10 (6%) cases where the participant had moved abroad; in 3 (2%) cases an automated response indicated that the phone could not receive incoming calls; and in 2 (1%) cases the participant did not want to talk.

We conducted an exploratory analysis to identify predictors of LTFU from the variables collected at trial recruitment to assess whether data were MAR or NMAR. To maximise statistical power, binary variables were created from some of the categorical variables. First we undertook univariable analysis to examine the crude association between each baseline variable and LTFU, expressed as an OR with a 95% CI. In the univariable analysis, age < 25 years, lower socio-economic status (SES; i.e., no access to motorised transport), not planning to use post-abortion contraception and not providing an additional friend/family contact number were associated with LTFU at 4 months (Table 1). Age < 25 years, lower SES and usually or sometimes having phone credit were associated with LTFU at 12 months (Table 2). There was no evidence of differential LTFU between those assigned to the intervention or control arm at 4 or 12 months.

Table 1 Predictors of missing data at 4 months
Full size table
Table 2 Predictors of missing data at 12 months
Full size table

We then undertook multivariable analysis to explore the association between variables and LTFU, controlling for confounding, using logistic regression analysis. Age and SES are associated with contraception use and may be associated with seeking LTFU [8, 9]. We therefore considered these variables as confounders in the analysis. In the multivariable analysis, age < 25 years, lower SES, not planning to use post-abortion contraception and not providing an additional friend/family contact number remained as predictors of LTFU at 4 months; LTFU was significantly increased amongst women aged < 25 years compared with those aged > 25 years (OR 2.31; p = 0.002), amongst women without access to motorised transport compared with women with access to motorised transport (adjusted OR 2.44; p = 0.008), amongst women not planning to use post-abortion contraception compared with women planning to use it (OR 2.34; p = 0.05), and amongst women who did not provide additional contact numbers compared with those who did (OR 2.40; p = 0.002) (Table 1).

At 12 months, age < 25 years, lower SES, and phone credit remained as predictors of LTFU; LTFU was significantly increased amongst women aged < 25 years compared with those aged > 25 years (OR 1.59; p = 0.023), amongst women without access to motorised transport compared with women with access to motorised transport (OR 2.04; p = 0.009), and amongst women who usually (OR 1.85; p = 0.014) or sometimes (OR 1.59; p = 0.046) had phone credit compared with women who always had phone credit (Table 2).

We then undertook two different analyses to compare with the complete case analysis (Table 3). First, we undertook multiple imputation to estimate the effect of the intervention on the primary outcome at 4 and 12 months. This involved replacing each missing primary outcome value with a plausible imputation based on the characteristics of missing values observed [10]. Because LTFU was NMAR, variables associated with LTFU were used in the multiple imputation model. Thus, logistic regression with the predictor variables age group, SES, post-abortion family planning intentions and whether an additional friend or family contact was provided was used for univariate imputation of the outcome at 4 months and the predictor variables age group, SES and phone credit for imputation of the outcome at 12 months. Multiple imputation was conducted separately for the outcomes at 4 and 12 months with 100 imputed datasets created for each outcome, respectively. Using multiple imputation, the intervention also showed an effect at 4 months (63% vs. 45%; RR 1.42, 95% CI 1.19–1.69; p = 0.001) but not at 12 months (48% vs. 43%; RR 1.12, 95% CI 0.89–1.40; p = 0.349). Second, we conducted sensitivity analyses treating all participants LTFU as non-users of contraception (extreme case scenario) to estimate the effect of the intervention on the primary outcome at 4 and 12 months [11]. In this analysis self-reported effective contraceptive use also remained significantly increased when we considered participants LTFU as non-users at 4 months (54% vs. 40%; RR 1.35, 95% CI 1.12–1.63; p = 0.002) but not at 12 months (34% vs. 27%; RR 1.25, 95% CI 0.95–1.63; p = 0.106).

Table 3 Effect measure variation according to analytical method used
Full size table

Table 3 shows how the effect measure varies according to the analytical method used. Using multiple imputation, we found that the RR was slightly increased at 4 months and slightly decreased at 12 months compared with the complete case analysis. When counting all participants LTFU as non-users of contraception, we observed that, compared with the complete case analysis, the RR was slightly decreased at 4 months and slightly increased at 12 months. Despite the changes in the RR, use of the different analytical methods did not result in an effect becoming statistically significant or vice versa.

Conclusion

Factors associated with LTFU in the MOTIF trial were young age, lower SES, not planning to use post-abortion contraception, phone credit and not providing additional contact numbers. Reasons for LTFU amongst younger women might be that they changed phone numbers (could not be found) or perhaps the potential stigma of contraceptive use in young people (no longer wanting to participate in the trial). Younger participants were also found to be at increased risk of LTFU in a head injury trial [2]. LTFU amongst women of lower SES might be due to changing phone numbers or no longer having access to a phone. Women not planning to use contraception might have been less motivated to participate in the trial follow-up. The finding that LTFU was increased in those not providing multiple contacts is consistent with existing evidence [4]. Although the RR showed slight variation using multiple imputation and counting those LTFU as non-users, there was no significant change in the principal findings.

Whilst this study provides some insights into factors associated with LTFU in this trial, it may have limited generalisability to other studies. This study had a relatively small sample size and was an exploratory rather than a pre-specified analysis.

Future studies assessing contraception use might anticipate increased attrition amongst younger participants and those of lower SES or who do not provide additional contact details. Attrition could be reduced by collecting as many contact details as possible, providing incentives, and possibly providing enhanced counselling to groups at higher risk of LTFU on recruitment [4]. Multiple imputation should be considered in addition to complete case analysis if LTFU NMAR is expected or observed.