Introduction

Knowledge of HIV status is key to earlier access to HIV treatment and prevention services. As an HIV prevention strategy, HIV testing is cost effective, estimated to cost US$249 per HIV-1 infection averted in Kenya [1, 2]. It is also the fundamental entry point to an effective seek, test, treat and retain (STTR) paradigm, which has the potential to bend the curve of the HIV pandemic [3]. In resource limited settings such as in sub-Saharan Africa, the shortage of health care workers has been identified as a barrier in the effort to scale up HIV prevention and treatment services. Other barriers include stigma, health seeking behavior among undiagnosed persons living with HIV (PLWH), especially men, and an overemphasis on facility based approaches to HIV testing services [4]. According to population-based surveys in low- and middle-income countries (LMIC), the median percentage of people living with HIV who know their status is estimated at under 50 % [5]. Given the public health implications of unknown HIV status, especially among undiagnosed PLWH, the availability of self-testing for rapid scale up of HIV testing is compelling [6]; increasing awareness of HIV status is an important step towards reducing HIV transmission and enabling antiretroviral therapy (ART) that reduces mortality as well as secondary HIV transmission. HIV testing guidelines recently released by the World Health Organization (July 2015) highlighted the potential contribution of HIV self-testing (HIVST) to close critical gaps in HIV testing coverage, and to support retesting efforts, worldwide [7].

Data from both blood and oral fluid HIV self-testing studies conducted in Canada [8], China [9], Kenya [10], Malawi [11], Singapore [12, 13], Spain [14, 15], South Africa [16], Uganda [17], and the U.S. [18–21] show self-testing is feasible, acceptable, and accurate. Availability of self-testing is one of several options to increase access to testing, especially in higher-risk subpopulations [9, 21] that may not access current forms of HIV testing, such as HIV-discordant couples, men who have sex with men (MSM), sex workers (SWs), people who inject drugs (PWID), and high-risk youth. Currently available options include voluntary counseling and testing (VCT), provider-initiated counseling and testing (PICT), home-based counseling and testing (HBCT), self-testing home specimen collection, and mobile and outreach HIV testing and counseling. HIV self-test results do not confer diagnosis, as a reactive (positive) HIV self-test must be confirmed according to national algorithms; individuals with non-reactive (negative) HIV self-test results should be provided with information about re-testing, especially if they had a recent exposure or are at on-going risk. In the US, where approximately 1 in 5 people do not know they are infected [22], the Food and Drug Administration (FDA) approved the OraQuick In-Home HIV test kit, the first HIV self-test (HIVST) kit for sale directly to consumers over-the-counter (OTC) and online [19]. However, such an approach has not yet been implemented as a standard option for non-health professionals in LMIC countries, e.g., sub-Saharan Africa, where two-thirds of all HIV infection globally exists.

Kenya has been a leader in innovative approaches to HIV prevention and care. The recent KAIS [23] national survey found that levels “of HIV testing have increased with 72 % of adults aged 15–64 years in 2012 reporting ever having been tested for HIV, a significant increase from 34 % in 2007.” Ambitious population coverage targets for HIV serostatus knowledge have been set, yet even with large-scale HBCT programs, VCT scale up, and other approaches; there still remains a coverage gap as noted by KAIS (one that is larger for men than women). This suggests HIVST has a place as part of a comprehensive testing strategy. Kenya has successfully piloted HIVST among health workers [10] and is the first African country (and one of the first countries globally) to develop policy guidelines [24] around HIVST for the general public. Our objective is to evaluate the performance and accuracy parameters of unsupervised oral fluid HIV self-testing among adult lay users in the general population of Kenya. The study aims were to determine (1) the ability of participants with unknown HIV status to correctly perform and interpret a rapid oral fluid (OF) HIV test and compare their results to staff/lab testing (i.e., sensitivity and specificity measures) and (2) participant attitudes and preferences towards OF self-testing (i.e., OF HIVST acceptability and feasibility).

Methods

Participants

Study participants were recruited from three sites: a health care facility that has expanded beyond HIV-only care provision and two community workplace settings in Eldoret, Kenya from November 11–29, 2013. Study flyers with mobile phone contact numbers were used to recruit at all three sites, along with provider word of mouth (clinical site) and employer information with referral to our study staff at the occupational sites. Eligibility criteria included (1) adults (≥18 years old), (2) who do not know their HIV status because they have never tested or their last HIV test was negative, and (3) live within 1 h of public transit travel time to Eldoret town. Participants received 250 Kenyan shillings (Ksh, equivalent to 3.00 USD) for their time and to reimburse their transportation costs.

Study Procedures

We conducted a prospective validation study comparing unsupervised self-administered OF HIVST among n = 240 individuals to staff-administered OF and rapid fingerstick whole blood (FS) test; all reactive, discrepant, invalid, and a proportion of the negative results were validated using an enzyme-linked immunosorbent assay (ELISA). We also assessed HIVST steps to understand errors in a subset of the 240 participants, who agreed to be videotaped while performing the steps of the self-testing. All participants conducted an OF self-test, followed by staff administered OF (to separate errors due to the lower accuracy of oral specimens versus errors due to participants’ lack of knowledge or experience with the self-testing kit [25] ) and FS rapid diagnostic test. A subset of 20 participants’ self-testing performance was video streamed which enabled study staffers to monitor the steps, in a separate area, in real time using a standardized checklist of steps and errors noted [26]. Two study staff reviewed and compared their checklists, and resolved discrepant observations. Only participants enrolled from the health facility site were asked to be part of the videotaped self-test usability component of the study; every participant was invited at the time of informed consent until 20 were enrolled.

The study was conducted over a single session and enrollment was done with written informed consent of each participant. All study procedures were approved by Moi University College of Health Sciences and Moi Teaching and Referral Hospital’s Institutional Research and Ethics Committee (#1029) and NYU’s University Committee on Activities Involving Human Subjects (#13-9670). Following informed consent, study staff administered a pre-HIVST questionnaire to collect basic demographic information and to assess risks, HIV test history, and HIV self-testing importance, confidence, and concerns. Afterwards, participants performed the self-testing in a private space, using a pictorial instruction sheet with both English and Kiswahili language text and icons, without supervision from study staff members. The OraQuick ADVANCE Rapid HIV 1/2 Antibody Test (OraSure Technologies), a rapid diagnostic oral fluid test kit approved for use by the Kenya Ministry of Health was used. After OF self-specimen collection, participants informed study staffer they completed HIVST by interpreting their results aloud. Afterwards, study staff administered a post-HIVST questionnaire to collect information about participant experience using the self-test kit, how HIVST should be packaged or made available, possible reasons people may or may not use HIV prevention and treatment services, as well as some of the same information collected during the pre-test questionnaire. Study staff then collected an OF sample and a blood FS sample (The Alere Determine HIV-1/2; Alere Medical Co.) and conducted parallel OF and FS rapid HIV tests. A blood specimen was drawn for ELISA confirmatory testing (Vironostika HIV Uni-Form II Ag/Ab; bioMérieux Inc.) at the time of any reactive, discrepant, or indeterminate/invalid OF/FS test results (except for one participant with an invalid OF who refused) and a percentage (29 %) of OF negative results for quality control purposes. The ELISA was selected as the reference standard as it is used in Kenya’s current national guidelines (at the time of the study) as the final serostatus determination assay and we followed these guidelines for the index value/cutoff. A single laboratory staff person conducted and read the ELISA who was trained to national standards in the AMPATH lab, which ISO 15189:2007 certified. Laboratory staff person was blinded to the results of the other tests and did not have access to other clinical information. Post-test counseling was provided according to Kenya Ministry of Health guidelines, after completion of all study procedures. All participants with a reactive test result were notified of their confirmed HIV-positive status once ELISA results were available and referrals/linkage to appropriate HIV care were made.

Sample Size Determination

Assuming sensitivity is 96 % and specificity is 99 %, and an undiagnosed HIV prevalence of 20 %, the total sample size of up to N ≤ 240 (minimum N = 180) provides adequate (≥80 %) power to detect key errors in HIV self-testing steps during this usability/validation pilot, including specimen collection and interpretation. For sensitivity, a null hypothesis of Se = .80 can be rejected with 82 % power when the true value is .96; for specificity, a null hypothesis of Sp = .80 can be rejected with 99 % power when the true value is .99.

For the videotaping we set out to attain a sample of 10 males and 10 females. Samples of this size have been shown to have sufficient power to detect the large majority of usability problems [27].

Statistical Analysis

Statistical analysis was conducted using Stata 13.1 (Stata Corporation, College Station, Texas, USA). Distributions of participants’ characteristics and OF HIVST, FS, and ELISA laboratory results were assessed using contingency tables. With ELISA and staff FS as gold standards, the performance of the rapid OF HIVST diagnostic test was estimated using the diagt module in Stata [28]. Cohen’s kappa [29] also was reported in order to convey the strength of agreement between different HIV diagnostic tests.

Data from the observation checklist for videotaped participants were entered into an excel database and data were analyzed descriptively for common user errors.

Results

We enrolled a total of 240 participants. Figure 1 outlines the status of enrollees. One study participant enrolled was HIV-positive and diagnosed prior to the study; this individual was removed from the analysis.

Fig. 1
figure 1

Enrollment

Full size image

As seen in Table 1, the sample was approximately one third female and two-thirds males. Most participants (90 %) had tested for HIV, while 10 % (n = 23) reported having never tested previously. A handful (n = 8 females, 3 %) said they had performed any kind of self-test before, in this case an over the counter pregnancy test. Mean age was 33, and mean education was 12 years. Two thirds of the sample were laborers. Most participants (88 %) were in a relationship, though we saw only six out of this group who came in with a partner as part of a HIV-discordant couple.

Table 1 Demographic characteristics and sexual risk behaviors
Full size table

Sexual risk-taking was common (Table 1), with 81 % acknowledging sex without a condom in the last month. Of those with a main sex partner, two-thirds said their partners had been tested for HIV, with 7 % of those reporting the partner’s status was HIV-positive (8/9 of those were on ART). Mean number of sex partners in the last month was 1.35, indicating some possible sexual concurrency, a factor thought to contribute to HIV epidemic spread. Around 6 % of participants said they had traded sex for drugs, money, food, clothing, shelter, or any other goods in the last 30 days. Almost half (45 %) said they were less concerned about HIV than they used to be, though 91 % acknowledged being “very worried about getting HIV; HIV stigma was still prevalent (60 % “worried about what people in the community will think if I have HIV”) but not universal.

The majority had a previous history of HIV testing with a mean of 3.77 tests taken (SD 2.30), and most had tested within the last year (Table 2). Previous test sites included VCT (61 %), with 31 % tested at a hospital and 4 % at home. Reasons for not testing included fear and perceived low risk. Concerns about testing HIV positive included stigma, abandonment and violence (all over 50 %), with confidentiality as the main/top reason for not having tested for HIV before.

Table 2 Exposure to services, HIV test history, perceptions and concerns
Full size table

Prevalence of HIV infection was based on staff FS results, since staff FS results were equivalent to the ELISA laboratory blood test (sensitivity and specificity both 100 %) and available for all participants. A total of 35 of 239 participants were positive for HIV infection, indicating prevalence of just under 15 % (14.6 %; 95 % CI 10.7–19.7 %).

Sensitivity and Specificity

Determination of the accuracy of OF HIV self-testing was based on comparison with the ELISA blood test and staff FS rapid HIV testing. Sensitivity was estimated as the proportion of positive ELISA results that were also positive by OF HIVST. Specificity was estimated as the proportion of negative ELISA (or staff FS) results that were also negative by OF HIVST. Among the 239 OF HIV self-testing results, 36 (15.1 %; 95 % CI 11.1–20.1 %) were invalid. More men than women had invalid tests; 7.7 % (n = 6) women and 18.6 % (n = 30) men (p < 0.03, OR 2.7). All individuals with an invalid test did recognize that something had gone wrong with their test and did not misinterpret it as either a negative or positive result. Given that invalids thus represented a form of missing data, and in order to document the most direct estimate of test performance, we excluded invalid test results from analysis of accuracy.

Table 3 compares positive test results by type of test; it is worth noting that the fingerstick results were the same as the lab-confirmed ELISAs in all cases where both were available. Among 29 participants with positive ELISA results, 3 false negatives were observed for OF HIVST (Sensitivity = 89.7 %; 95 % CI 72.6–97.8 %). Among 49 participants with negative ELISA results, 1 false positive was observed for OF HIVST (Specificity = 98.0 %; 95 % CI 89.1–99.9 %). Among 29 participants with positive staff FS results, 3 false negatives were observed for OF HIVST (Sensitivity = 89.7 %; 95 % CI 72.6–97.8 %). Among 174 participants with negative staff FS results, one false positive was observed for OF HIVST (Specificity = 99.4 %; 95 % CI 96.8–99.9 %). All participants positive by FS were confirmed by ELISA, which identified exactly the same people as having HIV infection. However, only a sample of negatives was confirmed by ELISA, hence the slight difference in specificity. When the videotaped cohort participants (n = 20) were excluded from analysis, OF HIVST sensitivity was slightly higher (Sensitivity = 92.9 %; 95 % CI 76.5–99.1 %), and specificity slightly lower (Specificity = 97.8; 95 % CI 88.5–99.9 %).

Table 3 Results by type of test
Full size table

Negative and positive predictive values for OF HIVST relative to ELISA and staff FS results were estimated as well. Among 51 participants with a negative OF HIVST result, 48 were also negative by ELISA (NPV = 94.1 %; 95 % CI 83.8–98.8 %). Among 27 participants with a positive OF HIVST result, 26 were also positive by ELISA (PPV = 96.3 %; CI 81.0–99.9 %). Among 176 participants with a negative OF HIVST result, 173 were also negative by staff FS (NPV = 98.3 %; 95 % CI 95.1–99.6 %). Among 27 participants with a positive OF HIVST result, 26 were also positive by staff FS (PPV = 96.3 %; 95 % CI 81.0–99.9 %).

Cohen’s kappa was .89 (95 % CI .78–.99) for agreement between OF HIVST and ELISA and was .92 (95 % CI .84–.99) for agreement between OF HIVST and staff FS results.

Test Performance Video Data

Twenty individuals were videotaped during their HIVST. Video observation demonstrated that all participants reviewed the instruction sheet prior to performing the self-test. Problems noted covered a range of issues, some minor (e.g., twisting rather than popping the cap, n = 14) and others less so (e.g., placing swab in the buffer solution before sample collection, n = 3). These errors are similar to those that have been noted in a 3-country videotaped HIVST study conducted by PATH [26]. In addition, although not captured during the videotaping session, or from the post-HIVST questionnaire, research assistant field notes anecdotally describe one participant drinking the buffer solution. Outcomes from the test performance video data are summarized in Table 4. The same individual could have made multiple errors.

Table 4 List of HIVST user errors
Full size table

Invalids

By design, samples positive by OF HIVST were over-sampled for ELISA confirmatory testing, so it is not feasible to use ELISA results to compare HIV prevalence among participants with and without invalid OF HIVST results. When staff finger-stick results (available for all participants) are used as a gold standard for HIV infection, those with invalid self-testing results had slightly increased odds of infection, but the difference was not statistically significant (OR = 1.20; p = 0.7977 by Fisher’s Exact Test).

In our study sample, participants recruited from Site 3 had increased odds of an invalid test result (OR = 6.76; p < 0.0001 by Fisher’s Exact Test), and being male also increased the odds of an invalid test result (OR = 2.74; p = 0.0329 by Fisher’s Exact Test). Age (>25 years) and education (<10 years) were not significantly associated with the odds of an invalid OF HIVST result (OR = 2.13; p = 0.3129 by Fisher’s Exact Test, and OR = 1.92; p = 0.1179, by Fisher’s Exact Test, respectively). Participants who had never tested before had increased odds of an invalid result, but this finding was only marginally significant (OR = 2.81; p = 0.0579 by Fisher’s Exact Test).

HIVST Acceptability

As seen in Table 5, results drawn from the post-HIVST behavioral questionnaire closed-ended/quantified questions, the importance of having access to a HIVST increased before vs. after the person conducted their own self-test, going from a mean of 8.1–8.6 (ascending scale 0–10). Confidence in the ability to perform and interpret the self-test appeared to increase with exposure to actually doing the test. Confidence in doing the self-test correctly rose from 6.0 to 7.9 and ability to read results correctly from 6.1 to 8.1. Only 11 participants (4.6 % overall) thought that doing the HIVST was ‘very difficult or difficult’ while 94 % ‘strongly agreed or agreed’ that HIVST was acceptable.

Table 5 Pre and post HIVST importance, confidence, and acceptability
Full size table

Affordability was a main theme in the behavioral survey data, mentioned by n = 192/239. The mean price people were willing to pay for a HIVST was 111 Ksh, around US$1.25 (range 0–1000 Ksh). Female mean price was 78 Ksh and male mean was 158 Ksh, though median price was 50 Ksh for both. The mean price among those under age 25 was 56 Ksh and for those 25 and older was 120 Ksh. (Of note, at the time of the study, the cost of the Oraquick self-test kit was US$7.50 per kit or ~600 Kenyan shillings at an exchange rate of 80. However, the Oraquick test is not yet publicly available for use in Kenya).

Discussion

This HIV self-test validation study conducted among a general population sample of unsupervised lay users in western Kenya found a sensitivity of 89.7–92.9 % and specificity of 97.8–98.0 %. Agreement between OF HIVST and both ELISA and staff FS results was very good. Of note, around one in seven users had invalid results. The study population had an overall HIV prevalence of just under 15 %. By comparison, the KAIS 2012 national household survey found HIV prevalence in the North Rift Province where the study took place (Eldoret is in Uasin Gishu County) was 3.1 %. The higher prevalence seen in our sample as compared with that found in the province during the KAIS 2012 survey suggests somewhat higher-risk individuals self-selected to participate in the study, which may be a selection bias but that at least indicates an interest in HIVST among this epidemiologically and clinically important subgroup of the general adult population.

Of note, 7.7 % (n = 6) women and 18.6 % (n = 30) of the men had invalid HIVSTs. This may suggest less familiarity with health tests among men. Potentially, factors such as literacy and previous exposure to health tests may influence HIVST performance, perhaps suggesting the need for educational campaigns and improved instructions for the self-test kits, particularly for men who had an invalid test rate more than double that of women.

Study limitations include the small sample size and potential volunteer bias. Power to detect differences in sensitivity or specificity based on participant characteristics is low. The sample size for cases with infection is 29, and for cases without infection is 49. Also, because the number of diagnostic errors among those with a valid OF HIVST result is very small (one false positive and three false negatives), it is nearly impossible to identify statistically significant predictors of diagnostic error/accuracy.

Supervised oral self-testing in Malawi found an overall sensitivity of 97.9 % (95 % CI 87.9 %–100.0 %) and specificity of 100 % (95 % CI 97.8 %–100.0 %) [11]. Although sensitivity in our study was lower than expected in comparison, in the overall sample it was nearly 90 % for unsupervised HIVST, similar to the 90 % reported for an unsupervised HIVST study conducted in Uganda where each participant first received a 10-minute HIVST demonstration [17]. When the videotaped subjects were excluded, sensitivity (92.9 %) was similar to that reported to the Food and Drug Administration (phase 3, 92.9 %) when they approved self-testing for general population use in the US. In our study, positive predictive value at this prevalence was reasonable at 96 %, though negative predictive value was lower at 94.1 %. Acceptability of HIVST was high (94 %) and similar to what was found in studies conducted in Malawi [11], Singapore [12], and the US [18]. In conclusion, there is clear interest in, and good acceptability of, HIV self-testing among the general population in western Kenya that may help expand knowledge of serostatus. Any rollout of HIVST should be accompanied by education around appropriate use to reduce invalid results and to encourage confirmatory testing and linkage to HIV care where appropriate.