The feasibility of cell phone based electronic diaries for STI/HIV research
- First Online:
- Cite this article as:
- Hensel, D.J., Fortenberry, J.D., Harezlak, J. et al. BMC Med Res Methodol (2012) 12: 75. doi:10.1186/1471-2288-12-75
- 3.6k Downloads
Self-reports of sensitive, socially stigmatized or illegal behavior are common in STI/HIV research, but can raise challenges in terms of data reliability and validity. The use of electronic data collection tools, including ecological momentary assessment (EMA), can increase the accuracy of this information by allowing a participant to self-administer a survey or diary entry, in their own environment, as close to the occurrence of the behavior as possible. In this paper, we evaluate the feasibility of using cell phone-based EMA as a tool for understanding sexual risk and STI among adult men and women.
As part of a larger prospective clinical study on sexual risk behavior and incident STI in clinically recruited adult men and women, using study-provided cell phones, participants (N = 243) completed thrice–daily EMA diaries monitoring individual and partner-specific emotional attributes, non-sexual activities, non-coital or coital sexual behaviors, and contraceptive behaviors. Using these data, we assess feasibility in terms of participant compliance, behavior reactivity, general method acceptability and method efficacy for capturing behaviors.
Participants were highly compliant with diary entry protocol and schedule: over the entire 12 study weeks, participants submitted 89.7% (54,914/61,236) of the expected diary entries, with an average of 18.86 of the 21 expected diaries (85.7%) each week. Submission did not differ substantially across gender, race/ethnicity and baseline sexually transmitted infection status. A sufficient volume and range of sexual behaviors were captured, with reporting trends in different legal and illegal behaviors showing small variation over time. Participants found the methodology to be acceptable, enjoyed and felt comfortable participating in the study.
Achieving the correct medium of data collection can drastically improve, or degrade, the timeliness and quality of an individual’s self-reported sexual risk behavior, which in turn, is a key factor in the success of intervention or education programs relying on this information. Our findings demonstrate that completion of electronic diaries via cellular phone is feasible way to describe STI/HIV risk among clinically recruited adult men and women.
Obtaining participant self-reports is common in STI/HIV research, not only because it is difficult to directly observe sexual risk behavior , but also because it is important to understand how an individual’s own circumstances and subjective experiences may increase or decrease the likelihood of a given behavior [2, 3]. Eliciting accurate information is therefore to the vital to the correct design and implementation of intervention and education programs [3, 4, 5]; however, many of the self-reported sensitive, socially stigmatized or illegal behaviors collected in STI/HIV research can raise challenges in terms of the validity and reliability of data collected [1, 6], particularly if behaviors are underreported out of participant concern for privacy or interviewer responses [6, 7, 8].
Recent studies suggest that the use of electronic data collection tools can increase the accuracy of self-disclosed risk behaviors by allowing a participant to self-administer a survey or diary, in their own environment, as close to the occurrence of the behavior as possible . Ecological momentary assessment (EMA) [2, 4, 10, 11, 12] is one such tool, in which participants respond to pre-programmed signals on an electronic device (e.g., a PDA or cellular phone) prompting them to complete diaries, in various within-day frequencies, related to recent or immediate social environment and behavior [13, 14]. This arrangement allows joint assessment of a risk behavior, and the context in which it occurs, in near real time . Data collection typically proceeds over weeks or months [16, 17], providing a better understanding of how social context differs when a behavior does and does not occur, as well as how the relationship between context and behavior may change over time.
EMA lends a number of advantages for use in clinical research. Compared to other collection modalities, it garners less missing data, higher reporting levels, stronger internal data validity and low behavior reactivity [4, 8, 12, 14, 15, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28]. The pervasive use of cell phones and other electronic devices in the study population may make protocol training and data entry easier for participants [29, 30]. Electronic platforms can time-stamp all data, from reminders to start/finish diaries, to actual initiation and completion, allows a precise analysis of participant compliance [4, 8, 14, 18]. Moreover, researchers can standardize measures across participants and have increased flexibility, including using branching/contingency questions, or multilingual versions of instruments [4, 12, 18, 19]. Finally, EMA strengthens the security of sensitive or stigmatizing information, allowing information to be vacated from the device immediately upon data entry for storage on a remote server, increasing participant valued privacy .
EMA has been utilized to examine a variety of clinical outcomes, in a number of high-risk populations, including cocaine relapse in homeless patients in treatment , adolescent mood variability and smoking , adolescent affect before and after sexual intercourse , prevention behavior in HIV-infected individuals , affect and drug craving in methamphetamine-dependent individuals  and women’s compliance to bacterial vaginosis (BV) treatment . In this paper, we evaluate the use of cell-phone based, thrice-daily EMA over 12 weeks as a means of prospectively studying sexual risk behavior and incident STI among a sample of clinically recruited adult men and women at high risk for HIV/STI. Parallel to other work , we assess feasibility in terms of (a) compliance, or data entry which is consistent with study protocol and substantially complete; (b) reactivity, or behavior change attributed to study participation; (c) participant acceptability, or convenience and ease of method beneficial to compliance. We also assess (d) data capture, or the volume of sexual and risk behaviors collected. We also comment on participant recruitment and retention within a high-risk, clinically recruited population, as well as provide recommendations for future method application.
Data were collected as part of a larger 12-week study prospectively examining sexual risk behavior and incident STI in clinically recruited adult men and women. In this larger study, data collection was organized around three components: 1) an audio computer-assisted self-interview (ACASI),administered at enrollment and exit visits, assessing individual and partner-specific demographics, sexual and contraceptive history, general attitudes and relationship content for up to five partners; 2) thrice daily EMA diaries monitoring individual and partner-specific emotional attributes, non-sexual activities, non-coital or coital sexual behaviors, and contraceptive behaviors; 3) self-obtained urine (males) or vaginal specimens for STI diagnosis. The current study evaluates data taken primarily from the within-day diaries.
Participants in the larger study were recruited from the patient population of the Bell Flower Clinic (BFC), a sexually transmitted diseases clinic operated by the Marion County Health Department in Indianapolis, Indiana. BFC serves primarily lower- and middle-income individuals residing in areas with high rates of unintended pregnancy and STI. Eligibility for the larger study included being 18 to 29 years of age and having primary residence in Marion County, Indiana for the 90 days of study duration. Both criteria were chosen to recruit a sample with a broad number of types of sexual relationships and reasons for sexual activity, as well as high rates of sexually transmitted infections, to facilitate follow up and to reduce sample attrition. Exclusion criteria included primary language other than English, ongoing participation in other research protocols at BFC, intoxication or psychiatric illness at enrollment, being HIV positive or being homeless.
At the time of analyses, all participants self-referred into the study based on study advertisements in BFC, as well or word of mouth within the patient population. All interested individuals contacted the research nurse practitioner (RNP), who provided additional study information, answered questions and performed initial screening. Very few (10%) of those calling were ineligible; of these, most exceeded the study age requirements, and one was HIV positive. Individuals who passed screening either scheduled an enrollment appointment at BFC within the next week, or were placed on a waiting list until a spot was available. The waiting list volume has ranged from 50 – 150 individuals, with a median time from placement on the waiting list until enrollment of four months. About 50% of individuals on the waiting list at any given time were never enrolled, most commonly due to an individual’s contact information changing between initial call and call-back. Very few were not enrolled due to a change in eligibility criterion. Retention and completion of enrolled participants is described in the results section.
EMA diary technology
EMA diary data were entered on an internet enabled Palm Centro cellular phone, measuring 4.2 in × 2.1 in × 0.7 in. Each phone was equipped with a 1.3 megapixel digital camera, touch screen navigation, a QWERTY keyboard, as well as a carrying case, wall charger and a stylus to facilitate data entry. The diaries ran on software (Pendragon SyncServer) allowing Palm OS handhelds and smartphones to synchronize forms and data securely over a TCP/IP connection. Because the smartphones access a nationwide cellular network, internet connection is possible throughout the continental U.S. with access to cellular signals, considerably increasing efficiency in speed of data transfer. The software allowed users to connect and transfer information to the study database at any time, with multiple study participants able to connect simultaneously. The software enabled study personnel to amend data collection forms automatically, without any action from the user. Additionally, in the event of equipment or program failure, study personnel were able to reinstall the study application remotely.
Participants had an availability window to complete the diary from 30 minute prior to the scheduled start to four hours after the scheduled start. Prior to, or after this window, data collection was disabled. Participants also received up to four SMS text message reminders about completing diary entries, sent at the scheduled start time, one hour after the scheduled start time, two and a half hours after the scheduled start time and fifteen minutes before the 4-hour availability window following the scheduled start time closed. Reminders were sent until the data were entered and successfully downloaded. Thus, a participant who had taken their diary, but whose data did not successfully transfer off their phone would still receive a diary reminder, indicating they needed to troubleshoot connectivity with the RNP.
Diaries were formatted to display one question on the screen at a time, requiring participants to select from a drop down list, or check appropriate boxes. Navigation from question to question was facilitated by the use of a “previous” and “next” button on each screen; however, for most questions, participants could not advance until the current question had been answered. For some sensitive questions, a “Prefer Not to Answer” option was available. These features were built into the diary program to reduce data errors and omissions.
In all data-related interactions, including actual data entry and reminders, time and date recorded and stored in the phone until transfer in real time to a remote server. Upon survey completion, participants were directed back to Home Screen (Figure 1; left) and clicked to synchronize data and initiate this transfer. Once transferred, participants could not enter the survey again until their next scheduled entry. In the event that connectivity issues prevented data transfer, information remained on the device, encrypted and password protected, until synchronization was next possible. In the event of equipment failure, the participant was instructed to contact the RNP, who verbally walked the participant through a procedure to clear the data from the phone and begin the data entry process over again. Information related to synchronization issues and data clearing was also time and date stamped for storage on the remote server.
EMA diary protocol
At enrollment, following the informed consent process and ACASI, the RNP provided a face-to-face, in-depth orientation to basic cell phone operation, diary access and data entry, as well as policies on cellular service usage and troubleshooting procedures for equipment or connectivity issues. Participants were also given a detailed information sheet concerning all policies, as well as contact information for the RNP and research field staff. Additionally, weekly contact with research field staff during specimen collection enabled reinforcement of study procedures.
Participants completed EMA diaries three times per day, at eight hour intervals (e.g., 6am, 2pm and 10pm), over approximately 84 days, for a total of 252 entries per person. Although some EMA researchers suggest prompting diary completion at random intervals each day (e.g. ), recent work suggests that data collection at consistent times each day is equally valid , and may increase the convenience of, and compliance to, the data collection procedure for participants. Thus, participants selected one of four pre-set 8-hour diary entry schedules which fit best within their daily lives. They were also allowed to change their schedule at any time, however, very few (N = 4) opted to do so. All diary procedures, including data collection schedule and instrument comprehension were piloted with an initial sample (N = 10) of individuals from the target population for two weeks, with modification to a small number of items before total field deployment.
As part of the larger compensation structure for the study, participants received $1.00 per on–time diary entry, earning up to $252.00 for the diary portion during the entire study. Compensation was delivered via check in monthly installments, delivered by the research field staff. Additionally, entries which were delayed or missed due to connectivity issues or equipment failure were also compensated on a case-by-case basis. Participants also had access to unlimited domestic calling, text messages and phone-based internet use during the study. Those who successfully completed the study were given the option of retaining their study cell phone and paying for service themselves. Decisions to keep or not to keep the cell phone did not influence compensation from other parts of the study. The majority of fee-for-service options (e.g., games, directory assistance, international calling) was disabled from the phone; however, any charge incurred was deducted from the participant’s final paycheck.
Partnered sexual behaviors (no/yes) included: received oral sex, gave oral sex, penile-vaginal sex, penile-anal sex and other sex. We also examined individual and partnered substance use (no/yes) including marijuana, alcohol or cocaine/methamphetamine use, as well as individual and partnered daily activities (no/yes) including taking a walk and going to the mall.
Descriptive statistics were used to examine compliance at the subject and partner level, completion time, acceptability and data capture. Reactivity was assessed using regression analyses adjusted for multiple diary entries within individuals .
Overall participant retention
Demographic Characteristics and Infection Status, Completed Participants (N = 243)
More than one race
Less than 20 year
28 or older
STI at Enrollment
STI Acquisition during Study
The complex structure of data collection required compliance analysis at three levels. First, participants were asked to complete three diary entries per day, resulting in 21 expected submissions per week during all 12 weeks of study participation. Second, each participant was expected to complete a total of 252 entries by the end of the study. Third, at the study level, the 243 completed participants were expected to contribute 61,236 (243 × 252) entries by the end of the study.
Compliance in the completing group was excellent. At the weekly level, participants submitted an overall average of 18.86 (SD = 2.69; Median = 20) of the 21 (85.7%) expected weekly diaries during all 12 weeks. At the participant level, the majority of the completing group (82.3%: 200/243) submitted nearly all (99%: 250/252) of the expected diaries at the end of the study. At the study level, completing participants submitted 89.7% (54,914/61,236) of the total diaries expected. We also evaluated study level compliance in the non-completing group, both in terms of the influence on total diary submission rates when included with the completing group, as well as when examining submission rates only among those who dropped out. At the study level, the completing and non-completing together (N = 277 total) submitted 85.4% (N = 59,967) of the 69,804 (277 × 252) diaries expected. Focusing solely on those who dropped out (N = 19), the average time from enrollment to dropout was 48.7 days (Median: 54.5 days; range: 12.0 to 84.0 days); at the study level, they submitted 69.6% (1424/2046) of the diaries expected at the point in the study at which they dropped.
Thus, based on the description, both demographic and completion rate, of the completer sample (N = 243) and the non-completer sample (N = 34), we concentrated the remaining analyses only on the completing group, as they provided full information at each study period in terms of answers to the survey questions. This full information did not require us to make assumptions about the nature of any missingness patterns (e.g., at random vs. non-ignorable), as it would with data not obtained from the non-completing group. This choice does not reduce the validity of our conclusions less valid; rather, it allows comparisons conditional on study completions.
Weekly Number of Diaries Submitted, Completed Participants (N = 243)
No. of participants
Diary submission patterns by completed participant (N = 243) gender, race, age and infection status
Weekly submitted diaries
Weeks in Study
Less than 20 years
28 year or older
Any STI at Enrollment
We assessed completion time as the number of minutes elapsed from initiation of subject diary to synchronization with the server. For participants with submitted diaries, the median time to completion was 1.77 minutes, with slightly less time required for subjects who reported no partner interaction within a diary period (median: 1.68) and greater time required when partners were reported (median: 3.00). The median time from the start of the subject portion of the diary to the start of the partner portion of the diary was 0.68 minutes, with about 2.25 minutes required to complete and synchronize this latter part. We additionally conducted analyses to assess completion time as a function of sexual behavior, finding that reporting any type of sex added about a minute to overall diary completion time.
Similar significant declines (Figure 3) in vaginal sex were also noted within monogamous relationships. Controlling for gender, age and STI status, women experienced a significantly lower drop (rate: -0.50%) as compared to men (rate:-0.80%; p < .05) and those with no STI at enrollment (rate: -0.68%) decreased significantly more than those with an STI at enrollment (−0.36%; p < .05). Those of youngest age (18 to 20 years; rate: -0.91%) and oldest age (27 to 29 years; rate: -0.83%) reported significantly less frequently as compared to other age groups (21 to 23 years; rate: -0.41%; 24 to 26 years; rate: -0.49%; all p < .05).
Sexual event summary, by participants and events
Total number participants reported*
Total number events reported**
Median events per participants
Received oral sex
Gave oral sex
Participant study experience evaluation
I enjoyed participating in the study*
I felt comfortable answering questions about sexual activities*
I did not change my behavior due to participating in the study*
The daily PDA questions were too long*
The specimen procedure was convenient*
Questions were answered in a timely manner by the research team*
Commensurate with our presentation of the compliance data, we evaluated EMA’s capabilities as a data collection tool by examining the volume of sexual behavior data captured at the participant- and event-level (Table 4). At the participant level, all individuals reported vaginal sex some time during the 12 weeks. The majority also reported receiving oral sex (94.2%) or giving oral sex (79.8%). About 62% reported engaging in other types of sex, including about a third (35.3%) reporting any anal sex across the 12 weeks.
Similar patterns were observed in the event level data. Overall, participants submitted 4423 total entries noting with any partner interaction. Of these, the majority (81.1%: 3587/4423) were associated with vaginal sex. Oral genital sex, both receiving (43.5%: 1927/4423) and giving (32.8%: 1927/4423) were also common. Other sex represented about a third of all (1311/4423) of all interactions, while very few (6.2%) were associated with anal sex.
Ease of use
Participants reported technical issues around connectivity and equipment loss/failure. Related to connectivity, the most common problems were inability to access the survey at some point (about 5% of participants) and difficulty synchronization of completed data with the remote server at some point (about 20% of participants). Upon experiencing connectivity issues, participants contacted either the RNP or other field staff, who verbally walked them through a reset procedure designed to save data until services were restored. In the rare event that the reset procedure did not work, the RNP contacted the information technology staff assigned to the study, who explored on site reasons for connectivity problems. Participants were compensated for missed surveys associated with connectivity.
Related to equipment loss/failure, the most common reasons for phone replacement were due to theft (about 10%) or damage to the phone (about 15%). About 11% of participants experienced random equipment failure not associated with negligence, theft or misuse. In all instances, participants were given a replacement phone and compensated for missed surveys, provided they contacted the RNP or field staff as soon as possible, and, in the case of theft, arranged to receive a police report on the stolen phone. If the participant was unable to verify theft with a police report, they were removed from the study and not compensated for the missed surveys. The average time from report of issue to replacement phone was two days, with the most common resolution of field staff’s bringing the phone to the participant. Despite these technical issues, no participants left the study specifically due to issues using the cell phone and only 6 participants had to be removed because they could not produce a police report. All other technical problems were described by participants as minor issues associated with study participation.
Achieving the correct medium of data collection can drastically improve, or degrade, the timeliness and quality of an individual’s self-reported sexual risk behavior [20, 38, 39, 40], which in turn can play a key factor in the success of intervention or education programs relying on this information. This is one of the first studies to evaluate the use of EMA to prospectively capture risk behavior in a high-risk, clinically recruited sample of adult men and women. Contrary to assumptions about challenges associated with recruitment and retention of this population in high-involvement studies [16, 41], our findings demonstrate that completion of thrice daily, electronic diaries via cellular phone is feasible way to describe STI/HIV risk in clinical and/or high risk population, capturing an ample participant- and event-level volume of different sexual behaviors. Participants were highly compliant with diary entry protocol and schedule, found the methodology to be acceptable, enjoyed and felt comfortable participating in the study, and did not alter their risk behaviors as a result of frequent reporting. In addition, study provided phones and cellular service provided an added participation incentive and facilitated communication between participants and study staff.
Behavior reactivity can be a concern in EMA studies, particularly because behavior assessments are completed repeatedly and in proximity to the behavior of interest, putting them in a prime position to affect behavior . Sexual behavior, drug or alcohol use and other “undesirable” behavior may be particularly vulnerable to reactivity . However, most EMA studies show minimal to non-existent reactivity, even for sensitive and stigmatized behaviors [16, 21, 34, 43]. Our data do suggest small reporting differences over time in a variety of in a variety of partnered and unpartnered, as well as legal and illegal behaviors, with most exhibiting less than one-half percent change per week. The most marked trend was a significant decrease in reports of vaginal sex, with these trends holding consistent across relationship types, age, gender and STI status. While no studies have assessed reactivity to sexual behaviors using EMA, our data are in line with reported reductions in longer EMA studies of alcohol use  and drug use . Some work suggests that the same EMA data collection intensity that causes concern may in fact attenuate reactivity by causing habituation ; however, it is likely that these findings reflect subtle shifts in day-to-day routines, given both the high level of diary submission as well as the majority of those who reported no alternation of behavior as a result of study participation.
With EMA, a maximum of 20% missed or missing entries, or, 80% compliance, is considered satisfactory , as the chance of systematic bias in the data increases with higher noncompliance . Our weekly (85.7%), subject (82.3%) and study (89.7%) diary completion levels are on par with or are higher than other shorter-term EMA studies with similar [32, 47, 48] or more frequent [17, 49] reporting schedules, as well as EMA studies with once or twice per day reporting [16, 34, 50] and retrospective  or end-of-day  electronic diaries. Moreover, even with a larger number of items to complete as compared to other studies(e.g., , participants did not feel the diaries were too long, and they completed diaries at similar levels regardless of whether any partner interaction occurred and/or if any sexual behavior was reported.
It is well established in multiple samples that participant acceptability is generally not a problem . The same can be said about our clinically recruited, high-STI risk sample of adult men and women, as the majority of individuals did not find the intense diary collection schedule to be burdensome. However, we argue that it is important to understand method acceptability in light of other aspects of study acceptability. Existing work emphasizes that participants value regular communication, health and social support from research/field staff, as well as compensation and study convenience, as important pieces in the quality of their study participation and their long term retention [41, 51, 52, 53, 54, 55, 56, 57]. From this perspective, we believe that neither our sample’s high rating of within-day diary collection, nor their high compliance, is surprising, as they also rated their overall study experience, their connection with field staff and the convenience of data collection as high.
The ability to keep participants engaged in HIV/STI research over an extended period of time is vital to capturing not only a sufficient volume of behavioral and epidemiologic data, but also to collect data that accurately reflect the habitude of the participants themselves. In contrast to assumptions about the difficulty of electronic and/or technically demanding data collection in clinical and high-risk samples [16, 41], we suggest that, with the correct support structure, EMA is a feasible, highly acceptable, and effective tool for prospectively capturing sexual risk behavior and STI in these populations.
Comment is warranted on the representativeness of this sample. Recruitment did not occur exclusively from the active patient population at the county health clinic (BFC). Individuals, many of whom had sought services from BFC in the past, self-referred for enrollment in the study. All of these individuals resided in low-income areas associated with high levels of unintended pregnancy and STI. Based on the sample characteristics in prior studies conducted at BFC, as well as within the general BFC patient population, our participants do not substantially differ in terms of education, sexual behaviors, early child bearing or STI. Moreover, between 5% and 20% of participants did experience a variety of expected technical and equipment issues, mainly related to loss of connectivity, inability to synchronize data, random equipment failure or equipment loss/theft. While these issues all impact a participant’s ability to complete diary entries, and may influence their desire for continued study participation, we were able to minimize the impact of these issues by not only providing them prompt resources to address the concern, but also compensating them for entries missed due to connectivity. The majority of those who experienced issues were resolved within the same day. Additionally, our attrition rate is similar to, or lower than, other short term follow up studies in this clinic . From this perspective, our exclusion criteria notwithstanding, we believe our data provide information that is generalizable to other clinic-based samples, particularly as a means of providing additional information on the processes of sexuality, sexual behavior and STI in higher risk persons (those economically disadvantaged, racial/ethnic minority, or both) whose risk is epidemiologically emphasized, but whose sexual relationships are largely ignored.
This project was supported by grant R01 HD053231-01A1 to Dr. J. Dennis Fortenberry from the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD). Its contents are solely the responsibility of the authors and do not necessarily reflect the official views of the NICHD.
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.