Maximizing Digital Interventions for Youth in the Midst of Covid-19: Lessons from the Adolescent Trials Network for HIV Interventions


Mobile technologies and social media offer powerful tools to reach, engage, and retain youth in HIV prevention and care interventions and deliver personalized, theory-based health content [1,2,3]. Technology use is ubiquitous among youth [4], from a variety of backgrounds and offers many opportunities for connecting youth to digital health interventions (DHIs), including those that address HIV prevention and care behaviors. HIV-focused DHIs are feasible and acceptable to youth, including sexual and gender minority youth, necessitating larger, randomized controlled trials (RCTs) to demonstrate efficacy [2, 5, 6].

While many DHIs are designed to be wholly delivered online, to evaluate efficacy with high fidelity, most RCTs still employ traditional human interaction for many research procedures. This applies to all study components from recruitment through implementation and follow-up. These in-person procedures have been disrupted due to the 2019 novel coronavirus (COVID-19) pandemic. What COVID-19 does not alter, however, is the critical need for youth-focused, HIV research to address the HIV epidemic. In this paper, we discuss COVID-19-driven alterations to studies conducted within the Adolescent Trials Network for HIV Interventions (ATN) supported UNC/Emory Center for Innovative Technology (iTech) [7]. iTech includes 11 sub-studies (eight active RCTs, two of which are delivered fully remotely) focused on improving HIV prevention and care among adolescents and young adults using DHIs and relying on harmonized assessments and engagement metrics. As many research groups pivot to virtual delivery, iTech’s experience implementing youth-focused virtual RCTs provides an opportunity to discuss best practices, potential pitfalls and unique considerations to maximize DHIs for youth in the midst of COVID-19. Table 1 provides information about current iTech studies, including current enrollment status. All studies have been described previously [8,9,10,11,12,13,14]; additional information about iTech can be found at

Table 1 Overview and current status of iTech randomized controlled trials

Best Practices and Considerations During Study Implementation


Prior to COVID-19, iTech utilized advertising on social networking sites and apps to recruit participants, as well as traditional clinic recruitment through iTech’s subject recruitment venues (SRVs). Clinic recruitment was particularly important for RCTs with youth on HIV pre-exposure prophylaxis (PrEP) or antiretroviral therapy (ART). While some youth may still attend in-person medical visits during the pandemic, most SRVs are shifting non-urgent visits to telehealth and prioritizing care for youth living with HIV over prevention-focused visits (e.g. new PrEP starts). These factors will likely hinder recruitment, even if studies are fully transitioned to virtual procedures. iTech currently has a centralized recruitment strategy for reaching audiences of potentially eligible participants online and tracking important metrics (e.g. clickthrough rates, cost per recruit) at the study and SRV level. Moving forward, we need to be thoughtful about how to adapt these strategies to a changing digital ecosystem in order to reach and engage participants that meet eligibility criteria, including those who would otherwise be reached through clinic outreach.


iTech developed comprehensive procedures to administer online consent to youth—including minors—and to remotely verify unique, valid participants [15, 16]. However, some enrollment steps require modification when the initial in-person visit becomes virtual. For example, in P3 and YouTHrive, participants must have proof of an active prescription (PrEP or ART, respectively) through medical record review by SRV staff and presenting their prescription bottle at their enrollment visit. While SRV staff can still remotely access electronic medical records from home, it is likely that the emphasis on social media recruitment will result in more screening of youth receiving care outside their clinical catchement areas, and acquiring medical data may be more challenging. Participants can consent to medical records release but even prior to COVID-19, gaining access in a timely manner could be difficult. Other participant-controlled options must be considered including asking participants to self-verify by showing their pill bottle to staff through videoconferencing or uploading a picture via a secure portal.

Intervention Implementation

The precise factors that ensure effective engagement in DHIs are still under study [17, 18]. However, private sector experts emphasize the importance of an intuitive onboarding experience introducing and educating users on the essential components of the technology, and quickly exposing them to the products’ benefits [19]. Many of iTech’s DHIs intentionally required in-person onboarding by staff to ensure that all components are demonstrated, to generate excitement about the intervention, and address any technical difficulties. When transitioning away from in-person onboarding, our goal is to recreate the same level of support and enthusiasm virtually. In addition to videoconference-assisted onboarding, youth will receive links to animated or video tours of the intervention technology. Staff will be provided with detailed onboarding guides highlighting key features and how to maximize use.

Moving iTech studies fully online and enrolling youth into these interventions could be particularly valuable during the disruptions of COVID-19. Given that youth often rely on online technologies to build social and sexual networks, receive social support, and obtain relevant health information [3, 5], these interventions may provide social support at a time when access to community support is limited. Many of these interventions provide youth with spaces to interact socially and gain support from other youth (e.g. P3, YouTHrive, TechStep) or providers (LifeSteps), which may be especially needed at this time. However, a key benefit of online interventions is the ability for youth to access them when and where they feel most comfortable. For some youth, this may be outside their homes. With current restrictions, youth may lack private spaces to access the intervention. Providing youth with tips on which pieces of the intervention to avoid using in these situations (e.g. videos with sound, videoconferencing sessions with study staff) is important to prevent inadvertent disclosure. Further, we felt it was important to provide factual, relevant COVID-19 information to participants. We have developed a series of COVID-19 related articles and resources tailored for youth (e.g. living with HIV during COVID-19, COVID-19 misinformation, engaging while socially distancing, COVID-19 and grief, ways to stay sane during COVID-19, and finding healthcare coverage if you lose your job) and will make these available to be included as part of the intervention content or in standard -of -care materials.

Laboratory Testing

The majority of iTech’s studies focus on improving the prevention continuum—including promoting home-based HIV testing as part of the digital intervention [20,21,22]. Within three iTech interventions (MyChoices, LYNX and We Prevent) participants can request home-based HIV/STI test kits via study app. Both product availability and processing services were impacted negatively by COVID-19. In addition, COVID-19 has disrupted staff’s ability to collect in-person biologic specimens (e.g. tenofovir diphosphate levels for PrEP adherence among those in P3 or LifeSteps; HIV viral load for ART adherence in YouThrive) which is of great concern given these measures serve as study efficacy endpoints. Finding alternative means to collect these outcome measures is critical, particularly for youth already enrolled. To address this, iTech is working with a commercial laboratory experienced in home-based testing, to continue providing youth the ability to order home HIV/STI kits within the study intervention platforms and to collect biologic outcomes. While resuming currently paused iTech studies is of vital importance, we also recognize our work with vulnerable youth populations requires ensuring that participant safety is protected. We are working to ensure that as we transition to home-based sample collection that: (1) we have secure systems in place to allow staff to easily order tests and receive results (including protocols for delivering these results to participants and reporting positive results to the appropriate public health authorities); (2) youth are provided with clear instructions on sample collection (written, videos) and a way to contact study staff if assistance is needed; (3) a clear plan has been delineated at each SRV to help youth deal with any positive results (including ways to refer them to care or provide them with treatment) which may, during COVID-19, be particularly challenging; (4) prior to sending any kit to youth, study staff will describe the process in detail, including exactly what the kit contents will contain, and will assist youth in deciding whether it is safe to mail the kits to their homes. Although the packaging is plain, a caregiver may open the package and discover the contents. Participants do have the option to receive kits at addresses other than their homes, but this may not be practical or possible with COVID-19 stay-at-home orders in place in all iTech SRV cities.


Ensuring good retention during COVID-19 is critical which means even greater efforts to maintain regular contact with participants through texting, email and/or phone calls and continuing to provide incentives for completing virtual study activities, including increasing incentives for additional self-collected specimens. Many SRVs already provided incentives from afar through reloadable credit cards (e.g. ClinCards) given to participants during their initial enrollment visit or sent directly from the financial institution. We are also considering other options that minimize face-to-face interactions or having to have staff purchase and mail incentives. Incentives being considered include virtual currency systems (e.g. Venmo or Zelle) or gift cards from online retailers (e.g. Amazon, Tango).

Impacts on Measurement

It is a certainty that COVID-19 and the plans to manage it are directly effecting the local HIV prevention and care environments on which digital HIV intervention RCTs are still dependent in many ways. For instance, the control arm of the RCTs is standard-of-care typically involving referral to local services. If those services are disrupted and control arm participants cannot access them, then we may find a larger intervention effect than would be normally expected. This is a direct challenge to the generalizability of our RCT findings. Some digital HIV inteventions also still require engaging with local services, such as our PrEP and ART adherence interventions, which still require keeping medical appointments and getting prescriptions filled. If those services see substantial disruptions, then the DHI efficacy could be siginficantly attenuated. Finally, changes in sexual and substance use behaviors are very likely and could reduce our ability to identify intervention effects on risk-reduction. There are no perfect solutions to these issues, but we are adding questions to all baseline and follow-up surveys to determine whether there are temporal COVID-19 related changes in access to services and engagement in sexual and substance use behaviors. This self-reported impact data may then be used in sensitivity analyses of our primary and secondary RCT outcomes.


iTech has been proactive at maximizing lessons learned from our fully remote RCTs to transition our remaining studies to full virtual implementation. Changes have been needed across all aspects of intervention implementation—from the intervention technology itself to measurement of study endpoints. Similar to other research teams, iTech developed a set of harmonized COVID-19 questions that are being added to all iTech studies and have been provided for use throughout the larger ATN. iTech is not only changing our studies to address disruptions caused by the virus but is also directly addressing and measuring the impact of COVID-19 among participants, which will allow better understanding of how the pandemic impacts youth both at-risk for and living with HIV.


  1. 1.

    Hightow-Weidman LB, Muessig KE, Pike EC, LeGrand S, Baltierra N, Rucker AJ, et al. building community through a mobile-optimized, online health promotion intervention. Health Educ Behav. 2015;42(4):493–9.

    PubMed  PubMed Central  Article  Google Scholar 

  2. 2.

    Muessig KE, Nekkanti M, Bauermeister J, Bull S, Hightow-Weidman LB. A systematic review of recent smartphone, Internet and Web 2.0 interventions to address the HIV continuum of care. Curr HIV/AIDS Rep. 2015;12(1):173–90.

    PubMed  PubMed Central  Article  Google Scholar 

  3. 3.

    Mulawa MI, LeGrand S, Hightow-Weidman LB. eHealth to enhance treatment adherence among youth living with HIV. Curr HIV/AIDS Rep. 2018;15(4):336–49.

    PubMed  PubMed Central  Article  Google Scholar 

  4. 4.

    Teens, Social Media and Technology 2018. Accessed 9 Apr 2020.

  5. 5.

    Hightow-Weidman LB, Muessig KE, Bauermeister J, Zhang C, LeGrand S. Youth, technology, and HIV: recent advances and future directions. Curr HIV/AIDS Rep. 2015;12(4):500–15.

    PubMed  PubMed Central  Article  Google Scholar 

  6. 6.

    Holloway IW, Rice E, Gibbs J, Winetrobe H, Dunlap S, Rhoades H. Acceptability of smartphone application-based HIV prevention among young men who have sex with men. AIDS Behav. 2014;18(2):285–96.

    PubMed  PubMed Central  Article  Google Scholar 

  7. 7.

    Hightow-Weidman LB, Muessig K, Rosenberg E, Sanchez T, LeGrand S, Gravens L, et al. University of North Carolina/Emory Center for Innovative Technology (iTech) for addressing the HIV epidemic among adolescents and young adults in the United States: protocol and rationale for center development. JMIR Res Protoc. 2018;7(8):e10365.

    PubMed  PubMed Central  Article  Google Scholar 

  8. 8.

    Bauermeister JA, Golinkoff JM, Horvath KJ, Hightow-Weidman LB, Sullivan PS, Stephenson R. A multilevel tailored web app-based intervention for linking young men who have sex with men to quality care (get connected): protocol for a randomized controlled trial. JMIR Res Protoc. 2018;7(8):e10444.

    PubMed  PubMed Central  Article  Google Scholar 

  9. 9.

    LeGrand S, Knudtson K, Benkeser D, Muessig K, McGee A, Sullivan PS, et al. Testing the efficacy of a social networking gamification app to improve pre-exposure prophylaxis adherence (P3: prepared, protected, empowered): protocol for a randomized controlled trial. JMIR Res Protoc. 2018;7(12):e10448.

    PubMed  PubMed Central  Article  Google Scholar 

  10. 10.

    Biello KB, Marrow E, Mimiaga MJ, Sullivan P, Hightow-Weidman L, Mayer KH. A mobile-based app (mychoices) to increase uptake of HIV testing and pre-exposure prophylaxis by young men who have sex with men: protocol for a pilot randomized controlled trial. JMIR Res Protoc. 2019;8(1):e10694.

    PubMed  PubMed Central  Article  Google Scholar 

  11. 11.

    Biello KB, Psaros C, Krakower DS, Marrow E, Safren SA, Mimiaga MJ, et al. A pre-exposure prophylaxis adherence intervention (lifesteps) for young men who have sex with men: protocol for a pilot randomized controlled trial. JMIR Res Protoc. 2019;8(1):e10661.

    PubMed  PubMed Central  Article  Google Scholar 

  12. 12.

    Gamarel KE, Darbes LA, Hightow-Weidman L, Sullivan P, Stephenson R. The development and testing of a relationship skills intervention to improve HIV prevention uptake among young gay, bisexual, and other men who have sex with men and their primary partners (we prevent): protocol for a randomized controlled trial. JMIR Res Protoc. 2019;8(1):e10370.

    PubMed  PubMed Central  Article  Google Scholar 

  13. 13.

    Liu A, Coleman K, Bojan K, Serrano PA, Oyedele T, Garcia A, et al. Developing a mobile app (LYNX) to support linkage to HIV/sexually transmitted infection testing and pre-exposure prophylaxis for young men who have sex with men: protocol for a randomized controlled trial. JMIR Res Protoc. 2019;8(1):e10659.

    PubMed  PubMed Central  Article  Google Scholar 

  14. 14.

    Siegler AJ, Brock JB, Hurt CB, Ahlschlager L, Dominguez K, Kelley CF, et al. An electronic pre-exposure prophylaxis Initiation and maintenance home care system for Nonurban young men who have sex with men: protocol for a randomized controlled trial. JMIR Res Protoc. 2019;8(6):e13982.

    PubMed  PubMed Central  Article  Google Scholar 

  15. 15.

    Teitcher JE, Bockting WO, Bauermeister JA, Hoefer CJ, Miner MH, Klitzman RL. Detecting, preventing, and responding to "fraudsters" in internet research: ethics and tradeoffs. J Law Med Eth. 2015;43(1):116–33.

    Article  Google Scholar 

  16. 16.

    Ballard AM, Cardwell T, Young AM. Fraud detection protocol for web-based research among men who have sex With men: development and descriptive evaluation. JMIR Public Health Surveill. 2019;5(1):e12344.

    PubMed  PubMed Central  Article  Google Scholar 

  17. 17.

    Hightow-Weidman LB, Bauermeister JA. Engagement in mHealth behavioral interventions for HIV prevention and care: making sense of the metrics. Mhealth. 2020;6:7.

    PubMed  PubMed Central  Article  Google Scholar 

  18. 18.

    Bauermeister JA, Golinkoff JM, Muessig KE, Horvath KJ, Hightow-Weidman LB. Addressing engagement in technology-based behavioural HIV interventions through paradata metrics. Curr Opin HIV AIDS. 2017;12(5):442–6.

    PubMed  PubMed Central  Article  Google Scholar 

  19. 19.

    App Onboarding Best Practices 2020 for Better User Engagement 2020. Accessed 6 Apr 2020

  20. 20.

    Cao B, Gupta S, Wang J, Hightow-Weidman LB, Muessig KE, Tang W, et al. Social media interventions to promote HIV testing, linkage, adherence, and retention: systematic review and meta-analysis. J Med Internet Res. 2017;19(11):e394.

    PubMed  PubMed Central  Article  Google Scholar 

  21. 21.

    Tang W, Wei C, Cao B, Wu D, Li KT, Lu H, et al. Crowdsourcing to expand HIV testing among men who have sex with men in China: a closed cohort stepped wedge cluster randomized controlled trial. PLoS Med. 2018;15(8):e1002645.

    PubMed  PubMed Central  Article  Google Scholar 

  22. 22.

    Stephenson R, Todd K, Kahle E, Sullivan SP, Miller-Perusse M, Sharma A, et al. Project Moxie: results of a feasibility study of a telehealth intervention to increase HIV testing among binary and nonbinary transgender youth. AIDS Behav. 2019.

    PubMed  Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Lisa Hightow-Weidman.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hightow-Weidman, L., Muessig, K., Claude, K. et al. Maximizing Digital Interventions for Youth in the Midst of Covid-19: Lessons from the Adolescent Trials Network for HIV Interventions. AIDS Behav 24, 2239–2243 (2020).

Download citation