Overview of study design
Our proposed study will link to an ongoing NIMH-funded scale up study of the YRI delivered via the EPP among high risk youth in Sierra Leone (U19MH109989;) wherein a sub-sample of U19 participants will be recruited and enrolled as index participants in the current study. We will recruit and enroll 140 youth who received the YRI and 140 control youth from the U19-funded scale up study as index participants. YRI youth will have also received EPP training. Participant recruitment will take place in communities in 3 rural districts where the YRI is currently being scaled up: Kailahun, Kono, and Koinadugu. Staff will describe the study to U19 study participants and obtain informed consent to serve as index participants for the current study. All study procedures received Institutional Review Board and Sierra Leone Ethics and Scientific Review Committee approvals.
Peer social network data will be collected using an egocentric fixed choice study design, which samples a population of interest and collects information on each respondent and his/her contacts, resulting in a collection of egocentric networks [25]. At baseline assessment, we will administer a standardized ego network survey for enrolled U19 index participants (“egos” in the context of social networks) to generate a list of peers (“alters” in the context of social networks) and explore their corresponding tie strengths. We will collect social network data in a 3-step survey. First, our survey will begin with name generator questions to prompt identification of 3 peers aged 18 or older. Second, after eliciting peers, name inter-relater survey questions will evaluate the strength of connections (tie strength) between YRI participants (egos) and identified peers (alters). Third, name interpreter survey questions will solicit contact information (i.e., name, cell #, residence) and a range of characteristics of peers to aid in identification (e.g., nicknames, demographics, height, etc.). We will contact identified peers, verify that they know the U19 participant, and invite them to participate in the current study via informed consent.
Primary caregivers will be defined as the cohabitating household member primarily responsible for looking after the participant’s wellbeing. Prior to beginning the YRI, youth who consent to participate in the current study will be asked to provide information about their household composition, including number of individuals in the household, age and sex of all household members, relationship of each household member to the participant, and to identify their primary caregiver (i.e., name, contact information, relationship). If participants are unable to identify such a member, they will be asked to identify the person they are emotionally closest to in their household—under the assumption that this member would be the most invested in their wellbeing.
Settings and participants
Inclusion criteria for peers are (a) be a part of a YRI participant’s social network; and (b) male or female aged 18 or older. Exclusion criteria are (a) be a current YRI participant; (b) severe, active suicidality or psychosis as assessed via the MINI-SCID diagnostic assessment administered by a study social worker; and (c) serious cognitive impairments that might preclude one’s ability to comprehend informed consent and complete study assessments as determined by a study social worker. Youth who report active suicidality or psychosis will be referred for immediate individual mental health services. If one or more of the YRI participants’ three identified peers do not consent to participate or is ineligible, we will recruit additional YRI participants, administer the standardized ego-network survey, and enroll their identified peers (alters) to reach our sample size. We will repeat this procedure to recruit control only peers.
Inclusion criteria for caregivers are (a) identified as a primary adult caregiver (aged 18 or older) of a YRI participant; and (b) residing in the household of a YRI participant. Exclusion criteria are (a) not residing in the household of a YRI participant; (b) severe, active suicidality or psychosis as assessed via the MINI-SCID administered by a study social worker; and (c) serious cognitive impairments that might preclude one’s ability to comprehend informed consent and complete study assessments as determined by a study social worker. Caregivers who report active suicidality or symptoms of psychosis requiring more intensive mental health care will be referred for immediate individual mental health services.
Recruitment strategy
We will recruit and enroll 840 peers (n = 420 YRI peers; n = 420 control peers) and 280 cohabitating primary caregivers (n = 140 YRI caregivers; n = 140 control caregivers). A list of peers and caregivers will be elicited through the ego network survey that will be administered to U19 index participants (n = 140 YRI youth; n = 140 control youth).
Index participants
Index participants will be randomly sampled from the total sample of enrolled U19 study participants (n = 1200). Random sampling will be executed at the district level: Kailahun (n = 140, Kono (n = 70) and Koinadugu (n = 70). The number of youth sampled per district will be roughly proportional to the number of U19 participants in the respective districts.
Clusters in the U19 study were defined based on geographic and logistical considerations related to the implementation of the YRI and EPP (20 YRI + EPP clusters, 20 EPP-only clusters, and 20 no intervention control clusters). We will use a two-stage sampling strategy to select index participants. First, YRI + EPP and control clusters will be randomly selected at the district level. Then, we will randomly select an equal number of index participants from each cluster. The number of youth selected per cluster will be defined based on the district level recruitment targets described in the previous section.
Peers and caregivers
Detailed contact information for nominated peers and caregivers will be collected during administration of the ego network survey and used to locate peers and caregivers. We will employ a three-level tracking strategy to locate peers and caregivers. First, field team members will attempt to establish contact through mobile phone numbers listed by index participants. Multiple attempts will be made to establish contact over the phone. In cases where phone contact cannot be established, field team members will attempt to locate peers and caregivers in person based on the address listed by index participants. If an individual is not located at the listed address, field team members will attempt to contact the index participants to gather additional tracking information. If we are unable to locate a peer or caregiver through the three steps listed above, the entire procedure will be repeated within a period of 5 days. Individuals who cannot be located after four attempts of the above procedure will be classified as attrition.
Measures
Peer outcomes
We will collect quantitative data on emotion regulation, daily functioning, prosocial attitudes, mental health, and risk behaviors among peers at baseline and 8-month follow-up (see Table 1). All quantitative measures underwent a thorough development, translation, and validation process in a prior randomized controlled trial for youth facing adversity in Sierra Leone and have demonstrated good reliability. The WHO process of translation and adaptation of instruments [26] was used to translate measures from English to Sierra Leonean Krio. Measures have been subjected to cognitive testing to ensure comprehension and cultural relevance in assessing satisfaction, social support, and fidelity as well as mental health, daily functioning, and other life outcomes. The following quantitative measures will be used in the current study: the Difficulties in Emotion Regulation Scale (DERS) [27] (α = .65); WHO Disability Assessment Schedule (WHODAS) [28] (α = .91); Brief COPE Scale [29] (α = .72–.84); the Conflict Tactics Scale [30] (α = .72–.86); and Inventory of Socially Supportive Behaviors [31] (α = .87); Hopkins Symptom Checklist [32] (α = .92); and the Adapted Youth Risk Behavior Survey [33] (α = .74–.86).
Table 1 Diffusion Study (Aim 1) Data Collection Methods for Peers Caregiver quantitative outcomes
We will collect quantitative data on burden of care, daily functioning, coping, social support, emotion regulation, mental health, and quality of life among cohabitating caregivers at baseline and 8-month follow-up (see Table 2). The following measures will be used: the Difficulties in Emotion Regulation (DERS) [27] (α = .65); WHO Disability Assessment Schedule (WHODAS) [28] (α = .91); Brief COPE Scale [29] (α = .72–.84); the Conflict Tactics Scale [30] (α = .72–.86); Inventory of Socially Supportive Behaviors [31] (α = .87); Hopkins Symptom Checklist [32] (α = .92); and WHO Health-Related Quality of Life-Inventory (BREF) [34,35,36] (α = .85); Burden Assessment Scale [37] (α = .89–91).
Table 2 Spillover Study (Aim 2) Data Collection Methods for Caregivers Implementation science outcomes
We will use common indicators of implementation science [38, 39] to allow comparison of findings to other contexts and advance implementation science across low-resource settings. We will assess penetration [38] of the YRI through the proportion of peers and caregivers directly and indirectly exposed to the YRI. We will assess cost through ROI and cost-effectiveness analysis. For scale up, we will broaden the number of individuals exposed to the innovation (peers, caregivers). For context, we will test a new strategy for future implementation efforts to include both the target population and the penetration of an intervention beyond the primary clients to their peers and families. We will also assess frequency of contact with key stakeholders.
Quantitative data collection
YRI participants’ peers and cohabitating caregivers will complete quantitative assessments at baseline and 8-month follow-up. Trained research staff will administer assessments electronically using a tablet. To investigate diffusion of YRI effects, peers will complete the quantitative assessment battery to measure mental health, emotion regulation, prosocial behaviors, and daily functioning. To investigate mental health spillover effects, cohabitating caregivers will complete a similar quantitative assessment battery to measure mental health, daily functioning, and burden of care.
Qualitative data collection
We will conduct active focus groups with 4 YRI index participants (2 males, 2 females) and 2 peers (1 male, 1 female) randomly selected from YRI sites at baseline (Post-intervention for YRI participants) and 8-month follow-up (12-month follow-up for YRI index participants). There will be four focus groups held at each time point. A research team member will guide group discussions in Krio, focusing on questions such as: What components of the YRI have you shared (e.g., deep belly breathing)? Who have you shared YRI components with? How did you share YRI components and skills (e.g., demonstration, discussion)? How did sharing of YRI skills take place (e.g., face to face, mobile device)? To assess fidelity of peer-delivered components, we will ask focus group participants who report sharing YRI components with their peers to demonstrate (role play) how they shared. We will videotape the role-plays and use them to rate the fidelity of the YRI component with a fidelity checklist developed from the Youth FORWARD study tools. We will also conduct 12 dyadic interviews with pairs of YRI participants and their first-nominated peers (1 peer and 1 YRI participant per dyad) at baseline. Dyadic interviews will target the same topics areas as focus group discussions, but will focus more specifically on sharing and receiving of information within the dyad. Audio-recorded qualitative data, transcribed and translated into English with identifying information removed, will be analyzed using grounded theory [40, 41] and an analytical strategy derived from thematic content analysis [42]. We will subject all data to an open-coding process without prior assumption of a theoretical framework for diffusion. We will identify key themes and iteratively develop a coding scheme, organized according to key themes (e.g., sharing through demonstration). Two team members trained in coding scheme will independently code 10% of transcripts to examine reliability. Poor agreement (i.e., low kappa ratings) will be grounds for refining the codebook. Reliability testing will be repeated until all coding is at 80% reliability. Team members will code the full qualitative dataset in MaxQDA using the robust coding scheme [43]. YRI content knowledge will be assessed using a quantitative scale administered at the conclusion of focus groups and dyadic interviews.
We will conduct Key Informant Interviews with a sample of cohabitating caregivers of YRI participants to gain a deeper understanding of how spillover effects occur. Qualitative data will deepen our understanding of the mechanisms by which youth participation in the YRI may lead to reduced burden and improved mental health among cohabitating caregivers. We will conduct key informant interviews with 12 caregivers of YRI participants randomly selected from YRI sites (6 males; 6 females) at baseline (post-intervention for YRI index participants) and 8-month follow-up (12-month follow-up for YRI index participants). Trained research staff will guide interviews, with questions such as: What, if any, change did you observe from their participation in the YRI? How did having them in the group affect how you felt? How did having them in the group affect your sense of caregiving burden?
Sample size and statistical power
Due to the egocentric study design, we have several small subgraphs, where each subgraph has the YRI participant youth (ego) at the “center” and the peers (alters) at the “periphery” forming what is often called a k-star in network analysis. Power was driven by the number of subgraphs (i.e., the number of ego-alter clusters). Plots for power versus standard effect size were generated for a social network size of 3 peers (alters) per youth receiving the YRI (egos). This totals 14 youth per YRI + EPP site (n = 140 total youth) and control sites (n = 140 total youth). Power was calculated for a 3-level hierarchical model with 3 peers clustered within index subject and index subjects clustered within site. At a power of .8 and an alpha of p < .05, we have the power to detect an effect size (standardized mean difference) in the range of .30, with an ICC of .03. Based on our earlier pilot study on caregiver spillover effects showing effects sizes from .31–.51 [15], a minimum detectable effect of .30 would be within range. Power considerations will allow us to generate insights on potential spillover effects and the range of implications they may have on cost-effectiveness. Using these same assumptions, the minimal detectable standardized effect size for caregivers (n = 280) is .41.
Statistical analyses: peers
We will conduct social network analysis to investigate how intervention effects are diffused through peer networks to non-participating youth. We will examine key dimensions of social network structure: participation rate, representativeness, and social integration. Participation rate will be calculated from the proportion of YRI sessions attended by youth (egos). Representativeness will be determined by comparing average characteristics of YRI participants and their peers; representative YRI participants (egos) should be more similar to peers (alters) in behavioral and demographic characteristics than to external controls. Demographic characteristics will include gender and age; behavioral characteristics will include emotion regulation and conduct problems. We will investigate the extent to which alters are similar to the egos in terms of demographics and other individual-level characteristics. Because of homophily—the tendency of connected individuals to be similar to one another—we expect ties to occur more frequently between similar (e.g., same sex, similar age) vs. dissimilar (e.g., opposite sex, different age) pairs of individuals. We will examine to what extent pairwise homophily is associated with greater diffusion or adoption of YRI components. The network analysis will attempt to identify factors that are associated with the existence of ego-alter ties. We will use a simple logistic regression as a starting point to modeling. To model the effect of covariates, we propose to use bivariate differences of nodal covariates as predictors;
To investigate diffusion of mental health effects among YRI participant peer networks, we will use growth modeling (mixed models) to assess whether changes in peer emotion regulation, mental health, and daily functioning are associated with uptake of YRI components. We will fit a linear growth model to the data using either a standard 2-level model by incorporating a measurement model for outcome variables or expanding it to a 3-level model [44,45,46]. In a 2-level model, the 2 time points (baseline and 8-month follow-up) are clustered within persons at level 1, where time-varying predictors (e.g., uptake of YRI content) are also included, and alters are included at level 2, where time-invariant predictors associated with the subjects (e.g., demographics) are included. These models will be used to assess the degree to which mental health trajectories among peers are associated with uptake of YRI components.
Statistical analyses: caregivers
We will use growth modeling to investigate mental health spillover effects among cohabitating caregivers. Growth modeling by means of multilevel modeling (mixed models) will be used to assess whether changes in caregivers’ sense of burden, functional impairment, and mental health over time are associated with their child/charge’s YRI participation. We will also compare changes in mental health and caregiver burden among YRI participants' cohabitating caregivers to changes among caregivers of youth in the control group. We also will examine whether changes in YRI participant mental health and functioning are associated with changes among cohabitating caregivers. With 2 time points, we will fit a linear growth model to the data using a standard 2-level model; the 2 time points are clustered within persons at level 1, where time-varying predictors (e.g., uptake of YRI skills) are also included. Level 2 is for caregivers, where time-invariant predictors (e.g., demographics) are included.
Economic analysis
The ROI framework allows for the inclusion and comparison of multiple types of outcomes by using dollars as a common metric [47]. ROI is quantified as the ratio of the sum of all benefits minus the costs, divided by the costs (ROI = (benefits-costs)/costs). Our economic analysis will focus on the labor market performance of peers and caregivers of the YRI youth versus those of control youth. We will use a one-month recall period and a one-year recall period. Primary outcomes will be engagement (hours worked) and income, measured through both wage employment and self-employment/enterprise.
We will also conduct a cost-effectiveness analysis to estimate the societal cost per QALY gained. We will calculate the costs of delivering the YRI to youth in the U19 scale up study to estimate intervention costs, including salaries of staff, intervention site operating costs, and overhead costs related to the YRI. We will use an ingredients approach in which implementation activities are first identified, and then the financial and economic costs of carrying out these activities are quantified [48]. Financial costs account for actual monetary expenses while economic costs encompass all resources, including those volunteered, and opportunity costs incurred by participants. Activities include staff and interventionist trainings, participant recruitment, YRI session delivery, and review of work with the research team. We will calculate the estimated time invested by each YRI participant to derive a value of this time in dollars, based on median wages in Sierra Leone. We will get cost estimates from a range of sources (e.g., invoices, receipts); staff time will be full salary cost; clinical space costs will be estimated based on local rents; and all costs will be expressed in United States dollars.
The economic impacts of the YRI will be assessed for YRI participants and their peers and primary cohabitating caregivers, including improved mental health and labor market outcomes. Mental health effects of the YRI will be defined as the standardized mean difference for primary outcome measures between peers and caregivers of YRI beneficiaries and the control group. Labor market outcomes include income, hours worked, number of jobs held, job search effort and other work quality measures via standard questions added to the survey batteries for YRI youth participants control youth. The principal outputs of the ROI calculations will be total estimated monetary value of benefits (better health and increased productivity) [49] for peers and caregivers (separately and combined) relative to cost.
We will convert the WHO-DAS, a measure of functional impairment, to QALYs [50]. To generate QALYs from the WHO-DAS, we will calculate a summary score using the polytomous scoring algorithm for WHO-DAS scores at each time point. Summary scores will be related to population-based health state preferences to derive a utility score or disability weight (in the range 0–1) using population norms from the WHO Multi-country Survey Study on Health and Health System Responsiveness Nigeria data set. We will plot utility scores/disability weights across the duration of the study and derive the QALY difference between peers and caregivers of YRI participants and those of control participants—an approach validated in previous studies [50]. For the intervention to be deemed cost-effective, each QALY gained must cost less than three times Sierra Leone’s GDP per capita at purchasing power parity [51]; this follows standards for good value for money outlined in global health interventions [50, 52]. We will calculate incremental cost-effectiveness ratios for peers and caregivers (separately and combined) by examining differences in costs between YRI participants’ peers and caregivers and those of control participants in the denominator, and differences in the effectiveness expressed in QALYs in the numerator. Sensitivity analyses varying discount rates and costs will provide a range of estimates for incremental cost-effectiveness ratios.
Study status
At the time of manuscript submission, the study was completing the baseline data collection phase. Table 3 presents information on the timeline of all study activities.
Table 3 Timeline and Milestones