Strengthening mental health and research training in Sub-Saharan Africa (SMART Africa): Uganda study protocol
Children in Sub-Saharan Africa (SSA) comprise half of the total regional population, yet existing mental health services are severely under-equipped to meet their needs. Although effective interventions for the treatment of disruptive behavioral disorders (DBDs) in youth have been tested in high-poverty and high-stress communities in developed countries, and are relevant for widespread dissemination in low- and middle-income countries (LMICs), most of these evidence-based practices (EBPs) have not been utilized in SSA, a region heavily impacted by poverty, diseases including HIV/AIDS, and violence. Thus, this paper presents a protocol for a scale-up longitudinal experimental study that uses a mixed-methods, hybrid type II, effectiveness implementation design to test the effectiveness of an EBP, called Multiple Family Group (MFG) aimed at improving child behavioral challenges in Uganda while concurrently examining the multi-level factors that influence uptake, implementation, sustainment, and youth outcomes.
The MFG intervention will be implemented and tested via a longitudinal experimental study conducted across 30 public primary schools located in both semi-urban and rural communities. The schools will be randomly assigned to three study conditions (n = 10 per study condition): (1) MFG delivered by trained family peers; (2) MFG delivered by community health workers; or; (3) comparison: usual care comprising mental health care support materials, bolstered with school support materials. A total of 3000 children (ages 8 to 13 years; grades 2 to 7) and their caregivers (N = 3000 dyads); 60 parent peers, and 60 community health workers will be recruited. Each study condition will comprise of 1000 child-caregiver dyads. Data will be collected at baseline, 8 and 16 weeks, and 6-month follow-up.
This project is the first to test the effectiveness of the MFG intervention while concurrently examining multi-level factors that influence overall implementation of a family-based intervention provided in schools and aimed at reaching the large child population with mental health service needs in Uganda. Moreover, the study draws upon an EBP that has already been tested for delivery by parent peers and community facilitators, and hence will take advantage of the advancing science behind task-shifting. If successful, the project has great potential to address global child mental health needs.
ClinicalTrials.gov, ID: NCT03081195. Registered on 16 March 2017.
KeywordsChild mental health services Multiple family group interventions Evidence-based practice Implementation science
Comparative Fit Index
Chi-squared Automatic Interaction Detector
Collaborative HIV prevention and Adolescent Mental Health Project
Community health workers
Disruptive behavioral disorders
Data Coordinating Unit
Data and Safety Monitoring Board
Knowledge and Skills Assessment Test
Low and middle-income countries
Multiple Family Group
Ministry of Education
Ministry of Health
Multiple Principal Investigators
Multi-level structural equation modeling
National Institute of Mental Health
Practical, Robust Implementation and Sustainability Model
Parent Teacher Association
Root Mean Square Residual
Root Mean Square Error of Approximation
Substance Abuse and Mental Health Services Administration
The World Health Organization
Currently, an estimated 450 million people, most of whom live in poverty and are from low- and middle-income countries (LMICs), experience serious mental health challenges . Children in Sub-Saharan Africa (SSA) comprise half of the total regional population, yet current mental health services are severely under-equipped to meet their needs [1, 2]. A recent systematic review estimated that one in seven children in SSA may struggle with a serious mental health issue. The World Health Organization (WHO) estimates prevalence rates may be even higher (20%) . Uganda (one of the poorest countries in SSA) reports 12 to 29% of children presenting mental health symptoms when screened in primary care clinics [4, 5] and found that one in five Ugandan adolescents evidences a serious mental health challenge. Given the large numbers of children in Uganda, child disruptive behavior disorders (DBDs), if untreated, are a particularly serious concern as they commonly persist through adolescence and adulthood [6, 7, 8, 9].
DBDs are chronic, impairing, and costly mental health problems that, when left untreated, carry a high price resulting from disruptions in school performance, friendships, and family relations. DBDs impair academic and social functioning, disrupt the parent-child relationship, are associated with child maltreatment and abuse, and commonly co-occur alongside depression, anxiety, and substance use disorders. The impairment associated with DBDs in childhood also place youth at increased risk for future school dropout, substance use/abuse, delinquency, incarceration, criminal behaviors, and premature death [6, 7, 8, 9]. Studies have identified specific risk factors for increased incidence of DBDs among children, including poverty, low parental educational attainment, maternal depression, harsh parenting, poor parent-child relationship, stress, and orphanhood (the death of one or both parents) [10, 11, 12, 13]. Studies emphasize adverse outcomes associated with DBDs, including academic problems, social impairment, a higher incidence of chronic physical problems, unemployment and legal problems, and substance abuse and violence among adults [9, 14, 15, 16, 17, 18, 19, 20].
Six SSA countries, namely Uganda, Nigeria, South Africa, Ethiopia, the Democratic Republic of Congo, and Kenya have DBD prevalence rates, ranging from 12 to 33% [21, 22, 23, 24]. Given the serious consequences of failing to intervene as DBDs emerge, it is imperative that effective and scalable solutions are discovered, while simultaneously recognizing the challenges facing these countries in meeting the educational and mental health care needs of their large youth populations. In Uganda, the focus of this paper, children make up about half (56%) of the total population (compared to 20% in the US) , and they most often present with multiple simultaneous physical, mental health, and educational challenges [25, 26]. Ugandan children live in disadvantaged communities with high rates of chronic poverty (38%), domestic violence (30%), physical violence toward children (80%), depression (33 to 39%), malaria (70 to 80%), and HIV or AIDS (6%) [27, 28, 29, 30, 31, 32]. The country also has a significant number of orphans [9, 30]. Hence, for DBDs to be addressed, poverty and family economic capacities, family and community safety, as well as health and mental health co-morbidities must be taken into account in any evidence-based practice (EBP) considered for scale-up.
Treatments for DBDs
The basic principles underlying effective parenting practices and child development are considered cross-culturally robust [33, 34, 35, 36, 37]. Positive behavioral supports, effective behavioral management, positive parent-child relationship, and parent involvement in a child’s life play critically important roles in the healthy development of all children, regardless of cultural or ethnic background. Many DBD focused EBPs are designed to enhance parenting skills [38, 39, 40, 41] via behavioral practice, modeling, coaching, goal setting, family communication, and building on family strengths [41, 42, 43, 44, 45]. These components are embedded within the MFG model. Although MFG and other similarly effective interventions for the treatment of DBDs in youth have been tested in developed countries and in similar high-poverty and high-stress communities, and they are relevant for widespread dissemination in LMICs, most of these EBPs are widely not used in SSA countries.
Rationale for applying 4Rs and 2Ss MFG intervention for DBDs
Kazdin and Whitley  describe how specific family factors tied to poverty (e.g., stress) may undermine parenting (e.g., family organization, discipline practices, family connectedness, support, communication) and contribute to serious childhood behavior problems [47, 48, 49]. In collaboration with parents and service providers in the US, this body of research was summarized to encourage transparency of the evidence base for families and to provide an “easy to remember” means of organizing existing science for family peer MFG facilitators [50, 51, 52, 53, 54]. Specifically, four broad conceptual categories were created and became the family level targets for MFG: Rules, Responsibility, Relationships and Respectful communication (4Rs). Stress and Social support (2Ss) were added as these impact service engagement and outcome. This same review of the evidence regarding the influence of parenting, quality of family life and youth behavioral challenges was also conducted in South Africa as part of the CHAMP (Collaborative HIV prevention and Adolescent Mental Health Project), an EBP listed on SAMHSA’s National Registry of Evidence-based Programs and Practices.
The MFG (guided by the 4Rs and 2Ss) involves six to eight families in the US and 12 to 20 families in South Africa [40, 55, 56, 57, 58, 59, 60, 61, 62, 63]. At least two generations of a family are present in each session. Content and practice activities foster both within-family and between-family learning and interaction . The MFG targets primary school age children aged 7 up to 13 years in the US and South Africa. In the US, the MFG is designed for children struggling behaviorally and their families. In South Africa, MFG has targeted youth at high contextual risk for behavioral challenges (high rates of poverty, violence, family loss due to HIV and other health threats). In both countries, children and their families (including adult caregivers and siblings over 6 years of age) are invited to attend 16 sessions.
MFGs are also the core of SUUBI-Maka (Hope for families in Luganda), a family focused, economic-strengthening program tested in Uganda, [10, 11, 12, 13, 65, 66, 67, 68, 69, 70, 71] furthering the evidence for their feasibility and acceptability in the specific context of this scale-up study. In addition, the development of MFG was guided by recommendations offered by Weisz  and Hoagwood  who argued that to improve implementation of EBPs, development and testing must occur in the setting where the service will ultimately be embedded, with the prior work in South Africa (SA) giving the MFG an important head start in learning about factors that influence implementation and scaling [40, 61, 70, 74, 75]. Also, both in the US and South Africa, MFGs have been facilitated by lay counselors or community health workers/village health teams, and parent peers, consistent with task-shifting strategies.
Contextual influences on implementation and scale-up
To examine short-term and longitudinal outcomes associated with the MFG (disruptive child behavior, behavioral functioning)
To compare the uptake and implementation of MFGs by trained existing family peers and CHWs
To elucidate multi-level factors that influences uptake, implementation and youth outcomes
The proposed study will target youth attending primary schools in Rakai District and the greater Masaka region (southwestern Uganda) as well as their caregivers. The region is heavily affected by poverty with high HIV prevalence rates (at least 8.5% vs. 6.5% found in the national average) . The area also has large number of children with no living biological father, mother or both (also known as orphaned children). An estimated two in five children are orphaned to AIDS each year in these districts .
Study population, recruitment, and retention
Sampling and recruitment
The study involves the active collaboration of the Masaka Diocese which operates slightly over 420 public primary schools in Rakai District and the greater Masaka region. We will recruit medium-sized schools to participate in the study. All schools with a total enrollment of more than 900 students will be eliminated as they are considered above average enrollment. Similarly, all schools with a total enrollment of less than 350 students will be dropped because they are considered below the average normal enrollment of at least 50 students per grade. Schools that remain on the list after this stratification will be contacted to schedule informational meetings with the school leadership and staff (school heads/directors, head teachers, and teachers) to present the research study and gauge interest in study participation. Follow-up meetings will be scheduled when necessary. The research team will compile a list of the schools that express commitment to participating in the study (see the “Randomization” section for further details).
Children and caregivers
Schools will be asked to organize meetings for all families to learn about the study. All research staff will be on hand at each meeting to provide the opportunity for parents to ask questions about the study individually or as part of a larger group of parents. If parents and children are interested in participating, each of them will have an opportunity to meet individually with research staff and provide informed consent and child assent.
Following consent procedures, parents will be asked to complete screening tools for child DBDs (see Table 3). The screening will be conducted by project coordinators and research assistants. If the child meets the criteria for a clinically meaningful behavioral problem on any of the screening measures, they will be considered “positive”. They and their caregivers will be invited to enroll in the study and an appointment to complete a full baseline assessment (for both child and caregiver; see the “Measures” section) will be made prior to beginning to attend MFG meetings. If a child does not evidence a DBD, they will be considered “negative.” The family will be enrolled to attend the MFG meetings (all families in the school community will have the opportunity to attend to decrease stigma, but also enhance functioning of all parents/families). An appointment to complete a shortened version of the baseline assessment (for both child and caregiver; see the “Measures” section) will be made prior to beginning to attend MFG meetings. The child and the caregiver will be monitored at each time point with shortened versions of the assessment batteries. If at any point during these assessments, the child meets criteria for DBD, both the child and the caregiver will complete full version of the assessment batteries.
School meetings is the first way that we will reach out to parents of children enrolled in the potential schools. Research staff will also conduct home visits (as has been typically done in other studies in the area). In total, we will recruit 3000 children and their adult caregivers across three study conditions. We will continuously monitor study participants’ recruitment and enrollment during weekly staff meetings to ensure equivalent involvement across conditions by: gender of youth, age and adjust if imbalance occurs accordingly.
Parent peers and CHWs to facilitate the adapted MFG
At each of the school sites assigned to the parent-peer-delivered condition (n = 10), the head of school and the Chair of the Parents Teachers Association (PTA) will identify up to six parent peers to receive the training to deliver the adapted MFG. Parent peers will have already received some training for their current roles (e.g., workshops in methods to reach out to parents whose children are not performing well in school or not attending; facilitation skills in relation to organizing parent meetings). We expect to recruit 60 parent peers from the 10 schools in the parent-delivered condition.
At each of the school sites assigned to the CHW-delivered condition (n = 10), we will collaborate with the village local leaders, the district health officials, and the Ministry of Health (MOH) to identify up to 60 existing outreach workers associated with local primary care clinics in the study area. CHWs have already received training in health assessment, home and school outreach and facilitation of health-related meetings. Many of the CHWs are also local leaders within their communities. We expect to recruit 60 community health outreach workers to deliver the adapted MFG to children from 10 schools assigned to their condition.
Inclusion criteria are as follows: (1) school head teacher willing to participate; (2) parent peers or CHWs willing to participate; (3) adult caregiver of a child in primary 2 through 7, 8 to 13 years, willing to consent and available for research and intervention activities; (4) a child screened for oppositional defiant disorder (ODD) or conduct disorder (CD) and willing to assent. Exclusion criteria include lack of understanding of the study and participant rights and refusal to participate. If the youth or adult caregiver presents with emergency needs (e.g., hospitalization), needed care will be secured, rather than study participation.
Sample attrition and retention
We believe that attrition will be minimal because the investigative team is known in the schools and communities, and has substantial experience recruiting and tracking “hard to reach” families. The study will take place in a highly stable region of Uganda, where geographical moves are rare. Using the same protocols that we have used in our prior studies, we will ask the participants to give their postal box number and telephone number (if they have one), and names, addresses, and contact information for three people who will always know how to reach them. Participants will be reminded that if we are to contact the people listed, we would never discuss any details about their involvement in the study. We will use these records to track the location of study participants over time. We have effectively used these methods in other research programs, resulting in very low attrition rates (for SUUBI-Uganda, 2.4% at 10-month follow-up and 7.3% at 24-month follow-up; Suubi+Adherence, 4.3% at 48 months, and Bridges, 4.1% at 12 months, 9.3% at 48 months) [10, 11, 12, 13, 65, 66, 67, 68, 69, 70, 71]. Hence, we expect attrition from baseline to the end of follow-up to be no more than 20%. We will keep careful records for those who drop out of the study and test for attrition bias based on data that we will have prior to study dropout. To the extent that such bias is present, we will limit generalizations accordingly, or where possible, introduce statistical adjustments (hot deck imputation methods [83, 84, 85, 86] to complete missing observations, and thus to address bias.
Statistical power calculations
Unit (n per group)
4 time points
School (n = 30)
Facilitators (n = 120)
Family (n = 3000)
Multiple Family Group (MFG) targets, family skills, and goals
Empirically supported family skill
Family organization; consistent discipline
Clarify rules, consequences, rewards
Clarify responsibilities, expectations, rewards
Family warmth; within-family support
Schedule for positive family interaction
Family communication; family conflict
Listening/ talking skills (parents/children)
Parenting hassles and stress; life events
Identify stressors undermining family change
Within-family and external support plan
Description of MFG intervention protocol
The protocols have been designed to provide multiple opportunities during each session to directly apply content to the realities of family life, emergent cultural and values perspectives, as well as to tailor messages to age of child. We have built in redundancy for missed appointments and opportunities for reinforcement. We aim for families to attend at least seven to eight meetings, as findings suggest that this dose is needed for reductions in child behavioral difficulties and prior studies in the US and SA suggest that most families will reach this goal [57, 58].
We will train at least six facilitators per school site to deliver the MFGs in both MFG-parent-peer-delivered and MFG-CHW-delivered condition. Training for parent peers and CHWs will occur separately, by condition. Incentives to participate and complete training will include: receiving certification in child mental health competency to be offered by the local district leaders—including the LC5 Chairperson and the district health official, and transport facilitation needed to complete project tasks and a community-wide recognition event thanking facilitators for their contributions.
Training and supervision
Training will consist of up to six modules. Training focuses on childhood conduct difficulties, family-level factors that have been linked to child outcomes, strategies to enhance engagement and motivation, and group facilitation skills and processes specific to MFGs. At the end of the MFG training, we will administer a Knowledge and Skills Assessment Test (KSAT), to assess mastery of the content (live competence demonstrations and knowledge questions read aloud to facilitators). The criteria of mastery for the KSAT will be set at 85%. During the MFG implementation period, facilitators will receive 2 h per month of group supervision across sites while the MFGs are in progress.
Description of the comparison condition
Mental health wellness materials from the MOH will be made available to all children and families recruited in the study (across all the three study conditions). We will adapt printed materials on child mental health based on those recently created by the MOH.
Adaptation of MFG manual
During phase I (months 1 to 12), the study team will review the existing MFG intervention protocol, as well as the South African training and delivery methods used in the CHAMP program of research (e.g., cartoon format of information delivery and family practice activities, training protocols for lay counselors and community facilitators), and the SUUBI family economic strengthening approach. Further, these working groups will review the existing evidence related to the relationship between specific parent management practices, family organizational processes and livelihood approaches, and positive youth behavioral outcomes in order to align with perspectives and organization of families in SSA. The intervention adaptation team will comprise the in-country research team as well as community stakeholder volunteers (one to two teachers, one to two parents, one to two CHWs) recruited to assist with intervention adaptation.
In the Ugandan context, faith-based institutions, including the church and religious leaders, are longstanding partners to the investigative team. The same guidelines for partnerships that organize our work with local, regional and national policy-makers, and non-governmental organizations (NGOs) also guide our intensive collaboration with religious, cultural, and community leaders. These partnerships are characterized by intensive, ongoing involvement and communication with substantial effort invested in developing shared goals, understanding and feedback processes across the entire span of the project and country contexts.
The adaptation process will follow steps previously outlined by Petersen and McKay [40, 61, 75]. Specifically, interaction between in-country and cross-country teams will be intensive and systematic. Team members will identify core elements of the existing science on parenting and family life that need to be addressed within each country context (for example, in prior studies set in SA, rapid ethnographic studies were conducted to identify elements of family life that were missing from existing evidence-based, particularly perceptions of disempowerment of parents, harsh discipline strategies, parents’ and children’s rights) and incorporate evidence-informed intervention elements into the adapted MFG intervention and training protocols.
Stratified cluster randomization of schools to conditions will be used, with schools stratified into four strata based on two variables: (1) student population size (medium size vs. large) and; (2) geographical location (rural vs. semi-urban), to ensure balance on those variables. The restricted randomization technique of Hayes and Moulton  will be used to assure overall school balance across the three experimental groups. Each school will be randomly assigned to one of the three study conditions such that all selected children from a particular school will receive the same intervention (reduce contamination). These schools are geographically apart, with limited transportation. Hence, the risk for contamination will be minimal (See the “Contamination” section).
A list of the primary schools in the area will be obtained from the Masaka Diocese that oversees all the 420 church-founded but government-supported public primary schools in the Rakai District and Masaka regions; and the District Education Office
All schools with a total enrollment of more than 900 students will be eliminated as they are considered above average enrollment. Similarly, all schools with a total enrollment of less than 350 students will be dropped because they are considered below the average normal enrollment of at least 50 students per grade
Schools that remain on the list after this stratification will be contacted to schedule informational meetings with the school leadership and staff (school head teachers, and teachers) to present the research study and gauge interest in the study. During these meetings, the purpose of the study will be explained and what the school’s participation entails will be laid out. All study-related questions will be answered. Follow-up meetings will be scheduled when necessary
The research team will compile a list of the schools that express commitment to participating in the study
Thirty schools from this list will be randomly chosen
Using SPSS software version 24, each of these 30 schools will be randomly assigned to one of the three study conditions, i.e., 10 to the adapted MFG delivered by parent peers; 10 to the adapted MFG delivered by CHWs; and 10 to the comparison condition
We estimate that a total of 30 schools will allow us to identify sufficient number of youth and nest them within trained program facilitators. This will allow the research team to detect a small to moderate effect based upon power calculations. Thirty schools is our best estimate of the total student population needed to draw a sufficient sample. The number of schools will be adjusted if needed.
Because the experimental-control manipulations are applied across schools that are a considerable distance from each other, and CHWs and parent peers have their own unique designated site, we expect contamination to be minimal. Specifically, all children from the same school and their families will be involved in the same study condition. The distribution of materials is unlikely to influence implementation given that the schools are geographically separated, with limited transportation opportunities in between schools. However, the in-country research team will monitor the extent of any such “seepage” and make statistical adjustments for it, if needed. Monitoring will be established by directly asking facilitators whether any intervention materials were shared with staff/caregivers from other schools every time they complete the delivery of a 16-week intervention and their answers will be carefully documented by the research team (including when, how many people, which materials, and which schools, when relevant). More specifically, this assessment will take place when the research team is in the field to complete the 16-week assessment with caregivers. Again, we do not anticipate contamination given the limited interaction across schools.
Aim 1. To examine the short- (8 and 16 weeks) and long-term (6-month) outcomes (primary outcomes) associated with the Multiple Family Group (MFG)
1. Disruptive Behavior Disorder Rating Scale 
S, T2, T3, T4
3. Alabama Parenting Questionnaire (APQ)-short form 
T1, T2, T3, T4
4. Parent Stress Index-short form 
T1, T2, T3, T4
5. Brief Symptom Checklist-short form 
T1, T2, T3, T4
6. Strengths and Difficulties Questionnaire 
T1, T2, T3, T4
7. Family Relations and Cohesion scale (adapted from Parent Child Relationship Inventory, Child Caregiver Communication Scale, Family Assessment Measure, and Family Adaptability and Cohesion Scale) [10, 11, 12, 13, 65, 66, 67, 68, 69, 70, 71]
T1, T2, T3, T4
T1, T2, T3, T4
9. Multidimensional Students’ Life Satisfaction Scale 
T1, T2, T3, T4
T1, T2, T3, T4
T1, T2, T3, T4
T1, T2, T3, T4
13. Child Depression Inventory (CDI-short form) 
T1, T2, T3, T4
14. Tennessee Self-Concept Scale 
Aim 2. To examine the uptake, implementation, fidelity, and sustainment of two MFG implementation approaches.
4. Knowledge, Skill, Attitude Test (KSAT) for MFG facilitators 
5. Attendance logs
every MFG session
6. MFG Intervention Fidelity Assessment 
F, C, R
every MFG session for F & C, every other session for R
7. Implementation and Feasibility Assessment 
T2, T3, T4 for F; T3 for SH
8. Program Sustainability Assessment 
T2, T3, T4 for F; T3 for SH
Aim 3. To elucidate multi-level factors that influences uptake, implementation, sustainment, and youth outcomes.
MACS process measures 
T2, T3, T4
Data analysis plan
Qualitative data analysis
All qualitative interviews will be digitally recorded and transcribed. Interviewers will then compare transcripts with digital records to insure accuracy of transcription. All field notes, interview transcripts, and interviewer notes summarizing interviews will be entered into NVivo . Accuracy of information will be assessed through triangulation in which accounts of specific events and behaviors obtained from field notes, interviews, surveys, and archival material are compared . To insure credibility of findings and enhance the validity and reliability of data, all interviews will be reviewed by at least two members of the research team. Consensus on coding and coding procedures and modifications to coding books will occur through regular team meetings. Study results will be presented to informants, enabling them to provide comment of results and suggest modifications or additional avenues of investigation.
Using a methodology of “Coding Consensus, Co-occurrence, and Comparison” , implementation team logs and interview transcripts will be analyzed in the following manner. Each investigator will review this material to develop a broad understanding of content as it relates to the project’s specific aims and to identify topics of discussion and observation. Investigators will prepare short descriptive statements or “memos” to document initial impressions of topics and themes and their relationships and to define the boundaries of specific codes (i.e., the inclusion and exclusion criteria for assigning a specific code) . The empirical material contained in the field notes and interview transcripts will then be independently coded by the project investigators to condense the data into analyzable units. Segments of text ranging from a phrase to several paragraphs will be assigned codes based on a priori (i.e., from the interview guide) or emergent themes (also known as open coding) . Following the open-coding, codes will be assigned to describe connections between categories and between categories and subcategories (also known as axial coding) . Codes will also be assigned to material to reflect the social and demographic characteristics of study participants. Lists of codes developed by each investigator will be matched and integrated into a single codebook.
Five transcripts will be independently coded by at least two investigators. Disagreements in assignment or description of codes will be resolved through discussion between investigators and enhanced definition of codes. The final list of codes or codebook, constructed through a consensus of team members, will consist of a numbered list of themes, issues, accounts of behaviors, and opinions that relate to coalition structure, function, development, and sustainment. With the final coding structure, two investigators will separately review a sample of transcripts to determine level of agreement in the codes applied. A level of agreement ranging from 66 to 97%, depending on level of coding (general, intermediate, specific), indicates good reliability in qualitative research .
Upon completion of the coding of the remaining transcripts, NVivo will be used to generate a series of categories arranged in a treelike structure connecting text segments grouped into separate categories of codes or “nodes.” These nodes and trees will be used to further the process of axial or pattern coding to examine the association between different a priori and emergent categories. They will also be used in selective coding of material to identify the existence of new, previously unrecognized categories. The number of times these categories occur together, either as duplicate codes assigned to the same text or as codes assigned to adjacent texts in the same conversation, will be recorded, and specific examples of co-occurrence illustrated with transcripts. Through the process of constantly comparing categories with each other, the different categories will be further condensed into broad themes.
Quantitative data management and analyses
Exploratory scale structure analysis. The research will use robust estimators (in the form of Huber-White estimation) to accommodate non-normality and variance heterogeneity. We will conduct formal tests of outliers and be sensitive to specification error. Inter-item reliability analyses for all scales will be conducted using confirmatory factor analytic (CFA) perspectives; correlations across time and construct will be examined
Data-reduction procedures. Most of the proposed instruments have summary scoring procedures; however, multiple scores in specific domains may result in multicollinearity, requiring further data reduction. Various data-reduction strategies will be conducted, including factor analyses with oblique rotation  and Chi-squared Automatic Interaction Detector (CHAID) algorithms, to identify the strongest variables in the pool. The Exhaustive CHAID method, developed by Biggs, deVille and Suen  will be used to explore various groups of predictor variables as they relate to outcomes
Tests of sample equivalence across study conditions. We will use baseline data to examine potential biases related to possible attrition by condition or selection biases not addressed by random assignment. Preliminary analyses will be performed to compare the schools, facilitators, and adult caregivers in the three study conditions on descriptive/clinical characteristics at baseline to ensure that randomization succeeded. Any ancillary variables that are not the focus of hypotheses, but relate meaningfully to outcomes within treatment groups are candidates for inclusion as covariates. When covariates differentiate treatment groups, they will be included in all analyses to remove confounding effects
General statistical considerations. To address aims # 1 and 2, we will use random-effects models to examine scores over the four assessments. Random-effects models have characteristics that provide solutions to commonly observed problems in the analysis of longitudinal data; missing data, serial correlation, and time-varying covariates . These analyses can model systematic person-specific deviations from the average time trend. Just as the nesting of participants within providers is an example of multi-level data, the nesting of perspectives within teacher reports can also be viewed as multi-level data. By treating treatment perspective as a “repeated measurement” we can estimate the difference between the three conditions on outcomes as a synthesis of the three perspectives. Divergence between the perspectives can also be examined by including interactions between covariates, condition and the rater effect. We will model each outcome as a function of group, time and group-by-time interaction. We will use Supermix software for the mixed-effects linear regression .
To address aim 3, our analyses will use variants of structural equation modeling (SEM) with Mplus. The models we test often will be statistically over-identified. We will evaluate model fit using both global fit indices (e.g., relative/normed chi-square χ2/df with range 2.0–5.0) [115, 116], comparative fit index (CFI) (with value ≥ 0.95) , standardized root mean square residual (RMR (below 0.08) , root mean square error of approximation (RMSEA) (with the lower limit is close to 0 and the upper limit less than 0.07)  and indices of fit that are diagnostic of specific points of ill fit in the model (e.g., standardized residuals, modification indices). If poor model fit, then we will examine diagnostics that suggest revisions to the model that are meaningful and that will significantly improve model fit.
Aim 1. To examine short-term and longitudinal behavioral outcomes associated with the MFG
Children who participate in MFG with their families will display significantly reduced conduct difficulties and increased functioning over time compared to those involved in the comparison condition. We expect that parent peers will evidence significantly more success engaging families to attend MFG sessions; thus, children in the MFG-parent-peer-delivered condition will evidence greater improvement relative to the other two study conditions.
We will consider a random-effects model for the child outcome variables across time as a function of treatment group, time and the interaction of group by time. A number of features will be common to all models. First, two levels of nesting will be explicitly modeled (caregiver and facilitator perspectives). At the third level of nesting, models may condition on a series of dummy codes capturing possible school-level effects. Second, the effect of primary interest is the test of the study condition (e.g., MFG-parent peer, MFG-community worker, comparison) by outcome trajectory (child behavioral symptoms and functioning) interaction. Third, time will be coded judiciously in accordance with a priori hypotheses about the time lags of effects of the experimental interventions. To adequately model hypothesized trajectories, higher-order polynomials can be estimated under different centering specifications, providing for multiple interpretations of the intercept and low-order trend components. Time contrasts can also be orthonormalized to reduce potential problems of multicollinearity in models specifying higher-order growth forms. Relevant trajectory variance components will also be evaluated to determine if there is variation (which would then be modeled as a function of both constant and time-varying covariates).
Aim 2. To compare the uptake and implementation of MFGs by trained family peers and CHWs
Given the level of training that CHWs receive prior to the study, they will evidence higher-fidelity initially, yet with training and ongoing supervision, we expect these differences to decrease over time.
To evaluate the fidelity, quality of implementation for MFGs implemented by family peers and CHWs, we will carry out a series of descriptive analyses and examine level of fidelity in four domains (described above). We will examine fidelity by year first to understand any change patterns over time, and then will combine analyses across years if no meaningful differences are found. Finally, see random-effects modeling to be used to examine differences by implementers over time.
Aim 3. To elucidate multi-level factors that influence uptake, implementation, and youth outcomes
The overall analytic structure uses a three-level, multi-level SEM (MSEM) framework with family level variables at level 1, facilitator variables at level 2, and school variables at level 3. The analytic structure is complex, so we highlight our approach using Fig. 1 as our primary reference. We will employ mediation analysis with two-level MSEM models as explicated by Preacher, Zhang and Zyphur  and extended to three level models by Preacher . We will use a combination of limited information estimation frameworks (focused on sub-portions of the full model) and full information estimation frameworks (focused on the entire three-level model) to strategically answer questions. At level 1, child outcomes are modeled as a function of the family predictors (see Fig. 1), taking into account the clustering at the higher levels of the model. The follow-up (6-month) measures of the mediators and the outcomes are included within the model in a classic SEM panel model with autoregressive effects, thereby linking the follow-up data to the immediate post-test data. This is an advantage of using MSEM over traditional multi-level modeling. Different centering strategies (e.g., grand mean vs. group mean) can be explored to garner various perspectives on the data. This feature of the model provides perspectives on the relative importance of paths in Fig. 1. To ascertain perspectives of the effects of school-level variables on facilitator variables, random intercept MSEM with the facilitator variables can be estimated as a function of the school variables that include dummy-coded treatment variables (MFG-parent peer vs. MFG-CHW) impacting the school mediators of readiness, leadership support and climate, which in turn, affect the facilitator-level intercepts. This modeling, or variants of it, address aim 3.
Integration of qualitative and quantitative analyses
We will use three different formats as described by Creswell et al.  for merging or converging the two datasets by bringing them together, connecting them by having one build upon the other; and embedding one dataset within the other so that one plays a supportive role for the other. Merging will occur through data triangulation in which results of quantitative and qualitative analyses are placed side by side to determine whether each provides the same answer to the same question (convergence). Results of our qualitative analyses will be connected to results from our quantitative analyses when we use the former to provide explanations for unanticipated findings produced by the latter (expansion). Finally, results of qualitative analysis can be embedded within the analysis of quantitative data by helping to provide context for outcomes (complementarity).
Data quality and management
The study will be overseen by local principal investigators. They will monitor the trial on a continuous basis and report to the Multiple Principal Investigators (MPIs) on a weekly basis via email, telephone or in person. Moreover, the study data will be reviewed by the National Institute of Mental Health Data and Safety Monitoring Board (DSMB) twice a year. The DSMB is established independent of the researchers involved in the study. The Board will oversee conduct of the study, particularly participant safety and data integrity. The Board has access to all the safety and data quality information collected and has the authority to stop the study if the Board believes the study is leading to unacceptable risks to participants. The study team will submit a data report to the DSMB twice yearly and upon request. The data report and other submissions to the DSMB will adhere to the established reporting format developed in consultation with National Institute of Mental Health (NIMH) collaborator(s). The report will include the major variables necessary for monitoring study participant safety, data integrity, and protocol fidelity, including study participants’ enrollment and retention. All the data reported to DSMB will be de-identified data to protect participants’ privacy. The DSMB will also review the study protocol, informed consent forms and procedures, and all relevant documents before the onset of the study, and will review and approve amendments to these documents.
The MPIs and Data Coordinating Unit (DCU) will prepare updates to the DSMB twice a year. In collaboration with DCU, the in-country data manager will organize data reports regarding preliminary process and outcome analyses (data collection at 8 weeks) and share with the MPIs. If preliminary analyses reveal harm (outcomes of child behavior are disproportionately negative), the results will be thoroughly reviewed by the MPIs and DSMB to assess whether stopping is warranted. However, no negative results have been identified in prior studies using the MFG intervention.
The intervention is associated with adverse effects that call into question the safety of the intervention
Difficulty in study participant recruitment or retention will adversely affect the ability to evaluate the study endpoints
Any new information becomes available during the trial that necessitates stopping the trial; or
Other situations occur that might warrant stopping the trial
Data Coordinating Unit (DCU)
This unit will be housed at Washington University in St. Louis and will be dedicated to the proposed research study. Data Coordinating Unit responsibilities include: (1) Creating a shared infrastructure that will facilitate the development and implementation of high quality study design, data collection, analysis, and development of publications across multiple projects (e.g., creation of a participant-tracking data base for scale-up study and small-scale implementation trials; assistance in creating data bases to organize data; assistance in planning and carrying out data entry and analyses of pilot data); (2) Developing core assessment battery data system (e.g., organize existing instruments; facilitate review of measures by collaborators to provide feedback); (3) Organizing consultation with on issues regarding protocol design, research methodology, measurement instruments and models, and data-analysis considerations.
Quality assurance plan
The MPIs and site PI are responsible at all phases of project implementation and for ensuring that the validity and integrity of the data, including study participants’ recruitment, enrollment, enrollment targets, and data collection procedures. MPIs, in collaboration with the site PI will implement quality assurance measures, including ongoing monitoring of the scale-up and pilot study activities, weekly correspondence with the local PI, and the timely reporting of any unexpected or expected adverse events that might occur during the study period. Moreover, the study will be monitored by external monitors twice a year (independent from investigators and the sponsor). The external monitors will write a report upon completing their audit, which will be shared with the local PI, MPIs, and the sponsoring institution (NIMH).
It is unlikely, though possible, that intervention facilitators may feel coerced into participating. We will assure all potential study participants that their participation is voluntary. Voluntary participation will be discussed with all potential participants and they will be instructed that if they choose not to participate, it will not affect their employment status
A second risk is the potential loss of confidentiality. Respondents may reveal sensitive information during family groups and/or data collection. All participating groups will be encouraged to respect and protect the confidentiality of information shared during groups. All research staff and facilitators will sign confidentiality pledges. If the research team detects breach of confidentiality by research staff or facilitator, their responsibilities related to the study will be terminated.
We will also take the utmost caution to protect the confidentiality of responses to measures. Hard copies of documents and audiotapes will be maintained in locked file cabinets of a locked office in a secure building at the International Center for Child Health and Asset Development (ICHAD) in Uganda, at the School of Public Health at University of Ghana in Ghana, and at the Department of Psychiatry at University of Nairobi in Kenya. All data will be coded with ID numbers and stored in computerized datasets. All documents and audiotapes will be transported in locked carrier bags. Participant consent forms and ID logs will be kept in two separate locked cabinets in a separate in-country location. Tracking information also will be kept on a password-protected computer. The master list will only be used to coordinate data collection and all staff will be required to receive training on both study participants protections, as well as maintaining the confidentiality of participants
For those in MFG groups, though highly unlikely, there may be some risk of physically challenging behavior because the program involves having children with behavioral challenges in a group setting together. The PI has conducted research studies using this intervention for over a decade and never experienced a physically aggressive incident. However, to minimize risk, both research staff and facilitators will be trained to diffuse these circumstances if they occur. The research team will make appropriate referrals to the families who need additional support
Among families, the most serious risk in the proposed research is any adverse outcome that may jeopardize the welfare of a participating child or family. In general, when conducting child mental health service research with children and their families, potential risks of adverse outcomes exist as a consequence of the progression of a serious mental health issue or response to a service provided. The principle investigators will be directly involved in training all research staff (including senior project coordinator, research coordinator(s), and research assistants) and facilitators to identify key indicators of any increase in physical or mental health problems among the children and families participating in the studies across the three countries. In addition to facilitators’ observations in each session, the study participants will be monitored throughout the study and data on targeted outcomes will be collected at the mid-point of the intervention as well as at 16 weeks, the end of the intervention. That will give the research team the opportunity to track potential worsening of symptoms. Research staff will be on site at least every other session, which will give facilitators the opportunity to communicate any observation of worsening symptoms. All reporting will be documented
Participants may feel discomfort in answering some questions in the measures. They will be reminded during data collection, that they can skip any questions that they do not feel comfortable answering
There is an increasing interest in and use of social safety nets to achieve health outcomes for children. At the same time, there is a need to understand how psychosocial interventions can complement these economic programs for enhanced impact on mental and behavioral health. The present study protocol describes a scale-up experimental study using a mixed-methods, hybrid type II effectiveness implementation design to examine the implementation and effectiveness of an MFG family strengthening intervention, aimed at improving child disruptive behavioral challenges in Uganda. The study will also examine multi-level factors that influence uptake, implementation, sustainment, and youth outcomes.
We hypothesize that children participating in MFG will display significantly reduced behavioral difficulties and increased functioning over time compared to those involved in the comparison condition. We expect that parent peers will evidence significantly more success engaging families to attend MFG sessions, and thus children in the MFG-parent-peer-delivered condition will evidence the greatest improvement relative to the other two study conditions. We expect that, given the level of professional training that CHWs have received prior to the study, they will evidence higher protocol fidelity than parent peers, at least initially. Yet, with training and ongoing supervision for parent peers, we expect these differences across groups to decrease over time. Hence, the study findings will yield important insights on the effectiveness of MFG intervention in reducing child behavioral challenges while also exploring contextual factors that may influence MFG implementation, sustainability, and impact.
We do not anticipate any major threats to study implementation though we recognize potential concerns and have adapted accordingly. We have a stringent retention plan for attendance at the MFG sessions. We expect to achieve our enrollment and retention goals based on our current and previous studies among school going children in the same study area. However, we are aware that as children grow, they are more likely to leave school, and some may migrate to work before completing the intervention. As such, should our study participants’ recruitment, enrollment or retention rates deviate from anticipated goals, we will consult with the research and implementing team to adjust our outreach activities accordingly.
Despite the potential limitations identified above, the proposed project is highly innovative in multiple ways. First, the study has been designed to align with the priorities and preferences of Ugandan policy-makers in the MOH and MOE. Second, building on existing child and family focused EBPs, [38, 55, 56, 65, 66, 67, 68, 69, 70, 71] this is the first investigation of a family centered approach of DBD interventions provided within the lower primary school context (before children drop out of school), with the aim of reaching a large child population with mental health service needs in SSA countries. If successful, the potential public health impact is high. Third, our study draws upon an EBP package that has already been tested with delivery by parent peers and community facilitators and will take advantage of the advancing science behind task-shifting, defined as “engaging non-specialists in the provision of effective psychosocial treatments under the supervision of mental health specialists” . The task-shifting approach to supporting lay workers and peers already exists in health and education systems, and thus is promising as a cost-efficient and feasible model for SSA countries. This study tests two task-shifting approaches (task-shifting intervention skills to CHWs and parent peers). It is possible that testing a task-shifting, large-scale implementation strategy in low-resource SSA settings can facilitate “reverse innovation.” Effective services and implementation strategies identified in developing countries may facilitate new innovations to address similar CAMH disparities in US populations or in other developed countries. Fourth, the collaborative approach is innovative as it includes capacity building and collaboration among regional health and education policy stakeholders, experts from multiple disciplines, and community members. The approach will help stimulate application of science-based policy decisions, enhance local community leaders’ and members’ participation in resolving CAMH service gaps, and increase the chance for program sustainment, dissemination, and policy change. Finally, several methods employed in this study are innovative. This study blends in clinical effectiveness and implementation research methods , and incorporates strategies (utilizing PRISM) to investigate scale-up-related mechanisms and implementation research questions, which can have more rapid translational gains, inform more effective implementation strategies, and provide more useful information for decision-makers. In summary, this study has great potential to address global child mental health needs.
In sum, this study has been carefully designed with contextual issues at the forefront. An in-country study team consisting of academic researchers, NGOs, government officials, and community/cultural leaders) will lead MFG systemic adaptation efforts so that, if findings warrant, there will be an increased chance that the intervention can be rolled out easily to an expanded range of child-serving settings and implemented by existing workforces. Our partnership with the Ministries and Departments of Health and Education also increases the likelihood that findings will influence large-scale roll outs, especially since the MFGs in the US and SA have already been delivered by lay counselors, family peers, and trained community workers in schools and public health clinics.
At the time of manuscript submission, the trial had received DSMB and Institutional Review Board (IRB) approvals and was in the process of study participant recruitment and baseline assessment. The study is ongoing.
We are grateful to Gertrude Nakigozi at Rakai Health Sciences Program in Uganda; Abel Mwebembezi at Reach the Youth Uganda; Apollo Kivumbi, and Phionah Namatovu at the International Center for Child Health and Development (ICHAD) who are key members of the research and implementation in Uganda. We would like to thank the 30 primary schools that have agreed to participate in the study. We are also grateful to the parents and teachers who have contributed to the adaptation of the MFG content and delivery to the Uganda context.
The study outlined in this protocol is supported by the National Institute of Mental Health (NIMH) under Award Number U19 MH110001 (MPIs: Fred Ssewamala, PhD; Mary McKay, PhD; Kimberly Hoagwood, PhD). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIMH or the National Institutes of Health.
Availability of data and materials
The data for this study has not been collected in full and, therefore, is not yet available to the public. Once complete, the dataset will be sent to the NIMH, which will make it publicly available.
FS, MM, and KH are the lead/principal investigators for the grant. OSB and KH serve as co-investigators. BP is the project scientist from the National Institute of Mental Health. FS and OSB drafted the manuscript. All authors have reviewed and commented on drafts and have approved the final manuscript prior to submission.
Ethics approval and consent to participate
All study procedures were approved by the Washington University in St. Louis Review Board (#2016011088) and the in-country local IRBs in Uganda: Uganda Virus Institute (UVRI – GC/127/16/05/555), and the Uganda National Council of Science and Technology (UNCST – SS4090). The study procedures were also approved by the Data Safety and Monitoring Board at the National Institute of Mental Health. Informed consent/assent will be obtained from all study participants. Any protocol modifications will be submitted to Washington University in St. Louis Review Board, in-country local IRBs, as well as the Data Safety and Monitoring Board for approval.
Consent for publication
Any results that will be published in academic journals or conferences will adhere to IRB guidelines.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
- 3.World Health Organization, World Psychiatric Association, International Association for Child, Adolescent Psychiatry, Allied Professions. Atlas: child and adolescent mental health resources, global concerns, implications for the future. Geneva: World Health Organization; 2005.Google Scholar
- 5.Nalugya J. Depression amongst secondary school adolescents in Mukono district, Uganda. Doctoral dissertation, dissertation. Kampala: Makerere University; 2004.Google Scholar
- 12.Curley J, Ssewamala FM, Nabunya P, Ilic V, Han CK. Child development accounts (CDAs): an asset-building strategy to empower girls in Uganda. Int Soc Work. 2014;57:1–14.Google Scholar
- 21.Apkan MU, Ojinnaka NC, Ekanem E. Behavioral problems among schoolchildren in Nigeria. South Afr J Psychiatry. 2010;16:50–5.Google Scholar
- 25.UNICEF. State of the world’s children 2015 country statistical tables: Uganda statistics. 2015. http://www.unicef.org/infobycountry/uganda_statistics.html. Accessed 15 July 2015.Google Scholar
- 27.World Health Organization. Country profile of environmental burden of disease: Uganda. 2009. http://www.who.int/quantifying_ehimpacts/national/countryprofile/uganda.pdf?ua=1. Accessed 2 July 2015.Google Scholar
- 29.Naker D. Violence against children: The voices of Uganda children and adults. Kampala: Raising Voices and Save the Children in Uganda; 2005.Google Scholar
- 32.WHO: Uganda Country Profile. 2013. http://www.who.int/countries/uga/en/. Accessed 30 June 2015.
- 33.Loeber R, Farrington DP, Stouthamer-Loeber M, Van Kammen WB. Antisocial behavior and mental health problems: explanatory factors in childhood and adolescence. Mahwah: Lawrence Erlbaum; 1998.Google Scholar
- 34.Dishion TJ, French DC, Patterson GR. The development and ecology of antisocial behavior. In: Cicchetti D, Cohen DJ, editors. Developmental psychopathology, vol. 2: risk, disorder, and adaptation. Oxford: Wiley; 1995.Google Scholar
- 39.McKay M, Alicea S, Elwyn L, McClain Z, Parker G, Small L, Mellins C. Addressing the need for theory-driven programs capable of impacting urban children’s health, mental health, and prevention needs: CHAMP and CHAMP+, evidence-informed, family-based interventions to address HIV risk and care. J Child Adolesc Psychol. 2014;43:428–41.CrossRefGoogle Scholar
- 40.Bhana A, Mellins C, Petersen I, Alicea S, Myeza N, Holst H, Abrams E, Johns S, Chhagan M, Nestadt D, McKay M. The VUKA Family Program: piloting a family-based psychosocial intervention to promote health and mental health among HIV+ early adolescents in South Africa. AIDS Care. 2013;26:1–11.PubMedCrossRefGoogle Scholar
- 42.WHO: Mental Health Action Plan 2013–2020. 2013. http://apps.who.int/iris/bitstream/10665/89966/1/9789241506021_eng.pdf. Accessed 29 June 2015.
- 44.Masseli D, Lys JA, Schmid J. Improving impacts of research partnerships. Bern: Swiss Commission for Research Partnerships with Developing Countries (KFPE); 2005.Google Scholar
- 59.McKay MM, Gonzales J, Quintana E, Kim L, Abdul-Adil J. Multiple family groups: an alternative for reducing disruptive behavioral difficulties of urban children. Res Soc Work Pract. 1999;9(5):593–607.Google Scholar
- 60.McKay M, Jensen P, the CHAMP Collaborative Board. Collaborative child mental health services research: theoretical perspectives and practical guidelines. In: Hoagwood K, Jensen P, McKay M, Olin S, editors. Collaborative research to improve child mental health services. New York: Oxford University Press; 2010. p. 14–39.Google Scholar
- 61.Bhana A, McKay MM, Mellins C, Petersen I, Bell C. Family-based HIV prevention and intervention services for youth living in poverty-affected contexts: the CHAMP model of collaborative, evidence-informed programme development. J Int AIDS Soc. 2010;13(Suppl 2):S8.PubMedPubMedCentralCrossRefGoogle Scholar
- 73.Hoagwood KE, Burns BJ, Weisz JR. A profitable conjunction: from science to service in children’s mental health. In: Burns BJ, Hoagwood K, editors. Community treatment for youth: evidence-based interventions for youth with severe emotional and emotional disorders. Oxford: Oxford University Press; 2002. p. 327–38.CrossRefGoogle Scholar
- 80.McKay M, Paikoff R. Community collaborative partnerships: the foundation for HIV prevention research efforts in the United States and internationally. West Hazleton: Haworth Press; 2007.Google Scholar
- 82.Uganda AIDS Commission. HIV and AIDS Uganda country progress report. http://www.unaids.org/sites/default/files/country/documents/UGA_narrative_report_2014.pdf. Accessed 29 Mar 2015.
- 85.Ono M, Miller HP. Income nonresponses in the current population survey. In Proceedings of the Social Statistics Section, American Statistical Association. 1969. p. 277–288.Google Scholar
- 86.U.S. Bureau of the Census. Tech rep. Vol. 63. U.S. Government Printing Office; 2002. Technical paper.Google Scholar
- 89.Wolf EJ, Harrington KM, Clark SL, Miller MW. Sample size requirements for structural equation models: an evaluation of power, bias, and solution propriety. Educ Psychol Meas. 2013;76(6):913–34. https://doi.org/10.1177/0013164413495237.
- 90.Hox JJ. Multilevel regression and multilevel structural equation modeling. The Oxford handbook of quantitative methods in psychology: Vol. 2: Statistical analysis.:Oxford University Press, 2013–03-21. Oxford Handbooks Online. 2013–10-01. Date Accessed 4 June 2018.Google Scholar
- 91.Gwadz M, Rotheram-Borus MJ. Tracking high-risk adolescents longitudinally. AIDS Educ Prev. 1992;Suppl:69–82.Google Scholar
- 104.Creswell JW. Research design: qualitative, quantitative, and mixed methods approach. Thousand Oaks: SAGE; 2013.Google Scholar
- 107.NVivo qualitative data analysis software; QSR International Pty Ltd. Version 10. 2012.Google Scholar
- 110.Miles MB, Huberman AM. Qualitative data analysis: an expanded sourcebook. Thousand Oaks: SAGE; 1994.Google Scholar
- 111.Strauss A, Corbin J. Basics of qualitative research: techniques and procedures for developing grounded theory. 2nd ed. Thousand Oaks: SAGE; 1998.Google Scholar
- 112.Boyatzis R. Transforming qualitative information: thematic analysis and code development. London: SAGE; 1998.Google Scholar
- 113.Costello AB. Getting the most from your analysis. Pract Assess Res Eval. 2009;12:131–46.Google Scholar
- 116.Tabachnick BG, Fidell LS. Using multivariate statistics. 5th ed. New York: Allyn and Bacon; 2007.Google Scholar
- 118.Hooper D, Coughlan J, Mullen MR. Structural equation modelling: guidelines for determining model fit. Electron J Bus Res Methods. 2008;6(1):53–60.Google Scholar
- 120.Creswell JW, Klassen AC, Plano Clark VL, Smith KC. Best practices for mixed methods research in the health sciences. https://www2.jabsom.hawaii.edu/native/docs/tsudocs/Best_Practices_for_Mixed_Methods_Research_Aug2011.pdf. Accessed 30 July 2015.
- 123.Fabiano GA, Pelham WE Jr, Waschbusch DA, Gnagy EM, Lahey BB, Chronis AM, et al. A practical measure of impairment: psychometric properties of the Impairment Rating Scale in samples of children with attention deficit hyperactivity disorder and two school-based samples. J Clin Child Adolesc Psychol. 2006;35:369–85.PubMedCrossRefGoogle Scholar
- 125.Abidin RR. Parenting Stress Index-short form. Charlottesville: Pediatric Psychology Press; 1990.Google Scholar
- 126.Derogatis LR, Fitzpatrick M. The SCL-90-R, the Brief Symptom Inventory (BSI), and the BSI-18. In: Maruish ME, editor. The use of psychological testing for treatment planning and outcomes assessment: instruments for adults. New Jersey: Lawrence Erlbaum Associates Publishers; 2004. p. 1–41.Google Scholar
- 129.Kovacs M. Child Depression Inventory. North Tonawanda: Multi-Health Systems, Inc; 1992.Google Scholar
- 130.Fitts WH, Warren WL. Tennessee Self-Concept Scale: TSCS-2. Los Angeles: Western Psychological Services; 1996.Google Scholar
- 131.McKay M, Block M, Mellins C, Traube DE, Brackis-Cott E, Minott D, et al. Adapting a family-based HIV prevention program for HIV-infected preadolescents and their families: youth, families and health care providers coming together to address complex needs. Soc Work Ment Health. 2017;5:355–78.CrossRefGoogle Scholar
- 132.Luke DA, Calhoun A, Robichaux CB, Elliott MB, Moreland-Russell S. The program sustainability assessment tool: a new instrument for public health programs. Prev Chronic Dis. 2014;11:E12. Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.