Background

Risk behaviour by young people, including substance misuse (alcohol, tobacco and illegal drugs), antisocial behaviour and crime has a substantial impact on the UK economy and the health of its population. These behaviours are associated with morbidity and mortality among young people, poor education, social exclusion, teenage pregnancy, conduct disorders and poor health over the life course [1–3]. The 2009/10 HBSC survey in Wales indicated that at age 15, 50% of girls and 47% of boys reported that they had been drunk twice or more (the third highest of 38 countries); 16% of girls and 11% of boys reported that they smoked cigarettes at least once a week; and 20% of girls and 22% of boys reported that they had ever used cannabis. Concerning drunkenness and cannabis use prevalence, Wales, Scotland and England all had rates above the HBSC average [4].

Intervention efforts have largely focussed on individual risk behaviours, ignoring the co-occurrence of multiple health risk behaviours [5]. In the 2012 survey of drug use, smoking and drinking among 11–15 year olds in England, 6% of respondents had smoked during the last week (rising to 15% by age 15) [6]. Past week smoking was strongly associated with other risk behaviours, and “[o]f the 6% of pupils who reported smoking in the last week, most (5% of all pupils) had also drunk alcohol or taken drugs recently, or had done both” (p185).

The survey also found that truanting from school was linked to these behaviours. For instance, “[p]upils who had truanted from school were more likely to be regular smokers compared with pupils who had never truanted (odds ratio = 2.02)” (p36).

Understanding the co-occurrence of multiple health risk behaviours and their association with poor educational outcomes, social disadvantage, conduct disorders and poor physical and emotional health is therefore an important area of current public health research [5, 7–10]. Similarly, there is increased interest in prevention efforts which focus on risk and protective factors such as pro-social behaviour, resilience and positive youth development [11–13], and which therefore have the potential to impact upon multiple risk outcomes. Family-based risk and protective factors have been identified as important targets for intervention [14], including the modelling of parents’ alcohol use [15], rules and monitoring around substance use [16, 17], and more general parenting styles and relationships within families [18–20]. Parenting and family interventions form a central part of many governments’ health, welfare and education policies, and prevention of substance misuse is also a key priority [21–25]. Yet interventions are often developed without reference to the existing evidence base or theoretical frameworks, and implemented without rigorous evaluation [26].

A promising programme which has targeted multiple risk behaviours by addressing risk and protective factors located in the family is the Strengthening Families Programme 10–14 (SFP 10–14). It draws on theories of bio-psychosocial vulnerability, resiliency and family process, and was initially developed as the 14-session Strengthening Families Programme for high-risk families with children aged 6–12 years whose parents were misusing alcohol or drugs. The revised SFP 10–14 is a seven week universal programme targeting young people aged 10 to 14 years and their parents [27]. SFP 10–14 has been reported to be effective over a six-year follow-up period in promoting family integration, delaying the onset of alcohol use, reducing uptake of smoking, the incidence of harder drug use (methamphetamine) [28–31] and substance use at 10 year follow-up (aged 21) [32]. Delaying the age of initiation of substance use is an important goal, since epidemiological research has indicated that later onset of alcohol and drug use is associated with reduced lifetime prevalence of alcohol and drug problems [33–35].

US-based cost-benefit assessments indicate that SFP 10–14 can be cost-effective at the population level for preventing alcohol misuse, other alcohol problems and also for tobacco use [36–38], cannabis and other drug use prevention. A potential ten-fold return for every dollar invested in the programme has been reported [36], although the authors conclude that economic analysis had largely been unexplored in the evaluation of such interventions. Another US study that examined the SFP 10–14 alongside other school-based programmes has shown that the costs of participation amounted to US$150 while the benefits realised could be as much as US$1000 per participant, depending on the extent of drug use [38]. The emphasis in these studies has been on the cost-benefit ratio and whether prevention can generate returns on investment from avoidance of other costs in other sectors and in future years. The results of such studies are highly dependent on the perspective taken and the nature and range of assumptions made. Most critically, they are dependent on the existence of high quality investigations that utilise reliable and valid outcome measures to produce indications of effect and that yield detailed and relevant cost information from a range of perspectives. Well conducted studies of this nature can contribute to subsequent meta-analyses and modelling exercises to identify efficient policy options.

The current evidence base for SFP 10–14 is derived exclusively from the US from two trials conducted by the same research team. Recent Cochrane systematic reviews and a 2006 technical report to the World Health Organisation (WHO) Expert Committee on Problems Related to Alcohol Consumption recommend that the SFP 10–14 is adapted and tested in other cultures and settings before widespread implementation [39, 40]. Recent methodological critiques of the two existing RCTs of SFP 10–14 point to limitations in the statistical analyses by the original research team, namely the use of multiple and one sided statistical tests leading to claims of statistical significance at p < 0.05 that may not be justified [41–45]. In these critiques the key issue is statistical precision, and an intention-to-treat re-analysis of the original data for a Cochrane review highlighted that the results of the SFP 10–14 trials showed an important effect with low precision [39]. For example, in this re-analysis the Absolute Risk Reduction and 95% confidence interval for drunkenness was 11.27% (0.31% to 22.24%). This underlines the need for additional, high quality trials to assess effectiveness and cost-effectiveness with precision and determine whether the programme has long-term public health benefits. This view is reinforced by a recent independent UK Department of Health-commissioned review of prevention and behaviour change in young people, which concluded that “More RCTs [are needed] of the Strengthening Families Program” [46].

It is not clear whether the apparent effectiveness of the SFP 10–14 in reducing alcohol and drug problems at the population level, as shown in research from the United States, will translate to other cultures, settings and countries. The SFP 10–14 has recently been culturally adapted for use in the UK [47, 48] and there is substantial UK policy interest in the programme. This includes the use of the SFP 10–14 as part of the UK Government’s Family Intervention Projects across a small number of selected sites in England, which target high risk families [49]. An evaluation of the programme’s implementation in Barnsley, England (which mainly used the original US programme materials), highlighted the need for some cultural adaptation (now completed) and consideration of the best approach with regards to universal or high-risk targeting [50, 51]. Many studies have highlighted the challenges associated with targeted programmes for young people, including recruitment, stigmatisation, and managing group dynamics, with the potential for harmful peer modelling effects [52–58]. Such targeting also risks losing the beneficial group effects which are achieved through the universal delivery of programmes such as SFP 10–14 to whole school populations, as in the US SFP10-14. In Cardiff, Wales, UK, these problems led to the development of an innovative approach of providing the SFP 10–14 as a universal intervention (available to any family via practitioner referral or family application), and running mixed groups of families from the ‘general population’ in combination with families with challenges (in the context of a group setting) [59], with the aim of promoting positive group dynamics. Groups of families attending the SFP therefore normally comprise around 70% from the general population, and 30% who may experience or present challenges in a group setting (families with challenges). It should be noted that in this ‘mixed families’ approach, such challenges relate specifically to the delivery and receipt of the group sessions, rather than general characteristics of participants’ support needs, family functioning or other risk factors. Examples of the challenges experienced or presented by families therefore include issues such as low literacy levels, behavioural problems (e.g. anger/aggression) and exclusion of children from school (and who may find working in a group setting challenging). An earlier paper describes the mixed families approach in more detail [59]. These mixed groups demonstrated evidence of high levels of participant retention and acceptability, and social effects outside the programme, with participating families supporting each other between sessions and after the end of the programme. Our trial of the SFP 10–14 UK in Wales (Project SFP Cymru) uses this ‘mixed families’ approach to group recruitment and composition, as well as the fully culturally adapted materials. The trial therefore builds on the completed formative evaluation, adaptation and exploratory evaluation of the UK version of the programme [47, 48, 50, 51, 59], and will generate important new evidence on the programme’s long-term effectiveness and cost-effectiveness in the UK.

Methods/design

Study design

The trial comprises a pragmatic cluster randomised controlled effectiveness trial with families as the unit of randomisation, with embedded process and economic evaluations.

Recruitment

SFP 10–14 UK will be implemented in seven geographical areas of Wales by local agencies that are independent of the research team (e.g. local authority parenting teams, children’s charities). Trial areas will run the programme on approximately 8 occasions over a two year period, with a maximum capacity of 10–12 intervention families per programme.

SFP 10–14 UK is designed for delivery to families with children aged 10–14, and in the Wales adaptation is targeted towards a mix of children and their families. Each programme is open to families from a loosely defined and fairly large geographical area. In each area, self-referrals to the Programme come forward in response to awareness-raising in community and educational settings. Other referrals come from agencies such as education, health and social services which identify families that may benefit from participating in the Programme.

Potential referrers and families are informed that the Programme is being run as part of a trial and are provided with brief information about the trial. When a family is referred or applies to the SFP 10–14 they will be visited by a member of the programme delivery team (normally the programme coordinator) who will undertake a needs and eligibility assessment. Based on the information contained in the family referral/application form and the needs and eligibility assessment the coordinator determines if eligible families are from the ‘general population’ or a family with challenges (in the context of a group setting). If families are deemed eligible to attend the programme they will be asked if they are willing for a member of the research team to have access to their referral notes and to visit them. Where the family agrees a research fieldworker will then visit them to obtain informed consent to participate in the trial, collect baseline data using a computer assisted personal interview (CAPI), and inform the family of the treatment group (C or I) to which they have been randomised. Figure 1 shows the anticipated flow of participants through the trial.

Figure 1
figure 1

Participant flow diagram.

Full size image

We will offer all parents and young people who provide data at 2 year follow-up a £10 gift voucher. All adults who take part in the 9 and/or 15 month interviews will be entered in to a prize draw with the chance to win a £50 family outing. Adults have a 1 in 20 chance of winning.

Participants

Inclusion and exclusion criteria

Table 1 describes the trial’s inclusion and exclusion criteria. Any families deemed eligible to attend SFP 10–14 UK will be included in the research trial, subject to their giving consent. Only families agreeing to participate in the trial and randomised to receive the intervention will have access to SFP 10–14 UK.

Table 1 Trial inclusion and exclusion criteria
Full size table

Ethical approval

Ethical approval for the trial was given by the Research Ethics Committee for Wales (reference 09/MRE09/53).

Informed consent

Prior to being visited by a research fieldworker, all families who are invited to participate in the research trial will be mailed an information sheet. This will include specific documents for parents/carers and young people. There will also be information for parents/carers about our request to involve their children in the trial. The fieldworker will go through this information with families and address any queries or concerns that they may have.

Informed consent will be obtained from all participants. For a family to be included in the research trial at least one parent/carer must consent to participate, and parent/carer consent for the inclusion of at least one young person in the trial is required. Young people will also need to give consent for their participation in the research trial. The fieldworker will be careful not to offer, or in any way imply access to the programme.

Randomisation

Families agreeing to participate in the research trial and who provide baseline data will be randomised using a computerised randomisation facility that will randomise families within strata defined by area (7) and use minimisation on the following variables:

  • the category to which the family have been assigned (general population/family with challenges);

  • age of children wishing to attend the programme (<12/12+). A mean will be taken if there is more than one child; and

  • the number of children in the family wishing to attend the programme (1/>1).

There will be complete concealment of the randomisation sequence, set up securely within the randomisation database by the statistician, from the field recruitment staff and other trial team members. A random element, set at 80%, will also be used to increase the integrity and reduce the deterministic nature of the randomisation process.

Intervention

Participating families will be randomised to one of two treatment groups. Either they will receive usual care with full access to existing services and a minimal information leaflet (control group-C), or they will receive the SFP 10–14 UK programme in addition to usual care with full access to existing services (intervention group – I). There will not be a defined programme of usual care, and the existing variation in services available to participants across the participating areas will continue throughout the trial period.

SFP 10–14 is open to families from a loosely defined and fairly large geographical area. The ‘mixed families’ approach aims to recruit about 4 families with challenges per group, and 6–8 families from the ‘general population’. Self-referrals (applications by families) come forward from awareness-raising in community and educational settings. Families are therefore drawn from the same area facilitating mutual support between sessions and at the end of the programme, but are not normally from a small or defined community such as a school or local neighbourhood. The programme comprises seven weekly sessions of two hours and is delivered in a range of community venues by a multi-agency team of trained professionals. In each session there is an hour during which parent sessions and young people sessions are conducted separately, followed by a second combined family hour. Typically the first hour focuses on skills (e.g. peer resistance for the young people, parenting for the parents), with the second hour designed to enable parents and young people to focus on communication skills, recognise family strengths, and practice skills covered in the first hour [27].

Objectives

The trial’s primary objective is to assess the effectiveness of the SFP 10–14 UK in preventing alcohol misuse in young people. Its secondary objectives are to assess the programme’s impact on drug misuse, smoking behaviour, alcohol initiation and drink-related problems and school performance, among young people. Tertiary objectives are to measure the extent to which SFP 10–14 UK has effects on mental health and wellbeing, and protective factors for alcohol and tobacco use/misuse located in the family (e.g. family functioning, parenting and young people’s peer resistance skills). The trial will measure the costs associated with programme delivery and whether it is cost effective. It will assess if there are important variations in delivery and receipt across trial sites, and identify key programme theory, content and processes.

Outcome measures

The trial has two primary outcomes - the number of occasions that young people report having drunk alcohol in the last 30 days; and drunkenness during the last 30 days, dichotomised as ‘never’ and ‘1-2 times or more’. Secondary outcomes (mainly concerned with long-term alcohol/tobacco/substance behaviours) are: reported use of cannabis (ever vs. never); weekly smoking (yes vs no, validated by salivary cotinine measures); age of alcohol use initiation; frequency of drinking more than 5 drinks in a row in the last 30 days; frequency of different types of alcoholic drinks; alcohol-related problems; and General Certificate of Secondary Education (GCSE) performance at age 15/16 (number of GCSEs passed and grades achieved, measured as a continuous outcome). All primary and secondary outcomes are collected from children at 2 year follow-up, with the exception of GCSE results, which we propose to collect via the Secure Anonymised Information Linkage (SAIL) Databank [60], once all participants have completed GCSEs/left compulsory education. The trial’s tertiary outcomes (mainly concerned with health and family wellbeing, and also substance use initiation) are: parenting (General Child Management Scale child report); family functioning (Family Relationship Index); children’s wellbeing and stress (Strengths and Difficulties Questionnaire scores); children’s health status (SF-36); parents/carers’ health status (GHQ and EQ5D); indicators of relative cost-effectiveness (derived from the health data); children’s smoking status (i.e. whether they have ever smoked/smoke now); young people’s self-efficacy; age of first cigarette; and age of first drug use. Tables 2, 3, 4, 5, and 6 provide a description of the trial’s primary, secondary and tertiary outcomes, and the specific measures used in each of the data collections. Selection of outcomes at 9, 15 and 24 months was informed by a literature review undertaken as part of the process evaluation.

Table 2 Measures collected at baseline from young people
Full size table
Table 3 Measures collected at baseline from parents/carers
Full size table
Table 4 Measures collected at 9 and 15 month follow-up from parents/carers
Full size table
Table 5 Measures collected at 24 month follow-up from young people
Full size table
Table 6 Measures collected at 24 month follow-up from parents/carers
Full size table

Assessments and data collection

Baseline

Computer assisted personal interviews (CAPIs) are conducted by research fieldworkers with parents/carers and children. Children’s baseline CAPIs cover: substance use behaviours, and aggressive and destructive behaviour; strengths and difficulties (using the Strengths and Difficulties Questionnaire); health, and sleeping patterns. Five sets of questions assess aspects of participants’ home life, namely: possessions the participants have in their bedroom; family activities and free time activities; family functioning; and parenting/child management. Full details are provided in Table 2.

Parent/carer interviews cover: drug and tobacco use, and parental reports of children’s substance use; parent–child bonding; parenting and family relationships and activities: and parents/carers’ health status. Participants also complete the parent version of the Strengths and Difficulties Questionnaire (SDQ). Table 3 provides a full list of questions and measures included in the interviews.

Follow-up

9 and 15 months

Parents/carers are followed up at 9, and 15 months past baseline, using computer aided telephone interviews (CATIs) conducted by the Participant Resource Centre at Cardiff University. Interviews at 9 and 15 months (Table 4) are identical, and collect data on hypothesised short term and intermediate outcomes. Baseline measures of parent–child bonding and parenting/child management are repeated. Parental expectations of their child(ren) and opportunities for the latter to participate in household tasks are assessed. Health, criminal justice and social care services utilisation is captured.

24 months

At two years past baseline CAPIs are conducted by research fieldworkers with parents/carers and children (Tables 5 and 6). Interviews with parents/carers cover participants’ substance use, health status and health and social care service utilisation. Young people’s interviews repeat baseline measures of smoking, alcohol and drug use. Additional questions examine age of first cigarette, age of first drunkenness, last month drinking frequency (including types of drinks), binge drinking and drunkenness frequency, cannabis use, and alcohol-related problems. The interview also assesses child reports of short term/intermediate outcomes, opportunities for involvement in pro-social activities in participants’ families, and attachment to parents, parenting/child management, and parental involvement in school work/activities. The extent to which young people help around the home is captured using suitably adapted versions of the questions asked to parents at 9/15 months. Other questions assess self-efficacy, school bonding, and interaction with anti-social peers. The child version of the SDQ is also administered. Measures of participants’ health used at baseline are repeated. Saliva samples are collected from consenting young people to assess inhalation of tobacco smoke in the previous few days, using a cotton wool swab of a salivette [88].

Sample size

We will aim to recruit 378 families in each arm of the trial (756 families in total). This will provide us with at least 80% power at the 2.5% level (halved to account for our two primary outcomes) to detect either a 12% difference in young people reporting having drunk alcohol (assuming a control group prevalence of 48% [4]), or a 10% difference in young people reporting having been drunk (assuming a prevalence of 22% [4]), at two year follow-up. This sample size is based on an average family in the trial having 1.25 young people, an intracluster correlation coefficient (ICC) of 0.2 and is adjusted to allow for 25% loss to follow-up (including withdrawals from the trial by participants prior to 24 month follow-up).

Analysis

Main analysis

The primary analyses will be based on the intention-to-treat principle and will involve fitting two-level logistic regression models to our primary outcomes (reported drinking at two year follow-up and reported drunkenness at two-year follow-up), with responses from young people nested within families. The covariates in the models include those balanced at randomisation (type of family – general population/family with challenges, and average age of young people) and baseline levels of drinking and drunkenness (depending on the outcome). Trial area (used as a stratification variable at randomisation) will be adjusted for. For each primary outcome, a statistically significant result will be concluded if the p-value for the trial arm explanatory variable is <0.025. While our power calculation is based on the above, a combined test will be performed if both p-values lie between 0.025 and 0.05. This will take the correlation of the outcomes into account and provide an overall p-value for the two outcomes.

The primary analyses will also be repeated

  • controlling for any baseline imbalances;

  • without controlling for any of the above mentioned covariates;

  • respecting the original ordinal scale (the primary questions are asked on an ordinal scale ranging from 0 occasions to 40+ occasions) using an ordinal regression model; and

  • using a complier average causal effect (CACE) approach, to investigate the treatment effect in the treatment adherent.

Secondary analysis includes investigating the effect of the intervention on substance use (reported use of tobacco and cannabis) at two year follow-up by fitting two-level logistic regression models, controlling for baseline reports of substance use, as well as the variables balanced at randomisation (all models will control for the variables that were balanced on at randomisation). The effect of the intervention on the proportion of young people reporting alcohol-related problems will be analysed similarly. The effect of the intervention on substance initiation (alcohol, tobacco and cannabis) will be investigated by fitting two-level Cox proportional hazards models (young people nested within families) for those participants who had not already initiated at baseline.

Educational achievement at 16 will be compared between trial arms by fitting a two-level logistic regression model (young people nested within families) comparing the proportion of young people who obtain five or more GCSEs at a grade C or above (GCSE Grades use an A*-G scale, with A* being the highest score). The proportion of young people who continue in education beyond 16 will be similarly compared between trial arms by fitting a logistic regression model. This analysis will be performed on the whole sample following the date that all participants (young people) should have completed school Year 11 when GCSEs are normally awarded. Other secondary and tertiary outcomes (e.g. parenting and child management, parent–child affective behaviour and wellbeing and stress) will be analysed by fitting similar regression models, controlling for clustering of responses within families (as appropriate) and controlling for variables that were balanced on at randomisation.

Sub-group & interim analysis

There is no planned interim analysis. Exploratory sub-group analyses are planned for:

  • age of young person at baseline;

  • gender of young person;

  • smoking behaviour of young person at baseline;

  • parental drinking behaviour at baseline;

  • socio-economic status (using both the Family Affluence Scale and occupational status, which will be analysed separately);

  • family status (general population/with challenges); and

  • trial area.

Up to two subgroup analyses will be performed to test hypotheses generated from the process evaluation. These will be specified in a later version of the Statistical Analysis Plan (SAP) before any trial analysis takes place, and will be generated by researchers with no access to the trial outcome data. Subgroup analyses will also be performed to assess differential treatment effects for those who may be at high risk of substance initiation and misuse (based on baseline scores for SDQ, the Family Relationship Index and the General Child Management Scale).

Process evaluation

An embedded process evaluation is examining how the programme is implemented and will facilitate interpretation of outcome effects [89]. In line with MRC’s guidelines [90], this component of the trial will enable the development and refinement of a programme logic model of the SFP10-14 UK by the research team, including its key processes, impacts and outcomes. The process evaluation has two key research questions: (1) How is the SFP 10–14 UK thought to influence social and individual behaviour of family members so that young people are less likely to use tobacco, misuse alcohol or to demonstrate other kinds of antisocial behaviour?; and (2) What are the best ways of implementing SFP 10–14 UK, and is there important variation in delivery and receipt? Following the framework proposed by Linnan and Steckler [91], the process evaluation will examine four key issues: trial implementation and context; trial arm fidelity; participation, reach and dose received/delivered; and reception and responsiveness. The process evaluation has the following aims, namely to:

  • identify key programme content and processes;

  • assess trial arm implementation and context;

  • evaluate fidelity and completeness of programme delivery;

  • assess participation and reach;

  • calculate the extent of families’ attendance at SFP 10–14 UK sessions; and

  • evaluate reception and response by families.

Multiple sources of evidence will be used to answer the research questions. Data collection will comprise:

  • interviews with Welsh Government staff involved in programme commissioning (n = 1) and SFP training staff (n = 2), programme coordinators (n = 6), and coordinator managers (n = 6). Apart from coordinator managers, all these participants will be asked to take part in two interviews – once during the early period of programme delivery, and again towards the end;

  • six focus groups (one in each delivery site) with staff involved in programme delivery (n = 48-60);

  • completion of reflection sheets by facilitators after each session (n ≤ 336) to assess fidelity of delivery;

  • researcher observation of programme sessions (three in each area, spread across the evaluation period). Two facilitators’ meetings (used to review and plan programme delivery) will be observed in each area; and

  • 8–10 intervention group families will be invited to take part in focus groups in each area. Separate groups will be conducted with young people and parents/carers from the same families. Parents/carers from 8–10 control group families will also be invited to take part in a focus group in each area; and

  • collection of routine data on programme recruitment, staffing and retention.

Table 7 shows how each of the data collection methods maps on to the aims and objectives.

Table 7 Process evaluation aims, objectives and methods
Full size table

Process evaluation analysis

Qualitative data from the process evaluation will be subjected to a thematic content analysis [92]. Key themes will be developed into an analytical framework. Each interview transcript will be entered into Atlas.ti 6, which will be used as a data management tool, permitting data coded to the same theme to be accessed quickly for further analysis. Each transcript will be coded to indicate the programme delivery location and type of participant, allowing analytical themes to be explored in relation to different groups’ experiences and to compare implementation across the seven areas and trial arms. Quantitative data on fidelity, participation and dose will be used in secondary analysis of outcome effects.

Health economic evaluation

Aim

The aim of the economic evaluation is to assess the costs and effects associated with SFP 10–14 UK from the perspective of the UK Treasury - encompassing health and social services, education and criminal justice, and participating families, in a cost-consequences analysis. The cost-consequences analysis provides a comparative analysis of the alternative programmes available to participants (usual service provision only versus SFP 10–14 UK plus access to usual service provision) and a clear descriptive summary of the costs of SFP 10–14 UK and the consequences (outcomes for participants, service utilization, and utility gains). Effects will be reported separately enabling a full evaluation of the different outcome components (main trial outcomes, service utilisation, and utility gains) and related to the costs for both of the trial arms. Further, distinction will be made between the costs incurred in the delivery of SFP 10–14 UK and outcomes generated by it in each trial area (programme delivery site), so as to assess variation and potential for efficiency gains, if ‘lowest costs’ and ‘best outcomes’ were achieved across the board.

Methods

Costs will be categorised according to whether they are research or programme related, with discussions held with relevant staff to agree the attribution factors used to determine costs of SFP 10–14 UK. Further, the agencies that incur costs will be clearly specified, as will the agencies that benefit from reductions in resource utilisation, so as to enable inter-sectoral comparisons to be undertaken.

Based on service utilisation data collected from adult trial participants at 9, 15 and 24 month follow-up, all inputs and services provided by agencies will be documented in descriptive terms and wherever possible translated into monetary terms using appropriate published unit cost data (e.g. BNF [93], PSSRU [94], NHS Reference Costs [95]). A series of cost modules will be developed, to establish a profile of costs relating to the agency that incurred them. Overall costs of providing SFP 10–14 will be computed and will include programme set up costs, and costs of implementing and delivering the programme. The cost per participant and cost per family of attending SFP 10–14 UK will be computed for the trial and for each programme area. The costs of setting up SFP 10–14 UK will include promotional materials and resources involved in participant recruitment, in particular the number of hours spent promoting/raising awareness of the programme.

The costs of implementing and delivering SFP 10–14 UK will include staff time, venue and equipment costs, provision of support facilities, materials utilised etc. Participation in SFP 10–14 UK is likely to result in changes in the utilisation of other services provided by a range of agencies. These will be captured within the trial so as to identify the extent of costs offset as a result of the programme, and which represent one aspect of the outcomes generated by the programme. This will be done by collecting data on service utilisation from participants (and changes over time) at 9, 15 and 24 month follow-up. The combination of changes in service utilisation over time and costs of SFP 10–14 UK delivery will result in a net cost per participating family, which will be used in the assessment of the relative cost-effectiveness of the programme. The extent to which these changes in service utilisation are likely to be sustained over time will be modelled and quantified to produce scenarios for assessing the longer term efficiency of the programme.

Information regarding the number of participants requiring childcare support and transport to attend SFP 10–14 UK, and details of the job title and employer of programme staff who deliver each programme will be accessed from a monthly data return proforma submitted by each programme delivery team. Most data on the cost of delivering SFP 10–14 will be accessed via financial monitoring forms, submitted to the research team by the agencies with responsibility for leading programme delivery in each trial site. These forms capture all costs incurred by programme delivery teams, including those relating to specific SFP programmes (e.g. venue hire, refreshments, payment of participant travel expenses, provision of childcare for younger siblings taking part in the programme). As most programme facilitators are employed by a network of partner agencies (not the agency with primary responsibility for implementation), the associated staffing costs may not be available. If actual salary costs of facilitators cannot be obtained they will be derived from the hours worked documented on the Financial Monitoring Form and matched to the job title and employer information gathered via the Monthly Data Form. The salary associated with professional grade of each facilitator will be derived from relevant PSSRU unit cost data and multiplied by the hours worked to derive an estimate of staffing costs for facilitators. Where members of the research trial team organise or contribute to promotional events (e.g. information stands at school parents’ evenings) the costs incurred will be captured using a ‘Record of costs: promotional events' proforma. The provision of services other than SFP 10–14 UK to trial participants will be measured as part of 9, 15 and 24 month follow-up interviews with parents/carers. Table 8 provides a summary of cost categories and data sources and measures.

Table 8 Summary of cost categories and data sources and measures
Full size table

Health economic analysis

Effects

The differences in primary, secondary and tertiary outcomes will represent the consequences of SFP 10–14 UK programme delivery to be used in conducting the cost consequences analysis. As part of this process the responses to EQ5D, collected from adults at baseline and 24 month follow-up, will be used to generate QALYs gained as a result of SFP 10–14 UK and compute the cost/QALY ratio.

Cost-consequences analysis

The cost consequences will be assessed once all necessary data has been collected, and post-trial modelling will be employed to assess the cost consequences over longer time horizons than is possible within the trial period. Changes in resources utilised over time will be calculated and used in conjunction with the costs of setting up and delivering SFP 10–14 UK to generate the overall cost of programme delivery per family, which will represent the incremental cost of providing the programme relative to usual service provision. The differences in primary, secondary and tertiary outcomes (including differences in utility scores derived from the EQ5D responses at each follow-up) will be used alongside the net cost of programme delivery in the cost consequences analysis to generate a set of indicators of relative cost-effectiveness within the trial period, based on incremental cost and incremental outcomes. Missing data will be dealt with by employing an appropriate imputation-based method for effectiveness and quality of life data [96], while the usual method for dealing with censored data relating to costs will be to employ the weighted cost method with known cost histories [97].

Sensitivity analysis

A series of one-way sensitivity analyses will be undertaken to assess the extent to which changes in the variables affect the baseline estimates. The variables will be adjusted in line with emerging distributions of data values, and impact on baseline estimates computed. A threshold analysis will also be undertaken to determine the degree of parameter variation required to alter conclusions derived from baseline findings. A series of scenarios will be developed and utilised within an economic model to assess the relative efficiency of SFP 10–14 UK over longer time horizons. Probabilistic sensitivity analysis will be undertaken to produce cost-effectiveness acceptability curves to assess the probability that SFP 10–14 UK represents value for money. Future costs and benefits will be discounted at the prevailing rate (currently 3.5% pa) to bring into present values. Costs and benefits will be discounted at 0% and 6% in the sensitivity analysis.

Modelling

The evaluation will also include a post-trial modelling phase, using a decision-analytic model, which will reflect longer time horizons than those available from within the trial period. The model will be populated with relevant information from within the trial period, but will also create a series of scenarios that reflect longer term costs and outcomes, based on parameter variation and discussion with experts.

Public involvement

The MRC guidance on complex interventions recommends that “appropriate users should be involved at all stages of the development, process and outcome analysis of a complex intervention, as this is likely to result in better, more relevant science and a higher chance of producing implementable data” (p59) [90]. The involvement of the public has also been advocated in research policies to ensure that research is relevant, reliable and understandable [98, 99]. Project SFP Cymru sits within the DECIPHer research centre which employs an Involving Young People Research Officer (IYPRO). This officer will be responsible for advising how public involvement is conducted throughout the trial.

All trial participants will be invited at baseline interview to be part of stakeholder groups. Parents and young people who consent will be contacted about the stakeholder events in their area. The frequency and content of the stakeholder events will vary dependent on the stage of the research and the need for lay input. It is envisioned that lay members will be consulted on trial materials such as questionnaires, the project website and newsletter, and identifying approaches to disseminating the findings of the trial to a lay audience, although flexibility will be built into the stakeholder events so that individuals can raise other issues they think are important for the research. The use of both top-down and bottom-up mechanisms of public involvement is designed to create a meaningful involvement process for all research participants. As recommended by INVOLVE, a national advisory organisation that aims to increase public involvement in NHS, public health and social care research, all lay members’ expenses will be covered, including travel and childcare costs, and vouchers will be given to compensate for time contributed [100]. We will also establish a stakeholders group to which all SFP coordinators will be invited, in order to promote effective communication between the research team and local practitioners.

Discussion

Substance misuse is an important public health challenge and is associated with a range of short-term and long term harms. Positive family relationships and aspects of parenting (such as consistency, emotional warmth and monitoring) have been identified as key protective factors against young people’s misuse of alcohol, drugs and tobacco [101]. Prevention interventions which promote these protective factors therefore have the potential to make a significant impact on the health and wellbeing of young people. Such potential impacts cannot be assumed, and social interventions, like their clinical counterparts, may be ineffective in achieving their long term goals, or have unintended or sometimes harmful effects [54, 102, 103]. Prevention interventions therefore need to be theoretically grounded in their development, and subjected to rigorous evaluation before decisions about widespread implementation are made. This trial will assess the long-term effectiveness of the recently adapted UK version of the Strengthening Families Programme 10–14, including its impact on rates of alcohol consumption and drunkenness in young people aged 12–16 in a UK context. It seeks to understand how and why any identified outcomes occur, and to inform the future development of the programme in the UK.

Whilst parenting and family interventions are a key focus of UK social and public health policy, the majority of the interventions in this area have been developed in the United States [104–106]. It cannot be assumed that the effectiveness of an intervention in the country it is developed in will be replicated in new settings, or that programme logic models will operate in identical ways across different cultural contexts. Prevention interventions which are transported to new countries may require adaptation (which needs to balance fidelity with cultural relevance), and should be subject to rigorous evaluation to assess their effectiveness, and how key programme change processes function in new settings [107]. The results from this trial will help inform future decisions about the implementation of SFP 10-14 in the UK, and will demonstrate if the effectiveness of the programme in the US is replicated in a UK context using culturally adapted materials. It therefore contributes to parallel research endeavours on SFP 10-14 in a number of other countries [108–110].

As well as assessing the impact of the culturally adapted SFP 10-14 in the UK on long-term outcomes, the current trial is evaluating the implementation model developed in and being employed in Wales. A core aspect of this model is the strategy of retaining the SFP 10–14 as a universal prevention intervention (accessible to any family in a local community), and of forming programme groups to comprise a mix of families from the general population, and those who may experience/present challenges in a group setting. This model seeks to avoid some of the harmful effects of targeted interventions, including anti-social peer contagion [58]. Relatively few previous studies of parenting/family-based prevention programmes have examined strategies to enhance group dynamics, implementation fidelity or family engagement through group composition strategies [59]. Our process evaluation will allow us to understand whether the programme was delivered as intended (including the composition of SFP groups), what factors influenced this, and relationships between quality/adherence of programme delivery and outcomes (at a trial, area and family level). By embedding process and economic evaluations within the main trial we therefore aim to contribute to current knowledge about the value of universal prevention interventions, and the challenges of implementing family-based programmes.

In this trial we also aim to address some of the key methodological critiques of previous randomised controlled trials of the SFP10-14, particularly those concerning the use of multiple statistical tests, and the absence of a pre-specified primary outcome [41–43]. A key challenge when evaluating complex interventions such as SFP 10-14 is that they are designed to impact on multiple long-term health behaviours through enhancing manifold protective factors (parenting style, family communication processes, young people’s life skills), making the traditional trial design of a single primary outcome and small number of secondary outcomes inappropriate [90]. Thus we have specified two primary outcomes (related to drinking frequency and drunkenness frequency), alongside secondary and tertiary outcomes, and this is a strength of the trial design. The outcome measures map onto, and are designed to test a logic model for the programme developed by the research team, which has also driven the selection of short-term and intermediate outcomes. In this way the trial is assessing both the overall effectiveness of the SFP10-14, and its predicted causal pathways.

In order to maximise the external validity and utility of our findings we have adopted a pragmatic effectiveness trial design [111, 112], in which recruitment procedures and delivery systems replicate ‘real world’ implementation. Thus in our trial of SFP 10–14 UK the programme is provided by local agencies that are likely to implement it in the future, and the research team does not seek to impose inclusion criteria with the aim of reducing the heterogeneity of the trial population (as might happen in an explanatory trial which attempts to recruit homogenous groups of participants). Participating families therefore comprise children from across the 10–14 age band which the programme is aimed at (12–16 at follow-up), and it is possible that the SFP10-14’s potential impact on health behaviours such as alcohol could be patterned according to age.

The trial team will need to obtain support from local agencies delivering SFP 10–14 for the trial, many of whom may have little or no previous experience of running parenting/family/substance misuse prevention programmes as part of an RCT. Practitioners (and others) may object to random allocation on the grounds that they believe that the intervention is likely to be effective [113], and that participants have unmet needs which SFP 10–14 UK addresses. Whilst central to many practitioners’ concerns may be the sense that participants randomised to the control condition are ‘given nothing’ (what Wong, et al. describe as ‘resentful demoralisation’ in the context of no treatment control groups) [114], in pragmatic trials such as ours, both intervention and control group families continue to receive ‘care as usual’ (i.e. whatever services or programmes are usually available), though usual care/services may differ across trial sites. One potential response of practitioners to allocation of participants to a control group is to offer a compensatory package of treatment or support [115], which can undermine the equivalence of the two arms of a trial. It will also be important for us to avoid contamination of the control group, and therefore ensure that families allocated to the control group are not offered SFP 10–14 UK prior to their 24 month follow-up interviews. We aim to build strong relationships with agencies involved in referring families to, and delivering SFP 10–14 through embedding research staff within these agencies, organising presentations to explain the rationale for the RCT design (and the need to maintain intervention and control groups and equivalence of access to usual care), and engaging the support of influential individuals (such as senior managers and team leaders). We have also sought to build strong partnerships with key policy makers, with the aim of generating high level support for the trial, and also increasing the likelihood that the results can be translated into policy and practice. In common with other comparable trials [116–118], we are likely to experience challenges with recruitment and retention of families. We plan to use a number of strategies to retain young people and parents/carers in the trial, including regular communication with participants, using agencies and schools to reach families who may have moved house after baseline data collection, forming groups of trial participants to advise us on the acceptability and likely impact of potential approaches, and offering incentives (e.g. vouchers, prize draws) at follow-up.

In conclusion, this trial aims to evaluate a promising family-based substance misuse prevention programme following its adaptation for the UK. The trial is designed to assess the effectiveness of SFP 10-14 UK, the extent to which it is delivered as intended (and key influences), and the costs and consequences of the programme. We aim to examine the relationship between trial outcomes and fidelity of implementation, and the value of the group composition strategies being used by SFP 10–14 UK practitioners in Wales.