Using Correlational Evidence to Select Youth for Prevention Programming

Abstract

In a period of increased accountability and reduced prevention resources, the effective targeting of those limited resources is critical. One way in which limited resources are focused is to identify and provide services to those most at risk for later substance use. Risk status, or propensity, is typically estimated from correlational evidence. Using meta-analytic techniques this paper examines the evidence that 29 of the 35 constructs specified by the CTC risk and protective factor model are related to alcohol, tobacco, or marijuana use. While these factors are generally demonstrated to be predictive of substance use, the strength of relation is modest. Ten factors show a significantly different strength of relation with tobacco than with alcohol and marijuana. Given the correlations observed and the rate of substance use in the population, providing only selective intervention services likely ignores the majority of those who will later use substances. Although selection improves the percentage of those receiving services who are likely to benefit from services, the evidence summarized in this study suggests selective interventions will omit many of those who will likely use substances. Given typical base and selection rates, smaller program effects on universal populations may keep a greater number of youth from becoming alcohol, tobacco, or marijuana involved. Editors’ Strategic Implications: The data make a strong and provocative argument for primary prevention of youth substance abuse that should be heard by policymakers and service providers involved in strategic planning and appropriate deployment of resources.

Appendix A: Meta-Analysis Procedures Applied in this Study

Criteria for Selecting Studies

• 1. The study used a panel design with at least two waves of measures on the same persons and contained sufficient information to allow a reasonable inference of the age of the sample at each time of measurement.

• 2. The study reported quantitative results for the relationship of at least one predictor variable with some concurrent or later (‘outcome’) measure of alcohol, tobacco, or marijuana use.

• 3. The study was conducted in a western, economically developed culture, and the findings were reported in English.

• 4. Studies evaluating the effectiveness of interventions were excluded. Also excluded were studies in which samples were selected on the basis of subjects’ status on the outcome behavior and then contrasted on various predictors, even when those antecedent predictors were prospectively measured.

Effect Sizes

The primary effect size index used for this meta-analysis was the product-moment correlation coefficient between a predictor variable and an outcome variable representing marijuana use by the same subject sample. Many studies reported the relationships of interest directly as correlation coefficients; others reported statistical results in a form that could be converted to correlation coefficients, most frequently as 2 × 2 cross-tabulations. It is often appropriate in meta-analysis to adjust some or all of the individual effect sizes in a distribution prior to analysis. These adjustments may transform effect sizes into more convenient forms, correct known biases, recode outliers, and adjust for various artifacts. The adjustments that applied to the effect sizes in the present meta-analysis were as follows.

Dichotomized Variables

Each relevant statistical relationship was first converted to a correlation coefficient if it was not already in that form. This was accomplished using standard formulas such as those found in Becker and Hedges (1989), Hunter and Schmidt (1990a), and Rosenthal (1991). In cases where correlation coefficients were derived from dichotomized interval or ratio data, the resulting phi coefficients are known to be attenuated (Hunter and Schmidt 1990a, b). To correct this downward bias, the 147 effect sizes that were coded from dichotomized predictor variables whose underlying distributions were believed to be normally distributed were statistically adjusted to estimate the correlation that would be obtained if the predictor had not been dichotomized. Given that the underlying distribution of marijuana use is not normally distributed, effect sizes were not adjusted for dichotomization of the outcome. The formula used to adjust the 147 effect sizes for predictor dichotomization was as follows:

$$ \hat{r} = \frac{r} {{\phi (z)}}{\sqrt {PQ} } $$

Where $\hat{r}$ = Adjusted correlation, r = Coded correlation with dichotomous predictor variable, P = Predictor variable baserate (marginal probability), Q = 1−P, ϕ(z) = y ordinate of the normal distribution at the point along z that corresponds to a cumulative P equal to the predictor baserate.

Data Reduction Procedures

To avoid statistical dependencies, multiple effect sizes coded for this synthesis were aggregated within each unique study-sample prior to averaging across study-samples. Achieving this reduction required several steps, beginning with an aggregation procedure that averaged each unique combination of predictor factor with age at time of predictor measurement, age at time of outcome measurement and study-sample. This reduced the dataset from 2,090 effect sizes to 931 sample size weighted aggregated effect sizes. Next, these 931 effect sizes were examined to determine if there were any remaining dependencies from studies contributing data from both the study population and from breakouts within that study population (e.g., separate estimates for males and females). When multiple estimates were identified, those mean estimates based on the most observed data were retained. Finally, we removed the dependencies caused by having multiple study-sample effect sizes from different periods of measurement. This final aggregation reduced the original 931 effect sizes to 535 independent effect sizes, with each study-sample contributing no more than a single aggregated effect size to each outcome by predictor relationship.

Z-transformation and Weighting

Following the advice of Becker and Hedges (1989) and Rosenthal (1994), the aggregated correlations were transformed prior to analysis using Fisher’s Z _r transformation. The Z-transformation takes the following form:

$$ Z_{r} = .5\log _{e} {\left[ {\frac{{1 + r}} {{1 - r}}} \right]} $$

Where; Z _r  = the Fisher’s Z _r transformed value, r = the correlation coefficient, log _e  = the natural logarithm.

The Z-transformed correlation can be converted back to a correlation via the inverse of the Z _r -transformation:

$$ r = \frac{{{\left( {e^{{2z}} - 1} \right)}}} {{{\left( {e^{{2z}} + 1} \right)}}} $$

Where; r = the correlation coefficient, Z _r  = the Z-transformed correlation, e = the base of the natural logarithm (approximately 2.71828).

All computations and analyses used the Z-transformed aggregated effect sizes and, additionally, weighted those effect sizes to reflect the greater reliability of effect sizes based on larger samples than those with smaller samples. However, the effect sizes were computed from samples ranging from 5 to 49,458 subjects. To keep studies with exceptionally large sample sizes from dominating the weighting procedure, sample size weights exceeding n = 699 were recoded to 700.

The sample size weights applied to the Z-transformed aggregated effect sizes were calculated simply as w _i  = (n _i −3), where w _i is the weight that is multiplied times each effect size and n _i is the sample size upon which that effect size is based (Becker and Hedges 1989; Hunter and Schmidt 1990b).