Appendix
Measuring effect sizes
In order to carry out a meta-analysis, there has to be a comparable measure of effect size in every evaluation. Welsh and Farrington (2002) found that the only comparable data that were reported in every CCTV evaluation was the number of crimes in time periods before and after the implementation of CCTV schemes. As far as possible, they analyzed the numbers of crimes in 12-month periods before and after:
|
|
Before
|
After
|
|
Experimental
|
a
|
b
|
|
Control
|
c
|
d
|
They used the odds ratio (OR) as the measure of effect size:
$$ OR = {{\left( {a*d} \right)}} \mathord{\left/ {\vphantom {{{\left( {a*d} \right)}} {{\left( {b*c} \right)}}}} \right. \kern-\nulldelimiterspace} {{\left( {b*c} \right)}} $$
This measure is intuitively meaningful because it indicates the relative change in crimes in the control area compared with the experimental area. In this article, it is termed the “relative effect size” (RES).Footnote 7 RES = 2 indicates that d/c (control after/control before) is twice as great as b/a (experimental after/experimental before). This value could be obtained, for example, if crimes doubled in the control area and stayed constant in the experimental area, or if crimes decreased by half in the experimental area and stayed constant in the control area, or in numerous other ways.
The variance of the OR is usually calculated from its natural logarithm LOR:
$$ VAR{\left( {LOR} \right)} = 1 \mathord{\left/ {\vphantom {1 a}} \right. \kern-\nulldelimiterspace} a + 1 \mathord{\left/ {\vphantom {1 b}} \right. \kern-\nulldelimiterspace} b + 1 \mathord{\left/ {\vphantom {1 c}} \right. \kern-\nulldelimiterspace} c + 1 \mathord{\left/ {\vphantom {1 d}} \right. \kern-\nulldelimiterspace} d $$
In this article, we use LRES, the natural logarithm of RES, and refer to VAR (LRES). This calculation of VAR (LRES) is based on the assumption that crimes occur at random, according to a Poisson process. This assumption is plausible because 30 years of mathematical models of criminal careers have been dominated by the assumption that the commission of crimes can be accurately modeled by a Poisson process (Blumstein et al., 1986; Piquero et al., 2003). In a Poisson process, the variance of the number of crimes is the same as the number of crimes. Unfortunately, the distribution from one year to the next of the number of crimes in small areas is not known.
The problem is how to distinguish year-to-year changes in crime that are caused by CCTV from changes caused by numerous other influencing factors. In comparisons of only one small experimental area with one small control area, it is likely that other interventions (e.g., new policing strategies) and other influencing factors (e.g., the release of a prolific offender from prison) will impinge differentially on the experimental and control areas. These other factors will increase the variance of the number of crimes in an area and will make it more difficult to detect any effect of CCTV.
Our estimate of VAR (LRES) was based on the variance of the monthly number of crimes. The yearly variance was estimated as 12-times the monthly variance. If the yearly variances are denoted by Va, Vb, Vc and Vd:
$$VAR{\left( {LRES} \right)} = {\left[ {\frac{{Va}}{{a^{2} }} + \frac{{Vb}}{{b^{2} }} + \frac{{Vc}}{{c^{2} }} + \frac{{Vd}}{{d^{2} }}} \right]}$$
In a Poisson process, Va = a (etc.), and this equation reduces to the usual one above. This estimate of VAR (LRES) is too high, because, unlike yearly figures, monthly numbers of crimes are influenced by seasonal factors. Hence, using this estimate means that the significance tests are conservative.
Excluding Hawkeye (where the number of crimes is based on 57 car parks), 70 sets of monthly crime figures were available for before and after periods in target, control and buffer areas and Divisions.Footnote 8 For each area in each year, the total number of crimes N was compared with V/N, where V is the estimated variance of the number of crimes (based on monthly numbers). In a Poisson process, V/N = 1.
It was clear that V/N increased with the total number of crimes. The correlation between V/N and N was 0.77 (P <0 .0001). A linear regression analysis showed that \( V \mathord{\left/ {\vphantom {V N}} \right. \kern-\nulldelimiterspace} N = .0008*N + 1.2 \)
This equation suggests that Poisson (random) variation is more important at small values of N, but that influencing factors become more important as N increases. For values of N up to 1000, V/N was between 1 and 2. Given the median value of N of 760, it might be expected that the median value of V/N would be 1.8, but in fact it was 2.3. The average value of V/N was 1.45 for the lowest ten numbers of crimes (average 105 crimes), 1.58 for the next ten (average 297), 1.97 for the next ten (average 485), 2.86 for the next ten (average 769), 4.56 for the next ten (average 2,650), 5.76 for the next ten (average 7,295) and 19.20 for the highest ten (average 24,327). Therefore, the Poisson assumption is most applicable for comparisons of small areas (target vs control or buffer), not for comparisons of target areas with Divisions.