Sample survey
A 25-item (including branching questions) questionnaire (see online Appendix B) was developed for residents who either received the intervention, or would have done so if they had lived in one of the treatment areas (these acted as the control group). The questionnaire included items concerning their recent victimisation experience; their satisfaction with, and likelihood of, reporting crime to the police; whether they had been visited by the police following their most recent burglary (if relevant); and whether they had been provided with a crime prevention “kit” or given crime prevention advice.
Prior to conducting the survey, statistical power calculations were conducted to determine how many surveys would need to be returned to enable us to detect a given difference between the treatment and control groups for the analysis of a contingency table. Given that the expected distributions were unknown (there was no prior expectation about how people might respond to the questions across the two groups), we examined what size of sample would be needed to detect a 0.1 difference (a 10% difference in opinion) for a 2 × 2 contingency table. This indicated that a sample size of 400 would be sufficient.
Prior research (Shih and Fan 2008) suggests that the typical response rate for postal surveys is about 45% (range 10–89%). However, the Shih and Fan (2008) study examined response rates for published studies, and those for unpublished studies may be lower. Further, it might be the case that response rates differ across treatment and control groups. Consequently, we assumed a fairly conservative response rate of 20%. On this basis, we estimated that surveys would have to be distributed to a total of 2000 respondents to generate the necessary sample. To encourage response rates, in accordance with advice given by Edwards et al. (2002), we offered an incentive to respondents in the form of participation in a raffle (with an iPod as a prize), minimised the length of the questionnaire, and indicated on the postal envelope that the survey was being conducted by a University.
Procedure
Survey questions were designed by a team of three academics and were loosely based on questions from the Crime Survey of England and Wales (see, e.g., Jansson 2007 for a review). A total of 2016 surveys, each with an accompanying covering letter and a return (post-paid) envelope, were posted to a sample of homes, consisting of burglary victims and their neighbours, selected randomly from the available sampling frame. A total of 672 surveys were sent to households that had been burgled in the experimental and control areas (336 in each type of area). A further 1344 surveys were sent to the neighbours of burglary victims that did, or in the case of the control group, would have received either bronze or silver treatments (672 in each type of area). This was to ensure that we sampled similar numbers of respondents that were (or would have been in the case of the control groups) allocated to the gold, silver and bronze conditions. All survey responses were transcribed by a research assistant, and a random sample cross-checked by a member of the research team. All responses were coded exactly as they were written and unanswered questions were left as blank cells.
Sample
A total of 246 residents completed the survey—132 from the treatment group and 114 from the control group. Since there were a possible 1008 responses in each group, these constitute response rates of 13% and 11%, respectively, and an overall rate of 12%. This indicates a very slightly higher response rate for the treatment group. Not all respondents answered each question. In what follows, those who did not answer a particular question are excluded from the analyses of the results for that question, but the total number of respondents who did reply is reported in each case.
Survey results
Police contact
Around 80% of respondents from the treatment group and 60% from the control group reported being contacted by the police in the last 12 months (regardless of whether they had been a victim or not) to provide them with crime prevention advice. Respondents from the treatment group were more likely to report that they had been contacted by the police than those in the control condition and the difference was statistically significant (n = 152; X
2(1) = 5.02, p = 0.03).
Awareness of burglary
To prevent repeat and near-repeat victimisation, one aim of the intervention was to raise residents’ awareness of burglaries in their area (but without increasing their fear of crime). Consequently, survey respondents were asked if they were aware of burglaries that had occurred in their neighbourhood within the last 12 months. Those in the treatment area (58%) were slightly less likely to be aware of burglaries in their area than were those in the control areas (68%), although the difference was non-significant (n = 246; X
2(1) = 2.10, p = 0.14). For those who were aware of a burglary in their neighbourhood, responses suggested that, in the treatment areas, residents were more likely to have been told about this burglary by the police (80%) than those in the control areas (60%), (n = 246; X
2(1) = 5.01, p < 0.05).
As well as being asked if they had been contacted by the police about a burglary in their neighbourhood, respondents were also asked if they had been provided with further information about the offence(s). Results suggest that respondents in the treatment area (60%) were more likely to have been provided with additional information about a burglary in their area than those in the control areas (25%) (n = 151, X
2(1) = 16.9, p < 0.001).
Respondents who were aware of burglaries in their area were also asked if they had been apprehensive about being burgled after the crimes concerned. Respondents were more likely to be apprehensive than they were not (approximately 70% of both groups), but there were no differences in reported apprehension for those in the treatment and control areas (n = 156 X
2(1) = 0.01, p = 0.91). Finally, respondents who were aware of burglaries in their area were also asked if they were aware of an increased police presence immediately after them. Results suggest that most were not. The small difference observed between groups (70% unaware in the treatment and 82% in the control) was not statistically significant (n = 157; X
2(1) = 2.39, p = 0.12).
To summarise, respondents in both the treatment and control areas were equally likely to claim to be aware of recent burglaries in their neighbourhoods. Those in the treatment areas reported that they were more likely to have learned about such offences from the police and to have been given more information about the offences than just that they had simply taken place. Respondents reported being apprehensive following burglaries in their areas, but, despite those in the treatment area being contacted by the police and being provided with more information, they were no more apprehensive than those in the control areas. One interpretation of this finding is that (on the basis of the current sample of data) it appears that providing residents with information about recent burglaries is unlikely to increase their fear of crime. This early indication is important, as fear of crime is often cited as a reason for not communicating specific information to potential victims (e.g. Winkel 1991). In light of reduced resources, it could be argued that informing and mobilising the citizenry is more important than ever. If crimes predict future crimes, as Johnson and Bowers (2004) show is the case for burglary, it is imperative that residents are informed not only of the crime type but also the modus operandi. Further, our results suggest that they can be informed of what can be successful in combatting repeat/near-repeat victimisation without experiencing a backfire by way of fear of crime.
In terms of implementation fidelity, the conclusions appear to be mixed. That is, while those in the treatment area appear to have received most of their information about burglaries from the police, if all respondents in the treatment area had been contacted about burglaries at neighbouring addresses (as intended), then those in the treatment areas would have had greater awareness of recent offences than those in the control areas. They should also have been aware of more police presence in the area, which was not the case here. This suggests that not all homes may have been visited. However, a few caveats are necessary. First, respondents were only asked if they were aware of burglaries that occurred in the last 12 months. In some cases, burglary offences of which they were aware may have occurred post intervention. Second, some burglaries of which respondents were aware may not have been reported to the police. And third, it is possible that while a survey respondent may not have been contacted about recent offences, another member of the household might have been.
Likelihood of reporting crime
Figure 2 shows that the likelihood of a respondent claiming that they would report a burglary to the police was high, regardless of condition. The small difference apparent was non-significant (n = 244; X
2(4) = 8.26, p = 0.08).
Levels of satisfaction with the police
Figure 3 shows reported levels of satisfaction with the police for the treatment (n = 130) and control (n = 111) groups for those who answered this question. Relative to the control group, a higher proportion of those in the treatment areas indicated that they were either fairly or very satisfied with the police (70% in the treatment group and 57% in the control group). Thus, while the overall difference observed across the five categories shown in Fig. 3 was not statistically significant (X
2(4) = 4.84, p = 0.31), the difference in those that expressed that they were either fairly or very satisfied with the police (versus those that were not) was (X
2(1) = 3.99 p < 0.05).
Who is responsible for crime control?
When asked who they felt were responsible for preventing crime, most respondents (around 75%) felt that this was the responsibility of the police and residents (see Online Appendix C). Fewer than one-third of respondents felt that the local authority had such a responsibility. None of the differences between the two groups was statistically significant.
In summary, for the sample obtained, those in both the treatment and control groups were highly likely to report an incident to the police. It appears that those in the treatment groups showed slightly higher levels of satisfaction with the police, presumably because they felt the police had invested time in discussing incidents and prevention with them. However, differences between groups were quite modest. Respondents in both groups thought residents and police were most responsible for prevention, with more in the treatment group naming the police (vice versa in the control group). Again, differences were small.
Police recorded crime data analysis
In this section, we examine the outcomes of Operation Swordfish in terms of changes in levels of recorded crime. In doing this, we test for effects at two levels—that of the individual houses that received treatment and that of the treatment areas as a whole. If the treatment was successful, we might plausibly expect effects at both these levels. To explain, the idea of the strategy is to secure not only victimised homes (thereby reducing repeat victimisation), but also potential future targets of burglary in the form of burglary victims’ neighbours. If implemented with sufficiently high levels of intensity, this could lead to offenders avoiding entire street segments. It is also possible that the intervention could lead to the geographical displacement of offender activity within the treatment areas, or a diffusion of crime control benefit (see, e.g., Guerette and Bowers 2009). We return to this issue later. For now, we begin by looking at the individual effects on those homes that were directly in receipt of the treatment implemented to see if and how rates of repeat victimisation were affected.
Repeat victimisation
To examine the direct influence of the intervention on the homes treated—burgled homes and their immediate neighbours—we examine the likelihood of subsequent victimisations and the time to victimisation following treatment for those households that received (or would have had they not been in the control areas) target-hardening. That is, we consider how much time elapsed between the burglary that triggered intervention at treated households (and their neighbours) and any subsequent burglaries. We then perform a survival analysis to see if the observed patterns differ for homes in the treatment and control areas.
To generate data for the comparison group, we performed an artificial run of the Swordfish software on the real crime data for the control areas during the experimental period, that is, we found which properties would have been treated had Swordfish been operational in those areas. Comparing the time to re-victimisation for these properties with that for their counterparts that actually received treatment enables us to estimate the effect of treatment.
In terms of treated households, a spreadsheet was maintained by each police area during the course of implementation. Data were available for 5140 household treatments, of which 648 represented gold treatments, 2395 were silver, and 2097 were bronze treatments. Data for some trigger events (burglaries that occurred in the treatment areas that initiated treatment) were not provided or were unsuitable for analysis. In some cases, this was due to recording issues encountered at the beginning of the treatment period (83 burglaries), while some missing data appear to be due to a drop in recording towards the end of the intervention period. In other cases, data were simply incomplete insofar as we could not confirm that the addresses were treated even though they qualified for assistance (259 burglaries). Here, the software prescribed treatment but there was no record in the dosage spreadsheet, and we can assume that either treatment was not given or that it was given and not recorded. To assess the effect of these situations, the analysis described below was also run with this broader intention-to-treat subset, but since we found no substantial differences for this analysis, these results are not discussed any further.
Police recorded crime results
We first compare survival rates for all treated properties, no matter what the level of treatment. Of the 5140 prescribed treatments, 166 properties were re-victimised by the end of the observation period, compared to 260 in the control group. The survival times for these are shown in Fig. 4, in which the vertical axis indicates the proportion of properties that had not been victimised x days after the trigger burglary that instigated the treatment. The plot shows the survival curves for each group, along with bands representing 95% confidence intervals in each case (for details of their calculation, see Kalbfleisch and Prentice 2002). Visual inspection suggests that burgled homes (that would have been treated) in the control group were more likely to be re-victimised than target-hardened homes in the treatment group (the green line showing treatment group survival remains above the blue line for the control group), such that, by the end of the evaluation period, fewer homes had been re-victimised in the treatment group. The difference in survival rates appears to be greatest around 1.5 years post-intervention. However, a formal analysis showed that the difference in the two distributions was not statistically significant (p = 0.39).
It is possible that the treatment effect might also depend on the type of area in which measures are implemented (see Pawson and Tilley 1997). Since the treatment and control areas were matched on most factors through the process of randomisation, we consider here if the effects differ by the rate of burglary, a factor that, as mentioned above, we matched across groups through the blocking manipulation (rather than simple randomisation). To do this, in Fig. 5, the plot on the left shows the survival curves for homes in the 50% of areas with the highest burglary rates, while that on the right shows them for those with the lowest. This shows the same trend as that observed in Fig. 4, but the effect appears to be larger in the low-crime areas. In this case, the survival curve for the treatment group is outside of the confidence intervals for the control group, and the effect is only marginally non-significant (p < 0.10). In the high-crime areas, the effect is non-significant (p = 0.90).
Treatment groups and moderators
In what follows, we examine these patterns in a little more detail by looking at the “survival” curves for homes that had been burgled and had received the gold intervention (or would have if they had been in the treatment group) and those for their neighbours (silver and bronze treatments). Figure 6 shows the survival curves for properties assigned to gold treatment. There appears to be a positive treatment effect for homes in the high-crime areas, but not in the low-crime areas (the survival lines show the reverse pattern). However, the differences are not statistically significant, and it is important to note that the number of homes included in this analysis is low (n = 318 for the high-crime areas, n = 152 for low-crime), making the analysis less reliable.
In the case of neighbours of burgled homes (that is, those in the bronze or silver treatment groups), there appears to be little effect in the high-crime neighbourhoods (Fig. 7, left), but in the neighbourhoods with the lowest burglary rates (Fig. 7, right), there does appear to be a preventative effect, and this is statistically significant (p = 0.002).
In short, the intervention overall did not have a significant effect on survival rates. In separating by area, it would appear that there was a (marginally non-significant) trend that suggested that across all treated houses burglary dropped in the low-crime areas. Looking at the effects by level of treatment, there was no effect for gold group households, but in the case of the neighbours of victims (silver and bronze groups), the risk of burglary victimisation was reduced in the lower-crime neighbourhoods and this effect was statistically significant.
Time series
In this section, we examine trends at the area level. We use time series data to do this because—unlike interventions such as a hotspots policing initiative—the intervention was implemented gradually over a large area over time. Figure 8 shows a time series plot of the total number of completed recorded burglaries (aggregated) across the treatment and control areas per week. Implementation intensity was measured in two ways. In the first instance, we use the number of incidents that were flagged as requiring treatment, regardless of whether treatment was actually recorded. This is a measure of intention to treat and does not necessarily reflect the number of homes which actually received the intervention. The dotted line represents a cumulative measure of this type of intervention intensity, indicating that implementation increased steadily over a period of 27 weeks. Looking at Fig. 8 it is difficult to determine if there was an effect of intervention. Consequently, we conducted a formal time series analysis to see if—after controlling for other factors (measured and estimated)—there was an association between the weekly count of burglary in the treatment areas and the cumulative dosage of intervention. To test this, the simplest model that could be employed would be an Ordinary Least Squares (OLS) regression. For such a model, we could regress the weekly count of burglaries in the treatment areas against that in the control areas, and the cumulative estimate of intervention dosage. However, a problem with analysing time series data using OLS regression is that an assumption of the test—that the observations are independent—will generally be violated (as is the case here). Consequently, we use more formal time series analyses.
A number of time series model specifications are possible. Autoregressive (AR) terms account for the fact that the value of a series at one time point is a function of those immediately before, and hence that observations are not independent. Moving average (MA) models measure the effect of stochastic shocks in a time series on future observations and then account for these, while ARCH models can be used to deal with the fact that the error associated with a model may vary across the time series (heteroskedasticity). To ensure that any conclusions were not sensitive to model specification, we tested all possibilities, starting with ARCH models.
Table 1 shows that while the ARCH term was not significant, the AR (up to two lags but not more) and MA terms were. In all cases, after controlling for the trend observed in the control areas, there was a negative association between recorded burglaries and the cumulative intensity of intervention. That is, as more homes were treated, on average the weekly count of burglary in the treatment areas appears to have decreased relative to the trend observed in the control areas. However, two issues are important to note. First, the estimated effect was relatively small, suggesting a reduction of about 6 burglaries per week for every 1000 homes that had triggered treatment. Second, in the case of the ARMA model, this association was marginally non-significant (with a p value of 0.072, two-tailed).
Table 1 Time series regression models for intention to treat data (z-scores in parentheses, standard errors are robust)
The second type of intervention intensity modelled was the total number of homes (burgled homes and their neighbours) that were recorded as having actually received the intervention. The analyses presented above were repeated using these data (see Online Appendix D) and revealed an identical pattern of results.
In summary, for all models estimated, the association between the weekly count of burglaries in the treatment areas and the cumulative intensity measure was in the right direction. In other words, as more homes were treated in the intervention areas, on average the weekly count of burglary appears to have decreased relative to the trend observed in the control areas (after accounting for the dependency in the data). However, as the intervention coefficient for the ARMA (1,1) model was marginally non-significant (see Table 1 and Online Appendix D), caution should be exercised when interpreting the results.