Introduction

Domestic abuse (DA), by which we refer to incidents of controlling, threatening behaviour, violence or abuse between those aged 16 or over who are or have been intimate partners or family members (Home Office, 2012), is an extensive social problem, marked by high levels of repeat victimization and under-reporting. One out of three women in the UK report having experienced domestic abuse at one point in their lives (Office for National Statistics (ONS (2019), Hirschel, McCormack and Buzawa (2017)). Yet, domestic abuse remains widely under-reported to police authorities and social services. Estimates around the world gauge that reported cases of domestic abuse against women represent only a very small part of the problem when compared with prevalence data. The share of unreported abuse, for example is estimated to be between 50 and 95% of the abuse occurring (Office for National Statistics (ONS) (2020), Gracia (2004)).

While domestic abuse reporting to police by third parties rose throughout the Covid-19 quarantine in Greater London, particularly in areas with high population density, there is concern that actual incidents rose even more, since under-reporting may have also increased in households where the abuse could not be reported by an outsider and the victim was unable to leave the proximity of the perpetrator due to lockdown measures (Ivandic, Kirchmaier and Linton (2020)). Abreu Minero and Nivette (2023) found that levels of domestic homicide in England and Wales remained stable, due to compositional changes of a drop in ex-partner homicides and an increase in parent and child homicides. Part of this finding could be explained by the cancelling out effects of the increased costs of committing abuse against ex-partners due to social distancing rules, as Ivandic, Kirchmaier and Linton (2021) find that domestic abuse crime committed by current partners and family members increased, while crimes by ex-partners decreased. Relatedly, Aebi et al. (2021) found that femicide did not increase during the pandemic in Latin American countries.

When victim and abuser are quarantined together, calling the police poses great risk to the victim’s safety, causing more victims to become increasingly isolated and vulnerable. Direct messaging is also dangerous since texts from the police, like phone calls, may provoke a controlling abuser. The challenging and precarious environment in which many victims of abuse found themselves during quarantine increased the urgency to reach them and motivated the study at hand.

Our research first considered whether it was even possible to reach isolated DA victims using a nontraditional social media campaign. It aimed to see if posting social media ads on Facebook and Instagram feeds could increase dissemination of domestic abuse information and determine for which demographic groups the reach is higher. Secondly, we examined whether providing victims with more information and a safer means of contacting police eventually resulted in an increase in domestic abuse reporting.

To start, the study combined two randomized control trials with uniquely detailed and confidential high-frequency administrative data from two major police forces in the UK. The goal of the campaign was to inform potential victims about Silent Solution, a safer means of reporting instances of DA to police currently in place in the UK. Our approach leveraged the wide use of social media platforms to display empowering yet simple advertising visuals and messages designed to promote Silent Solution. In collaboration with two police forces, we randomized the treatment across high-risk geographic areas in one force and across high-risk individuals in the other force. We identified potential high-risk areas and potential high-risk victims using historical police administrative records. These records contained detailed information on the timing, location, description, type of relationship, information on the victim and information on the perpetrator, among other factors. Finally, we used the high-frequency detailed confidential records to test whether there was a significant difference during the period of treatment for the treated individuals and areas as compared to the ones in the control group—or those who received only ambient messaging.

Our study yields two main results. First, social media campaigns can be a powerful tool for engaging with potential domestic abuse victims, particularly among younger cohorts. Second, promoting reporting through social media platforms did not lead to an increase in reporting among the treatment groups relative to the controls. This was true also for younger cohorts of victims, even though they were more engaged with the campaign.

There are two reasons we believe explain our null result findings. First, our social media targeting relied on Facebook’s internal algorithms to deliver ads to the treated groups. Using survey evidence, we found that Facebook failed to correctly target the treatment areas and hence did not execute geographically randomized control trial according to our design. Second, Facebook and Instagram provide no way to limit sharing of sponsored social media posts, meaning we could not control spillovers between treatment and control groups. Other possibilities are that Silent Solution is not an adequate method of reporting, or that domestic abuse reporting requires a series of more complex interactions beyond just promoting more information.

Our research contributes to the scarce experimental literature on how to increase domestic abuse reporting among victims. To the best of our knowledge, this is the first randomized test of the effects of a social media campaign on engagement and reporting. As police forces across the UK, but also worldwide, start using social media more to engage with the citizens they serve, these results provide important and valuable implications for their use and the role of technology in the future policing. Our results contribute to the understanding of how police forces can use social media to reach specific groups of people, such as younger cohorts.

The paper is organized as follows. The “Theoretical Framework” section provides a literature review of the existing evidence on the determinants of reporting domestic abuse and the role of social media. The subsequent “Data” and “Research Design: Randomized Control Trial” sections describe the institutional background, data and research design. The “Results” and “Understanding null effects” sections depict the results, and the “Conclusion” section concludes with brief discussion of our contribution and its implications.

Theoretical Framework

Domestic abuse is different from many other crimes, in part due to the nature of the close relationship between victim and offender, which makes its reporting much less likely when compared to other types of crime (Gracia (2004)). Among the factors that may discourage reporting, intimidatory and retaliatory violence have been emphasized (Hoyle and Sanders (2000)), as have the “private nature” and stigmatization of domestic abuse, the emotional and economic dependency on the perpetrator, isolation and lack of support networks and other measures of control (Hoyle and Sanders (2000)), including embarrassment, victim blaming and the perception of it as a purely private issue (Gracia (2004)). Another critical factor is the criminal justice system’s inability to stop the abuse: imposed economic sanctions (fines) are often counterproductive as the victim and offender share finances. There is also mixed evidence as to whether arresting the offender reduces violence or prevents re-victimization (Davis and Smith (1995), Angrist (2006), Xie and Lynch (2016)). Though an arrest may have a small deterrent effect, research suggests that reporting by the victim itself does more to reduce the probability of repeat victimization (Xie and Lynch (2016)). Moreover, a lack of awareness of domestic violence laws and available resources (Kim and Ferraresso (2021)) also correlate to a lower likelihood of reporting.

Factors associated with a higher likelihood of domestic abuse reporting include the following: higher education level of the victim (Coker et al. (2000); more severe violence, sustained injuries or life-risking violence (Fanslow and Robinson (2009), Park and Ko (2020), Kim and Ferraresso (2021))); existence of children exposed to violence (Fanslow and Robinson (2009)); damage to property (Birdsey and Snowball (2013)); and age of the victim (younger victims tend to report more often than older victims) (Rivara et al. (2009)).

While existing literature emphasizes the long-term benefits of reporting for the victim, there is less theoretical understanding of which policies can encourage reporting. This question is particularly important when thinking about what police forces can do to encourage existing victims to seek out help, especially as reporting to the police is often the main gateway to additional resources and support (Zaykowski (2014), Xie and Lynch (2016)). Victim support services are severely under-utilized because victims often do not know about the resources available to them (Sims et al. (2005), Fugate et al. (2005)), especially in the case of immigrant women who may not know their legal rights and may have language barriers (McCart et al. (2010)).

Social media platforms have been increasingly used as a source for information (Wester-man, Spence and Heide (2013)), and this is particularly the case among younger generations, where 96% of 16–24 and 88% of 25–34 years old use social media as a source of daily information. With the rise of social media and increased role of technology in the recent decade, policing has started to adapt and increasingly uses social media both as a channel of communication and an investigation tool. Walsh and O’Connor (2019) provide an overview of social media’s implications for practices and perceptions of policing, with specific emphasis on its benefits and costs. They argue that “social media also transform how the police communicate with citizens and manage the visibility of their personnel and activities. Traditionally dependent on the mass media to engage the public, law enforcement agencies have embraced sites like Facebook, Twitter, and YouTube, creating social media teams to produce their own news and directly broadcast content” (Walsh and O’Connor (2019)). Lee and McGovern (2013) acknowledge that social media provides the police stricter control over their appearance and may come to represent “one of the most powerful tools” of their public relations. Williams et al. (2018) show that when the police uses social media interactively and caters to local geographical priorities, it is perceived beneficial and advances community policing goals. However, Walsh and O’Connor (2019) also state that the effectiveness of police’s use of social media relies on whether the police are perceived as legitimate.

We believe there is a glaring gap in the literature when trying to understand whether social media can engage individuals on topics around domestic abuse and whether this, in turn, can directly result in changes in policing outcomes—namely whether it can lead to more reporting.

Hence, our research aims to test the following research hypotheses:

  • H1: Individuals will engage with domestic abuse information on social media, and police forces can use social media to increase public awareness.

  • H2: Giving more information to victims increases their likelihood of reporting the abuse.

  • H3: Social media can be particularly effective when reaching younger cohorts and encouraging them to report abuse to the police.

Data

In this section, we describe the data sources from the two police forces—the Thames Valley Police (TVP) and the Metropolitan Police Service (MPS)—that informed both the design of the randomized control trial and its evaluation. Across both forces, we received high frequency and confidential administrative data on the population of domestic abuse calls, where domestic abuse is defined as any incident of controlling, threatening behaviour, violence or abuse (physical, emotional, psychological, sexual or financial) between those aged 16 or over who are or have been intimate partners or family members. Domestic violence is a subcategory of domestic abuse where violence is present at the incident (Home Office, 2012)Footnote 1.

Data on Domestic Abuse: Metropolitan Police Service

The calls-for-service data from the Metropolitan Police Service has 328,385 observations spanning from 1st January 2019 to 14th June 2020. It contains data on the date, time of the call, a classification of the reason for the call, location identifiers, level of urgency of the call, the helpline number contacted, its criminal classification identifier (if applicable), the type of caller (victim, witness, third party, etc.) and the associated response time. Using the classification codes in the calls-to-service data, the total sample of domestic abuse calls is identified.Footnote 2 The information on location identifiers in the dataset was converted into postcode sectors allowing us to identify postcode sectorsFootnote 3 .

The MPS covers a population of around 9 million people in Greater London. It received about 2.5 million calls-for-service in 2019, out of which 7% were related to domestic abuse. In the 11 weeks from the beginning of the first lockdown on the 26th of March 2020, there were a total of around 45,000 calls to the MPS related to domestic abuse, and about half (43%) of total crimes recorded were DA related, emphasizing the importance of domestic abuse in policing work as other demand for policing declined during the Covid-19 pandemic.

The MPS, by its standard recording procedure, does not record the line through which the calls come in. As a result, the data does not contain separate indicators if the “Silent Solution” is triggered. Hence, we are unable to directly calculate the daily number of “Silent Solution” uses. Therefore, to study the impact of the intervention with the MPS data, our outcome variable of interest is the total number of domestic abuse calls-for-service. Assuming our hypotheses hold true, this implies that we would expect to see an increase in total DA calls-for-service from treatment areas in response to the intervention relative to the control areas if the intervention was successful.

Data on Domestic Abuse: Thames Valley Police

The Thames Valley Police (TVP) sample is formed by all calls flagged as domestic abuse by the Thames Valley Police from March 2019 to August 2020 and includes data from three different confidential datasets provided by that police force: the calls-for-service from the command and control system, the crime register and a victim dataset.

The master dataset contains all domestic abuse incidents and crimesFootnote 4. In addition to detailed time information, this dataset includes information on whether the incident was a crime, the type of offence, its Home Office classification, a set of labels for hate incidents, a domestic abuse flag if the incident fulfills the domestic abuse definition of the Home Office, and where it happened measured at the output area (OA) level. Moreover, it contains the victim risk level and the possible involvement of alcohol or drug use during the incident. When merged to the command-and-control register, additional information on the urgency grade of the incidents and the location of the incident (measured by OA) are added. It also has depersonalized but unique ID numbers for the victim and suspect that link to their respective registers, which include additional demographic information like age, gender and ethnicity.

Similar to the MPS, TVP does not record the line through which the calls come in; hence, we cannot observe in the data for which calls the “Silent Solution” was triggered.

Therefore, our outcome variable of interest is the total number of domestic abuse calls-for-service. To conduct the RCT evaluation, the sample is collapsed into a weekly panel of individual victim by week, across 77 weeks from January 2019 to August 2020. The outcome variable of interest is the weekly number of domestic abuse calls.

Research Design: Randomized Control Trial

Due to the Covid-19 pandemic and resulting quarantine, domestic abuse incidents have risen, and the share of reporting has likely declined (Ivandic, Kirchmaier and Linton (2020)). The reason is that traditional methods of reporting require the victim to speak over the phone and, when a victim and abuser are quarantined together, calling the police may jeopardize the victim’s safety. In the wake of the start of the pandemic in late March 2020, this led us to design and launch a social media campaign to promote the Silent Solution, an under-used method of reporting that could be particularly valuable when the victim is near her abuser.

Silent Solution is a system currently in place in the UK that allows people who are unable to speak but need police assistance to call the police and safely alert them of their situation. It works as follows: when a person at risk calls 999 (the UK’s emergency number), they can press 55 instead of talking aloud and be automatically transferred to a police call handler. The handler will then attempt to communicate with the caller by asking simple “yes” or “no” questions that can be answered with the phone keypad. While trying to assess the situation, the handler can also digitally trace the caller’s location so they can deploy officers if necessary. This system provides a safer option for domestic abuse victims in close contact with their perpetrators who would be otherwise unable to contact the police without putting themselves in more danger and fear of retaliation.

Out of 3.2 million calls for service to the Metropolitan Police Service during 2019, only 120 utilized the Silent Solution. While some ambient public service campaigns by the UK Government provided information about the Silent Solution, they were not specifically targeted toward high-risk groups, and the publicity it received was verbose and overly detailed. Furthermore, some of the official online material about the programme was out of date and did not explain the ability of the system’s use of mobile phones’ locations to send help if needed.

Domestic abuse victims, therefore, may not have known of the system’s existence or its benefits. If Hypothesis 2 is true, giving information of its existence to isolated high-risk victims should increase the probability of them reporting to the police. In order to test this hypothesis, we conducted a randomized control trial that consisted of using adverts featuring Silent Solution on social media and targeted them to people who were potentially at risk of domestic abuse victimization.

The reason behind using social media advertising as opposed to a more conventional approach—such as direct messaging by the police via text or WhatsApp—is that domestic abusers often monitor their victim’s communications. Thus, conventional means may have put victims at greater risk for retaliation. By using the more indirect approach of social media ads, information appears as part of the victim’s regular social media feed with no obvious attribution. Facebook and Instagram were our chosen apps as their reach makes them ideal for our messaging. As of March 2020, there were 44.8 million Facebook users and 24.5 million Instagram users in the UK. The Instagram audience skews younger than Facebook’s (Newman et al. (2021)). We ran a randomized control trial (RCT) to measure the effect of the social media adverts on reporting, as random assignment to treatment and control groups should make the intervention exogenous to any of the subject characteristics and thus prevent these endogeneity concerns.

An additional novelty to our design was that our targeting strategies differed between MPS and TVP, which allowed us to test their relative effectiveness. Targeting at MPS focused on high-risk areas, that is, areas with persistently high levels of DA-related calls for service. Our units of analysis in the MPS evaluation were postal sectors, a small geographical unit recognized by Facebook’s advertising engine. At TVP, targeting focused on high-risk individuals. We used TVP’s administrative data to identify recent DA victims who were at high risk of repeat victimization. Hence, our unit of analysis in the TVP evaluation was the individual. In both cases, only the treatment groups were targeted for advertising on Facebook and Instagram, while the control groups received no additional information, that is, they were not targeted for the Silent Solution ads.

Finally, the ads themselves were designed to be simple and clear. We designed ads in collaboration with advertising experts that involved visuals which promoted the reporting of domestic abuse as an empowering choice. Figure 7 in Appendix shows the adverts that were shown in social media feeds. Additionally, we created a short survey on the Silent Solution website, which was reached by voluntarily clicking through on the ads, which allowed us to test who was reached and who engaged with the Silent Solution advertisements.

We preregistered the study on the American Economic Association RCT Registry under the ID AEARCTR-0005724. Our pre-analysis plan was approved by the LSE Research Ethics Committee under REC ref. 1117.

Metropolitan Police Service

For the intervention in the Metropolitan Police serviced areas, the social media advertisements targeted postal sectors in the Greater London area. While the MPS services in total 1046 postal sectors, we randomized a total 523 sectors categorized as high-risk areas between treatment and control areas. These high-risk areas were defined as being above median level of the average monthly DA calls in 2019 by postal sector areas. These high-risk areas have persistently high levels of DA-related calls for service, averaging around 22 calls per month in 2019 (as compared to the below median postal sectors averaging only around 5 calls per month).

Of these 523 postal sector areas, initially 174 were assigned to treatment and 174 to control. The remaining 175 postal sectors were randomized 3 weeks later, out of which 87 into treatment and 88 into control. This led to a total of 261 postal sectors assigned to treatment and a total of 262 control sectors. The treatment and control sectors are plotted in Fig. 1. The intervention ran from 21st April, 2020 until 31st May, 2020. Adverts were shown across Instagram and Facebook to any women older than age 18 living in the given postal sector.

Fig. 1
figure 1

Treatment and control areas in MPS serviced postal sectors. a Initial assignment. b Final assignment. The treatment postal sectors are in dark blue (1), the control sectors in light blue (0) and the areas that were randomized later are marked grey (− 1). The outlines show the total area serviced by the Metropolitan Police Service, and all the white areas represent non-randomized sectors

Full size image

To ensure that the randomization was unbiased, we also computed a balance test in Table 5 in Appendix. We tested the balance of our randomization along a number of dimensions that we believe could have impacted DA levels during the Covid-19 pandemic: history of DA in the sectors (2019 and 2020, before and after the Covid-19 pandemic hit), the number of Covid-19 cases in the sectors, the population density, deprivation levels and demographics. Domestic abuse call data was used to compute mean differences in DA calling history in 2019 and 2020, prior to the start of the intervention. The Covid-19 cases data was obtained from publicly available coronavirus (Covid-19) data.Footnote 5 The remaining demographic characteristics were drawn from the 2011 Census for England and Wales.Footnote 6 Each of these factors must be balanced between our treatment and control groups to ensure internal validity of our study. Table 5 in Appendix shows that the two groups do not differ significantly along any of these characteristics, and that the randomization allows us to retrieve unbiased estimates.

Thames Valley Police

For the intervention in Thames Valley Police serviced areas, our treatment and control groups consisted of individuals at higher risk of becoming victims to domestic abuse. Based on existing research that recent victimization increases the likelihood of revictimization (Grogger et al. (2021), Bland and Ariel (2015)), we narrowed down the randomization sample to individuals that had been victims in at least one domestic abuse report during the period between 15th March 2019 and 15th March 2020. Out of those 12,946 potential subjects, we restricted the experiment to only female victims (10,460 persons) who had a mobile number recorded (7034 persons). These 7034 individuals were randomly assigned to either the treatment group or the control group. The TVP intervention ran from the 4th of May 2020 (first day of treatment) to 1st of July 2020 (last day of treatment).

Table 6 in Appendix shows the results for the balance test comparing victim police records and demographic characteristics of the victims in the treatment and control groups. The first and second columns are the average values of the treatment and control groups before the start of the intervention, from the 15th March 2019 to the 15th of March 2020. Domestic abuse calls are average total calls flagged as a domestic abuse incident where the subject was the victim, in the sample period. Total calls are average total incidents reported to the police where the subject was the victim. Violence without injury calls, violence with injury calls, criminal damage calls, other theft calls and rape calls are the equivalent corresponding to those types of crimes. Average victim age in last incident is the victim age in the last call reported in that period. The rest of the demographic variables in the table are the mean of the indicator variables equal to 1 if the victim fulfills that demographic trait. None of the differences between the treatment and control group outcomes is found to be significant at 5% significance level, allowing us to conclude the randomization ensures a causal interpretation of the intervention on the differential outcomes between the treated and control victims.

Estimation Specification

For the main analysis of both interventions, we estimate the intent-to-treat (ITT) effect using ordinary least squares (OLS):

$$DA_{it}=\alpha+{{\beta T}_{i,t}\;+S}_{it}$$
(1)

Where DAit is domestic abuse calls in an area/individual i in week t, Tit is a treatment dummy and Sit is a random error. A direct outcome of interest would be a self-reported measure of knowledge about how to report domestic abuse in the presence of a perpetrator. Our choice of the outcome measure, however, was informed by existing data availability and measures of policy interest given budget constraints. Recall that the level of assignment of treatment and the level of outcome measurement overlap, each distinct in the MET and TVP interventions.

Metropolitan Police Service

For this intervention, the assignment of the treatment and the measurement of outcomes are at the postal sector area i level. Additionally, to the specification in Eq. 1, we also ran the same regression adding a linear time trend and density controls. Residential density controls were added because Table 5 in Appendix found the treatment and control groups to differ significantly across their density levels.Footnote 7 Linear time trends were added to control for an increasing time trend across the period and seasonality effects. Since time trends required larger time periods, the results from the weekly regressions with these controls were estimated using data from 2019 to 2020, while the one without controls in Eq. 1 was estimated using only the data from the intervention period. Moreover, in each regression, in order to account for the re-randomization of sectors 3 weeks after initial randomization, the treatment dummy Tit changes to 1 if a postal sector received treatment that week and 0 otherwise. Our data also allowed us to identify whether the callers were victims or third parties. Therefore, each regression specification was estimated for all DA-related calls-for-service, calls-for-service by victims and those by third-party callers.

Thames Valley Police

For this intervention, the assignment of the treatment and the measurement of outcomes is at the individual victim i level.

Since randomization was done prior to the start of the ad campaign, the treatment status is constant throughout the intervention period, hence Tit is equal to 1 if the individual was part of the treatment group. The model is estimated on a victim by week panel starting in the week of the beginning of the intervention until the last week of data available, 2 months after the treatment window. We consider the individual to be treated in the aftermath of the intervention as well. In addition to the specification in Eq. 1, we add linear time trends to control for an increasing time trend across the period.

Results

In this section, we present the results of the randomized control trial of the social media campaign on two sets of outcomes: social engagement and domestic abuse reporting. We discuss these two sets of outcomes for both the Metropolitan Police Service and Thames Valley Police interventions separately.

Greater London Engagement

The MET campaign ran from 21st April to 31st May 2020, reaching a total of 1,236,367 people and generating a total of 30,544 link clicks. The reach and link clicksFootnote 8 peaked during the week of 27th April to 4th May, with the daily reach of advertisements reaching over 80,000 unique individuals and generating over 1200 link clicks on those days. On days when a larger number of people are reached, the number of clicks is also higher (Fig. 2).

Fig. 2
figure 2

Reach and link clicks, MET campaign. Note: In the left figure, the blue line denotes reach, and the orange line denotes link clicks. In the right figure, the blue line denotes reach, the yellow line impressions, and the orange line denotes the link click through rate

Full size image

The campaign also generated an average link click through rateFootnote 9 of about 2.5% on average. CTR is highest when the advertisements first reach people in the initial stages of the campaign and gradually dips off in later weeks. However, it rises again towards the end of the campaign, suggesting that repeated exposure might have caused people to engage with the campaign more.

Most of our link clicks were received on Wednesdays (a total of 5000), while the maximum number of people reached were on Fridays, and both reach and link clicks were especially low on Mondays and Tuesdays. The click through rate averaged at around 2.1% across all the days of the week, with the maximum click through rate being observed on Mondays, where actual number of link clicks was low (Figure 8 in Appendix).

We were also able to analyse the engagement with the campaign by age group. The majority of people reached by the campaign were younger than 35 years old: 40% of the people belonged to the 18–24 years age group, and 33.92% were between 25 and 34 years old, leading that about 80% of total link clicks came from these two groups. The highest link click through rate was 2.54% seen for 25–34-year age group (Fig. 3), followed by 18–24-year group and 65 + .

Fig. 3
figure 3

Link click through rate by age, MET campaign. a Click through rate by age group. b Total link clicks by age group

Full size image

Thames Valley Engagement

The TVP campaign targeted 3541 women who had experienced DA in the previous year, hence a smaller and more targeted group. The TVP campaign ran from 5th May to 31st June, 2020, generating a total of 761 link clicks over the period. An average link click through rate of about 0.3% was observed (Fig. 4).

Fig. 4
figure 4

Reach and link clicks, TVP campaign. Note: In the left figure, the blue line denotes reach, and the orange line denotes link clicks. In the right figure, the blue line denotes reach, the yellow line impressions and the orange line denotes the link click through rate

Full size image

In this case, we saw very high link clicks in the initial days of the treatment window, which gradually dropped off to average at about 15 daily clicks at the end. Contrary to the MPS campaign, prolonged exposure to adverts did not increase engagement as the link clicks towards the last days were at levels similar to those earlier on, and link click through rate remained at similar levels and averaged at 0.3%. It was the highest when the campaign was first introduced and remained approximately stable after dropping a few days into the campaign. Relative to the London experiment, the link click through rates in Thames Valley were much lower.

We find little variation in the number of people reached by day of the week, while variation in the link clicks by day of the week shows more discernible patterns. The highest link clicks were observed on Tuesdays, when link clicks were at about 160. Link clicks and the click through rate were the lowest on Saturdays (Figure 9 in Appendix).

Looking at age groups, we found an older audience compared to the London intervention. While people aged 18 to 34 years old made up the majority of the engagement, the 18–24 and 25–34 age groups did approximately 60% of the link clicks, a noticeably lower number than the 80% jointly reached by the same age groups in London. Another point of interest here is the high link click through rate by individuals who are 65 years of age or above. Distinctly different from the other campaign, we find that this group has a link click through rate of 0.46%—higher than that for all the other age groups (Fig. 5).

Fig. 5
figure 5

Link click through rate by age, TVP campaign. a Click through rate by age group. b Total link clicks by age group

Full size image

Metropolitan Police: Domestic Abuse Outcomes

We estimate the effect of the Silent Solution advertisement campaign by comparing the level of DA reporting during the period of the intervention using Ordinary Least Squares (OLS) as described in Eq. 1. Our data allowed us to distinguish victims from third-party callers. Therefore, each regression was estimated for all DA-related calls-for-service, calls-for-service by victims, and those by third-party callers. Theoretically, we would expect the intervention to affect only DA reporting by victims.

The results of this analysis are presented in Table 1. The analysis fails to show any difference in calls-for-service, calls by victim, or third-party calls between the treatment and control groups. In fact, the effect on outcome of most interest, calls by victims, is precisely estimated at zero. Even after adding controls for density and a linear trend over the longer time period in Tables 7 and 8 in Appendix, the treatment effect remains insignificant for DA calls by victims.Footnote 10 This suggests that the Silent Solution intervention did not have any effect on the reporting levels in the treatment sectors. Figures 6 and 12–14 in Appendix confirm this. The increase in calls from treatment and control sectors are similar. In Table 10 in Appendix, we also show the two-sample t-test for equal variances across treatment and control areas.

Table 1 Effect of the intervention on DA calls-for-service, MPS
Full size table

Thames Valley Police: Domestic Abuse Outcomes

For this analysis, the model in Eq. (1) is estimated by ordinary least squares (OLS) on a victim-by-week panel, starting in the week of the beginning of the experiment and lasting until the last week of data available or 2 months after the treatment window. Columns 1 and 2 of Table 2 show the regression results, with and without a linear time trend. There is no evidence that treatment in the form of targeted social media advertisements on the Silent Solution program helped increased reporting. Even though the coefficients are negative, they are not statistically significantly different from zero.

Table 2 Effect of the RCT on domestic abuse incidents suffered by the victims, TVP
Full size table

In Table 3, we estimate the same model including an age interaction term to see if the treatment effect is significantly different for those aged below 25 (column 1), those between 25 and 40 years old (column 2) and people over 40 (column 3). Neither the dummy variables for age nor the interaction terms are significantly different from zero, meaning there is no evidence of differences in the treatment effect depending on age.

Table 3 Effect of the RCT on domestic abuse incidents suffered by the victims, by age—TVP
Full size table

Understanding Null Effects

In the previous section, we established two main results. First, social media campaigns can be a powerful tool for engaging with potential domestic abuse victims and can serve as a channel of information for police forces. We found high engagement with the social media campaign, which is particularly pronounced among younger cohorts. This set of evidence confirms our first hypothesis: that individuals will engage with domestic abuse information on social media, and police forces can use social media to increase public awareness.

However, we did not find any increases in domestic abuse reporting, either among treated individuals in Thames Valley Police or among treated areas in the Metropolitan Police Service. We also did not find any statistically significant difference in reporting for younger cohorts of victims. These findings go against our second and third hypothesis, namely that giving more information to victims increases their likelihood of reporting the abuse, and that social media can be particularly effective when reaching younger cohorts and encouraging them to report abuse to the police.

There are a number of potential reasons for this. It could be due to Facebook’s imperfect compliance with our definitions of treatment groups, spillovers from the treatment to the control groups, or that the method of reporting promoted in this intervention (the Silent Solution) does not adequately meet the needs of the victims even when they are aware of it. In the following section, we discuss an analysis on data from the Metropolitan Police Service trying to isolate the first two reasons and discuss why we believe Facebook’s imperfect compliance with our definitions of treatment groups is the main reason of our null effect findings on reporting.

Survey Results on Targeting

Since the social media targeting relies on Facebook’s algorithms to correctly serve social ads, and those algorithms are not transparent to researchers, discrepancies between our designed targeting and Facebook’s implementation of that design could have resulted in imperfect compliance with our definitions of treatment groups. Anticipating potential imperfect compliance, we created a short survey on the Silent Solution websiteFootnote 11 that allowed us to better ascertain who engaged with the Silent Solution advertisements. We did this only for the MET arm of the intervention as it was done at the postal sector level (anonymity concerns with regard to the individual-level targeting in the TVP arm do not allow us to conduct a similar test). During the period of the Silent Solution social media campaign, we recorded a total of 155 responses. We limited the number of questions to four (to limit attrition) asking about gender, age, postcode, and information on how they reached the website. These survey observations allowed us to test whether the advertisements correctly targeted the desired gender and age range and, more importantly, the targeted postal sectors. We found that 96% of respondents were female (and another 3% identified as male and 1% as non-binary). Moreover, we found that 98% of respondents were in the targeted age group of 18 years old and more.Footnote 12 There were no missing responses for the questions asking about gender or age. Thus, Facebook targeted the correct age and gender groups.

To test whether the social media adverts correctly reached the treated postal sectors, we also asked about the postcode of the respondent’s home address. A total of 37% of respondents did not want to disclose the full postcode. This left 73 with location information, out of which 59 responses were recorded in the area serviced by the Metropolitan Police Service, and 14 were elsewhere. Out of the 59 in Greater London, 34 were in the treatment areas, 17 were in the control areas, and 8 were in the non-randomized postal sectors. Put differently, 42.5% of treated respondents were not in the treatment areas. These survey responses tell us that geographical targeting on Facebook’s advertising did not work. It suggests that the Facebook targeting was imprecise, and that while the targeted areas likely were exposed to the advertisements, there were also sizeable spillovers/leakages in the treatment to control areas. This is strong evidence explaining why we do not find effects on DA reporting.

Spillovers

A competing hypothesis would suggest that while Facebook overall correctly complies with the definitions of treatment groups across postal sectors, the very dense nature of Greater London might lead to small spillovers across neighbouring postal sectors (e.g. in the case of large residential blocks or high streets). If Facebook was unable to target the postal sectors finely, some control sectors might have accidentally been exposed to the Silent Solution advertisements as well. This is further exacerbated as there are many instances (as is clear from Fig. 1) where treatment sectors have control sectors as neighbours and vice versa. In order to check for this and account for (as well as identify) such spillovers of the “treatment,” we hypothesize that spillovers are likely to be minimal for postal sectors that are surrounded by postal sectors of the same assignment. For instance, control sectors surrounded mostly by other control sectors or treatment sectors surrounded primarily by other treatment sectors. Therefore, we re-estimated Eq. 1 limiting our analysis to subsets of 58 postal sectors of control and treatment sectors. These are sectors that share their treatment status with the majority of their neighbouring sectors (we define neighbours as postal sectors that share boundaries).Footnote 13

The results from this analysis are presented in Table 4. Once again, we find no effect of the Silent Solution advertising on domestic abuse reporting behaviour despite limiting our analysis to sectors where spillovers would be the least. This might be because spillovers were much larger than what our neighbour analysis allowed us to account for, or it might be driven by a lack of any treatment effect altogether.

Table 4 Effect on DA calls for service after accounting for spillovers, MPS
Full size table

Finally, there is another source of potential spillover. Facebook and Instagrams do not allow the sharing of sponsored social media posts to be restricted either privately or publicly, meaning we could not control this channel of spillover. We cannot test this notion empirically, since social media networks are largely dispersed around the whole metropolitan area. Whereas the first two analyses spoke to the null results behind the randomized control trial with geographic variation across the Metropolitan Police Service, this reason could also explain the null finding in the experiment across Thames Valley Police. As the randomization and treatment were done at the individual level within Thames Valley Police, from the initial design of the experiment, we decided not to administer any similar survey within this arm of the experiment due to anonymity and data privacy concerns. In Fig. 4, we observed that while the total daily clicks were on average about 15 per day, the daily reach was averaging around 2000, which plausibly means that the campaign reached more individuals than intended through users sharing the social media posts themselves and leading to spillovers to the control group within the Thames Valley Police arm of the experiment as well. Nonetheless, there could also be other explanations for the findings, including that Silent Solution may not be an adequate method of reporting, or that domestic abuse reporting requires a series of more complex interactions beyond just promoting more information.

Conclusion

The challenging environment for domestic abuse victims created by the Covid-19 lockdowns, which found them at greater risk for harm and increasingly isolated, motivated the study at hand. It combined two randomized control trials with uniquely detailed high-frequency administrative data from two major police forces in the UK. The goal of the campaign was to inform potential victims about the Silent Solution, a safer means of reporting to police. To do so, our approach leveraged the wide use of social media, which also poses less risk to victims than direct messaging.

Our research found that social media campaigns can engage with potential domestic abuse victims. We found high engagement with the social media campaign, particularly among younger cohorts. However, we did not find that the social media campaign led to increased domestic abuse reporting, either across treated individuals or across treated areas. Nor did we find any statistically significant difference in reporting for younger cohorts of victims. The reason behind this null effect may be Facebook’s noncompliance with our definitions of treatment groups. Using survey evidence, we concluded that Facebook targeting failed to restrict the ads to the target areas and hence did not enable a geographically randomized control trial. However, there could be other explanations for this null finding, including that the Silent Solution is just not the adequate method of reporting, or that domestic abuse reporting requires a series of more complex interactions rather than just exposure to more information.

Despite the null findings, we believe our research has important implications for future police work. As police forces across the UK, but also worldwide, increase their use of social media to engage with the citizens they serve, these results show that social media can be a cost-effective and useful tool for informing citizens and exposing them to new technologies and available services. Our second contribution, perhaps more relevant for the research community, sheds light into the limitations of experimental design using Facebook and social media, particularly at fine geographical levels.