Increasing the take-up of the housing allowance among Swedish pensioners: a field experiment

Abstract

Using a randomized field experiment in the Swedish pension system, we investigate whether receiving an information letter affects the take-up rate of the housing allowance for pensioners. We also investigate whether the framing of the information letter affects take-up. The results show that simple information letters had a significant effect on the application rate and subsequent take-up rate: The baseline application rate in the targeted control population was only 1.4%, while the corresponding rates in the different treatment groups were between 9.9 and 12.1%. However, while the applications in the control group were accepted in almost 3 out of 4 cases, up to 50% of the applications in the treatment group were declined. The lower conditional acceptance rate in the treatment group seems to be largely driven by wealth, which the Pensions Agency cannot observe prior to submission. Information campaigns aimed at increasing benefit take-up therefore need careful design in situations with imperfect targeting.

1 Introduction

A common feature of various means-tested social benefit programs is that the targeted individuals fail to take up their benefits (Currie 2006). Empirical estimates typically indicate low take-up (see, e.g., Blank and Ruggles 1996; U.S. Department of Health & Human Services 2015; Department for Work and Pensions 2008; Riphahn 2001; Gustafsson 2002).¹ Incomplete take-up reduces the effectiveness of social programs and limits the ability of the government to reduce poverty. It is therefore of central policy importance to understand how take-up can be affected. The various factors that might explain why individuals do not apply for the benefits they are entitled to have generally been categorized into lack of information, information costs, transaction costs, stigma and complexity/non-transparency (Currie 2006; Remler et al. 2001). However, little is known about the relative importance of these factors in different parts of the population.

Using a randomized field experiment in the Swedish pension system, we investigate whether information letters with different framing affect the take-up rate of housing allowance for pensioners. Specifically, from a sample of about 95,000 single pensioners (10% of all single individuals in the age group $65+$ in Sweden) with sufficiently low income to potentially qualify for the housing allowance, about 10,000 individuals were randomly selected to receive any of four different information letters.² One of the letters contained basic information about the housing allowance, which was reproduced in the other three letters with a slight addition that relates to the hypothesis we wish to test. One of the framed letters addressed the role of eligibility myths and stigma while the remaining two letters aimed at clarifying the eligibility criteria. An application form was enclosed in all mailings. Therefore, the letters simultaneously lowered both information costs and transaction costs related to applying for the benefit. Individuals who received no information mailing make up the control group.

The housing allowance is a means-tested benefit for pensioners with low income and wealth that pays maximum SEK 5000 (SEK 1 $\approx$ USD 0.11) per month with an average payment of SEK 2400. Previous studies indicate substantial under-utilization of this benefit and the foregone benefits are large. According to the Swedish National Audit Office, half of the non-claiming individuals are estimated to forego SEK 900 per month, an amount that individuals with a monthly pension income of SEK 10,000 typically could qualify for (Riksrevisionen 2013). While only 15% of the current pensioner population receive housing supplement, as many as 30–35% are likely to be eligible.

Our study is closely related to three previous studies in the empirical literature on benefit take-up. From a methodological perspective, our study is inspired by Bhargava and Manoli (2015) who conduct an extensive information intervention aiming to increase the take-up of the US EITC. We distinguish our project from Bhargava and Manoli (2015) by studying take-up among a group of non-working individuals in great economic need with low take-up, namely poor elderly. From a policy perspective, it is important to know whether the effectiveness of information letters varies across individual characteristics like work status and age. Furthermore, as noted by Korpi and Palme (1998), the pensioners are of particular interest since public transfers play a larger role in their income than they do for the working population. The second study is by Matikka and Paukkeri (2016) who analyze the effect on the take-up of the minimum guarantee for pensioners in Finland by sending information on eligibility and an application form to eligible individuals. Our study is similar to the Finnish study in the sense that they also focus on poor elderly. However, in contrast to Matikka and Paukkeri (2016) where the letter recipient status was based on pre-determined pension income status, we randomly assign individuals to different information letters. Furthermore, while Matikka and Paukkeri (2016) study the introduction of a new benefit, we study a benefit that has long been in place. As a result, we do not have to account for potential effects of media coverage and time on take-up.

Our study is also inspired by Guthmuller et al. (2014) who conduct a randomized information experiment to increase the take-up rate of a health-insurance voucher program for the poorest in France. An important distinction between our study and Guthmuller et al. (2014) is that they evaluate the effects of two treatments (varying the subsidy amount and invitations to information meetings) against a basic information letter. In our experiment, we have two control groups: one that received no letter and one that received a basic information letter. This allows us to evaluate both the effects of receiving any information at all as well as the specific contents of such a letter.

The results show that simple information letters had a significant effect on the application rate and subsequent take-up. The baseline application rate in the targeted control population was only 1.4%, while the corresponding rates in the different treatment groups were between 9.9 and 12.1%. The only letter that had a significantly different effect from the basic information letter was the letter that addressed common misconceptions about the benefit’s eligibility criteria. This group had the highest application rate of 12.1% compared to the basic letter of 9.9%. The effects are particularly large for old pensioners (above the age of 80). We also find that the economic impact of the treatment was substantial. Using an IV analysis, where application is instrumented by treatment, we estimate an income effect of approximately SEK 750. This means that the applicants, induced by the treatment (i.e., the compliers), increased their income by roughly 10%.

However, the information treatments also had negative effects in the form of substantially lower conditional acceptance rates. The applications in the control group were accepted in almost 3 out of 4 cases, while up to 50% of the applications in the treatment group were declined. These findings suggest that there is strong positive self-selection among the applicants in the treatment group and that the information letters pushed some individuals to apply even though they were not eligible. The lower conditional acceptance rate in the treatment group seems to be largely driven by wealth, which the Pensions Agency cannot observe prior to submission. However, all the other unobservables—occupational pension, cohabitation status and housing expenses—also seem to contribute to lower eligibility among the treated applicants.

Declined applications are costly both to the individuals in terms of false hope and the effort of applying, but also to taxpayers in general through the administrative costs of processing applications for ineligible applicants. The substantial decrease in conditional acceptance rates among the treated is somewhat surprising since all letters aimed at simplifying the pensioners’ own evaluations of eligibility. We can only speculate about the reasons for this decline in self-selection among the treated. One reason could be the well known fact that Swedes (and Scandinavians in general) report a very high level of trust toward public institutions [see, e.g., Rothstein and Uslaner (2005)]. It is possible that a high level of trust may misfire, so that an individual who is informed by an institution that she may be eligible simply disregards her own knowledge and presumes that the authorities know best. If this, arguably speculative, story is correct, public institutions with high levels of trust need to be extra careful when designing information campaigns aimed at increasing benefit take-up in situations with imprecise targeting. It could also be the case that the information treatments simply lowered the costs of submitting an application to the point where many applied even though they thought that they were not likely eligible.³

The results also indicate that far from all letter recipients applied (the non-application rate is close to 90%). Many probably refrained from applying because they knew that they were not eligible due to any of the aforementioned unobservables. Even though this group did not waste any effort on applying, the misdirected letters may have a cost in terms of bad publicity for the Pensions Agency. Of course, the high non-application rate also implies large direct costs of sending letters to non-eligible people. This adds to the necessity of more accurate assessment of the eligibility of individuals before scaling up these interventions to the broader population of (potentially eligible) non-claimants.

Our finding that information letters increase take-up is consistent with studies that examine the impact on benefit take-up of receiving an information letter compared to receiving no letter at all. Bhargava and Manoli (2015) find large effects on take-up from simple reminders about the EITC. Matikka and Paukkeri (2016) also find that the targeted information letters about a new guarantee pension system (introduced in 2011) significantly increased take-up and prompted pensioners to apply sooner.⁴ To our knowledge, the only related study with similar difficulties in reaching the eligible population is Guthmuller et al. (2014). While the submission rate in their experiment was slightly higher than in our case (16.7%), the conditional acceptance rate was very similar (55.2%).

Regarding the different explanations for why some individuals do not apply for the benefits they are entitled to, we conclude, based on the modest differences between the different letter framings, that the primary explanation for the low take-up rate among poor elderly is low awareness.⁵ An interesting exception, however, is that the conditional acceptance rate for the Rule of Thumb Letter was about 20% higher than that of the Base Letter. Demonstrating the eligibility criteria using simple examples might therefore be the most effective tool to get eligible individuals to apply. This is in line with the finding in Bhargava and Manoli (2015) that simplified information about the benefit had larger effects.

The remainder of the paper is organized as follows. Section 2 provides a background of the Swedish pension system and the housing allowance for pensioners. Section 3 describes the experimental setup, including the sample selection and the contents of the information letters. Section 4 reports the results from the experiment, and Sect. 5 concludes.

2 Housing allowance and take-up

Sweden’s pension system has two main pillars: a universal public pension system and an occupational pension system for workers whose employer is tied to some occupational pension plan. The public pension is the most important source of pension income for most people. The relative importance of the public pension typically decreases with the income level of the individual because of a progressive feature of the public pension (Hagen 2017).⁶

The public pension system has in itself three tiers. The first two tiers are income-related and are referred to as the income pension and the premium pension.⁷ The third tier is called the minimum guarantee and is paid out to pensioners above 65 years of age who have low or no earnings-related pension. As shown in Fig. 3, the reduction is taken in two steps: For low incomes, the minimum guarantee is decreased by the full amount of the earnings-related pension; for higher incomes, the guaranteed pension is decreased by only 48%. Thus, a single pensioner with a monthly earnings-related pension of SEK 11,343 or more received no guaranteed pension in 2016, the year of the experiment.

Another component of basic security in the Swedish pension system is called the housing allowance. The housing allowance is means-tested against income (pension, labor and capital income) and wealth. It is also a function of housing expenditures. If the individual is married or has a cohabiting partner, eligibility depends on the income and wealth situation of the household. The maximum and average benefit level per month is SEK 5000 and SEK 2400, respectively.

Take-up is more common among women than men (18 and 7% of the pensioner population, respectively). This reflects the fact that women have lower pensions on average. Around 80% of those who are entitled to the minimum guarantee are women.

Estimates of the share of non-applicants in the population of potentially eligible pensioners range between 25 and 35% (Riksrevisionen 2013; Pensionsmyndigheten 2019). A survey sent out by Statistics Sweden to potentially eligible non-applicants in 2007 offers some explanations for this low take-up (Försäkringskassan 2007). It showed that around half of the non-applicants did not know about the housing allowance. This suggests that an information letter, regardless of framing, is likely to have a large impact on the submission rate. The survey also showed that one-third of those who knew about the allowance falsely believed that they were automatically disqualified because they were house owners (as opposed to living in an apartment). A significant share also believed they were disqualified because they had too high income and/or wealth. Thus, the framing of an information letter, particularly with respect to the eligibility criteria, may affect submission (and acceptance) rates. The low take-up could also be due to social stigma associated with receiving means-tested benefits, which obviously is difficult to elicit from surveys.

The Pensions Agency is responsible for administrating the housing allowance. This means that they collect the incoming applications and decide whether the allowance should be granted or not. The application form can either be submitted by mail or online and a decision is typically made within 1–2 months (the average waiting time in the experiment is 34 days).

There are two application forms: one for single households and one for two-person households. Applicants are required to provide information on things that affect eligibility (i.e., household members, type of housing, housing costs, assets, debts and income). Before making a decision, the Pensions Agency verifies the reported information by requesting information about the applicant(s) from the relevant banks, mortgage lenders and the Tax Agency. The information is always double-checked against the banks if the reported wealth exceeds SEK 100,000. If reported wealth is below this level, the responsible desk officer at the Pensions Agency has the mandate to make a decision without verifying the information. Recipients should notify the Pensions Agency of any changes in the financial, living or family circumstances that might affect the housing allowance.

3 Experimental design

3.1 Sample

The sample for the field experiment was drawn in May 2016, three months prior to the mailings. The selection criteria for the original sample have been chosen so as to capture elderly who are likely to be eligible for the housing allowance, but for some reason never applied. The individuals in the original sample satisfy the following conditions as of May 2016. First, the pensioners are at least 65 years of age (the housing allowance eligibility age). Second, the pensioners are registered as unmarried (single, divorced or widowed). This restriction is motivated by the fact that more than 90% of the estimated eligible non-receiving population are made up of single households (Riksrevisionen 2013). Third, the pensioners have started to withdraw 100% of their public pension, which is also an eligibility criterion, which amounts to less than SEK 10,833 (average public pension in 2016 was SEK 12,300). This implies that virtually everyone in the sample is eligible for the minimum guarantee.⁸ Finally, the pensioners did not apply for the housing allowance in the two preceding years.

It is important to note that these selection criteria do not guarantee eligibility. There are four main limitations to the data that prevent the Pensions Agency from targeting the truly eligible. First, there is no information in the data on wealth. The size of the housing allowance decreases rather quickly with increasing levels of wealth.⁹ Second, there is no information on pension income from other sources than the public pension system, such as occupational pension. Third, the selection criteria made sure that no individuals had a (living) spouse but cohabitation status is not a priori known to the Pensions Agency. As a result, some individuals in the sample may be disqualified because their cohabiting partner has high income and/or wealth. A fourth limitation is the lack of information on housing expenditures. Thus, it is expected that some (ineligible) letter recipients choose not to apply and that some fraction of submissions are rejected. We get back to the issue of imprecise targeting in more detail in Sect. 4.3.

The resulting sample consisted of 96,481 individuals, which corresponds to about 10% of the Swedish population of single individuals 65 years old or older. We then dropped 969 individuals who were deceased by August 5 from the sample, i.e., roughly a month before the first letter was sent out. From the remaining sample of 95,512 individuals, henceforth referred to as the original sample, 10,013 individuals were randomly selected to receive any of the four different information letters during the fall of 2016 (week 35–week 39).¹⁰ The remaining subjects received no letter and hence make up the control group. Figure 1 provides a timeline of the events in the experiment.

Fig. 1

Timeline. Note: The sample for the field experiment was drawn in May 2016. The Pensions Agency did not send letters to sampled subjects who were deceased before August 5, 2016. Therefore, subjects that deceased before August 5, 2016, were dropped from the sample. Three months after the sampling, during week 35 (August 29–September 4) to week 39 (September 26–October 2) 2016, the four different information letters were sent out. Finally, subjects with a missing value for pension are dropped. Most of these deceased before end of April 2017, while 59 subjects (of which 7 receive a letter) have missing value for pension for an unknown reason

We make two restrictions on the original sample to arrive at our analysis sample. First, 3822 individuals were dropped because information on their public pension was missing. Of these, 3763 had deceased before May 2017, when the Pensions Agency collected information on pension, while 59 individuals had missing information for pension for an unknown reason. Second, 933 subjects were dropped because they submitted an application before the experiment started.¹¹

The final sample, called the main sample, consists of 90,757 individuals. Of these, 9534 received a letter, henceforth referred to as the treatment sample, and the remaining 81,223 are henceforth referred to as the control sample. We include applications that arrived at the Pensions Agency before January 1, 2017, i.e., 4 months after the first set of letters were sent out. A decision is generally taken within 1–2 months after the submission.

3.2 Interventions

During the fall of 2016 (week 35–week 39), the Swedish Pensions Agency sent out letters to the treatment sample.

Four types of letters were sent out, approximately to 2500 subjects each and 2000 letters each week, i.e., about 500 of each treatment each week. The information in the most simple letter, which we refer to as the Base Letter, was reproduced in the other three letters with a minor addition that relates to the hypothesis we wish to test. An English version of each letter is provided in the Appendix. The two application forms were enclosed in each mailing.

The Base Letter informed that many pensioners who might be eligible for the housing allowance had not yet applied for it and that the recipient might be one of those. Furthermore, there was information on an income level for eligibility, where to apply, how the allowance is paid out and what it is. The letter also included a link to an online tool that pensioners can use to get a preliminary check of their eligibility status.¹² The title of the Base Letter was “Have you heard about the housing allowance?”.

The Myths Letter had, in addition to the information provided in the Base Letter, information aiming to correct for four widespread myths about the housing allowance: The letter explained that the allowance does not depend on the type of housing (tenancy, condominium or own property) nor the value of the residence; that those with low income can have a certain wealth and still be eligible for the allowance; that those who share residence with others may be eligible; and that certain life changes that pertain to housing, income or marital status, might affect eligibility. The purpose of the Myths Letter was also to de-stigmatize the take-up of means-tested benefits. The title of this letter was: “280,000 pensioners receive housing allowance today—are you eligible, too?”

Table 1 Descriptive statistics by treatment status

The Rule of Thumb Letter showed, in addition to the information provided in the Base Letter, three examples of the possible level of allowance given three different combinations of income and wealth. The purpose of this letter was to make the eligibility criteria with respect to income and wealth more transparent. Previous studies have shown that a significant share of the eligible non-receiving population have false beliefs about how much income and wealth you may have and still qualify (Riksrevisionen 2013). The rules of thumb might help people overcome the rather complicated eligibility calculation.

Finally, the Table Letter showed nine potential combinations of income and wealth and the resulting housing allowance. Thus, this letter was similar in spirit to the Rule of Thumb Letter except that it included more detailed information on wealth and income criteria (and presented in table form).

3.3 Data and descriptive statistics

The data for the analysis were provided by the Swedish Pensions Agency. For all individuals in the main sample (see Sect. 3.1), we have information on age, gender, marital status, and public pension income. There is additional information for those who actually applied for the housing allowance. This information is obviously related to the eligibility criteria of the housing allowance and include second and third pillar pension income, income from capital, labor and self-employment, cohabitation status and housing costs.

Table 1 reports summary statistics by treatment status. As expected, the majority in the sample are women (around 73%). The average age is 76, but the spread is quite large (10% are 89 years or older). The average monthly public pension is approximately SEK 8500.

We run a number of balancing tests, implemented through a series of regressions, to ensure that the treatment samples are similar across the observed variables. The first hypothesis is that receiving a letter does not have predictive power for the level of covariates. Therefore, for each covariate $x_j$ and using the main sample, we run the following regression

$$\begin{aligned} x_{ji}&= \alpha + \mu _{j} \text{ Letter }_i + \epsilon _{i} , \end{aligned}$$

(1)

and the null hypothesis is that for each j, $H_0: \mu _{j}=0.$ The second hypothesis is that all letters have the same predictive power for the level of covariates. Therefore, for each covariate $x_j$ and using the treated sample, we run the following regression

$$\begin{aligned} x_{ji}&= \alpha + \mu _{j1} \text{ Myths }_i + \mu _{j2} \text{ RoT }_i+ \mu _{j3} \text{ Table }_i +\epsilon _{i}. \end{aligned}$$

(2)

and the null hypothesis is that for each j and k, $H_0:\mu _{jk}=0$.

Columns 1, 3 and 5 in Table 2 show the balancing of receiving a letter, that is, estimates of regression (1). While there are no significant differences between the control and the treatment group with respect to gender and age, the treatment group individuals receive a significantly higher public pension than the individuals in the control group. However, the difference, SEK 49.5, is economically insignificant, or less than 0.6% of the average monthly pension in the control group.¹³ We therefore conclude that these two groups are very similar and that randomization into control and treatment has been successful. When considering the randomization of letters separately, Eq. (2), the significant difference in pension disappears (see columns 2, 4 and 5 in Table 2). Women are, however, more likely to receive the Myths and the Rule of Thumb Letter, but again, the differences are economically insignificant, with women being 75% of those receiving the Myths and Rule of Thumb Letters as compared to 73% in the control sample. We therefore conclude that the randomization has in general been successful.

Table 2 Balancing tests

4 Results

Figure 2a shows the share of pensioners who submitted an application by treatment. There is a clear difference between the control group, on the one hand, and the different treatment groups, on the other hand. While only about 1% in the control group submitted an application during the experiment period, more than 10% of those who received a letter applied. The submission rate is highest among pensioners receiving the Myths Letter, but otherwise very similar.

Figure 2b shows the share of accepted applications conditional on submission, also by treatment. A striking result is that the conditional acceptance rate is much lower in all treatment groups compared to the control group. Thus, the substantial increase in application rate, induced by the treatments, has clearly come at a cost since the letters seem to have caused many ineligible pensioners to apply. We will get back to this issue in Sect. 4.3, in which we analyze the probable causes for ineligibility.

Fig. 2

The share of pensioners with submitted and accepted applications by treatment. Note: a shows the share with submitted applications, and b the share with accepted applications. Each bar represents the share of either the Control group or a treatment group. The treatment groups are subjects who receive the Base, the Myths, the Rule of Thumb (RoT) or the Table letter. The capped spikes show the 95% confidence interval of each share

Table 3 The effect of receiving a letter on application submission

4.1 Response to receiving a letter

In the first specification, we estimate the effect of receiving a letter in contrast to not receiving any letter,

$$\begin{aligned} y_{i}= \alpha + \delta \text{ Letter }_i + \varepsilon _i \end{aligned}$$

(3)

where $y_i$ is indicator for either an application submission or an acceptance of an application for subject i. The indicator variable takes the value 100 if the condition holds and zero otherwise, where the variable for acceptance is coded as 0 if a subject does not submit an application. $\text{ Letter }_i$ is a dummy indicating that a letter was received. The results from estimating specification (3) are shown in Table 3. In the first column, submission is used as the outcome, while acceptance of an application is the outcome in the second and third columns.

Receiving a letter almost sevenfolds the probability of a pensioner submitting an application. The probability increases by 9.3 percentage points, from 1.4% (column 1). Due to the increased submission rate, a letter fivefolds the probability of a pensioner receiving housing allowance. The probability increases by 4.4 percentage points, from 1.0% (column 2). The last column in Table 3 shows that the acceptance, conditional on submission, is significantly lower among subjects who received a letter than in the control group, 50% and 73%, respectively. That is, about half of the treated subjects who submitted an application for housing allowance were not eligible to receive an allowance. Point estimates are stable to the introduction of covariates (see Table 7 in Appendix A3).

Table 4 The effect of receiving a letter and submission on housing allowance

We also investigate the effect of receiving a letter on the benefit amount. The specification we estimate is

$$\begin{aligned} \text{ HA }_i = \alpha + \delta \text{ Letter }_i + \varepsilon _i, \end{aligned}$$

(4)

where HA is the monthly housing allowance amount in Swedish Krona (SEK).

The first column in Table 4 shows that the information intervention increased the average housing allowance by SEK 69. In line with our previous finding that individuals in the treatment group are less likely to have an accepted application, Table 4 also shows that the treatment group applicants on average receive smaller amounts than the control group applicants. Conditional on acceptance (column 3), the control group receives on average SEK 2380, in contrast to the treatment group’s average of SEK 1735. The difference of SEK 645 corresponds to almost 30% of the average housing allowance among the control group recipients.

The rather small effect of receiving a letter on the benefit amount (SEK 69) reflects the fact that only one-tenth of the letter recipients chose to apply. Another interesting outcome is therefore the corresponding effect on those who were induced by the treatment to apply. To investigate this, we run an IV estimate of submission on housing allowance and the ratio of housing allowance to pension, where submission is instrumented with treatment status. That is,

$$\begin{aligned} \text{ HA }_i = \alpha + \delta _{HA}\hat{s}_i + \epsilon _i, \end{aligned}$$

(5a)

where $\hat{s}_i$ is the predicted value of submission from the first stage,

$$\begin{aligned} s_i = \alpha _s + \delta _s \text{ Letter }_i + \epsilon _i. \end{aligned}$$

(5b)

We also use the fraction of housing allowance to public pension as a dependent variable.

The results, displayed in columns 4 and 5 in Table 4, show that those who were induced by the treatment to apply, on average received housing allowance of around SEK 743 (columns 4), or about 10% of the amount they had in public pension (column 5). Note that this is slightly lower than the average housing allowance of SEK 873 among treatment group applicants (column 2). The reason is that those who would have applied even without the treatment receive higher housing allowance than those who are induced to apply by the treatment. Point estimates are again stable to the introduction of covariates (see Table 8 in Appendix A3).

4.2 Response to each letter

We now turn to the specific responses to the different letters. To make the result tables more tractable, we drop the control group and use those who received the Base Letter as the reference group. That is, we separately estimate the effect of receiving the Myths, Rule of Thumb (RoT) and the Table letter, in contrast to receiving the Base Letter,

$$\begin{aligned} y_{i}= \alpha +\delta _1 \text{ Myths }_i + \delta _2 \text{ RoT }_i+ \delta _3 \text{ Table }_i +\varepsilon _i. \end{aligned}$$

(6)

The results from estimating equation (6) are shown in Table 5.

Subjects respond strongest to the Myths Letter and weakest to the Base Letter. The share of submitted applications among pensioners receiving the Myths Letter and the Base Letter is, respectively, 12.1% and 9.9%, and the difference is statistically significant. Furthermore, when compared to the Rule of Thumb and the Table Letters, the treatment effect of the Myths Letter is significantly different at the 5% and the 10% level, respectively (see results from F tests in column 1).

Turning to acceptance, column 2, we see again that the Myths Letter generates the largest response compared to the Base Letter. The average effect on the number of accepted applications is 32% larger among pensioners receiving the Myths Letter than among those receiving the Base Letter, 6.0% and 4.6%, respectively.

Finally, from the last column we see that the acceptance rate conditional on submission is significantly higher among treated subjects receiving the Rule of Thumb Letter compared to the other letters. This suggests that demonstrating the eligibility criteria using simple examples may be an effective tool to get eligible individuals to apply. Point estimates are stable to the introduction of covariates (see Table 9 in Appendix A3).

Table 5 The effect of receiving each letter on application submission

We have also examined whether the letters had a significant effect on take-up within various subgroups, including gender, age and public pension income. Appendix A4 presents the estimated equations and the corresponding estimates. In short, the elderly (80+) and the poor (below median pension) were more likely to apply and submit an accepted application. We show that these effects are more likely driven by age than income. The probability of submitting and getting accepted was highest among females who received the Myths Letter, while among males, the Base Letter had the largest effect on the probability of submission.

4.3 Imperfect targeting

As noted previously, the substantial increase in the application rate, induced by the treatment, clearly came at the cost of a drop in the conditional acceptance rate. In the control group, only one out of four applications was declined, while as much as half of the applications in the treatment group were declined. To understand what disqualifies the treatment group applicants, we compare them to the control group applicants in terms of the covariates that are most crucial for eligibility.

Table 6 shows five sets of results from regressing a treatment dummy (i.e., receiving a letter) on the amount of public pension, indicator for having positive occupational pension income, indicator for having positive wealth, indicator for joint submission with a cohabiting partner and annual housing expenditures, respectively.¹⁴ This comparison is only possible to make for the applicants since individual level information on occupational pension, wealth and cohabitation status is collected by the Pensions Agency upon application.

We first conclude that the higher rejection rate in the treatment group cannot be attributed to differences in public pension. The coefficient for belonging to the treatment group in the first column is insignificant and reflects a very small amount. The second column shows that treatment group applicants are on average more likely to have positive occupational pension income, 44% compared to 38% in the control group. The differences with respect to wealth are even more pronounced, however; the share of applicants with positive wealth is 16 percentage points higher in the treatment group than in the control group. Thus, the lower conditional acceptance rate in the treatment group seems to be largely driven by wealth, a conclusion also reached by the Pensions Agency (communicated through email correspondence).

Table 6 Differences between treatment and control group conditional on submission

A third explanation for the low conditional acceptance rate in the treatment group is that a substantially larger share of treated applicants reported that they had a cohabiting partner. As seen in the second last column of Table 6, 9% of the control group applicants used the application form for cohabiting couples. In the treatment group, the corresponding share is 15%. According to the Pensions Agency, the majority of applicants that jointly submitted an application with a cohabiting partner would be eligible on their own, so it should be the income and/or wealth situation of the partner that disqualify them for housing allowance. Furthermore, the conditional acceptance rates among cohabiting applicants in the treatment and control group are 15% and 42%, respectively. Acceptance rates are therefore generally lower among cohabiting applicants, but the difference is larger in the treatment group.

A final explanation for the higher rejection rate in the treatment group could be differences in housing expenditures. In fact, the last column in Table 6 shows that annual housing expenditures are on average 9% lower among treatment group applicants than among control group applicants (10% when controlling for cohabiting partner).

We want to emphasize two implications of the results that the treatment group applicants have higher rejection rates and lower benefit levels. First, many letters were sent to ineligible individuals. Basing selection only on public pension income and marital status clearly misses important financial information that might disqualify individuals for the housing allowance, such as wealth and actual cohabitation status. Second, many individuals applied despite being ineligible. This could either be due to difficulties in calculating eligibility status and taking the letters as a signal of being eligible (complex benefit criteria) or that the letters decreased the cost of applying, or a combination of both.

5 Discussions and conclusion

This paper reports the results from a randomized field experiment investigating whether receiving an information letter affects the submission and acceptance rate of a means-tested benefit in Sweden, the housing allowance for pensioners. We also investigate whether the framing of the information letter affects take-up. The experiment was carried out in collaboration with the Swedish Pensions Agency

The sample for the field experiment consisted of 96,481 single, low-income pensioners and was drawn in May 2016, three months prior to the mailings. Four different letters were sent out to 10,000 subjects (2500 subjects each), and those who received no letter make up the control group. The information in the most simple letter, the Base Letter, was reproduced in the other three letters with a slight addition about common misconceptions about the benefit (the Myths Letter), three simple examples of the possible level of allowance for different combinations of income and wealth (the Rule of Thumb letter), and nine potential combinations of income and wealth and the resulting housing allowance (the Table Letter). Applications were collected up until December 31, 2016.

The results from the experiment are mixed. On the positive side, the results show that simple information letters had a significant effect on the application rate and subsequent take-up. The baseline application rate in the targeted control population was only 1.4%, while the corresponding rates in the different treatment groups were between 9.9 and 12.1%. The only framed letter that had a significantly different effect from the Base Letter on the probability of submission was the Myths Letter. The unconditional acceptance rates also increased substantially, from around 1% in the control group to about 5% in the treatment groups. The economic consequences of the treatment were substantial. Our analysis shows that those individuals who were induced by the treatment to apply, increased their monthly income by around 10% on average.

However, the information treatments also had negative effects in the form of substantially lower conditional acceptance rates. The applications in the control group were accepted in almost 3 out of 4 cases, while up to 50% of the applications in the treatment group were declined. The reasons for decline seem to be that the applicants in the treatment group had higher occupational pension, higher wealth, lower housing expenditures and were more often cohabiting compared to the applicants in the control group. Even though the letters were supposed to give simple information about the eligibility criteria, they thus seem to have misled some applicants to believe that they were eligible even though they were not. Another interpretation is that the treatments simply lowered the cost of applying to such a low level that many applicants applied even though they were aware that the chance for acceptance was low.

It is reasonable to assume that a policy maker would like to increase take-up among the eligible population but at the same time also reduce the share of declined applications. Declined applications are costly both to the individuals in terms of false hope and the effort of applying, but also to taxpayers in general through the administrative costs of processing applications for ineligible applicants. There are also indirect costs associated with the non-application of eligible individuals because of uncertainty about their eligibility. It is very hard to put a price tag on any of these costs and benefits.

We can thus conclude that, from the perspective of the policy maker, these information interventions could be an effective tool to increase the take-up rate of the housing allowance. Scaling up these interventions to the broader population of (eligible) non-claimants can be done at relatively low cost, e.g., by including the information in the personalized Orange Letter that is sent out yearly by the Pensions Agency to all individuals who are part of the pension system (savers as well as pensioners). However, if this is to be done, it is important that the eligibility of individuals can be assessed properly.

One way of achieving better targeting would be to allow the Pensions Agency to gather more data from other public agencies and banks in order to make a more accurate a priori assessment of the eligibility status. Another option could be to send out this information to pensioners who experience an important life change which typically increases eligibility, such as the loss of a husband or wife. If enhanced precision is not feasible and one still wants to make use of information interventions, then our results suggest that demonstrating the eligibility criteria using simple examples may be the most effective tool to get eligible individuals to apply.

Appendix

1.1 Minimum guarantee in the public pension system

See Fig. 3.

Fig. 3

The relationship between the income pension and the minimum guarantee in the public pension system. Note: Annual pension in price-related base amounts (1 base amount in 2016 = SEK 44,300). The corresponding monthly amounts in SEK 2016 are: SEK 4208 (1.14), SEK 4652 (1.26), SEK 7014 (1.90), SEK 7863 (2.13), SEK 10,054 (2.72) and SEK 11,343 (3.07). The gray-shaded area denotes the size of the minimum guarantee (the vertical distance between the dashed $45^{\circ }$ line and the solid lines)

1.2 Letters

See Figs. 4, 5, 6 and 7.

Fig. 4

The Base Letter

Fig. 5

The Myths Letter

Fig. 6

The Rule of Thumb Letter

Fig. 7

The Table Letter

1.3 Robustness to introduction of additional covariates

To demonstrate robustness, we show that our main results are stable to the inclusion of additional controls. The additional covariates used are a female dummy, age dummies and public pension decile dummies.

Tables 7, 8 and 9 are replications of Tables 3, 4 and 5, respectively, except with the inclusion of the additional covariates. In Tables 7 and 9, the additional covariates have been added to the specifications (3) and (6), respectively. Columns 1 to 3 of Table 8 show results from OLS specification (4) with the additional covariates. Column 4 of Table 8 shows results from IV estimation of specification (5) with the additional covariates in the second stage, and the final column shows results the same IV specification, with housing allowance replaced by the fraction of housing allowance to public pension (P), scaled by 100.

The introduction of additional covariates does not significantly change any of the estimated treatment effects.

Table 7 The effect of receiving a letter on application submission conditional on covariates

Table 8 The effect of receiving a letter and submission on housing allowance conditional on covariates

Table 9 The effect of receiving each letter on application submission

1.4 Heterogeneity of responses

Tables 10 and 11 show the submission and acceptance heterogeneity in response to a letter with respect to gender, age and public pension income. We estimate the following specifications

$$\begin{aligned} \mathbb {P}(y_{i}=1)&= \alpha ^0 +\beta ^0 Z_{ji}+ \delta _1^0 \text{ Letter }_i + \delta _{j2}^0 \text{ Letter }_i Z_{ji} + \varepsilon _i, \end{aligned}$$

(7)

$$\begin{aligned} \mathbb {P}(y_{i}=1)&= \alpha + \varvec{Z}_i \varvec{\beta } + \delta _1 \text{ Letter }_i + \text{ Letter }_i \varvec{Z}_i \varvec{\delta }_2 + \varepsilon _i, \end{aligned}$$

(8)

where $Z_{ji}$, $j=1,2,3$ are the variables of interest; (1) female, (2) being older than 80 years and (3) having public pension above median, and $\varvec{Z}_i$ is a $1\times 3$ vector with the variables of interest. The treatment effect in the base groups ($Z_{ji}=0$ for each j and $\varvec{Z}_i = \varvec{0}$) is captured by $\delta _1$. The parameters of interest are $\delta _{j2}^0$ and the vector $\varvec{\delta }_2$, which tell us each group j’s treatment effect (receiving a letter vs. not receiving any letter at all) in comparison with the base group.

Table 10 The effect of receiving a letter on submission by subgroup

Table 11 The effect of receiving a letter on acceptance by subgroup

In general, older pensioners and those with lower income are more likely to respond to a treatment, while gender plays a smaller role. The relatively old (80+) pensioners receiving a letter are about 46% more likely to respond to a letter than those who are below 80 years old (13.7% and 9.4%, respectively).¹⁵ The $80+$ pensioners correspond to one-third of the individuals in our sample. The corresponding proportion in the Swedish population of single individuals, aged 65 or above, is also one-third.

Pensioners with pension income below median are about 21% more likely to respond to a letter than those that have income below median (12.1% and 10.0%, respectively). However, the larger response of low-income pensioners is possibly driven by them being older. Using simultaneously all the interaction terms mentioned above does not alter the point estimate on old age much, but the point estimate of the effect of having public pension income above median shrinks toward zero, and only the coefficient on old age remains significant (column 4). As seen in Table 11, the heterogeneity in acceptance rate follows a similar pattern, and the coefficient on old age is the only coefficient that remains significant when all interaction terms are simultaneously estimated.

Table 12 shows the results from estimation of specification (6) using subsamples of treated individuals specified above. That is; (1) by gender, (2) by age, namely older than 80 years (Old) and those who are 80 years old and younger (Young), and (3) by pension income, namely, pensioners with pension income above and below median (High and Low).

As seen from columns 1 and 2, females respond more strongly to the Myths Letter than any other letter. Comparing females’ treatment effect to the Myths Letter to other letters, we find a statistically significant difference at the 5% level for all letters except for the Table Letter, which is significant at the 10% level. To the contrary, we find very little treatment heterogeneity among males. When considering the relatively old (columns 5 and 6), the response rate is also highest among those who receive the Myths Letter, but not statistically significant (possibly due to lower statistical power). Furthermore, the response among the older group is lowest for those who receive the Table Letter, and statistically significantly different from the response to the Myths Letter. Similarly young, respond strongest to the Myths Letter, but otherwise they show little treatment heterogeneity. Finally, subjects with high pension income respond strongest to the Myths Letter, while there is little heterogeneity in the response of subjects with low pension. We have also made the corresponding analysis for the acceptance outcome. Similar patterns are revealed from that analysis, but with lower lever of significance (not reported).

Our general conclusion from the heterogeneity analysis is that the framed letters did not differentially affect different groups of pensioners compared to the Base Letter, with the possible exception of the Myths Letter which affected females and those with high income more strongly.

Table 12 The effect of receiving each letter on submission by subgroup