Introduction

Area-based measures of deprivation, which date back to the 1970s (e.g. Holtermann, 1975), have been used widely in the UK and in other countries to understand the geographies of deprivation, to design policies aimed at promoting regeneration and reducing spatial inequalities, and to target resources to communities in need (Blackman, 2006; McCartney et al., 2023). In addition to their use by central, devolved and local levels of government for these purposes, deprivation measures have also been utilised by research institutions and academics, and third sector and voluntary bodies (see Sect. 1.3 of Noble et al., 2019) to better understand the nature and extent of neighbourhood deprivation, and its causes and consequences. An array of measures of deprivation have been used in (parts of) the UK to capture different forms of deprivation, both material and social. For those measures based on census data, a common approach has been to create composite indices by combining percentages converted to standard scores.

Historically, these indices solely relied upon the use of census data, which was made available on a decennial basis. For example, the Carstairs index was originally created using 1981 Census data in Scotland (Carstairs & Morris, 1989, 1991), and incorporated the following input variables: male unemployment, car access, overcrowding, and social class. An alternative composite, the Townsend index (Townsend et al., 1988), was developed soon after, and incorporated census derived information on: unemployment, car access, overcrowding, and rented households. These were cross-sectional measures of deprivation, applicable to the specific census time point (Norman, 2010). Underpinned by census data but using other sources as appropriate, deprivation indexes have also been developed in Australia, Canada, France, New Zealand, the US and elsewhere (Eroğlu, 2007; Bell et al., 2007; Havard et al., 2008; Pornet et al., 2012; Fu et al., 2015; Norman et al., 2016b).

A recent study by Lloyd et al. (2023a) considered deprivation via a time series of snapshots, but did not assess the ways in which deprivation changed within each neighbourhood across multiple time points. That is, trajectories of deprivation in each area were not captured. Similarly, in the majority of applications of deprivation indices, measures only for some recent timepoint are used in analysing spatial trends, and for the purposes of resource allocation. Such approaches fail to acknowledge that places may change their characteristics and that, sometimes, there may be very different deprivation histories for communities whose deprivation levels may be similar at a single time point. The challenges associated with tackling inequalities in such areas may be quite distinct, and understanding deprivation trajectories is crucial when designing and implementing area-based interventions. The same long-run area deprivation information is needed for understanding outcomes for individuals who have lived in different kinds of places over time (e.g. Jivraj et al., 2020 and Murray et al., 2021).

This paper is part of a Nuffield Foundation project ‘Trajectories of Deprivation in the UK’,Footnote 1 and this element of the research focuses on long-term deprivation trends using data from the censuses of England and Wales, at decadal intervals, from 1971 to 2021 inclusive. We utilise the Townsend index, as this particular census-based measure incorporates sets of variables that can be accessed at each census from 1971 to 2021, thereby enabling an exploration of deprivation trends across a long time period. This has many advantages, including the possibility of charting how areas which were subject to major industrial decline 50 or more years ago have fared in recent periods. Importantly, these explorations are facilitated by converting the geographical units used to report the data to make them consistent over time and using methods of measuring change in deprivation. This study builds on a body of previous research which shows how deprivation has changed in neighbourhoods across the UK (e.g., Norman & Darlington-Pollock, 2017; Norman, 2010, 2016). The end result enables a novel assessment of how deprivation has evolved across England and Wales over a lengthy period of time, and, crucially, how far deprivation has persisted or increased in some localities. We argue that this knowledge provides an important evidence base which should be built into schemes for tackling spatial inequalities. In categorising areas which have seen a decrease in deprivation as distinct from those areas where high deprivation is entrenched, the data outputs from this research have the potential to help shape local area interventions by identifying factors which might be associated with decreasing deprivation.

This paper first steps through an explanation of the measurement of deprivation from our earliest period, 1971, through to 2021 and then provides overview analyses of ‘Changing deprivation 1971 to 2021’. We are emulating the original Townsend scheme but are aware that the input variables are not without their critics in their relevance for different time points and geographical contexts (e.g. differences in deprivation in urban and rural areas). We return to these aspects in the Discussion section.

Measuring Deprivation: 1971 to 2021

Employing the Townsend Index as a measure of deprivation, we can capture change over time for local areas from 1971 to 2021 in England and Wales. The Townsend Index was originally devised using 1981 Census data but has been reproduced (for a variety of geographical small areas) using data from the 1971, 1981, 1991, 2001 and 2011 Censuses (see, for example, Norman, 2010, 2016, 2017; Norman & Darlington-Pollock, 2017). Here we further extend the time-series using the most recent 2021 Census data.

To calculate a measure which tracks change over time in deprivation for small areas requires that:

  • All input variables are available and have sufficiently similar definitions at each census time point;

  • The geographies for which the data were originally released can be made compatible with common spatial units over time; and

  • Once adjusted to consistent geographies, the raw data can be standardised and combined to a composite index so that a change in deprivation score has a meaningful interpretation in terms of improving or worsening deprivation.

The use of census data meets these criteria. We discuss and illustrate these aspects below.

Input Variables to the Townsend Index

The Townsend Index focuses on ‘material’ deprivation which relates to the “lack of goods, services, resources, amenities and physical environment which are customary, or at least widely approved in the society under consideration” (Townsend et al., 1988, p.36). Since these aspects are not directly measurable, four proxy indicators identified from the Census, on unemployment, car ownership, home ownership and household overcrowding, were selected by Townsend and colleagues.

The input variable definitions, and extracts explaining their inclusion from Townsend et al. (1988), are shown in Table 1. The relevant raw data are all available for small areas from the 1971 Census through to the most recent 2021 Census. The numerators and denominators for unemployment are derived from individual responses on the census forms, and for non-home ownership, car ownership and overcrowding from the household returns. The unemployment variable is effectively comparable over time but has minor differences in the age boundaries of economic activity. This is due to a younger school leaving age in 1971 (15, thereafter 16) and an upper limit in the census tables of age 59 (female) and 64 (male) rising to age 74 in more recent censuses. The home ownership and car variables are consistent over time. We appraise differences in the overcrowding variable below.

Table 1 Input variables to the original Townsend Deprivation Index
Full size table

Reproducing the Townsend Index over time necessitates appraisal of the overcrowding measure for 2021. Counting total rooms in the household was required for the 1971, 1981, 1991 and 2001 Censuses. In 2011 there was a question about the number of rooms and an additional question specifically about the number of bedrooms. However, in the 2021 Census, respondents in England and Wales were not asked to count the number of rooms in the household: only the number of bedrooms. Thus, for the latest 2021 Census, it is not possible to calculate the percentage of households with more than one person per room.

The bedroom-based occupancy rating provides a measure of whether a household’s accommodation is overcrowded or under-occupied (ONS, 2023a, 2023b). A measure based on counting the number of bedrooms should be reliable since the census instructions on which rooms to count has varied over time (and between the UK’s constituent countries). A negative occupancy rating (-1 and -2) implies that a household has fewer bedrooms than the standard requirement based on the household’s demographic structure. Note that in 2021 the Valuation Office Agency (VOA) provided estimated counts of rooms by household which is available alongside the census counts. However, we prefer to use the bedroom occupancy measure since the VOA information may be subject to different biases to census respondent supplied information.

To determine the impact of different ‘household overcrowding’ measures for measuring deprivation and change over time using the Townsend Index, the previous ‘more than one persons per room’ variables in 1991, 2001 and 2011 were compared with the bedroom occupancy variables in 2011 and 2021 (for simplicity excluding the 1971 and 1981 data). The appraisal here is for England and Wales, and with all data converted from the previous census geographies to the 2021 Lower Super Output Areas (LSOAs) (see below).

Table 2 shows strong positive correlations across the years and between the differently defined measures indicating that the geography of household overcrowding is persistent, and the measures are sufficiently comparable. The variable names in bold are proposed to be included as overcrowding indicators in the calculation of comparable Townsend Scores over time. In 1991 and 2001 these are the standard ‘more than one persons per room’ definition. In 2011 this is an average of persons per room and the combined negative bedroom occupancies (-1 and -2). In 2021 the variable comprises the negative bedroom occupancies. The rationale for the use of the average in 2011 is to alleviate possible discontinuities but also since this correlates with the earlier and later censuses marginally more strongly than the single variables.

Table 2 Correlations between measures of household overcrowding: England and Wales 1991 to 2021
Full size table

Variables prefixed ‘ppr’ are based on persons per room. Variables prefixed ‘occ’ are based on bedroom occupancy. The variable prefixed ‘pprocc’ is an average of the two definitions available in 2011. The suffixes min1 and min2 imply that a household has -1 and -2 fewer bedrooms than the standard requirement. All correlations are statistically significant (p < 0.05).

Creating a Set of Consistent Geographical Units

Having assessed the comparability of input variables and definitions over time, the issue of geographical unit comparability is next addressed. The geographical units used for the release of census data are different at each time point. This is due to variations in strategies for the geography of data collection and release. The latter is affected by the decisions on threshold counts of population and household and geographic scale with respect to the protection of personal confidentiality (Cockings et al., 2011). Unless data are adjusted to a consistent set of zones, a time-series analysis will be hampered by any boundary changes (Norman et al., 2003).

The analytical units for the Townsend deprivation index over time are the Lower Super Output Areas (LSOAs) used for the release of the 2021 Census in England and Wales. The LSOAs are a statistical geography which has become a scale commonly used in deprivation applications (DCLG, 2015; Norman & Darlington-Pollock, 2017). In 2021 there were 35,672 LSOAs in England and Wales which comprise between 997 and 9,898 persons (mean 1,671) and between 400 and 1980 households (mean 695). We adjust the data from the previous censuses to this geography so that the data and findings are relevant to a contemporary policy setting. We term the units of data release at previous time points as the ‘source’ geography and the 2021 LSOAs the ‘target’ geography. For the reliability of the conversions, we use units which are inherently smaller than the target geography. In 1971, 1981 and 1991 these are termed Enumeration Districts (EDs) but are different zonal systems at each of these time points. For 2001 and 2011 we use the Output Area geography which nested into the LSOAs at each of those time points, but not necessarily in 2021.

The method of conversion is defined in Norman et al. (2003) and used subsequently in Norman (2010, 2016) and in Lloyd et al. (2023a). We use the residential postcode centroid locations to connect the source units to the target zones using GIS point and polygon linkages. Since there will be partial overlaps between the boundary systems in different census years, the postcodes which fall in the intersections are used to apportion the population counts from the source to target geographies. Conceptually, the method is a hybrid of areal interpolation (population distribution proxied by postcode distribution) and dasymetric mapping (postcode presence for where people live). The list of residential (i.e. not business premises) postcodes is defined to be as contemporary as possible with each census using the dates for when the postcode was ‘live’. As detailed in Norman and Riva (2012, p.489), this is feasible back to 1981, but not for 1971, and so the 1981 list is used in that case.

Figure 1 illustrates a geographic conversion scenario for the Roundhay area in Leeds, UK. Figure 1a shows the ‘source’ geography EDs which were used for the dissemination of the 1981 Census. Figure 1c shows the 2021 LSOAs, the target geography to which all data need to be adjusted. The ED and LSOA polygons have effectively no correspondence. Figure 1b illustrates both the postcode distribution (point location symbols weighted by address counts) over an Open Street Map (https://www.openstreetmap.org/) background. The postcode locations are associated with the residential areas, with very few in the area of Roundhay Park.

Fig. 1
figure 1

Linking source and target geographies: examples from Roundhay, Leeds, UK

Full size image

Table 3a has an extract from the GIS spatial join linked postcode locations which includes the source 1981 ED and target 2021 LSOA with which each postcode is associated, along with the number of addresses at each postcode. The number of postcodes and addresses are then summed for the intersections of the ED and LSOA polygons and also summed across the EDs. Dividing the intersection count by the total count in the ED gives the proportion of the raw population data which would be allocated to the target LSOA. Taking the first two rows on Table 3b as an example, for ED 08DAD03, ~ 0.27 of a population count will be apportioned to LSOA E01011650 and ~ 0.73 to LSOA E01011651. Table 3c lists the 1981 EDs which each contribute to E01011650 and those EDs to E01011651. The apportioned ED / LSOA intersection populations would be summed for these LSOAs.

Table 3 Extracts of postcode links and calculation of conversion weights, Roundhay, Leeds, UK
Full size table

Calculating Deprivation Change Over Time

As a composite measure of deprivation, the Townsend Index is the unweighted sum of the standardised (using z scores) percentages of unemployment (natural log, to account for the skewness in unemployment percentages), car ownership, home ownership and household overcrowding (natural log, again to account for skewness). The original version and calculations for other individual census years and geographies are cross-sectional measures of deprivation, applicable to the census year of the input variables. Crucially, a change in index score (or quantile) between censuses cannot be interpreted as an absolute change in deprivation because the cross-section measures are time point specific. Time comparable versions of the Townsend Index were developed in Norman (2010) to assess change between 1991 and 2001 (UK coverage), and later extended in time to cover each census from 1971 to 2011 (GB coverage) (Norman, 2016; Norman & Darlington-Pollock, 2017). An equivalent method has been applied to the Carstairs Deprivation Index for Scotland for 1981 to 2011 (Exeter et al., 2019).

Using the same two LSOAs in Roundhay, Leeds, as used to illustrate geographic data conversion (Fig. 1 / Table 3), Table 4 displays percentages of non-home ownership at each census from 1971 to 2021. For the cross-sectional standardisation to a z score in 1971, the mean across all LSOAs in England and Wales areas (48.44%) is subtracted from the observation (for the first LSOA here, 47.68%) and divided by the standard deviation (SD) for all LSOAs (26.31). The resulting score is -0.03. Negative z scores are for values which are lower than the national (England and Wales) average and positive z scores for those which are higher. E01011650 is very close to the mean. E01011651 at 27.54% is well below the national average, so the resulting z score is further from zero at -0.79. All the other ‘cross-sectional’ z scores are calculated accordingly. The difficulty with these cross-sectional values is that one year cannot be compared with another in a meaningful way. In E01011650, non-home ownership increased between 2001 and 2011 from 14.91% to 19.33%. However, the z scores (-0.79) remained the same because the national average also rose.

Table 4 Calculation of cross-sectional and time comparable z scores, LSOAs in Roundhay, Leeds, UK
Full size table

To overcome this issue, in the ‘Whole Time Period’ part of Table 4, the mean and standard deviation values are for all areas across all time points. The average non-home ownership over the six censuses was 37.77% and the standard deviation was 23.73. These are then used as inputs for the z score calculations with each of the percentages for the individual LSOAs results in scores which are comparable across space and time. For E01011650, in 1971, with non-home ownership above the average for the whole time period, the z score is positive, which then reduces by 1981 as the percentage non-home ownership reduced relative to the national average. Contrary to the counter-intuitive cross-sectional measures, in 2001 and 2011, the z scores for E01011650 move closer to zero as the percentages of non-home ownership rise.

Using this approach, for each LSOA and census year, the percentages of non-home ownership and no car and the log transformed percentages of unemployment and overcrowding have had time comparable z scores calculated across the six censuses. These are then summed, unweighted (as is standard for the Townsend measure), to a final deprivation score. The areas with higher levels of deprivation have scores which are positive, with negative scores representing areas with lower levels of deprivation. Since the scores are comparable over time, if the scores are reducing, then the area is becoming less deprived (and vice versa).

Many applications use the continuous scores categorised into quantiles (see Norman et al., 2023). We categorise the deprivation scores into quintiles such that the cut-offs partition the scores into groups of equal population size. Here LSOAs in quintile 1 are the least deprived areas and those in quintile 5 are the most deprived. Over time if an area changes quintile, then the logical interpretation can be made as to whether areas have become more or less deprived.

Using the same approach as Exeter et al. (2019), in addition to using quintiles, we also cluster areas into groups using k means classification using the z scores of each input variable. This results in five categories of LSOAs representing areas which were over the time period 1971 to 2021 in different deprivation trajectory groups. We provide further details about this classification in the analysis below.

Note that for this time-series of scores, quintiles and trajectory categories, if an area had a population of fewer than 100 people (as in both the original versions of the Townsend and Carstairs indexes), then it is excluded from any calculation for that time point. This may be the case where areas with small or no persons present at previous censuses may subsequently have been developed with residential housing. Some areas may previously have been populated but then the housing has been demolished and then redeveloped so the time series of census data is interrupted.

Changing Deprivation 1971 to 2021

In this section we first provide summaries of the change in Townsend Scores over a 50-year period spanning between1971 to 2021, along with the changes in levels of the four input variables. We then present mapped distributions of deprivation in 1971 and 2021 which are used to highlight spatial trends over time. We present the deprivation trajectories along with the trends of the input variables within each category. Finally, we use the ONS Supergroups Classification of Local Authorities to show how the deprivation trajectories differ across these area types.

Figure 2 shows that, on average, across all LSOAs in England and Wales, deprivation as measured by the composite Townsend Index decreased from 1971 through to 2001, before rising by 2011, and then falling again by 2021. The averages of the input variables do not, however, necessarily all move in the same direction. Across LSOAs in England and Wales, unemployment rises between 1971 and 1981 and remains relatively high in 1991 before falling relatively sharply between 1991 and 2001, and then more steadily in the subsequent two decades. On the contrary, levels of non-home ownership fell dramatically between 1971 and 1991, with little change to 2001, before rising through 2011 and 2021. Both the no car measure and the household overcrowding measure decreased between 1971 and 2011 but, whilst lack of access to a car continues to fall to 2021, percentage overcrowding increases between 2001 and 2021.

Fig. 2
figure 2

Mean Townsend Scores and Input Variables: 1971—2021

Full size image

Figure 3 maps levels of deprivation (Townsend index quintiles) for LSOAs expressed relative to the whole period 1971 to 2021 – thus, the number of LSOAs in each quintile is not the same in the two maps. In 1971, the most deprived areas are concentrated in the major urban centres of, for example, London, Birmingham, Manchester and Liverpool, and in the old industrial and coalfield areas such as south Wales, the North-East and south Yorkshire (Fig. 3b). In 1971, there are relatively deprived areas in both rural and coastal locations. By 2021 (Fig. 3c), underpinned by the changes in input variables noted above, there is an urban – rural contrast in levels of deprivation. In the main urban areas, relatively high deprivation has persisted, but the old industrial and coalfield areas are not as deprived as in 1971. Apart from some pockets of deprivation, more rural and coastal areas are also less deprived. Figure 1A in the Supplementary Materials (SM) presents maps of deprivation for each census year from 1971 to 2021 and therefore the changes at 10-year intervals between the maps in Figs. 3b and 3c.

Fig. 3
figure 3

Townsend Deprivation by Lower Super Output Areas: England and Wales 1971 and 2021. Note: The cartograms in panels b and c are based on the square root of each LSOA’s area. In the cartogram, the size of urban and rural areas are scaled up and down respectively (see Lloyd et al., 2023a p. 465 for more details)

Full size image

To determine the degree to which the geographic distribution of deprivation is similar over time, Table 5 reports the correlations between deprivation scores at each time point. These correlations are strongest between any successive pairs of censuses (greater than 0.8 in most cases) but despite a 50 year time difference, the correlation between deprivation in 1971 and in 2021 is 0.706. These correlations indicate that the spatial distribution of deprivation is largely entrenched (and see Norman et al., 2022).

Table 5 Correlations of Townsend Scores between Censuses 1971 to 2021
Full size table

To also evidence the extent of change in deprivation, Table 6 is a crosstabulation of deprivation quintiles in 1971 and in 2021 which shows counts of LSOAs falling in the specific combination of quintiles for these years. LSOAs which were in the same quintiles of deprivation in both years are recorded on the leading diagonal of the table (e.g. 1,220 LSOAs were in the least deprived quintile 1 in both 1971 and 2021). The 8,786 LSOAs in the same quintile in 1971 and 2021 comprise ~ 25% of the total number of areas (35,306). LSOAs below the diagonal are less deprived in 2021 and are the majority (69%). 6% of the LSOAs fall above the diagonal and are areas which became more deprived over time. This ‘start: end’ crosstabulation, which accounts only for deprivation in 1971 and 2021, does not account for any changes in quintile of deprivation between the intervening censuses.

Table 6 Crosstabulation of Deprivation Quintiles in 1971 and 2021
Full size table

Using the equivalent approach as in Table 6 and producing crosstabulations of quintiles between successive censuses can be useful to present changes in deprivation over time. Yet the crosstabulations between six censuses and the (5 × 5) quintiles of deprivation for each generates a cumbersome amount of data. Similarly to Exeter et al. (2019), we therefore cluster the time series of LSOAs into trajectories of deprivation using k-means classification of the four input variables. These groupings into deprivation trajectories make the trends easier to interpret. This approach accounts for the complex ways in which sets of LSOAs change which are impossible to capture using sets of crosstabulations each comparing two time points.

Figure 4 illustrates the deprivation trajectories of the mean Townsend scores for the five categories. Clearly, there is a range of levels of deprivation over time and, for four of the five categories, change over time is almost in parallel to the national trend in Fig. 2. The least deprived LSOAs are grouped into a ‘Persistently Not Deprived’ cluster. A somewhat ‘Less Deprived’ group started the time period in 1971 on average just above zero (so slightly deprived relative to the six censuses) but, by 1991, was clearly below the average. A group of LSOAs has Townsend scores above zero and is labelled here as ‘Moderately Deprived’ although the level of deprivation decreases over the fifty year period. The highest level of deprivation, though with an improvement, is for a cluster of LSOAs which are ‘Persistently Deprived’. A group of LSOAs has a rather different trend since these are deprived in 1971 and 1981 but below the level of the ‘Moderately Deprived’ group. After 2001, this cluster of LSOAs has increasing levels of deprivation and is thereby labelled a ‘Worsening Deprivation’ trajectory group. This is a relatively small group of LSOAs (2,319; 7%) compared with the others: Persistently Deprived (12%), Moderately Deprived (19%), Less Deprived (29%) and Persistently Not Deprived (34%). Due to areas having populations fewer than 100 in some census years, 847 (2.4%) LSOAs are excluded from the trajectory categories.

Fig. 4
figure 4

Changing deprivation by various trajectories: 1971 – 2021 England and Wales

Full size image

Figure 5 shows the trends in the means of the input variables by the trajectory groups. For unemployment (Fig. 5a), all groups follow similar trends, with 1981 and 1991 being higher than the other census years. The Persistently Deprived and Worsening Deprivation groups have, on average, higher unemployment in 1991 than in 1981. All trajectory categories have levels of non-home ownership (Fig. 5b), in parallel with the national mean across time, apart from the Worsening Deprivation group which sees non-home ownership rise steadily from 1991 onwards. Although there are minor variations in trends for Persistently Deprived and Worsening Deprivation areas between 2001 and 2011, all groups see a steady fall in levels of no car access over time.

Fig. 5
figure 5

Changing input variable trajectories: 1971 – 2021 England and Wales

Full size image

Differences in trends for household overcrowding are more marked. Figure 5d shows that for three categories – Moderately Deprived, Less Deprived and Persistently Not Deprived – there is a flattened U-shape, with mean rates falling from 1971 to 2001 but then rising by 2021 to rates similar to 1981. Although there are also falls in rates of overcrowding for the Persistently Deprived and Worsening Deprivation areas from 1971 to 1991, there are rises in the latter group between 1991 and 2001, to have rates above the former, and for both groups there are then substantial rises in household overcrowding through to 2021.

The mapped distribution of the trajectories (Fig. 6) shows that the Persistently Deprived areas are concentrated within the major urban areas of England and Wales. The areas with Worsening Deprivation trajectories tend to be in more suburban areas (such as the outer London Boroughs), largely driven by the observed increased in overcrowding. The Moderately Deprived areas are mainly located in the former industrial and coalfield areas. Urban coastal areas tend to be in the deprived trajectory, with both coastal and inland rural areas predominantly in the Less Deprived and Persistently Not Deprived categories.

Fig. 6
figure 6

Deprivation Trajectories: LSOAs in England and Wales 1971 to 2021

Full size image

To explore whether the deprivation trajectories vary by different geographical contexts, we utilise the ONS Supergroups Classification of Local Authorities (Fig. 2SM). This particular classification has relevance because it includes types of areas which are likely to relate to different levels of deprivation, e.g. ‘Mining and Manufacturing’ areas and ‘Prospering UK’. For these and the ‘Coastal and Countryside’ and ‘London’ areas, we have noted above observations of different deprivation levels and trends. The Supergroups were based on 2001 Census data and thereby represent a time point part way through our 50 year time-series. Although we are investigating change in deprivation here across a static classification, most areas have reasonable geodemographic stability (McLachlan & Norman, 2021) especially at local authority level.

The mean Townsend Scores across the Supergroups are illustrated in Fig. 7. The ‘London’ categories have the highest levels of deprivation in 2021. Prior to this, deprivation levels within the London Cosmopolitan and London Centre were continuously relatively high compared with other area types. The London Suburbs category has seen a marked rise in deprivation from 2001; previous research has shown that this change was driven by increased overcrowding (Lloyd et al., 2023a), which corresponds with our findings shown in Fig. 5d. The other area types have seen decreasing deprivation since 1971, although all experienced a rise between 2001 and 2011, before improvements by 2021.

Fig. 7
figure 7

ONS Supergroups Classification of Local Authorities, Townsend Scores 1971 to 2021

Full size image

It is important to note that the mean Townsend Scores by Supergroup may mask variation by deprivation trajectories within area types. (Table 1SM shows the distribution of the deprivation trajectories within each Supergroup type.) Therefore, Fig. 8 illustrates the deprivation trajectories by Townsend scores for each of the seven Supergroups. For the three London Supergroups, ‘Centre’ is distinctive by the Persistently Non-Deprived LSOAs increasing throughout the whole 50 year period, whereas for ‘Cosmopolitan’ and ‘Suburbs’ this deprivation category has decreasing Townsend Scores over time. Although there are minor variations, the mean Townsend Scores for the trajectories are very similar in all the other Supergroup types outside of London.

Fig. 8
figure 8

Deprivation trajectories 1971 to 2021 within each ONS Supergroup

Full size image

Discussion

For several decades, the Townsend Index has had widespread use in public health, epidemiology, regeneration and population geography (e.g. Adams et al., 2005; Higgs et al., 1998). In this paper, we have emulated the Townsend Index of ‘material’ deprivation for the six censuses from 1971 to 2021 to create a resource which can be used in a variety of research settings whether, for example, area based analyses of drivers and consequences of change, or for the modelling of outcomes for individuals as influenced by the types of places in which they have been living throughout their lives.

The input variables on unemployment, non-home ownership, lack of a car and household overcrowding, as originally justified by Townsend and colleagues (1988), can be obtained from each census from 1971 to 2021 at small area level and their definitions are fairly consistent. There are minor variations in the age range for which unemployment can be obtained which might affect differences in rates between censuses and comparability of unemployment over time but the overcrowding variable is not available in 2021 consistent with previous definitions. We have adjusted all data from the previous censuses to the 2021 version of the Lower Super Output areas (LSOAs) so that the 50 year deprivation history of areas can have contemporary relevance by enabling direct links to data for the most recent time point. This paper builds on previous work (e.g. Norman, 2016; Norman & Darlington-Pollock, 2017) which calculated deprivation measures which are comparable over time, for the census geographies relevant to previous time points.

On average, areas are less deprived in 2021 than in 1971. However, the trajectories of the input variables and of overall deprivation are not linear. The earlier decades are distinctive in rapid falls in non-home ownership and lack of car access but rises and falls in unemployment. The more recent decades have seen rises in non-home ownership and household overcrowding. Based on shared patterns of change, the LSOAs have been grouped into five clusters of LSOAs with similar deprivation trajectories over time. Four of these clusters have distinctly different levels of (lack of) deprivation but the changes over time for these groups are almost in parallel. The other group has experienced increasing levels of deprivation, particularly after 2001, largely associated with increasing non-home ownership and overcrowding.

Geographically, there has been a shift from a widespread level of deprivation, including in more rural areas in 1971, to being more concentrated in urban areas in the 21st Century. There is a distinctive improvement in deprivation for old industrial and coalfield areas (see Sinnett & Norman, 2024) and although this also appears to be the case for ‘Countryside and Coastal’ areas, visually this appears to be a mixed picture which is consistent with the case up to 2011 (Norman et al., 2022). London stands out for having relatively high levels of deprivation, including a relatively high proportion of LSOAs with a ‘Worsening Deprivation’ trajectory; itself consistent with the ‘suburbanisation of poverty’ (Bailey & Minton, 2018).

Although we are reproducing the scheme as closely as possible, we are aware of pros and cons of the individual input variables and the resulting composite index. For example, unemployment is ubiquitous for its inclusion in deprivation measures (Senior, 2002; Haynes et al., 1996). Non-home ownership may be refined if restricted to residents in social housing (Allik et al., 2020). Rather than being a proxy for income, car ownership might relate to accessibility issues and thereby lead to urban – rural biases (Bertin et al., 2014; Christie & Fone, 2003; Higgs & White, 2000; Martin et al., 2000), and decreases in car ownership could also partly reflect environmentally-motivated trends. For future work, there may be merit in comparing our current outputs with a scheme which excludes the car ownership variable (as in Exeter et al., 2007). Finally, the census itself is a ‘gold standard’ source providing comparable input variables and geographies (with adjustments, as demonstrated in this paper) and for crosstabulated variables which provide greater insights into the characteristics of deprived populations (e.g., see Lloyd et al., 2023a). However, there may still be geographical biases since respondents in more deprived areas may be less likely to respond to the requirement for their information (Norman et al., 2017).

We acknowledge the possibility that the rises in household overcrowding in recent censuses, as reported above, are due to the switch from a persons per room to a bedroom occupancy measure. As a check on this, the number of persons and number of households in each LSOA can be used, as a crude proxy, to calculate an overall ratio of persons per household (PPH). This ratio can then be used to assess population density changes at household level. Figure 9a illustrates persons and households aggregated across the deprivation quintiles and Fig. 9b across the deprivation trajectories. In the ‘moderately deprived’ category and quintiles 4 and 5, there are increases in PPH after 2001 and for the Persistently Deprived and Worsening Deprivation trajectory rises too. For the latter, prior to the increases, there was a stalling in the previous falls in PPH from 1991 to 2001. Although the change of overcrowding measure will have impacted to an extent on the comparability of our measure of deprivation, this persons per household measure also indicates that population densities at a household level are increasing in relatively deprived locations. These findings are consistent with Lloyd and Gleeson (2022), who observed marked increases in overcrowding in outer London between 1971 and 2011 (and subsequently used a persons per dwellings measure), and increases in the percentage of households overcrowded in the majority of regions, and in England as a whole between 2001 and 2011.

Fig. 9
figure 9

Persons per Household: 1971 – 2021 England and Wales

Full size image

As a cross-sectional measure of deprivation, the Townsend Index has to a large extent been superseded in the UK by the Indexes of Multiple Deprivation (IMDs) (Noble et al., 2006). The IMDs include a range of deprivation domains which cover a broader range of topics of contemporary relevance compared with the Townsend scheme. These topics are informed by the use of administrative data on a diverse range of characteristics which are not included in the census (for example, health conditions and crime). Moreover, these administrative sources are available at a much finer temporal granularity (often at least yearly) than is offered by the UK’s decadal census. These indices have been periodically produced, separately for each of the four UK nations (whereas the Townsend scheme can be calculated across the countries).

Despite its various strengths, using the IMD for the measurement of deprivation change over time and concurrently for multiple UK constituent countries is more challenging than for a census-based measure. There are various data types and measures within the published IMD data (e.g. ranks, composites, non-ratio variables), the years of release across the UK are different and the year of publication is not necessarily the same as the input administrative data and the different countries have context relevant indicators within some domains. Along with different years of release, this means that cross-country comparisons are less than ideal (Morelli & Seaman, 2007) although adjustments can be applied (Abel et al., 2016). Despite differences in the detail of the various measures of deprivation, strong correlations between schemes have invariably been found (Ajebon & Norman, 2016; D’Silva & Norman, 2015; Hoare, 2003; Mackenzie et al., 1998; Morris & Carstairs, 1991). ONS (2023a, 2023b) have released a household-based deprivation measure along with the 2021 Census outputs. Households are classified on how many adverse characteristics relating to education, employment, health and housing attributes were present. The 2021 Townsend Scores are positively correlated with the ONS measure (2 or more dimensions present) (correlation coefficient of 0.712) as are each of the inputs: unemployment (0.691), non-home ownership (0.677), no car (0.618) and overcrowding (0.519).

The great advantage of using the Townsend Index to track deprivation over time is that deprivation can be calculated for a consistent local level geography using a comparable measure with variables obtained from the census, the most reliable data source. Some might argue that the slight change in meaning, definition, and specific details of the Townsend input variables leads to geographical biases. However, emulating an index which is known and has a track record of use will allow comparison with any new census-based scheme for assessing changing deprivation over time. For example, for 1971 to 2021, there is potential for a composite of unemployment, public housing, lack of education and low quality employment (though definitions of the latter two aspects are inconsistent) to consider where there are differences. Other potential approaches could follow the geodemographic classification framework of Singleton and Longley (2015), calculating a London only measure, and another measure for the rest of England and Wales.

There are various extensions we are planning for this work. Firstly, we will extend the coverage to include Northern Ireland and Scotland (although there will be the challenge of including the Census which was taken in 2022 in Scotland). We will also explore further ways of grouping areas into trajectories. The classification approach employed here does not explicitly account for time sequences, and it takes a set of four variables for each time point treating them as independent of one another. Future work will assess the use of longitudinal methods which explicitly account for the sequence of the observations (e.g., McNicholas & Murphy, 2010; Patias et al., 2022)). We also need to consider how to analyse areas which had sparse populations at one or more census time points. Although a long-term administrative data based measure cannot be back-dated equivalently in time we will seek to link the very long-term trajectories observed here with separate trajectories constructed using the IMDs and other data sources including benefits claimant count data.

Conclusions

Whether areas become more or less deprived over time has previously been demonstrated to relate to the impact of net-migration on population size (Norman et al., 2016b), to variations in cancer incidence and survival (Basta et al., 2014; Blakey et al., 2014; McNally et al., 2012, 2014a, 2014b; 2015), to mortality (Exeter et al., 2011; Norman et al., 2011) and to environmental inequities (Mitchell & Norman, 2012; Mitchell et al., 2015). The general patterns indicate that, as areas become less deprived over time, good health and other benefits accrue and vice versa. If change in specific places can be temporally tracked, then the effect of a policy such as regeneration can be assessed for its success.

The Townsend Index for a contemporary geography and with a deprivation measure which is comparable over time provides novel insights into long-term deprivation trends. The data resources can be used for specific place-based analyses and so that individual records can be linked to assess how life course trajectories relate to area deprivation trajectories (updating, for example, cohort studies by Jivraj et al., 2021; Murray et al., 2021; Norman et al., 2005). We will extend the work to have a full UK coverage, experiment with further trajectory measures and will also investigate case studies of various local authority districts.