Evidencing Culture for Policy

Abstract

This chapter looks under the bonnet of research, interrogating data and evidence used in social and cultural policy. It looks at data in the culture-well-being relationship in three ways. First, if well-being data can indicate whether policy spend on culture is good for society. Second, a review of two projects that evaluate ‘cultural occupations’ and ‘artistic practice’ in the UK and the US. Despite ostensibly similar approaches with well-being data, different understandings of these categories affect findings. Third, Italian research found ‘cultural access’ was vital to well-being, but its operationalisations are curious in ways that affect the conclusions and recommendations. Understanding well-being data—and the contexts of their use—is critical in appreciating evidence, its limits and uses in social and cultural policy.

7.1 Well-being as Evidence for Social Policy

Now, of course we’ve already got some very strong instincts—even prejudices, sometimes—about what will improve people’s lives, and we act on those instincts … These are instincts we feel to the core, but it’s right that as far as possible we put them to the practical test, so we really know what matters to people. Every day, ministers, officials, people working throughout the public sector make decisions that affect people’s lives, and this is about helping to make sure those government decisions on policy and spending are made in a balanced way, taking account of what really matters. (David Cameron, Prime Minister’s Speech on Wellbeing, 25 November Cameron 2010)

Using well-being data is thought to improve how we understand human progress and development, as we discovered in the first half of this book (particularly Chap. 2). Chapter 6 looked at two further reasons to use well-being data: to evaluate policy decisions that have been made and to predict the impacts of possible policy change. In the case of cultural policy, the common rationale for using well-being data is to argue for more investment or to ‘defend’ (Belfiore 2012) the existing funding and status of the policy sector.

Shortly after the turn of this century, we saw an international commitment to well-being data that has been called ‘the second wave of well-being’ (Bache and Reardon 2013). The UK’s Office for National Statistics (ONS 2011a, 2011b) conducted a national debate so it could understand what people thought should be measured. The above quote is taken from a prime minister’s speech that launched the Measuring National Well-being (MNW) programme and this debate. He talks of having instincts about what matters, but these need to be put to the test. Chapter 6 concluded with how, when the UK began measuring well-being, there was no measure for culture. This was despite the instinct that culture is good for well-being. It was also in spite of advocacy to that effect and efforts to collect more robust data, analyse them better and present compelling evidence.

Various areas of social policy have claimed their contribution to personal or societal well-being to differing degrees over the last 25 years (Oman and Taylor 2018). Notably, these appeals are rarely evaluated on their own terms (Oakley et al. 2013). The previous chapter (Chap. 6) looked at the relationship between culture and well-being because of its reliance on data and because the cultural sector¹ has sought a clear identity through arguing its value to well-being (Oman and Taylor 2018). It also discussed how this policy sector in particular often adopts what has been called a ‘special case’ rhetoric (O’Brien 2013), meaning it argues that it has unique or exceptional qualities. These are enmeshed in claims to the historical traditions of ideas of culture and its relationship to societal well-being (Belfiore and Bennett 2008) that have become naturalised and popularised. In other words, the relationship between culture and well-being seems almost natural, and common sense, whilst also appealing and almost taken for granted.

Alongside these processes of naturalisation and popularisation described in the previous chapter, investment in forms of research to generate well-being evidence for advocacy has also increased (Oman and Taylor 2018; Oman 2020). This form of research is often commissioned to support an argument in policy or political arenas, and we have looked at this as ‘instrumentalisation’ in the area of culture as social policy. This type of commissioned research is common in the UK and is meant to build an argument that a particular activity or service is good for well-being (Oman and Taylor 2018).² However, commissioning research to make evidence to support the value of a service, and therefore maintain its subsidy, affects the relationship between data, researcher and evidence.

How does commissioning research to support the arguments people want to, and need to make, change the nature and role of evidence in different social policy areas? How does this affect overall knowledge of ‘what works for well-being’³ in terms of social policy? Importantly, how does ‘capitalising’ on well-being data affect its capacity to do social good or to be good data? Do the economic value of data and their analysis change the relationship between well-being data and a good society? It is important to ask questions about research that seeks to prove something which is of financial and political value to particular groups.

In this chapter, we will look at some examples of data and evidence used to make specific arguments about the relationship between culture and well-being (the culture–well-being relationship), alongside evidence that might trouble some of the assumptions outlined previously. The examples in this and Chap. 8 are primarily focussed on cultural policy as a form of social policy. These case studies present issues for well-being data, evidence, knowledge and understanding that can be generalised more broadly to other domains of social policy, but focussing on cultural policy as one area makes the contradictions starker.

When you encounter research findings in your day-to-day life, you are most likely to see them in the media. Journalists don’t often have time to sit and read a whole piece of research, and so you are likely to see the reproduction of a headline finding only. Sometimes this is directly from the researcher’s own writing up, and sometimes it is reproduced second hand in others’ summaries. There is an example of this in Sect. 7.4. It is less common to see the inclusion of caveats, methods, limitations and discussions when you see headline findings reproduced in the media, which limits how we understand well-being and data, as we shall go on to discover.

Can you think of a newspaper article you’ve read that says something like ‘Loneliness is killing us’ (e.g. Perry 2014) in its headline, which then moves on to clarify that this is actually not quite the case, the headline exaggerated the research that this article is based on and actually the research itself has many caveats? No, me neither. Media reporting of research is not renowned for this detail. Dramatic headlines are one thing in a newspaper article, where we have a shared understanding—to a degree—of how newspapers report information. Arguably, we have a different expectation when it comes to reading official reports. These can also lack detail on contextual information, caveats and limitations, as we discovered at the end of Chap. 4, with the testing of the ONS4 questions. This not only has a bearing on our understanding, but how we trust how data are reported. Often, it is just convenient to read headlines of research as they are presented to us, even believing they represent a body of evidence. The examples presented in the remainder of this chapter highlight that conclusive answers are difficult to find to questions about the well-being of any particular group of people, and the role of culture—or work—or leisure—in this. Crucially, looking at these examples, or ‘problems’ in detail, and putting them in context, generates additional well-being data to further improve understanding.

7.1.1 Data and Evidence in Cultural Policy

‘Facts about the Arts’ sets out to bring together some of the available statistics on the arts. Anyone who has the temerity to try to do this invites the scorn of those who believe that the concept of the arts itself is elusive and indefinable and any attempt to measure it cannot begin to represent its essential quality. Others, however, believe that the considerable body of material which does already exist can be gathered together and presented in such a way as to lead to a better understanding of the extent to which the arts contribute to the quality of life of the country. Amongst those potential users are Parliament, the media, the general public, and the many who have the power to influence and make decisions about the arts. (Nissel 1983, 1)

Muriel Nissel was a British statistician and civil servant, who collaboratively created ‘a national survey analysing trends in social welfare’ which was to become Social Trends. Social Trends (1st edition 1970) was a significant step in the history of UK statistics, as it symbolised a move away from tracking economic-only concerns to a more general concern with welfare.⁴ Nissel was, therefore, key to the social indicators’ movement, which coincides with what we have been describing as the ‘first wave of well-being’ (Bache and Reardon 2013). Nissel’s quote from her book, ‘Facts About the Arts: A Summary of Available Statistics’ (1983), points towards this imagined clash we have encountered between the arts and data⁵: that they somehow do not go together, and yet must be put together.

Evidence is a contentious idea for those working in or interacting with cultural policy (both narrowly and broadly defined). The idea that the arts and culture have a role to play in improving quality of life is inherent to the identity of cultural policy. We saw this, of course, when the Arts Council of Great Britain was created, as discussed in the previous chapter. This idea of the culture–well-being relationship has then become operationalised in policy, by which we mean, it has been ‘put to use’: in order to advocate for the social purpose and even the social value of the arts; even the value added of ‘culture’ for the well-being of the wider population in various ways. So, cultural policy research will often operationalise this assumed relationship between culture and well-being in terms of value (as social impact) using quantitative evaluations, and we will look at some attempts to do that in this chapter.

Box 7.1 Operationalisation as a Process in Research

Operationalisation in research has a slightly different meaning than in everyday speech. It is the process through which you decide what you are going to measure to understand a concept. Or, more formally, it involves identifying measurable dimensions of a concept.

In this book, the main concept is well-being, of course; but along the way, we have also encountered other concepts, like poverty, social value and in this chapter, of course, culture.

How do you identify measurable dimensions of a concept? This could be designing questions that you can ask survey respondents or identifying data that are already out there (administrative data like hospital admissions are a good example). Measurement is about getting from the questions to the answers.

In some cases, it’s simple: operationalisation could be, for example, deciding that in order to understand ‘hospital capacity’ you will use average A&E waiting times as the measure.

But sometimes it’s intermediate: you might be interested in A&E waiting times overall; or average A&E waiting times for people under 18; or the percentage of people who wait more than four hours; or the longest anyone ever waited in a four-week period.

And sometimes it’s complicated: for example, you may be calculating a scale based on responses to loads of survey questions—where the operationalisation is ‘we’re interested in all of these questions to get at this concept’. Think of something that looks like the PANAS Questionnaire (Fig. 4.3). Instead of lots of different feelings and emotions (as in the PANAS), imagine lots of questions that are more specific, yet similar, about your mood. This could be an operationalisation of ‘anxiety’ or of ‘depression’.

If we want to understand the culture–well-being relationship—as policy, or in social impact—there are a number of ways we might operationalise culture and a number of ways we might operationalise well-being.

In statistics, operationalise, more specifically, would mean we need to find a concept from well-being data that is something measurable (a variable).

The following two chapters will investigate how the idea of a ‘culture–well-being relationship’ has been operationalised in policy, also looking at how it has been operationalised in research that is used to advocate policy decisions. This chapter problematises a number of aspects of the assumed relationship, by reconsidering how these concepts are operationalised in data. It also poses questions about why some data are utilised to reinforce long-held beliefs and values, when other data are readily available, yet are not used. Could it be that they do not allow for such a positive narrative?

Given that the value of culture is promoted for its positive relationship to well-being, and that this is partly to assure policy investment, we begin by looking at the relationship between data that capture changes in government investment in culture, and data that capture change in an aspect of subjective well-being. This exercise has two aims: to review the relationship from a different angle and to demonstrate how data can be found and used on websites that are accessible by everyone. We then look at ideas of being an artist and cultural work and compare two reports that use a similar methodology to analyse data from different countries. Again, this not only reveals something about the relationship between ‘culture’ and ‘well-being’, but also demonstrates how we can interact with research and evidence . Finally, we examine one piece of academic research that looks at ‘cultural access’ (participating in cultural activities) and well-being, to observe how this rendering of the culture–well-being relationship is evidenced in an academic journal article. While far from exhaustive, this chapter takes the key concerns of cultural policy: what gets funded, and to do what; who makes culture; who consumes culture; to look at them all in their own terms.

7.2 Policy Spending on Culture as Good for Society

Wellbeing evidence can help policymakers to assess the impact of arts subsidy on wellbeing inequalities, and thus to ensure that the benefits of this spending are spread to those with lower wellbeing, including disadvantaged and underrepresented groups. (Berry 2014, 36)

The quote above is taken from a 2014 report that was written to the All-Party Parliamentary Group⁶ on Wellbeing Economics. The report addressed what it called ‘four policy areas’ that the authors labelled: Building a high wellbeing economy: Labour market policy; Building high wellbeing places: Planning and transport policy; Building personal resources: Mindfulness in health and education; Valuing what matters: Arts and culture policy. It may strike you that these ‘policy areas’ seem quite different from what we have seen before—particularly in the discussion on well-being indicators and policy domains (see Chap. 3). Putting planning and transport together, for example, and foregrounding work in the economy (rather than just the financial stuff). Of course, here, we will be looking at the fact that ‘arts and culture policy’ is called ‘valuing what matters’—recalling what we talked about in Chap. 6 and those before it, we might want to ask who is valuing what matters—and what matters to who?

On this point of what matters to who, the report advocates assessing and ensuring whether ‘benefits are spread to those with lowest well-being’ (cited above). Framing this statement in this way is interesting, as it seems to acknowledge that well-being (or, how different things impact on well-being) is not experienced universally. Notably, some argue that it is easier to improve the well-being of those with better well-being first (Oakley et al. 2013, 23),⁷ while, of course, the Easterlin paradox implies that it is easier to improve the lives of those who are poorer using money than it is those with higher incomes (see Chap. 4 for this discussion). As you can see, the relationship between money, identifying need and improving well-being is less clear-cut than we may be led to believe.

The report does not explicitly state that policy spend does not evenly impact on people’s well-being, citing evidence, so that it is clear this is a danger we should mitigate against. Instead it says we should assess whether it does. Its recommendations state that government should ‘seek to ensure that the benefits of arts spending reach those with the lowest wellbeing, including communities with high deprivation’. This is an important point that is often glossed over. In cultural policy, it is now acknowledged that the most privileged tend to consume the most culture, they therefore benefit most as a group from the largest subsidies (Neelands et al. 2015; Taylor 2016; Belfiore 2016). The intersection of well-being and inequalities and arts spending is more complex, and one deserving of its own book. However, it seems that investigating policy spending on the arts for well-being is an issue of empirical and moral concern.

7.2.1 Well-being Data and Investment in Culture

For now, let’s look at some well-being data to observe the relationship between culture and well-being. To be specific, we are not going to look at the concept of culture as a whole, or, as is normally investigated, the concept of participating in culture (in some way). Instead we are going to look at the money spent on culture. If advocacy for policy spending on culture is based on its positive impact on well-being, this implies that increased investment in culture is assumed to improve well-being. If this is the case, then this should be visible in some data, right? New Labour claimed a 90% increase in expenditure (in real terms) in its so-called cultural manifesto, ‘Creative Britain’ (in Labour 2010). You would maybe expect to be able to see a relationship between increased investment in cultural infrastructure and improved well-being as a result. You might also expect to see this demonstrated through statistics, whether they come from administrative data or from national-level surveys. Can we see this relationship in the data? How might we check?

We do not necessarily even need to find administrative data to answer the question ‘Did increased spending result in increased well-being?’ We can find sources that tell us about well-being over time and spending over time. The increase in spending is described in a number of other literatures, and specified in some as well, including Hesmondhalgh et al. (2015, 73):

New Labour increased central government grants to local government from £82 billion in 1999 to £173 billion in 2010 (UK Public Spending website). This enabled local government to invest, particularly in ‘cultural infrastructure’ such as refurbished or completely new galleries and concert halls.

So, this means we could use the numbers published elsewhere, and simply consult well-being data, or literature, to see whether the investment identified by Hesmondhalgh and his co-authors affected well-being. However, the reference we have here indicates a credible data source for data on cultural investment, so we can use data from the UK Public Spending website and the data on well-being that would be most appropriate.

Box 7.2 Primary, Secondary and Tertiary Data

Recall from Chap. 3 that…

Primary data are collected by you or a project you are working on. In Chap. 3 we used the example of a questionnaire outside a music event in a local park.

Secondary data refer to data collected by someone else or another organisation that is made available at individual level. They will almost always be either anonymous or de-identified.⁸ They are usually quantitative data but can be qualitative. In Chap. 3, I discussed reanalysing qualitative data from the Measuring National Well-being debate that was collected by the ONS.

Tertiary data consist of summaries of primary or secondary data, often called headline data. If you go to the ONS’ well-being pages (n.d.), you will find headline statistics, so you do not have to do the maths yourself.

Should you want or need to find data yourself, I am sure the idea of it can feel daunting, and for many reasons. I try to tackle the most obvious ones to me in Box 7.3.

Box 7.3 Concerns with Finding Appropriate Data

1) Where to look?: The UK Public Spending website offers figures for year-on-year spending (tertiary data) that is a good place to start. It can be difficult to have faith in your ability to find the right data, but you can always begin by referring to how someone else has gone about it. In our case, we have started with Hesmondhalgh et al. (2015).

2) Suitability: There are various funding streams that subsidise ‘culture’, so what are you looking for?⁹ As you will see in Table 7.2, I chose to use declared total government spend and Grant in Aid to ACE (being one of four arts councils in the UK). That is not to say that this is not complicated, but again, I followed how it was used in the literature and Hesmondhalgh et al. offer detailed descriptions of funding at this time (Hesmondhalgh et al. 2015, pp. 71–75) that can help you decide which is best to use. I used the clearest to me.

3) Availability: The availability of recent historical data that was readily available on websites may have gone through a process of archiving. This changes links and might make it difficult to find the data you have identified as useful from the literature. You can consult the UK government web archive (The National Archives n.d.) if it is government data, or data from a non-departmental government public body like the ONS or ACE. As we have already encountered, back when we were thinking about the role of methodology in data in Chap. 3, there are pros and cons to all data, but administrative data are easy to access and managed by public bodies, with strict guidelines. It is therefore a great place to explore possible relationships and patterns for further research.

4) Assurance: Knowing you have made the right choice can feel impossible. It is not always explicit that many choices are made in even a simple data process, like the one I describe here. The key thing is to know that most choices will have pros and cons and that there are limits to all claims of what can be known with the data and methods used. You just want to be sure to be aware of the limits, and state them when you describe your findings.

I have chosen to consult the ONS for well-being data, as their platform is most familiar to me, and therefore feels easiest to refer to. Going back to the choices we make about which data we choose (Table 3.1), there can be a trade-off between resources (skill, time, money) and robustness. In another situation, you might find other tertiary data more accessible. The data I use here are headline statistics, rather than the whole dataset of every response. Therefore, basic data practices (cleaning and aggregation) have already been done by those who administer the data, for ease of use by the media, government and indeed anyone who is interested. The same is true for the public spending data I have chosen.

As we have previously discovered, Life Satisfaction (LS) is probably the most popular measure of subjective well-being (see Sect. 4.5 for reasons why). While the UK’s Measuring National Well-being (MNW) programme did not officially begin until 2010, the UK had national-level surveys that had a question about life satisfaction for decades. Other national statistics offices, and international statistics bodies, have also administered surveys with life satisfaction questions in. The tertiary data I use here are from the British Household Panel Survey. It followed the same representative sample of individuals—the panel—over a period of years between 1991 and 2009. The same households who took part in BHPS were asked to participate in a larger survey, called Understanding Society.¹⁰ The same questions are asked of participants in the later survey, so data are available for after 2009.

Table 7.1 demonstrates that using data for satisfaction with life overall, as measured by the BHPS, does not show an increase in life satisfaction over time. While this is a somewhat crude attempt to use data that is readily available, it demonstrates that it can be easy to explore a fundamental question quickly and sensibly. In this case, the question might be: ‘if we know that investment in a particular policy initiative or policy domain has increased substantially over time (Hesmondhalgh et al. 2015), how can headline well-being statistics help us understand the influence of investment on well-being?’ As Table 7.1 shows, the increase in funding is not seen in an increase in LS scores.

Table 7.1 Life satisfaction data 2002/2003–2009/2010

There are many limits to what we can know from the data sourced—we know very little of its context in this table, for example, but it tells a clear story. As it was from an ONS summary (for ease), rather than LS data from the UK Data Service, the years represented (2002/2003–2009/2010) are those available and only a subset of New Labour’s time in government exactly (1997–2010). This does not mean they are not useful. The UK government’s changes in funding and policy are unlikely to see an instant impact on a national population’s life satisfaction. There are likely to be lags in effects. However, as noted with the poverty data in Chap. 1, selecting your timeframe can alter the narrative about the effects of government policy, be that life satisfaction or poverty. But we can check. Hesmondhalgh et al. kindly gave us the rest of the data for Grant in Aid to ACE, as follows:

• 1997–1998, £186.60 million

• 1998–1999, £189.95 million

• 1999–2000, £228.25 million

• 2000–2001, £237.155 million

• 2001–2002, £251.455 million

Therefore, the increase in Grant in Aid spending was about the same in the five years that we didn’t include, as in the eight years we did, and it increased quite steadily.

What if we want to ask a more complex question, or see if there is any pattern between well-being and funding? In Table 7.1 we were only exploring one dimension of data: life satisfaction over time. Table 7.2 uses the same LS data points over time with some additional rows to report data on arts funding too. This will let us see a relationship between ‘amount of funding’ from one set of data and the level of life satisfaction over time from another set of data. We can then plot these data over time as a line graph that looks like Fig. 7.1. A positive relationship between increase in funding and life satisfaction over time would see the lines on the graph charting a similar course, so to speak.

Fig. 7.1

Patterns between arts funding and life satisfaction over time. (Total spend data from UK public spending website https://www.ukpublicspending.co.uk/download_multi_year_1997_2010UKb_17c1li111mcn_F0t8nt. Grant in Aid data via Hesmondhalgh et al. 2015)

Table 7.2 Policy spending on the arts and life satisfaction

There is no obvious relationship between policy spend on culture in the data plotted and life satisfaction. Even if we account for the additional five years of data, life satisfaction does not appear to relate to policy spend. Interestingly, LS data from the BHPS from the longer timeframe¹¹ are even less inclined to show a steady increase than our subset. While the easily available data do not have all of the 13 years in which New Labour were in office, you might expect that 8 years’ data would be enough to find a relationship between policy spending on the arts and life satisfaction, if there is one to find.

So, what about the limits of what we can know about the relationship? Figure 7.1 may only report life satisfaction data, but we know some other things about cultural investment, based on the literature presented so far. For one, we have evidence that policy spend on the arts does not reach everyone equally, because not everyone participates in the culture that this money is spent on. This could mean that the way that culture was funded in this timeframe would, therefore, possibly limit potential increases in life satisfaction overall across a whole population. We need to acknowledge that there is a difference between cultural participation and investment in culture.

Let’s quickly return to what we have already learnt about life satisfaction data as a measure of subjective well-being. Firstly, let us consider the question: ‘How dissatisfied or satisfied are you with your life overall?’ This does not capture all aspects of subjective well-being. In fact, if we think back to Chap. 4 on subjective well-being and Table 4.2, with the ONS4 questions, you will remember that life satisfaction falls under one of the three dimensions of subjective well-being: evaluative. Then, it follows that there may be increases in aspects of well-being that were not captured by responses to this question. Life satisfaction data are therefore useful, and may be the most useful according to some (Layard 2006), but still limited in evaluating overall subjective well-being (if we are to follow the accepted reasoning presented so far).

So, we need to acknowledge that there are many limits to knowing the extent to which policy spending in one area can have a clear relationship with life satisfaction, and what that means for the culture–well-being relationship. There are, in fact, numerous limits to any claim that might be made for causation. The life satisfaction data could also include the effects of countless other things happening at the same time which could be counteracting the effect, if, indeed, it existed. Remember the conditions of a good measure of well-being in Chap. 3? It should

be sensitive to important changes in wellbeing and insensitive to spurious ones. In practice, distinguishing between the two is quite a challenge and often relies on judgement based on a priori expectations. (Dolan and Metcalfe 2012, 411)

Clearly, the process I have described is not seeking a metric. All I have done here is describe the data easily available to look for a relationship between arts funding and LS. Therefore, no attempts have been made to account for confounders (which we will come to in others’ research later). There are so many variables that might affect life satisfaction in a way that would be captured by life satisfaction data, that it is extremely difficult to pinpoint the impact of one aspect only in this descriptive way. People who analyse data, rather than simply describe it, will use a theory or hypothesis about pathways that shape well-being to help them create models that do this work. We will return to this in Chap. 8.

Life satisfaction is a very influential measure that we have encountered numerous times in this book. We have been measuring it for years, as it was in the first wave of well-being indicators (see Chap. 2). Realising that life satisfaction had not changed as expected with income over the years, resulted in Easterlin’s paradox that was influential in the second wave of well-being as happiness economics (Easterlin 1973; Chap. 4). Life satisfaction is also measured using Big Data technologies (Chap. 5) and is thought to be the measure of subjective well-being that people most readily understand (Chap. 4). Crucially, because questions about satisfaction with life (although worded slightly differently) have appeared in numerous surveys, and for decades, we have a lot of life satisfaction data to make simple comparisons over time, as we have just seen. LS can also be used to show very powerful relationships to outcomes of well-being, such as suicide rate and the familiarity of LS, together with the prevalence of the data, make it useful for simple exercises, as we have attempted here.

We’ve briefly looked at ways that the relationship between different variables (different policy spend data and life satisfaction) can be plotted. This will hopefully make it a bit easier for you to engage with similar representations in future. This section also demonstrated that it is quite easy to play with data that are publicly available. You can download the data into a table, like those featured, and use a simple function in Excel to plot line graphs to look for relationships over time.

Of course, there is another key point to this section, really, and that is to problematise the assumption that the arts and culture are a priority for policy spending if you want to improve well-being (Berry 2014). If you look at historic well-being data that coincide with previous increases in policy spend, you cannot find patterns in the data that prove that this relationship exists. There are many limitations to the claims that can be made with these data. The increase in arts funding coincides with a more general increase in public spending overall, therefore it is hard to disaggregate policy spend from other things that may affect life satisfaction in this time. Another issue is that life satisfaction data only capture one aspect of well-being. I’m sure you have thought of other limits, as well. What is key is that while using data in this way may not prove anything, sometimes exploring data can be good enough reasons to ask questions—remember this is what Easterlin did when he found that life satisfaction did not have the relationship to income that had been long-assumed in the data he had. This is said to have changed well-being research forever—even if people still argue about it. Sometimes data help us question the status quo in productive ways. They are not only there to help certain people answer certain questions. 

7.2.2 Policy Decisions and Investments Using Well-being Data

Lord Richard Layard (the Happiness Tsar from Chap. 4) has previously stated that ‘policy is not going to be framed around [well-being] for decades, but unless you have the index you’ll never get to a point where you can influence things’ (Rustin 2012).¹² This is a far more measured take on well-being data and evidence being used for policy-making than suggested by the prime minister’s speech that opens this chapter. Lord Gus O’Donnell, another major advocate for well-being in policy-making, is also an economist and an extremely influential civil servant.¹³ He explained that same year:

We now know much more about what drives the wellbeing of people and communities than we did 10 years ago, and our knowledge and understanding is set to increase significantly over the next few years. (O’Donnell in Legatum Institute 2012)

As recognised by the OECD and the ONS early on in their programmes to measure well-being (see Chap. 3), there was a general acknowledgement at this time that well-being measures were evolving and exploratory. So, while a simple visualisation of how life satisfaction over time might interact with arts funding or suicide rates, not all well-being measurements are equally robust, and all have limits that are not often made clear when data are expressed. This is also the case when the concept of well-being is operationalised with another concept, such as culture.

Well-being valuations are far more complex than the way tertiary or headline data were ‘described’ in the previous section’s simple line graphs. As we discovered at the end of Chap. 6, demands from and on government departments to evaluate the impact of their decisions, evolved from the descriptive to more complicated modelling in the 2010s. These models can analyse primary or secondary data and enable a more sophisticated reading of the data. A model helps researchers understand far more complex relationships, including what might be interfering with our understanding (confounders). It can also express a relationship between two things, such as culture and well-being, in monetary terms. We will look at an example of well-being valuation modelling, and how complex this is, in greater detail in the next chapter.

Box 7.4 What Is a Model?

Earlier in this book, I stated that data don’t just fall from the sky as facts. Neither do the models that analyse them. A model will probably contain assumptions about how concepts like ‘well-being’ and ‘culture’ are associated.

There are two main kinds of models: exploratory and confirmatory .

Exploratory models

These allow you to try numerous variables that may be associated, and see what emerges as of possible interest. In other words, you are exploring the possibilities of the data. Developments in machine learning have sped up this kind of exploratory modelling with Big Data, as we discovered in Chap. 5.

Confirmatory models

Most of the chapters in this book refer to work that aims to confirm a hypothesis. Statisticians and others who model quantitative data in this way don’t just throw a bunch of variables into a model and hope for the best. Their models are designed with a theoretical foundation and that will most likely be arrived at from what we already know from previous studies about how one thing (say income) affects another, well-being, for example.

Before a good confirmatory model is designed, it is important to establish ‘what counts’ in the issues you are considering, and how things are expected to fit together.

In exploratory analysis, you won’t need to guess how concepts fit together (although you might have an inkling), and won’t need the same level of attention to the variables you pick in relation to the concepts.

An example of what a model does

A simple model might be based on the hypothesis of a positive correlation. Say, between the average wealth of a nation and its average happiness (as with Easterlin). Imperfect measures tend to be used that represent far more complex concepts like wealth and happiness. For example, variables for life satisfaction and income will not tell us all we need to know about wealth or well-being. Also, resources dictate that it is unlikely a researcher will examine the entire path between income and well-being; instead it will examine whether the two measured concepts (variables) have a statistical association.

It is likely that the relationships examined in any one study represent only small parts of a larger theory. This is not unusual, but is it always explicit when research is presented?

As Chap. 6 describes, government departments including DCMS were indeed looking at how to use well-being data in valuations¹⁴ before Cameron’s speech in November 2010 from the official launch. This is because DCMS and the areas it funds were addressing HMT’s preference for valuation techniques (see O’Brien 2010). There are a couple of approaches that have been called well-being valuation. Fujiwara’s (2013) seems the most influential in the UK, but other examples (Sidney et al. 2017) called well-being valuation take a different approach. Following the increase in using subjective well-being data to value the impact of services, there has been a growing number of studies investigating the impact of the arts or specific cultural organisations in this way (such as Fujiwara et al. 2014a, 2014b and Fujiwara 2013 that we look at in the next chapter). These studies use responses to subjective well-being questions in national-level surveys, together with data on, say, theatre attendance, and estimate the impact of that artform. Such valuations assess data which can tell you that people who go to the theatre are more or less likely to have answered subjective well-being questions in a particular way. The magic is in the modelling.

Important questions remain, however, when it comes to the limits of the data and the extent that valuations can advise policy; particularly when it comes to stating one thing is more valuable than another. The practice of ordering the value of one thing over another does not seem to be presenting us with findings that corroborate each other. In one study, one artform is more important than another. As we saw in Chap. 4, ‘excessive TV watching’ is pitted against an unspecified amount of gardening when reporting on data collected to understand how people are spending their time in lockdown and measuring their well-being (Bu et al. 2020; Mak et al. 2020; Nuffield 2021). Bias is brought to the data, which means they can be read in ways that confirm prior beliefs about what is an excessive amount in one area, but not necessary to measure about another. Consequently, this bias will feed into the presentation of findings and shape recommendations to decision-makers. In other words, ostensibly rational, neutral decisions which are supposedly made on the basis of well-being data are in danger of reproducing prior judgements and beliefs of the researchers—especially if they confirm those of the policy-maker reading the recommendations.

7.3 Well-being Data and Cultural Practice

So, we know that culture is a tricky word to define and can be measured in different ways; we know the same is true of well-being. We have looked at how we might need to think about how the concept of well-being is ‘operationalised’. This is, of course, also true of culture, and the previous chapter spent some time covering different meanings and uses of culture.

What is it about culture that is being measured? This is, therefore, another question to think about, when we are trying to understand the relationship between culture and well-being. It is just as true whether we are reading the research of others, or, indeed, trying to design our own. Is it the specific activities that make culture? Different cultures? The culture wars? If it is measuring the activities that make up ‘culture’ (however defined), is it people who do things themselves or watch others? That is, are you producing culture (i.e. making art) or consuming it (i.e. watching Netflix)? Are you an artist or another kind of cultural practitioner who makes culture as their profession? Or a painter or singer in your spare time? Does singing along to the radio count the same way that being a member of a church choir does? Is it about participating with people? Does watching other people sing (because you are an audience member with people) count as participating in culture, just through watching? If so, does it make a difference if you watch it digitally—and with family or alone? What about the evidence we have seen that being outside seems to increase the relationship between different activities and well-being (MacKerron and Mourato 2013)? Should that mean that all outside arts get more money because they will have extra well-being value?

All these ways of thinking about what you might want to measure about culture for society or people actually involve quite different experiences for people. In this language of well-being valuation and data, you might find someone saying that how you operationalise culture matters for well-being effects. If you measure going to pubs or restaurants, how can you be sure that this is not a proxy for disposable income, leisure time or spending time with friends? The following might be the questions you might want to ask for cultural and social policy:

• What are you doing?

• Who are you doing it with?

• Where are you doing it?

• How long are you doing it for?

• How often are you doing it?

• How long do we expect an effect to last?

• How big should that effect be to count as impacting on well-being?

We’re not going to go into arguments for what the most important aspect of cultural participation is. We have touched on these debates in Chap. 6 and acknowledged they are comprehensively covered elsewhere. Instead, as this book is about well-being data, we are going to look at how data can help answer certain questions, and what the limits to these are. We are going to compare how two different research projects answered a question about being an artist or having a creative occupation, and how that might be related to well-being.

7.3.1 Being an Artist and Well-being

For those of you who didn’t watch Disney-Pixar’s Soul at Christmas in 2020 (and again for those of you who didn’t watch it, I’ll try to not spoil it), the film places a lot of emphasis on the meaning of music for the main character, Joe Gardner. He sees music—specifically jazz—as his purpose in life. The cruel twist is that, just as Joe gets his big break, and is on the cusp of being able to make music—in a real band—not just as an elementary school teacher, this big break is jeopardised. Ironically, it is the sheer joy at his big break that leads to this twist of fate. The unfairness of Joe not getting to fulfil his potential keeps us rooting for him through a meandering journey of self-discovery. Much of the journey is watching him strive to get back to where he was, so that he is able to enjoy that big break.

The over-riding feeling for most of the movie is that, for Joe, ‘making it’ in music is what will make his life worthwhile. The movie goes some way to explain the moment of getting lost in music, something that positive psychologists have described as ‘flow’,¹⁵ but which the movie describes as ‘in the zone’. You watch Joe reflect on what he thinks amounts to his meaningless existence, like the existential philosophers before him. There is also a moment where you watch Joe, sitting on a New York sidewalk, feel the sun on his face and wonder at a helicopter seed spiralling from a tree. This—‘being in the moment’—differs from flow. In flow, you are lost in your thoughts, in an activity, whereas being in the moment is about being present in your body, and is what mindfulness practice is based on. This Disney movie better describes some of the complex theoretical imaginings of well-being than thousands of years of philosophers we’ve come across before in this book—possibly this is of no surprise?

The drive to be able to do something creative as a job—and in the way you want—is not just the stuff of Disney films. In fact, being an artist of sorts has long been seen as desirable and holds much symbolic value.¹⁶ Idealised representations of creative and cultural jobs include creativity and expression, autonomy and passion—or doing something you love. The realities are often far harsher: with independence comes precarity of employment; there are inequalities in opportunities to ‘do what you love’. Often people end up working for money doing something associated to their creative practice—like our main character Joe being a music teacher, while awaiting his big break. Also, the rarity of opportunity to do what you love, and to be expressive and creative, often means you are expected to put up with being treated badly, or indeed to work for free, which is not an option for all.¹⁷

In short, the idea of being an artist is an ideal and the reality of creative occupations is quite different. While quality work is seen as important for well-being (What Works Wellbeing 2017), the actual quality of creative work and the anxieties that accompany the lifestyle necessary of such occupations make it an interesting case for well-being research. The idea of creative work or being an artist is filled with contradictions that deserve attention, and yet the well-being of ‘creatives’ and artists is less frequently looked at than you may imagine (as the publications we are about to look at tell us).

7.3.2 Two Reports on the Relationship Between Being an Artist or Working in a Creative Occupation and Well-being

The two reports we will turn to were published in subsequent years. Their titles and their named approaches suggest that they both contain findings from research using similar methods to answer a similar research question about the well-being of ‘creatives’. This enables us to see how ‘culture’ can be operationalised as being and working as an artist, and how this can relate to well-being. It also continues to allow us to familiarise ourselves with looking at others’ research as it appears in reports, and to think more about what might be happening under the bonnet.

Report 1, Artful Living : Examining the Relationship Between Artistic Practice and Subjective Wellbeing Across Three National Surveys was funded by the National Endowment for the Arts in the US (Tepper et al. 2014). The research looked at different cohorts of arts practitioners and graduates in the US, using three different surveys.¹⁸ Contrary to the received wisdom that music and the performing arts are associated with the largest increases in well-being (e.g. Fujiwara and MacKerron 2015), Tepper et al. found that fine arts and crafts consistently related to higher well-being; music did so for some groups and not others; and participating in theatre ‘seemed unrelated to wellbeing’¹⁹ in the data they had on arts practitioners and graduates (Tepper et al. 2014, 7). Overall, the authors say that there was ‘strong support’ that what they call ‘artistic practice’ is associated with higher life satisfaction and lower anxiety, as aspects of subjective well-being.

Report 2, Creative Occupations and Subjective Wellbeing is a working paper for NESTA, a UK Thinktank. This study used data from the UK’s Annual Population Survey (APS).²⁰ This research concurs with Tepper et al. (2014) that creative occupations are associated with higher than average life satisfaction , worthwhileness and happiness, ‘although most creative occupations also have higher than average levels of anxiety’ (Fujiwara et al. 2015, 1). This is contrary to Tepper et al.’s findings on anxiety from their data, but is corroborated by a number of studies, including the recent book, Can Music Make You Sick? (Gross and Musgrave 2020).

We are going to break down the ways that these studies may seem similar, yet differ. Both Tepper et al. and Fujiwara et al. use multiple regression of cross-sectional ‘national survey’ data that ask subjective well-being questions from people with an artistic practice in the case of the US or a creative occupation in the case of the UK. This means that these data include variables based on questions asked by the organisations who administer the survey; the named researchers (or authors) don’t ask these questions of the participants themselves. Some of the datasets used include creative practitioners and people who are not creative practitioners. This is fairly common, and the researchers simply distinguish which cases (people in the data) meet the criteria of their research question, meaning they analyse the people who have a creative occupation/artistic practice and remove those who do not form from the model.

Box 7.5 Multiple Regression and Cross-Sectional Data

What is multiple regression of cross-sectional data?

Let’s look at these separately.

Regression analysis is common in statistical analyses. It involves estimating the relationship between a dependent variable and one or more independent variables.

In an analysis (e.g. a regression) you distinguish between

(1) Independent variables: that can take different values. You use an independent variable to predict the dependent variable. That is why it is sometimes called a predictor variable.

(2) Dependent variables: that can take different values. When you are measuring your relationship, you are interested in how the dependent variable is affected by the independent variable. It is, therefore, sometimes called the outcome variable to reflect this.

Say you are interested in private music tuition in childhood and creative occupations. You are not expecting an adult professional occupation to retrospectively generate experience of music lessons, but you might want to understand if the opportunity of private tuition affected a later career. So, occupation would be your dependent variable, and music tuition in childhood would be your independent variable.

So, we are still interested in private music tuition in childhood and creative occupations. We have established we want to understand how the first affects the second (and not the other way around). You might decide on other things that you think predict being a creative, such as gender, which previous research may suggest affects the likelihood of entering a creative occupation. Therefore, you would bear this in mind as another possible independent variable.

This is what makes it ‘multiple’ because we have now got more than one independent variable to predict our dependent variable.

A regression to explain how many people work in creative occupations could be conducted with either cross-sectional or longitudinal data.

Cross-sectional data are collected from a survey from a specific point in time, or time period. The same survey questions can be repeated, but these questions will have been asked from different people.

Longitudinal data hold information on the same people over time. This means you can ask the same questions, year on year, to see change over time. For example, you can ask people year on year if they have private music lessons. You can also have data for different questions. This is useful for our example, as we might have data on private music tuition in childhood, and data on occupation in adulthood, should the participant be around that long.

DCMS’ Taking Part Survey (TPS) has a longitudinal component²¹ and a cross-sectional one.

Since 2005/2006, TPS has been run on a cross-sectional basis that involves a new sample of households, which is drawn annually, and a new group of respondents who are asked the same questions. This enables researchers who use this data to say ‘last year X% of the population had music lessons’. But it cannot, therefore account for change that happens to an individual, so you won’t know that ‘the people who stopped music lessons last year are like abc’. Given that change implies impact, this is a big deal for many of the studies we encounter in this book.

The two research projects on well-being from the US and UK that we are exploring use different samples and surveys. This means that in both studies the group in ‘creative occupations’ may not necessarily map onto those with an ‘artistic practice’ as neatly as the labels used suggest. We come back to this in the next paragraph. The UK report uses the Annual Population Survey, which contains information on people’s occupation and the ‘ONS4’ questions that we keep encountering. Creative occupations were defined using DCMS’ Creative Industries Economic Estimates (DCMS 2011) and then coded using the ONS’s standard occupational classifications, called SOC codes (ONS 2010b).²² The authors are therefore able to look at the four ONS measures: life satisfaction , worthwhileness, happiness and anxiety for the 30 creative occupations as defined by the DCMS (2011) in Table 7.3.

Table 7.3 Occupations in the creative industries

There are many discussions over what counts as a creative occupation using these classifications that we won’t get too caught up in here.²³ However, when you imagine a town planning officer, they probably feel quite different to you from a musician. Also, realistically, the day-to-day duties of one is likely to feel very different than the other. A town planning officer will probably have more regular hours and a more secure contract than a cellist. You might also imagine that a cellist may have more capacity for self-expression, and feeling, well, artistic, than a town planner. The differences in day-to-day tasks, security, income and so on are all important external factors that will affect well-being. Therefore, these discrepancies across creative occupations (some of which may not feel that creative) may limit improved understandings of the impact these professions have on well-being, if the model treats everyone with a job defined as ‘creative’ (using occupational codes) as equivalent. What is key here is that it is that the categories used to break down the data (from the APS), and how they have been coded into professions (using the ONS’ occupational classifications) is important context to knowing what we can understand about differences in well-being.

In contrast, the US case uses data from three surveys which target different groups. The Strategic National Arts Alumni Project (SNAAP) captures data about graduates of arts institutions. The Double Major Student Survey focusses on undergraduates who have two majors from four comprehensive institutions and five liberal arts colleges. The DDB Needham Life Style Survey (DDB) is the nation’s largest and longest running annual survey of consumer attitudes. The report states that the researchers ‘look specifically at responses to creative practice, life satisfaction, and “sense of control” in one’s life’, but it is not precisely clear whether they identified ‘creatives’ or looked at everyone who answered these questions. The participants across these surveys are classified as ‘having an artistic practice’ for different reasons. In fact, most of the secondary data analysis is of responses regarding how people do cultural activities in their spare time.

Crucially, and confusingly, the participants across the three surveys do not all actually have an ‘artistic practice’, in a professional sense. In fact, the authors ‘use the terms artistic practice, creative engagement, and creative practice interchangeably throughout this report’ (Tepper et al. 2014, 8). So, there is no analysis of the relationship between well-being and creative occupations, per se, or necessarily any differentiation between a professional artist or an amateur who ‘engages’ in artistic practice. Similarly, the questions used to establish aspects of subjective well-being are not the same across each survey. Table 7.4 shows the subjective well-being questions and how the ‘artists’ were identified across the three US surveys, alongside the UK case. Therefore, establishing what counts as ‘an artistic practice’ is one of the issues, and the other is establishing how subjective well-being is understood. There are therefore key differences in how these concepts were operationalised in these reports.

Table 7.4 A comparison of culture and well-being questions across the four surveys used in the two case studies

Table 7.4 is populated with text that has largely been cut and pasted from the two reports. It contains contextual information on the nature and purpose of the surveys used (you will see that in most cases the surveys have different aims) and the wording of the questions. I have attempted to categorise the US study into Evaluative, Experience, Eudaimonic, as per the categories in Chap. 4 and Table 4.1.²⁴ This was easy for the ONS4 from the UK case, as these have been categorised for us already. The US case proved more difficult. The question about what Tepper et al. call ‘positive self-image’, while not unrelated to well-being and anxiety, fell less neatly into our categories, as designated by Dolan et al. (2011a, 2011b), the ONS or those recommended by the OECD (OECD 2013; Smith and Exton 2013).

‘So what?’ you may ask. Well, these two reports came out in subsequent years and with titles that imply they are researching the same relationship between culture and well-being. They may appear to have used a similar approach, listed as multiple regressions of cross-sectional data. However, there are key differences in the data they investigate. 1, they report on different countries; 2, one uses three data sources, the other uses one; 3, their operationalisation of the ‘cultural occupation/artistic practice’ variables are very different; 4, as are the operationalisations of subjective well-being; 5, those running the regressions (the modellers) used slightly different controls (see Table 7.5). There are numerous reasons for these differences, but mainly, remember that theories of what is good for well-being are not entirely universal, which will affect what someone wants to control for, but also the data are different, which will limit what it is possible to control for.

Table 7.5 Controls used in the two studies looking at well-being and creatives

Box 7.6 Control Variables

Controls are control variables

Say there was a positive relationship between older people and enjoying jazz music, and a negative relationship between younger people and enjoying jazz music. A study to see if there is an association between increasing funding for jazz music and enjoyment of jazz music may find no significant difference. The differences by age would be masked because the negative (younger people) relationship and the positive (older people) relationship could cancel each other out, resulting in no overall observable relationship.

Controlling for age can better establish that ‘funding jazz is likely to have a positive effect on enjoyment in older people, but not younger people’.

When look back at Table 7.4, the survey questions generating the various forms of subjective well-being data are different. They do not use the same concepts of subjective well-being and the questions are not identically worded. The samples of creative practitioners appear to overlap conceptually at first, but they are far from identical. Therefore, we are not actually really looking at the relationship between identical things. Creative occupation or artistic practice do not strictly mean having a job that is creative in these studies, and the meanings and measures of subjective well-being are different in the data analysed.

Again, ‘so what?’ you may ask. Looking at the headline evidence together is the most typical way of understanding other people’s data analysis and findings to construct a body of evidence. Taking a moment to compare these two reports highlights how different two studies which may appear comparable really are, as well as the difficulties in finding conclusive answers to questions about the well-being of any particular group of people, and the role of culture—or work—or leisure—in this. Looking at differences in data sources, concepts, methodology, findings and motivations provides extra data that help establish how conclusions and headline findings may have been arrived at.

The studies differ in numerous ways: the questions asked, who was asked (or included), the nature of the sample—as well as the interpretation of what being creative involves. Furthermore, the research designs were analysing different subjective experience contexts: different places, and different relationships to creative cultural engagement (e.g. professional or amateur). The two reports were also commissioned by different organisations in different countries with undoubtedly different research agendas. Therefore, while in principle, these two studies are looking at the same social issue in the same ways, they have different research questions that are applied to different contexts.

While the two studies were not designed to test each other, the two headline findings can be used together in a literature or evidence review to make a statement about what is known about being a ‘creative practitioner’ well-being. Notably, the UK case states: ‘[t]o our knowledge this is the first quantitative study that specifically analyases the connection between creative jobs and wellbeing’ (Fujiwara et al. 2015, 2). The US case notes that ‘[a]s of yet, no one has examined the complicated relationship between creative practice and wellbeing within the US’ and ‘preliminary work has failed to demonstrate a robust relationship between creative practice and wellbeing in part because of limited sample sizes’ (Tepper et al. 2014, pp. 8, 10). Interestingly, neither of these reports seems to have been cited much.²⁵ When they are cited, for example by Tiller (2014, 43), the positive impacts tend to be reported. Also Tiller (2014) reports on the benefits of ‘artistic practice’ as cultural participation, rather than being an artist, and others interpret Tepper et al.’s results as follows:

• Researchers have found that the more individuals participate in artistic activity, the higher they score on a variety of wellbeing. (Kemp et al. 2018, 1)

• Tepper et al. (2014) found that creating crafts, gardening, and playing a musical instrument—or personal art‐making—were positively related to life satisfaction. (Kemp et al. 2018, 3)

Part of the nonsignificant relationship between active arts participation and life satisfaction may be due to a perceived lack of time individuals feel they have to engage in creative practice. Hence, if they feel that time is constrained such that they do not have sufficient time to engage in artistic creation, benefits related to SWB may be minimal. (Kemp et al. 2018, 6)

This final point is of interest, as neither Tepper et al. nor Kemp et al. really pick up on the fact that it may not be that those engaged in active arts participation, as described, do not have enough spare time to do enough creative practice, but instead, that they could be—like our friend in the Disney movie—dissatisfied with the job they have. Tepper et al. say that it may be better for some graduates to walk away from their artistic practice (Tepper et al. 2014, 28), but leaving ‘the industry’ seems to be attributable to a lack of time for ‘robust artistic life’ versus ‘simply dabbling in the arts’. This analysis does not incorporate what we know of the hardships of those who are full-time artists and those who are still aspiring (refer to Brook et al. 2020 for discussion on this). Given that the authors state: ‘this report represents an initial exploration of the thesis that the arts are essential to a high quality of life’ (Tepper et al. 2014, 28), we might question whether they were ready for an interpretation of the arts and their labour markets as bad for well-being in various ways.

Tepper et al.’s title Artful Living: Examining the Relationship Between Artistic Practice and Subjective Wellbeing Across Three National Surveys was misleading to some audiences, particularly in the UK, where artistic practice tends to mean working as a professional artist. Instead, it was more broadly defined to include practising an art as a hobby. Similarly, not all the creative occupations in Fujiwara et al.’s report were as closely aligned to having an artistic practice as you might assume by the term creative occupation. Ultimately, it can be more difficult to compare or synthesise studies than is obvious by the title of a report, or its headline findings. This is often not acknowledged and can limit the validity of comparisons when evidence is reviewed and synthesised.

The way that the idea of culture and well-being are operationalised in these two cases differs more than to be expected: the data and the contexts in which they were collected, or the surveys or questions which generate the variables, are not always as similar as might be assumed. When we describe findings from apparently comparable studies, it is just as important to account for the motivations and methods of these studies (their contexts) as it would be our own. This is because when we synthesise the research of others, we create new knowledge that is able to make grander claims as it appears more generalisable.

7.4 Well-being Data and ‘Cultural Access’

Once we put the culture/well-being link under the right set of analytical lenses, it turns out quite clearly that ‘culture counts’, namely, that there is clear evidence that cultural access has a definite impact on individual psychological well being (and particularly so if cultural access occurs in a well-balanced mind–body perspective), and moreover that culture provides for some of the most effective predictors of well-being. (Grossi et al. 2012: 147)

Among the various potential factors considered, cultural access unexpectedly rankes [sic] as the second most important determinant of psychological well-being, immediately after the absence or presence of diseases. (Grossi et al. 2012, 129)

Moving national contexts again, the Italian Culture and Well-being Project used what it called ‘data mining’²⁶ to understand the ‘interaction between culture, health and psychological well-being’ (Grossi et al. 2012). It is clear to see from its concluding lines that it is of interest to our exploration of how people understand what it calls the culture/well-being link. The headline outcome (also quoted above) foregrounds what it calls ‘cultural access’. Interestingly, the authors claim that ‘cultural access unexpectedly’ appears to be the second most important thing for people’s well-being, after physical health. We will return to finding the right set of lenses and a finding being unexpected at the end of this section. First, we will look at what the researchers mean by culture.

What does the report mean by ‘cultural access’? The 15 ‘cultural activities considered in the survey’ consist of ‘jazz music concerts; classic music concerts; opera/ballet; theatre; museums; rock concerts; disco dance; paintings exhibition; social activity; watching sport; sport practice; book reading; poetry reading; cinema; local community development’ (Grossi et al. 2012). Therefore, does ‘cultural access’ mean ‘can you access these activities?’ or does it mean ‘do you do these activities?’ This is a key question for cultural policy as social policy, as we have discovered a number of times in the last few chapters: for if taking part in culture becomes some kind of proxy for having access to things that improve our well-being, the word access—and the implications for fairness of who has access and who wants access are important to establish.

One of the concerns over using well-being metrics to value culture is that the models used do not include all forms of cultural life (Jones 2010; O’Brien 2010). As we know from Chap. 6, defining culture is complicated. Thus, the value of what has come to be described as ‘everyday participation’ (Miles and Gibson 2016), including activities, such as attending sporting events (Oakley 2011) or chatting in a local shop (Edwards and Gibson 2017) should be acknowledged in some way when valuing ‘culture’ as something broadly defined. Increasingly, evidence indicates that it is ‘participation per se’ that is good for well-being, irrespective of what one is participating in (Miles and Sullivan 2010). Likewise, when people describe what is important to them for well-being, arts and culture activities, such as formalised theatre attendance, appeared less frequently in the ONS data I analysed than a more general and everyday participation (Oman 2020). It is therefore important that well-being metrics include—or at least acknowledge if they exclude—everyday participation, together with recognised artforms, such as theatre.

The inclusion of various ‘everyday’ forms of participation in Grossi et al.’s model might address concerns about formal culture and everyday participation. However, can 15 activities address the concerns of O’Brien and Jones in 2010, that metrics miss some aspects of cultural life? The 15 aspects of ‘cultural access’ chosen by the authors are said to have resulted from a literature review. Incidentally, this review and its results are not mentioned in more than passing by the authors, so as readers we don’t know why or how they came upon these 15, how many documents were reviewed before the 15 were decided, and so on.

These 15 categories of cultural access were formulated into a question that was added to a questionnaire. There is also no detail on the decisions made in this respect. The survey was conducted by an Italian pollster company called Doxa, through telephone interviews, according to the CATI²⁷ system, with 1500 random participants of the National Statistical Survey conducted by the Italian Statistics Bureau (ISTAT 2015). You may remember in Chap. 3 that the ISTAT is one national organisation that uses the same dimensions of well-being as the OECD. This project didn’t use these dimensions of well-being.²⁸

Instead, the authors describe that ‘their survey collected information covering socio-demographic and health-related data’ (Grossi et al. 2012, 132), together with the 15 activities as a proxy for cultural access. See Table 7.6 for these categories, as described in the article. They also describe questions from the Psychological General Well-being Index (PGWBI), which has 22 self-administered items ordinarily, but they used a trialled and tested shorter version of six items (Grossi et al. 2012, 133). As you can see in Table 7.7, these psychological questions ask very similar things to the ONS4 that we have encountered multiple times before. They are however worded slightly differently, which will have an effect on the data which may or may not be relevant to the claims made about the findings.

Table 7.6 Variables used in Grossi et al. (2012)

Table 7.7 The Psychological General Well-being Index questions used in Grossi et al. (2012)

In order to analyse ‘cultural access’, the authors take the answers from the questions about how many times people have participated in a particular activity. What is intriguing is that the authors have then combined these activities into a single measure, without accounting for this in the paper’s definition of ‘cultural access’. Consequently, the authors seem less concerned with deciphering what it is that people do (i.e. the nature of cultural access) than the frequency of cultural participation.

If we follow the recommendation that it is participation per se that matters for well-being (Miles and Sullivan 2010), incorporating various types of activity into a single dimension of culture could be a positive research decision. As we have already encountered a number of times, valuing one activity over another is ethically, methodologically and politically problematic. Of course, the data in and of itself do not account for all ‘cultural access’, or as we have described before, cultural activity. The questions can only account for the 15 activities included, missing out many social and cultural concerns, but as we saw in Box 7.4, this is not unusual in and of itself.

The analysis includes variables for aspects of cultural activity which are undoubtedly important to some people’s well-being. It is in the descriptions, categories and claims where issues may arise. For example, a question on ‘social activity’ could end up with data including almost anything, depending on the wording of the question. We do not know the exact wording of the question, but the paper states:

Each subject being surveyed in the study had to go through a structured questionnaire asking about the daily frequency of access to all of the activities listed. (Grossi et al. 2012, 132)

This seems to imply that the participants could define social activity for themselves, which could include leaving the house and talking to someone in a shop, which while valuable (feeling all the more valuable as I edit this book in lockdown), is not able to argue the value of investment in opera, say.

Is that a problem in and of itself? Possibly not. However, to include all social activity, and then conflate all the results to a single measure, without making this explicit in the headlines of the research may be misleading. As a consequence of these decisions, the value of ‘cultural access’ potentially includes the value of all social activity, as defined by different people. The authors have decided upon such a list to act as ‘a proxy of individual levels of “cultural access”’ (Grossi et al. 2012, 132). However, they have then combined the 15 proxies into one measure of cultural access. This could considerably inflate the impact of ‘cultural access’. This is important, as, the authors state ‘that there is clear evidence that cultural access has a definite impact on individual psychological well being’ (Grossi et al. 2012, 147).

Combining variables into one category is an issue with the evidence base for culture and it confuses the well-being evidence base as well. The language used in findings, and reproduced in evidence reviews, assumes it argues the value of a particular idea of culture. This limits the reach of the ‘discussion’ aspects of academic journal articles, as much as it does our understanding. Here we see the slippage in the definitions of culture described in the previous chapter can be used to include many aspects to account for culture’s impact; yet ‘cultural access’ comes to mean the arts when this argument is reproduced, as we shall see.

Before we move towards our conclusion, let us remind ourselves of the headline findings, again:

The results show that, among the various potential factors considered, cultural access unexpectedly rankes as the second most important determinant of psychological well-being, immediately after the absence or presence of diseases, and outperforming factors such as job, age, income, civil status, education, place of living and other important factors. (Grossi et al. 2012, 129)

In spite of queries with the Italian Culture and Well-being Project, the headline results appear in other high-profile reviews. These include the ‘Understanding the Value of the Arts and Culture’ report from the AHRC’s Cultural Value project (Crossick and Kaszynska 2016) and a 2020 report to the Welsh government (Browne Gott 2020). The more findings are reproduced, the more credible they seem, and the more they are reproduced. One review (Taylor et al. 2015) was commissioned by the CASE programme, which you may remember from Chap. 6. The report describes the Culture and Sport Evidence (CASE ) programme as a joint programme of strategic research led by the Department for Culture, Media and Sport (DCMS) in collaboration with the Arts Council England (ACE), English Heritage (EH) and Sport England (SE). The report was of a systematic review of the literature and evidence (Taylor et al. 2015, 8) and it evaluates the above study as follows:

Grossi (2012F) offers arguably the most authoritative review based on quantitative research, linking participation in arts with better social outcomes and impacts, including health. (Taylor et al. 2015, 71)

[A]rts-related activities are seen as central to wellbeing by most people, according to a recent Italian study (Grossi 2012F). Among the various potential factors considered, cultural access ranked as the second most important determinant of psychological wellbeing, immediately after the absence or presence of diseases, and outperforming factors such as job, age, income and other important factors. (Taylor et al. 2015, 75)

Even without concerns about the category of cultural access, the methods of the study did not ask people whether culture was central to anything. It asked them what they did and how they felt. There is a concern, with all social research, that if you look for a particular outcome, you are more likely to find it. Hold that thought. Because, we might want to have a think when considering others’ research, whether it is putting the culture–well-being relationship under different ‘lenses’, until it finds the one it likes? That is until that lens, or series of lenses, finds that ‘culture counts’ in the way that is desired (Grossi et al. 2012: 147).

It is easy to see that the CASE review of the literature and evidence cut and pasted the findings directly from the article and, in fact, its abstract. The reason I mention this is that this is not abnormal practice. Instead, I want to highlight that it is not always clear that when a finding appears in a review commissioned by such significant body, that this does not actually qualify that the finding has been checked by that authority; there is no guarantee that the authors checked for robustness, or that it should be authoritative.

So, in presenting the impact of ‘cultural access’ (however defined) on well-being, research satisfies the hunger for those who want evidence of the culture–well-being relationship. This also has silly ends, fuelling the fires underneath claims such as culture can ‘reduce crime’ (Morris 2003) or ‘tackle poverty’ (National Assembly Wales 2019). The sad thing is these actions are a double-edged sword: they are popular because they seem to justify people’s feelings that the arts are good for us, while at the exact same time discrediting the good evidence that is available for advocacy.

This indicates both the value of, and requirement for, a review of rigour when it comes to data and their categorisation in the empirical work underway to understand the relationship between different activities and programmes on well-being. Perhaps, even more importantly, attention must be paid to the resource in the teams synthesising and evaluating the evidence base in order to direct future research, policy and practice. It is not simply a case of levels and areas of expertise, but the resource of time to review and evaluate evidence.

This chapter has revealed that it is not hard for everyone to look a bit further—beyond the headlines—and establish potential issues. If we acknowledge that culture and well-being are slippery concepts, then how a concept such as cultural access is defined and measured requires some clarity if the evidence is going to be used politically, whether that is to justify funding or as we are increasingly seeing in this book, how resources decided by policy-makers are related to inequities of resource in society more generally.

7.5 Conclusion: Using Well-being Data to Understand Policy Questions

We began this chapter with David Cameron promising to put ‘instincts we feel to the core’ to ‘the practical test’ so that those whose decisions on policy and spending, that affect people’s lives, take account of what matters. We end with concerns about impact and conflated variables. We considered data and evidence in cultural policy briefly, before looking at three components of the culture–well-being relationship that are relevant to our policy concerns. First, we looked at subjective well-being (measured as life satisfaction) over time and policy spending on culture in the UK over time. Second, we looked at different kinds of subjective well-being data and ‘creatives’ (broadly defined) in the UK and the US. Finally, we looked at subjective well-being and ‘cultural access’ (broadly defined) in Italy.

We had a play with different kinds of readily available data to look at the relationship between policy spend on culture and whether that impacts on national well-being. We considered the contexts of the data, the limits of what we can expect in terms of impact on life satisfaction as a measure and in terms of policy spend on a measure. Although these data were used descriptively, we found ourselves with questions as to why more research has not been done on the relationship between policy investment and well-being, given claims for investment based on improved well-being? This left us at a point of provocation: why are some data operationalised to understand the culture–well-being relationship, when other data are not?

We compared two studies that seemed to look at comparable groups, but reached different conclusions about the well-being of people who could be called creatives. Again, we reflected on the contexts of data, the ambitions of the researchers and the aims of the research to appreciate the limits and extents of claims that can be made. We spent some time breaking down how models and categories work, and why they are important for understanding what is being measured about culture and what is being measured about well-being. We also considered a much-cited study on the impact of what the authors call ‘cultural access’ on well-being. We discovered that ideas of culture and cultural access were slippery which enabled a favourable outcome. We reflected on how an outcome that might be popular, because it reinforces people’s beliefs about the culture–well-being relationship, can result in the study being frequently referred to in later, and influential literature reviews.

This chapter has tried to break down some features of how these different aspects of cultural policy (investment, labour, access) are measured. It also wanted to demonstrate that these relationships can be explored simply, using easily available data. The lack of relationship between life satisfaction and GDP (the Easterlin paradox) is lauded as the starting point for a whole new area of research in happiness economics and positive psychology. Yet, the lack of relationship between life satisfaction and arts subsidy is not discussed as an important research question. We might be similarly interested in how little research has happened since the two projects on being an artist or the creative occupations, to further understanding of the complex relationship between professional creative practice and well-being.

The final question for this chapter, though, is are we using data to establish evidence or finding data to suit arguments? There are frequent calls that more evidence is needed to support the cause of cultural policy to argue its value as social policy. Why are there not more analyses of the data already available, even if they reveal a possibly uncomfortable relationship, as in the case of cultural funding, or other aspects of delivering social policy and well-being? Perhaps this might be where more complex relationships between well-being, inequality and culture might be explored. Despite the crudeness of tracking arts funding and life satisfaction data together, they tell a simple and effective story and definitely warrant future research. Or, at least ask questions of existing research. In the next chapter we will explore one of a number of studies that use increasingly complex quantitative techniques to express the relationship between culture and well-being differently. Thus, continuing our exploration of evidencing culture for policy.