Looking at Well-being Data in Context

Abstract

Understanding the where, what, how, who and why is important to any social research. This chapter poses these questions about data and well-being in various ways. We look at well-being measurement, appraising the pros and cons of different forms of data and approaches, acknowledging that all data have limits and that context should drive any chosen approach. It presents examples of qualitative data available through interviews and ethnographies, and quantitative data through surveys, and administrative records. We focus on objective well-being data and a case study of the OECD reveals the volume of decision-making behind international objective indicators. Such human intervention is rarely visible, but is important and useful to improve understanding and comprehension of well-being data more generally.

3.1 Well-being Measurement (Other Data Are Available)

It measures neither our wit nor our courage, neither our wisdom nor our learning, neither our compassion nor our devotion to our country, it measures everything in short, except that which makes life worthwhile. And it can tell us everything about America except why we are proud that we are Americans. (Robert F. Kennedy 1968)

These remarks from Robert F. Kennedy are often found in arguments for measuring well-being,¹ as an alternative to gross national product (GNP, and what Kennedy calls ‘it’).² As touched on in the previous chapter, GNP (and GDP) are ‘national accounts’ and are administrative data that capture the economic activity of a country. Data on economic activity are used to measure financial success, compare countries against each other, and track progress over time.

Robert F. Kennedy’s comments are from a speech at the University of Kansas on 18 March 1968, forming part of his campaign for nomination for the US presidency.³ Fondly called ‘Bobby’, he is remembered for his advocacy for the civil rights movement. In this speech, he also declares support for student protests as good for society, and against the Vietnam War happening at the time (Kennedy 1968). Interestingly, his questioning of the value of GDP to measure human flourishing did not make much of an impact at the time. It is only retrospectively, and with hindsight, that this quote has gained notoriety, thus implying that it resonates more now than it perhaps did to American citizens in 1968.

Why is this speech important? Kennedy advocates changing priorities of public policy-making in line with altering values (both how we value and what we value). It indicates that it was politically prudent for a politician like Robert F. Kennedy to argue for replacing GDP as the main indicator of human progress at that time; it also suggests that believing in measuring well-being, rather than GDP, was ideologically aligned with supporting student protests and problematising the Vietnam War. Likewise, it tells us that there is an alternative to GDP or GNP to measure at that time. With the previous chapter, we can historicise this speech as coinciding with the social indicators movement that characterised what Bache and Reardon (2013) called the ‘first wave of well-being’. This means we can contextualise this political speech as from a time when different measures were called for—by people with particular values—to understand human flourishing, or how a nation was progressing. We are acknowledging that these comments were little repeated at that moment in time, but were later revisited to justify another ‘second wave of well-being’ (Bache and Reardon 2013).

So why are these historical and political settings for measuring well-being valuable for this chapter? Because they help contextualise well-being data. Context is key to recognising the role of methods in generating well-being data, as this chapter will show. Exploring the stories that lie behind data, and looking under the bonnet of how they are generated, is important to understanding: what they measure; whether they measure what they say; and the reasons why they have been collected and analysed in particular ways.

This is all part of what I call ‘data contexts’, arguing it is important to know how data work in what contexts (Oman n.d.). What do I mean by this? Well, understanding where data come from, and why they were generated, is important. Were they generated in a lab or in a real-world setting? Why do they exist? Were they collected for one purpose and are being used in another? Who has analysed them and how may that affect how we view the data? We also need to think about how different techniques of analysis are applied and how they are operationalised in different contexts. What do they achieve? Do they monitor people’s toilet breaks in a call centre or how many steps a day we take while working from home? Do people know these data are being collected and why? Do the data help to hold governments accountable for national poverty or are they used to decide welfare distribution?

Measuring well-being as a political and scientific project does not have a consistent historic arc. There are moments where various technical and intellectual disciplines, and people with differing political interests, gather around ‘the well-being agenda’ as a project. This results in different types of well-being data being foregrounded, even acting as the catalyst for political change, at different times. The UK’s national well-being measures are often called ‘Cameron’s happiness index’ (Clinton 2011; Mirror 2011) after the UK’s Prime Minister contributed to the launch of the Measuring National Well-being (MNW) project (Cameron 2010). As we shall see, the next section of this chapter opens with evidence that the idea of well-being measures for the UK (to become the MNW project) developed under the previous New Labour administration. The history of these measures is, therefore, not always obvious.

Similarly, it is not always clear what might be well-being data, and what are not. Data about well-being have long been valuable because they could help to understand how well a population was doing. Sometimes the data collected were believed to capture a specific aspect of happiness; other times to understand a particular part of a population, or indeed, one person’s quality of life. Therefore, data about well-being do not all look the same, do not have the same unit of analysis (individual people, nations or communities), are not used the same way and do not all exist for the same reason. Again, this is why context is important.

This chapter considers how well-being data is collected: the diversity of methods and the range of data that can be called well-being data. This includes background and context to the well-being statistics you might read in newspapers, online, or have seen in COVID-19 briefings and press conferences. It also begins to look at claims about what can possibly be concluded from different kinds of well-being research. We will continue to break down technical terms to show well-being data and measurement are complex, and their uses in policy are not universal. It aims to show that this language and these ideas can be more accessible when you know where they come from.⁴

Well-being data as a term most often describes well-being metrics or indicators. This chapter offers some examples of how many decisions are made when choosing an objective indicator of well-being. Despite the name ‘objective’, which implies they are not affected by feelings or opinions, they do not fall from the sky as facts. If truth be told, they are the product of a specific methodology , which means they must fulfil certain practical and theoretical criteria that satisfy often long-established opinions of what are the best methods to capture the most objective data, and then how to go about analysing them.

Objective well-being indicators predominantly originate from survey data (like the census) or administrative data (such as mortality rates). They also include some subjective data where people are asked about aspects of their lives, such as how satisfied they are with their health. We come to this later. These datasets will include enough of the population that it is sensible to analyse them numerically—as quantitative data. These quantitative analyses are not always conducted by the person or the organisation who collects these data. Similarly, secondary uses of data can make the data useful as well-being data, when it may not have been collected for such a purpose.

Whether objective or subjective, it is mostly agreed that:

• 1. all well-being measures must be theoretically grounded (Haybron 2008), meaning that there is a clear, agreed rationale as to what exactly is being measured, what for and how the data are collected and handled

• 2. the limited impact of previous attempts to measure well-being lies in deficient theoretical grounding, and therefore failed understanding of what the measures are for and who they benefit (Scott 2012)

• 3. assessing one’s own well-being is a subjective and aesthetic^5,6 experience (Rapley 2003)

• 4. well-being survey questions should involve concepts which are readily understandable and easy to relate to, such as ‘satisfaction’ and ‘happiness’ (Fleche et al. 2012, 9)

• 5. well-being measures need to be subject to harmonisation (GSS ), meaning that they should be able to work with other well-being measures

Not all well-being data are numbers, or the result of large-scale data collection, however. It can be easier than you may imagine to produce and use well-being data. To discover how accessible other methods are, we will explore other ways of collecting data, such as interviews and focus groups. We will also look at policy documents as data, like the speech above, finding that ideas of measurement and well-being are used together, and how that can reveal the all-important context to why data are used to make certain arguments. As with the quantitative data found in well-being indicators, it is also important to understand the limitations to what we can claim to know as a result of analysing qualitative data. Whether from a few policy documents or interviews with a community in a particular place (rather than a whole population), these data tell us a lot about a small number of people and may not describe how things work on a larger scale. These are matters of methodology .

Box 3.1 Methodology

Methodology is more than the methods used to collect data (e.g. a questionnaire or interview) or analyse data (i.e. statistical techniques or thematic analysis⁷). It is more than who is using methods, whether in academic research, in national-level surveys, or in evaluations of how much a policy decision or an individual project has impacted on well-being. It is the system behind methods: why people have decided to do these things in these ways. This is what makes data ‘theoretically grounded’ (see above).

As we go about our day-to-day activities, we don’t tend to consider the theory of what we are doing and why, but odd moments might make us stop and think about why we have done something in a certain way and whether that is the best possible, or the one most suited to our situation (how much time we have and where we are, for instance). Think about when we hear how other people do something, their tips or techniques might be different from ours and can be about something quite mundane.

Think about a cup of tea (English tea to non-native Brits, or depending on dialect: ‘a cuppa’ or ‘a brew’). It has different names, depending on where you come from, and there are often discussions about how to make tea the right way: milk first or second; let the bag stew or not; in a teapot, cup or mug, and for how long. There are also TikTok videos and Facebook posts on the issue, Reddit feeds exclaiming the crimes of others’ tea-making methods, and reports in the national press, saying certain methods ‘spark outrage’ (Morris 2020). What works best, and in which order, is therefore not a universal truth and there are opinions on how these all work together.

Methodology, similarly, involves the theory behind how stages of working with data work together. Working with theory doesn’t only mean reading philosophers, but more practically involves careful consideration of each process.

Some useful questions to ask about these stages include:

Was it appropriate to apply this particular approach to collecting and analysing data to the particular issue the researchers want or need to know more about?

Or would it have been more appropriate to analyse data already available or accessible in a different, perhaps easier, and less intrusive way?

Would people have been easily able to answer the questions?—we’ve all answered plenty of surveys where we cannot answer the questions truthfully, because the questions are badly designed. Or, indeed, because we do not want to tell the truth, exactly.

Is it fair to ask people to answer this question about themselves in this context (on the street, in a room full of others, at work where their screens might be viewed by colleagues, etc.)?

Is this ethical?

Methodology is often described as bringing theory to method. It is not so different from debating how tea is made, and how that affects the result. Methodology discussions are also often tribal, with in-fighting and disciplinary arguments—even disagreements over namings and meanings. In the case of data, this more simply involves thinking through what we do with data and how we have thought about collecting them. What order certain processes go in and what are our approaches to each process, and why that is best suited to the situation at hand. It is the foundations of why research has been done in a particular way.

There is often a tendency in the social sciences to feel the need for academics to take a position on the value of quantitative data over qualitative data or vice versa. This is colloquially called the ‘Quants-Quals debate’, which I had never heard of until I became an academic, but it is rife.⁸ Other academics have requested I make it clear where I stand in the past. So, I want to make it clear that in this chapter—and the whole book, in fact—I resist this assumption that any data is better than another because we read them as text or count them as numbers, or collect them differently. All well-being data might be valuable to understanding well-being. Whether they are qualitative or quantitative is not the issue at hand. Instead, context is the issue: where the data came from, are they used appropriately and how are they applied? Are their uses ethical and fair? What are the limitations to the data we have? What can we know as a result of the data? What happens next?

The chapter describes different sorts of data: a moment from my research, hypothetical examples, as well as case studies from international statistics agencies to reveal some of the contexts of data collection, interpretation and uses of well-being data. It does this to show that all data have origins of thought, process and practice and are therefore rarely completely neutral or objective. All methodologies have their limitations, which thereby limits the claims that can be made. These are not always fully recognised.

If limitations are acknowledged in one place, that place is often far removed from the headline findings⁹ to make caveats clear when interpreting results. The de-contextualising of data removes how we understand their limits and appropriateness. It must, therefore, impact on how ‘good’ the data can be in understanding society and well-being. It also affects the capacity for data to do good and inform societal change in such a way as to improve social, personal or national well-being. We need to account for the data used and we need to heed different accounts of what well-being means, as well as how we might understand it better.

3.2 Accounts of Well-being

Example 1 Wellbeing is a positive state that people experience when they are able to meet their needs for strong social relationships, equality of opportunity, rewarding work, economic and physical security, health, and opportunities to participate in cultural activities and enjoy contact with nature. It is enhanced when an individual is able to fulfil personal goals and achieve a sense of purpose and fulfilment in society.

Example 2 Wellbeing is a positive physical, social and mental state; it is not just the absence of pain, discomfort and incapacity. It arises not only from the action of individuals, but from a host of collective goods and relationships with other people. It requires that basic needs are met, important personal goals are achieved and people are able to achieve a sense of purpose and fulfilment in society, and that they are satisfied with their lives. (Levett-Therivel Sustainability Consultants’ Report to DEFRA 2007)

The above definitions are examples from a consultation across government and well-being experts, in response to the UK’s 2005 Sustainable Development Strategy. Called Securing the Future, the new strategy (HM Government 2005) committed the UK government to working towards new well-being indicators and to work towards policies with an explicit well-being focus (Levett-Therivel 2007).

The final definition that is often assumed as the working definition for the UK’s Measuring National Well-being programme combines these two examples (DEFRA 2007), also drawing heavily on the World Health Organization’s definition of health:

Health is a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity. The enjoyment of the highest attainable standard of health is one of the fundamental rights of every human being without distinction of race, religion, political belief, economic or social condition. (WHO 1946, 1)

When the UK’s Office for National Statistics (ONS ) started to produce working papers on well-being, they began with DEFRA’s final statement:

Wellbeing is a positive, social and mental state; it is not just the absence of pain, discomfort and incapacity. It arises not only from the action of individuals, but from a host of collective goods and relationships with other people. It requires that basic needs are met, that individuals have a sense of purpose, and that they feel able to achieve important personal goals and participate in society. It is enhanced by conditions that include supportive personal relationships, involvement in empowered communities, good health, financial security, rewarding employment and a healthy and attractive environment. (DEFRA in ONS 2009, 6)

As the previous chapter indicated, there are many definitions of well-being from different parts of the world and philosophical traditions. These different accounts of what well-being is have lineages: they are cumulative; learning from and adapting previous versions to suit who is using it, and for what: to suit its context. The same is true with policy.

Here we have traced the lineage of definitions across policy documents over a number of years, which can be a useful methodology to help understand how meanings adapt in policy documents to suit the context. In other words, we have ‘followed the data’ in a very different way, those data being textual. They are still important data about well-being, however, as they help us understand how well-being is understood and why.

The quotation immediately above is an example of the ONS establishing the lineage of their working definitions. They account for their categories before explaining how they might go about using them to measure well-being. In 2007, Paul Allin, who was to become Head of the ONS’ MNW programme, explained that well-being ‘can best be viewed as a multidimensional, shifting concept’ (Allin 2007, 49). Despite indications that the self-named new sciences of happiness (Layard 2006) were evolving (O’Donnell et al. 2014; Helliwell et al. 2015; ONS 2015; Dolan et al. 2011b), as we explore in the next chapter, some academics fear that the concept of well-being itself has lacked attention, as the ‘empirically-oriented field’ needs more theoretical input (Jugureanu 2016, 68). The lack of consensus on how to conceptualise well-being for policy and measurement is a concern, however, when policy-making (OECD 2013, 11). As is deciding on what the best methods might be for measuring well-being effects and outcomes (Dolan et al. 2011a). So, as you can see ‘objective well-being data’ involve many decisions: what to measure and how to measure it are key to understanding what are the best well-being data.

Before the UK started collecting well-being data to form its well-being national accounts, the MNW programme took a novel approach to making the decision on what to measure. The methodology chosen to inform this decision became a national well-being debate that was launched by then Prime Minister David Cameron (2010) and administered by the ONS. This large-scale exercise collected different kinds of data, using different methods, asking people what mattered to them about well-being; what to measure and how to measure. The UK’s ‘What Matters to You?’ debate received 34,000 responses and has been applauded for its democratic approach to meaning and measurement (Kroll 2011, 6), which we shall come to later.

So, GDP and GNP were ‘national accounts’¹⁰ that used economic activity to measure progress and the international well-being agenda was keen to replace these with new national accounts of well-being.¹¹ The UK’s MNW debate was to inform this work in the UK, alongside expert consultations, such as the one that wrote the report quoted at the opening of this section. In this context, national accounts are called this because they ordinarily track economic transactions, like an organisation’s accounts. The ONS still do not formally include well-being in its national accounts, a label they still reserve for transactional data.¹² Somewhat confusingly, the economists informing the MNW programme also talk of accounts of well-being too. Their meaning is slightly different. We encountered the two main traditions in the previous chapter: ‘Benthamite-subjective-hedonic-individualistic’ or ‘Aristotelian-objective-eudaimonic-rational’. The short-hand versions of these being pleasure (or feeling) and purpose (or flourishing). In addition to these traditions are three different ‘accounts’ of well-being that are used to understand well-being and inform policy (Dolan et al. 2011a). These are:

• 1. Objective lists

• 2. Preference satisfaction

• 3. Mental states (or what has come to be known as subjective well-being)¹³

Different ways that well-being might be captured and measured are therefore ‘accounts’ of well-being. These have informed the programme to devise ‘the national accounts of well-being’. We cover the ways that well-being is captured as an account below.

3.2.1 Objective Lists

Objective lists of well-being involve a list of assumptions regarding basic human needs, rights and conditions that are believed to impact on well-being. A simplified example is the Human Development Index (HDI), which is a composite index of three separate indicators: life expectancy, education and gross national income per capita. A composite index means a single number is calculated from these three indicators to make the data easier to use and visualise. This enables the HDI to rank countries into ‘tiers of human development’ (Human Development Reports 2020; United Nations n.d.). The key aspect of the HDI’s design is its simplicity. Rather than intending to capture all aspects of well-being, the idea is that it is simplified and made easy for a broad audience to read and understand.

The use of indices like the HDI to understand international development has been criticised. One source of criticism is that the dimensions that contribute to these indices are those things that are considered important in the richer countries in the global North, rather than those things that are considered important in the countries where these indices are being used.¹⁴ Another source of criticism is to do with what happens when these dimensions are combined. In the case of the HDI, the three dimensions are treated equally: for example, the income dimension is treated as holding the same importance as the two social dimensions (education and life expectancy). This assumes that all human beings value the three dimensions equally (United Nations 2020). However, this is not always the case and the representations of these various ‘achievements’ are sometimes criticised as being arbitrary, subjective or depending on a priori value judgements (OECD 2011b). In particular, because wealthier countries will always appear higher up the scale as a consequence of the importance placed on income.

Box 3.2 A Composite Index

A more familiar type of index might be from hearing or seeing things about the stock market on the news. The Dow Jones Industrial Average is a composite index. Ordinarily called the Dow or the Dow Jones, the index is made up of financial data from 30 large companies listed on US stock exchanges. Composite indexes are used to conduct investment analyses, measure economic trends, and forecast market activity in a way that is easy to read. ‘The Dow’ is criticised because it only includes 30 companies, as it was designed in the 1880s to represent the main markets at the time. Markets are, of course, now more complex. It is also criticised because it is weighted by price, when other indexes use alternative weights that capture more of the intricacy of the market.

As you can see, there is even methodological disagreement on how to best represent stock market data so they capture the most important aspects of the market (change) while remaining understandable. Interestingly, both in spite of and because of its age, the Dow is still the most used.

The objective list approach (or establishing a list of objective indicators) is mainly used by national and international statistics offices, with the aim of generating a complete list of what is necessary to satisfy a good life or ensure a good society. The OECD and ONS examples of well-being indexes are more comprehensive examples of these lists and have closer to 50 indicators (rather than the HDI’s three).

3.2.2 Preference Satisfaction

Preference satisfaction accounts work on the premise that ‘what is best for someone is what would best fulfil all of his¹⁵ desires’ (Parfit 1984, 494). This is how economists have long approached understanding well-being (Dolan and Peasgood 2008). The rationale behind expressing well-being like this for economists is that people’s preferences are revealed by what they purchase (see Chap. 2, Box 2.4 for a description). By extension, this means that the higher a person’s income, the more they are able to gain access to what they want. Also, the greater the choice available, the more able people are to satisfy their desires. The idea that choice is better is also a driving principle of new public management we also encountered in Chap. 2: the rationale being that people should be free to purchase from a wide variety of market providers, rather than public services being delivered by the public sector.

It is this account which has historically informed policy decisions at a monitoring level, using GDP as a proxy for well-being.¹⁶ It is also this account of well-being that the Easterlin paradox (1973) found wanting, as Easterlin’s analysis found that improved material living standards had not improved measured happiness in wealthy countries over time. This is largely assumed to be as a result of adaptation, in that as one preference becomes satisfied, we adapt and want more.¹⁷ This is ultimately seen as benefiting the economy, but bad for people and societies. Alongside empirical issues are concerns that ‘making preference satisfaction the measure of political health completely cuts out the possibility of public deliberation about the ends we should pursue as a self-governing people’ (Williamson 2010, 171). This latter issue was, of course, what the UK’s MNW debate aimed to overcome.

3.2.3 Mental States (or Subjective Well-being)

Subjective well-being is ‘an umbrella term’ (Hicks 2011, 3) which covers three strands of a person’s self-assessment of their happiness levels: life satisfaction , mood and meaning. The whole of Chap. 4 is about subjective well-being, so we only cover it briefly here. The term can also, confusingly, be used to just describe mood or happiness, rather than necessarily encompassing all concepts. Subjective well-being can be measured in various ways, like asking people about their happiness in any given moment, or about how satisfied they feel with their life overall. Along with preference satisfaction, subjective well-being measures have been thought to be more democratic than objective lists (Graham 2010), because they allow people to decide how well they are doing, without someone else assigning a level of well-being to them on their behalf. We will come to people deciding their own well-being later.

The above ‘accounts’ of well-being have been formulated with quantitative data in mind, collected through large samples and national-level surveys. It is these data that are used most in decision-making, especially in policy. However, other kinds of data allow you to derive preference satisfaction, subjective well-being—even objective lists. These methods collect people’s own accounts of well-being from them on a smaller scale. Various methods can be used, like interviews or diaries, and are designed to understand how people’s lives work in more detail. Owing to the smaller scale of these projects, they are more available to the researcher who does not have the resources of the United Nations (UN) or a national statistics office to understand well-being at national or regional levels. These methods also tend to present more detail about specific people and contexts, and so are often better for a project that wants to understand the well-being of the staff of an organisation or, perhaps, how one thing affects a small group of people in great depth. It is also especially useful for understanding people’s lives and experiences in the everyday.

As we touched on earlier in the chapter, there is a history of researchers gathering around their own preference for qualitative or quantitative approaches. This can result in habitual silos of research and a history of squabbling over the value of one kind of data over another. The tradition of a divide tends to obscure the fact you can make the most of both worlds. It is possible to take a mixed methods approach to research, using both qualitative and quantitative data generated by various methods. There are also ways of collecting data that can result in both sorts of data. Many surveys offer a chance to answer using tick boxes and text. What should be at the forefront of any research is what is most appropriate to the context of collection and the question at hand. We shall think a little bit more about everyday contexts of data in the next section.

3.3 Everyday Well-being Data: Asking People Questions About Their Lives

Well-being data are not only for policy-makers or international economic development agencies. They can be collected in various ways available to us in everyday situations. Many of us have seen an increase in emails popping into our inboxes or Facebook timelines asking us to complete some kind of questionnaire about our well-being. COVID-19 has seen collection of these kinds of data increase.

These are well-being data collected through a questionnaire not so different from a national-level survey, but on a smaller scale. Although most require good planning and ethical consideration of how the questions you ask people may have some negative impact on them. In short, could your research negatively impact on people’s well-being?

The following section offers a brief overview of methods that can collect ‘smaller data’ for different ends. Vignettes from my own research, a hypothetical questionnaire scenario, and the ethics of ethnography are presented to help you consider the different contexts and considerations of well-being data. Table 3.1 shows the advantages and challenges of different kinds of data for understanding well-being. While this section might help you design your own research, there are countless exhaustive textbooks out there that devote more space to that. Here the goal is to help you to imagine data contexts, and so better evaluate how other people have used data.

Table 3.1 Data sources and their uses

3.3.1 Questionnaire Data

If the idea of data collection is new to you, perhaps the easiest way to imagine well-being data being collected is by using a questionnaire that asks people how they feel about things related to their well-being. Questionnaires are easily distributed, and ask the same questions in the same way and can be repeated numerous times with the same or different people. This means their data are easily comparable, providing insights into well-being across a group or sub-population. If you ask the same people, you can understand their well-being over time. Online questionnaires distributed by organisations that have some responsibility for our well-being are increasingly familiar, for example, universities surveying their students and employers, their staff.²⁰ These tend to ask us questions about our well-being that are useful to the running of the organisation in some way. The data can be used ‘by management’ to decide if it is allocating resources well, or if HR needs to make an intervention, in the same way that policy-makers can use well-being data.

Another way to imagine the context of questionnaire data collection might be the market researchers who used to be on the streets with clipboards (that my mum would always desperately avoid at the shops). In our increasingly online world, people’s opinions are still sought using questionnaires in person (although, along with everything else, COVID-19 has compromised this, and we are yet to see how social research will find its new normal). Questionnaires could involve asking whether people would buy a product in the case of market research, but can also include questions about something they have just seen, an experience they just had, or how they feel about a particular place, like the park they are in. Some have had questionnaires after their COVID-19 vaccine, asking about their healthcare experiences. People can fill in the questionnaires themselves, or the researcher could complete the questionnaire on their behalf. If the research wanted to understand how people feel about a local, publicly subsidised event, the questions answered could look something like:

Q1 Have you just seen [specific subsidised concert]?	Y/N
Q2 Is this your local park?	Y/N
Q3 How are you feeling right now—out of 10, with 10 being the best you’ve ever felt, and 0 the worst?	0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

These questions would generate binary data (yes/no) that can be aggregated (totalled) alongside numeric data from the scale. These sorts of data are easy to work with quantitatively, as you can categorise easily across the binary questions. Q3 uses a Likert²¹ rating scale that presents a series of answers to choose from, ranging from one extreme attitude to another. It’s sometimes referred to as a satisfaction scale as it is ideal for measuring satisfaction, and is therefore often used to measure well-being. The numbers from the scale are used to establish trends or averages.

Say, a researcher was lucky enough to get 100 people to speak to them on their way home from a concert in a park, they would have a sample of 100 people, and would know that they saw the event (is that the same as attended, you may ask? We will see what to do about that shortly). The researchers could establish what percentage of those spoken with were local residents (although, note, that what is meant by ‘local’ is not specified, which is not ideal). They could then look for trends in how people felt having attended the concert using the numeric data.

Or, they could ask the question,

Having seen this subsidised concert in your local park, how are you feeling right now?

Yeah, good, ta. There was a great atmosphere.

This box, called a free text field or open text, allows people to answer a question in their own words. Whilst this is less easy to process and compare at scale, it can sometimes provide valuable information. In each case, the majority of the data collected would be subjective, as the numeric or textual answers would reflect the reported experience of the individual. Therefore, the answers collected—the data—may be considered a valuable reflection of how they are feeling.

However, not all textual or verbal responses are succinct. In fact, when you ask people how they feel in themselves or about something to do with well-being, their responses can contain much rich detail (Oman 2015, 2017, 2019). So they might say, something like:

I feel great! It was great to have the opportunity to go to a gig close by. Because I only earn £6000 a year, I don’t get to go to concerts any more. I think that because I never get to go, that made this all the more special

Yeah, good. There was a great atmosphere.

These qualitative data contain: objective data in the salary disclosure, an example of preference (in that they chose to spend their limited income on the subsidised concert), as well as what they think this means for their well-being and concert attendance.

However, there are confounders, too: their limited income = limited concert attendance, which means that they think their enjoyment of this concert was heightened. How does this compare to other people who attended, etc.? How could it compare? How might a valid argument be made for the impact of this concert (as a cultural product, or an arts event) rather than capturing ‘the social value’ (which we covered in the previous chapter) of going to an event in the local park? How could claims made be generalisable? Meaning how could what is learnt from 100 people in one context be used to understand different people who attend different kinds of concerts with different life circumstances in different places at different times? Also, the fact that this person lives locally to the concert is probably a factor impacting on their decision to go. How might we isolate the relationship between concert attendance and happiness from these confounders? Here we mean how much of an effect did proximity to the concert have on attendance versus wanting to go to the concert for another reason? How do you know they weren’t caught in a very limited moment of elation that meant they said they felt great, but which didn’t last? How do you know the people spoken to could possibly represent diverse opinions? Perhaps they were all picked because they were all wearing band T-shirts for those on the line-up? It may be that people who are more likely to stop and answer questions will also have more time to go to concerts? How do you know if you need to know these things, or indeed, which of them you need to know?

Perhaps, more importantly, how sure can we be that what people say is an accurate representation of their feelings and opinion more generally? There is evidence that people who are approached will say nice things to people because, despite popular belief, people are generally nice, and they don’t want to offend people. This may mean giving an answer they think the interviewer wants and is called the ‘interviewer effect’.²² In our case this would mean that people are inclined to say that something has improved their mood or happiness or well-being because they think that is what the person posing the question wants to hear. Asking the question, did you see the concert? followed by how do you feel right now? will suggest to the person asked that the researcher wants to understand if the concert has positively impacted on how they are feeling and is a leading question.

There are other aspects of situations like this which will affect people’s answers: can they be overheard, for example? Do they want to look like they like the music played, or do they want to suggest they have ‘better’ taste? Sometimes people answer for the benefit of others, rather than truthfully.

It is not only how truthful someone is in the moment, but also a question of how long that moment lasts. If you ask someone directly after the concert how they feel, are you able to argue for a longer-term effect on well-being? We don’t know how long such an effect will last. Can feeling great for five minutes be argued as a positive impact on well-being? These are contextual issues with data: often the context in which data have been collected compromises the claims which can be made through analysing them. These are issues of validity (see Box 3.3). Yet, when you read a local council’s report about an event like a park concert, it will rarely acknowledge the limits to what can be known.

Similarly, how do you also account for negative effects on well-being and social impacts that are less positive? What of the park being shut for the concert take down and put up? What of the noise pollution affecting older people, pets or babies sleeping? All of these are examples of confounders on the claims that what might seem a simple initiative, such as the local council subsiding a concert in a local park, can have social impact in a way that is simple to express. The negative impacts are not often accommodated in research which asserts social impact, yet is clearly important to account for these issues in any claims made for any positive effects.

Box 3.3 Validity

Researchers need to think in terms of validity to understand the limits of what can be known by what they are asking. There are two main types of validity.

Internal validity is concerned with how capable a research tool (say a survey question) is in enabling a researcher to answer their research question. For example, when you ask someone ‘how are you feeling right now’ without asking them to connect the feeling to the concert, you are unable to know that the feeling is linked to attending a concert. This will limit the claims you can make with validity.

External validity is concerned with how generalisable the results of a piece of research are outside of the study; by which we mean ‘can the findings of this study (speaking to 100 people outside X park) explain how people that we didn’t speak to feel about concerts?’

Limits to validity are not always bad, it depends on the context, but they should be accounted for.

It is not often acknowledged that good questionnaires that collect ‘good data’ are not easy to design or execute well. Questionnaire data therefore may be useful for many purposes and relatively easy to access, but need testing. One way of feeling more secure in the quality of questions, even on a small scale, as with our concert scenario, can involve the same questions and techniques as questionnaires used in large surveys. Of course, the claims cannot be generalisable, as you are less likely to speak to a range of people, but you can then compare your data with a representative sample.²³ Researchers should, therefore, think very carefully about the context of where they want to use the questionnaire, who and what they want to know about, and the limits of what can be known from specific questions asked of the people they are able to speak to. They also need to think about their impact: will they ruin the experience of the concert? Will they offend people in some way, or indeed, will the simple act of asking them if they enjoyed something affect their desire to say yes or no, and to communicate how much they enjoyed it? How much can be known from such a short-lived interaction with a hundred people? What use are these ‘snapshot’ data in answering bigger questions?

3.3.2 Interview Data

Interviewers are able to ask people what they think well-being means and what things are important in their life. We have already noted that questionnaires are used in national-level survey data collection; these usually use closed questions which can easily be added up quantitatively. The questions are asked by ‘interviewers’, whose job is to ask closed questions and make the experience of the questionnaire as similar as possible for all respondents.

Contrarily, interviews are a common feature of smaller data collection projects, where the questions can be more open-ended and may have very few questions at all. It is also common for an interviewer to develop a rapport²⁴ with an interviewee: something which you might hear talked about in positive terms when journalists interview key figures. Having a connection with your interviewee can, therefore, make the interview better, because people trust the person they are speaking to—because the data (the information) are richer and more detailed.

We tend to think of interviews as one-to-one situations, but you can do group interviews, often called focus groups.²⁵ In my PhD research, I started my focus groups with a question from the ONS’ MNW debate: ‘What Matters to You?’ They were designed as group discussions, where people had a lot of time to talk about a few questions at length, rather than asking lots of questions and people having less time to answer them. There are merits to both approaches, but I decided that it was more important to my research that people speak amongst themselves about what was important to them and think about how it related to well-being (Oman 2017). What we call ‘a structured interview’ has a strict set of questions which all interviewees answer, and these can be applied in a group setting. A ‘semi-structured interview’ is more fluid, allowing the interviewee to bring up whatever they want, which could be entirely unexpected, and so each discussion can take a completely different direction. Taking the former approach would have made my conversations as similar as possible for comparability; the latter allowed me to watch people chat away about anything they thought important.

The group discussions²⁶ I have organised in previous research projects have produced qualitative data that are largely subjective and about all different domains of people’s lives and experiences. For my well-being focus groups, people talked about all sorts: redundancy, bereavement, suicidal thoughts, loneliness, parenthood, their sexuality, education, careers, disabilities, dwindling community resources and transport and their hobbies. To return to the concert example, in the kind of well-being data I collected with open questions, people might talk amongst themselves about local events, without being asked a question about concerts at all. As it was, although many people talked about the value of their leisure activities (Oman 2020), the only occasion people talked about concerts specifically was not to say how much they enjoyed one in particular. Instead, one young person barely noticed and the other (in the exchange below) was highly critical of a large-scale cultural event in Northern Ireland. Here’s a snippet between a 17- and 18-year-old, who I’ve renamed James and Jack:

James: :

During the summer when they had the big concerts and stuff, that was like the only time I noticed that town was different. It kind of seemed like it was all decorated and everyone was kind of buzzing.

Jack: :

Yeah but I just think was kind of a distraction purpose to turn everyone’s heads away from the real issue. Like a home basically, we need a home to live, people die on the streets how many times a year? And they’re dressing up the city as, oh we’re a great city and people are lacking the basic human rights, that is not right.

One thing about research which aims to evaluate how people feel about things is that the longer you allow them to talk, the more comfortable they feel, which can mean that they become more honest. It can also mean that they deviate from the topic, and may not say what you anticipate. Another example from this research was a community arts project where I expected people to mention the arts project in relation to their well-being—especially as it was the one thing they had in common and the reason we were meeting. Yet, they did not refer to it, not even once. Instead, they held a very political discussion about the lack of community services for their families in their area. This may be that they thought that was what I was there to listen to, so I could report back in some way to an authority that would do something about these aspects of their lives. It is not always possible to conclusively know why an open conversation has followed a particular path, and part of qualitative research is to reflect on the possibilities of why that may be.

Another thing to bear in mind with these sorts of data collection is that through discussion, people find themselves agreeing with others in the group. This may mean that opinions expressed independently at the beginning of a focus group²⁷ have evolved through discussion and group ‘meaning-making’ (Freeman 2013) or it can mean that they feel pressurised to assimilate to ‘groupthink’. It can be hard to establish which of the two processes have provoked a changed opinion and how that affects your results. Again, aspects of the context can give you clues and are part of your methodology in group interviews and focus groups, as much as any other approach. Their limitations can be as much opportunity as confounder, as long as they are considered.

My PhD²⁸ focus groups enabled me to speak to over a hundred people and listen to them discuss what mattered to them about well-being. This was important to my research question which wanted to recreate a debate-like feel and therefore encourage people to talk—and debate—amongst themselves. But this can mean that quieter people’s views are not as audible and that it is not possible to understand how many of a group feel one way over another. As you can see, all decisions have pros and cons to weigh up.

One-to-one interviews enable you to understand the perspective of one person in detail and then compare that with the views from another interview, if appropriate. They can be used in evaluations and impact studies, providing testimonials of experience. Also, much like with focus groups, these are often transcribed into long pieces of text. This turns audio qualitative data into textual qualitative data and can take considerable time to analyse and compare. Interviews offer incredibly rich data on a person’s well-being, and with a well-thought-out strategy, can enable researchers to make some broader claims about how a particular group of people experience something like well-being, or indeed what is important to them about it. However, these claims must acknowledge the limits of context as discussed above.

3.3.3 Ethnographic Data

Another way that interview data might be useful for understanding well-being would be in the case of ethnographic research investigating the impact of a social policy. Ethnography involves a researcher spending a long time in their research site. This means they understand as much about the context in which they are collecting data as possible. For example, Kelly Bogue was embedded in her local community investigating the impact of ‘the bedroom tax’ (2019). ‘The bedroom tax’ was a nickname for an aspect of the Welfare Reform Act (DWP 2012) which meant that people living in social housing saw their benefits reduced by 14% if they have a spare room or 25% if they have two or more. The negative well-being implications of this policy on the community studied were multiple, with carers and those registered disabled being penalised for necessary home adaptations, alongside the anxiety and stress of people forced to leave the homes in the communities in which they had lived for decades (Bogue 2019). While this research was not seeking data to answer questions on well-being per se, the study produced much data that could inform well-being research for policy.

Some ways of doing ethnography allow you to participate in a context as a contributing member. This means practitioners, whether social workers, artists or people working in an office, might find it a useful way to examine well-being within their own work contexts. Overall, it’s rich, meaning that there is much detail gathered to deepen understanding, but time-intensive and gaining permission can be difficult to negotiate unless the researcher is already embedded in the community. Crucially, this kind of ethnography writes you into the context, so you affect it to an even greater extent than time delimited interviews. This requires thinking through in terms of whether it is ethical, or too intrusive. It also needs to be considered in terms of the claims that can be made: how would the particular context have been different had you not been there?

3.3.4 Secondary Qualitative Data

Qualitative data are increasingly collected with a view to the data being used again. This means those collecting data must be mindful of this when designing the questions asked and ensuring interviewees give permissions for storage, secondary access (used by someone else) and re-use (in publications or otherwise). Secondary data usage involves analysing data collected by someone else, as opposed to analysing primary data that you collect yourself and is more common with quantitative data.

It is sometimes possible to ask permission to access qualitative data that were not collected with the same questions in mind. This would mean that the same data, collected for a different purpose, could possibly be reanalysed to answer the specific question: ‘what were the impacts of X social policy on the well-being of a specific community between X and X date, for example?’ However, much qualitative data are too specific, in that they contain too much data and information about issues that are too personal to the people involved. For instance, given the sensitive nature of Bogue’s data on the bedroom tax, it would be unlikely that these data could be reanalysed to answer a broader question on well-being for ethical reasons, even if it were a practical possibility. It is unlikely that permissions for re-use would have been sought at the time of collection, and were people told the data might be placed in a repository, they may have not been as honest. These kinds of data are extremely difficult to anonymise in a way that completely protects participants and were one to try, perhaps there would be very little left to analyse. The benefits of qualitative data in capturing the specificities of people’s experiences, therefore, mean there can be barriers to secondary qualitative research.

Data collected by international bodies, such as the International Monetary Fund (IMF) or the UN, and national statistics agencies, such as the ONS in the UK, make their data publicly available for secondary analysis. These are primarily quantitative data and in addition to the findings that these bodies publish themselves (often presented as tertiary data). The ONS have pages and pages of findings and data on their website under well-being now, and there are a lot of data available from the UN’s HDI on its website.²⁹

Sometimes large surveys managed by national and international agencies, and available for secondary analysis, contain free text data (as shown above). If qualitative data has been collected at a large enough scale, then there is sometimes value in coding these and then adding up (aggregating) the answers which are similar, and turning this qualitative data into quantitative data. In 2013, I requested permission from the ONS to access free text fields from the Measuring National Well-being debate. I had developed a hypothesis from reading a report which contained quotes from the debate that I wanted to investigate.

My research question for these data, related to the issue we found outside the imaginary concert (described earlier in this chapter). If the evidence we have about the well-being impact of particular leisure and cultural activities can be argued as circumstantial, and from leading questions, the credibility of data is called into question—most specifically, its collection (Selwood 2002; Belfiore 2002). This is an issue that plagues arguments over the quality of the evidence on the relationship between aspects of culture and well-being that we return to in the second half of this book. If the data can be dismissed as resulting from leading questions and years of research projects that are therefore not able to offer generalisable results, then how might this issue be addressed?

I proposed we turn this question on its head. How does that help us overcome some of these issues with the context in which these data are collected? What if the question was more like: ‘When people describe well-being, how often do they talk about participating in different kinds of activities—and what might that tell us about aspects of social and cultural policy?’ I coded 6787 free text fields on well-being that were collected by the ONS and collated them into themes of all the things they talked about. I then quantified the themes (Oman 2015, 2020) and then ordered them in terms of prevalence of response.

Table 3.2 shows the difference in order according to what the ONS said it found in the overall debate (34,000 responses) and what I had found in the free text fields. Again, people did not refer directly to concerts (using the word) in the national debate, and only once in a subsequent consultation (Beaumont 2011, 29), but people did refer to the importance of broader concepts of social and cultural participation (Oman 2015, 2019, 2020).

Table 3.2 ‘A re-ordering’ of priorities in the Measuring National Wellbeing Debate Questionnaires

Well-being data include many sorts of data beyond those used in national indicators or the statistics we read in the media. They can all be extremely useful to inform work of many kinds from social work and policy, to arts administration, to the management of a particular company or understanding how to better care for students away from home at university. The data required, and how they are analysed, involve a balance of what needs or wants to be known (see Table 3.1). It is also a practical matter of preference of approach, skill and resource; all need to be balanced and there are various limits on different kinds of data to answer different questions. Table 3.3 offers an overview of how different data can help answer different questions for different reasons and/or audiences. 

Table 3.3 Overview of data types and possibilities for answering well-being questions

Understanding whether data are ‘good data’, as in good at the job you need it to do, requires an appreciation of all the many aspects of the context, situation and/or population you want to understand. It requires an understanding of what you want to know about well-being, which we have discovered is contestable and varied in different contexts. Thus, for them to be data for good (and thus good for well-being) requires context and reflection. There is a tendency to view and to use certain kinds of data as if they are objective and unaffected by human decisions. The next sections look at objective data and their issues.

3.4 Objective Well-being Data and Measures

In terms of quantitative data, you might imagine that the key question is how should well-being be measured? Really, this is a much bigger question, or series of inter-related questions, which are how should well-being be conceptualised, operationalised and measured? Or before well-being can be measured, we need to decide what we mean by well-being (conceptualise) and find measurable dimensions of our concept (operationalise),³⁰ and then we can decide on a way of measuring it.

We have discussed some of the methods of collecting well-being data. Many decisions are involved that are not always made obvious, but are all important. The point here is that the conceptualisation of ‘what is it we’re actually trying to get at when we want to understand well-being’ is distinct from its operationalisation. It is also worth noting that to operationalise a concept in research has a slightly different meaning than it does in everyday life. We come back to this in Chaps. 6, 7 and 8. If someone operationalises something, it generally means they put it to use, or bring it into use. In research, it is more a process of establishing how we can measure. So, conceptualisation is different from operationalisation, but connected. The operationalisation of ‘here is the form of words we’re using to ask the question’ is different again from ‘here are the options for the answers people can be provide (and if applicable, how we’ll combine these answers from different questions to give people an overall well-being score)’.

As we have hopefully established in the introduction to this chapter, money is important in most contexts, but is far from everything. There are many more features that shape people’s lives and that need to be understood if we aim to understand well-being as quality of life (Dodge et al. 2012). You could ask a population any number of questions to understand aspects of their quality of life. For example, is your housing adequate? How sanitary is your local environment?³¹ Do you have public institutions that respond to your needs? Would you say have an active social life? Are quality healthcare and education services easily available to you? You may note that all of these questions are phrased in such a way that they ask for people’s opinion on aspects that are thought to affect our quality of life. They are therefore going to produce data that are subjective.

All of these issues can also be measured using data that are objective indicators. For example, administrative data such as GP visits and hospital wait times could be used to generate a benchmark for ‘fair access to healthcare’, and then community-level data could be measured against this benchmark. These are proxy indicators because they do not directly answer the question ‘does this person have fair access to healthcare’, but are used to stand in for data that could.

Proxy indicators have a number of pros. They are not biased by people’s inaccurate memories of how long they waited in hospital, which, for obvious reasons, may be clouded with frustration. You do not have to worry about issues of sampling bias (see Box 3.4). Also, proxy data have often already been collected and cleaned by someone else, or a statistical organisation. So, while they can only partially answer the question of how many people have fair access to healthcare, the pros will have been thought to outweigh the cons. Similarly, being able to answer a research question on fair access to healthcare doesn’t tell us everything we need to know about well-being: it is one aspect of well-being. It only partially indicates someone’s quality of life, and so to understand quality of life more completely at population level, we need more indicators.

Objective measures of well-being are based on assumptions regarding human needs and rights, believed to impact on quality of life. Herein is the difference between quality of life and well-being. The academic literature tends to assume that quality of life involves material conditions, whereas well-being also involves life satisfaction, mood and meaning (although as we know from the previous chapter, this is not always clear-cut). It is the quality of life aspects of well-being that are measured with objective indicators using the objective list theory that most indexes are based on.

The existence of the list, of course, suggests that a person or people with expertise have decided what should go on the list: what is important and what standard measures should be used, or indeed to whose standard? There is even an ‘objective list theory of well-being’ (Rice 2013) that is pluralistic. This means that instead of identifying a single feature common to all states of well-being (think of an overarching argument for ‘what is the meaning of the good life?’), it identifies a number of characteristics of what makes for a good life. This philosophical theory is applied to lists of objective indicators, of what would be all the qualities needed for a good life. The key is that the aim is to cover all the important domains in life, so unlike a simple index, like the HDI, these tend to have lots of indicators. In other words, the well-being data are about lots of aspects of life.

The previous chapter explained a brief history of the move away from a single measure of progress (GDP ) towards multiple measures of well-being in the twentieth century. These tended to be an index of multiple objective indicators of quality of life, associated to different ‘domains’ of life. Some organisations and nations recognise the same six major objective and observable dimensions for the measurement of objective well-being. These include international organisations, such as the Organisation for Economic Co-operation and Development (OECD 2011a) and the United Nations Development Programme (UNDP 2015), as well as national statistics offices, such as the Italian Statistics Bureau (ISTAT 2015). Notably, within each dimension are multiple indicators (ordinarily two or three). Figure 3.1 shows just how many indicators there are within domains in the OECD’s index and per member country. As we shall discover in the following chapter, these bodies all heavily influence each other by way of advisory groups and drawing on perceived best practice.

Fig. 3.1

OECD well-being indicators. (Source: OECD Compendium of OECD Well-Being Indicators 2011)

Given that the theory behind the objective list approach means you need to analyse data from across all these dimensions, this can make it difficult to interpret these data, even at headline level (see Fig. 3.1), but also to compare them. Changing the unit of analysis from each indicator, to per country, or domain, makes them more readable. This is the same as with the Dow Jones, where the index is designed to have a single measure for readability. With the HDI,³² the three dimensions are combined into a single measure for easy comparison.

With more complex indexes than the HDI, such as the OECD’s (Fig. 3.1), decisions need to be made on balancing the importance of the different domains. As we know, each of the three domains contributes equally to a country’s overall HDI score (United Nations 2020), this is not the case with all indexes. If domains are not equally weighted, then decisions have to be made about the relative importance of each to overall well-being decisions. As Table 3.2 demonstrates, establishing the importance of different domains of well-being is not a neutral process.

To this end, these weights involve subjective decisions by experts on what is more important about the objective indicators. That is not to say it is not a rigorous process, that it is not based on much evidence, and that experts do not debate and review these processes to ensure robustness. Yet, the term objective can obscure what is going on behind the scenes, or underneath the hood, if you like, of what are called ‘objective indicators’ of well-being, or imply that they arrive at some sort of universal truth about well-being. As criticisms over the HDI surface, people do not value these aspects of life equally, or, indeed, the same as each other. Remember that there is a difference between measuring what is valuable and what is valuable to measure—to whom and why.

Box 3.4 Weights and Sampling Bias

Weights

The term ‘weighting’ is used in several different ways in the analysis of quantitative data, and it’s important to be clear about which way we’re talking about.

In this section, we are concerned with how different bits of information about countries are combined to give an overall score for those countries. Or, how important money is, as opposed to education or health. The HDI applies an equal weight to these categories.

Weighting is also used to describe a technique when working with survey data to correct for sampling bias. As we have discussed, it is rare to achieve a whole population, and so most survey data are a sample. No matter how large that sample is, your sample is unlikely to look the same as the whole population, so you need to adjust for different proportions who answered the survey. For example, younger people are often less likely to respond to surveys, so estimates based on surveys often weight young people’s responses more heavily to adjust for this difference.

These two different meanings of the term ‘weighting’ are applied in very different ways—in one case, to the questions that are being asked, and in another, to the people who are being asked the questions—and shouldn’t be confused.

An attempt to counter criticisms of weights applied by experts, The OECD states that its ‘Better Life Index is an interactive composite index that aggregates average measures of country’s well-being outcomes through weights defined by users’ (OECD 2018, 4). What does this mean, and why have the OECD attempted to do this? Let’s break this down.

The OECD’s Better Life Index website has an interactive dashboard, enabling people to use sliders to order and balance the importance of different aspects of well-being. When people use the sliders, they are effectively applying weights to the different aspects of well-being to construct an overall index that is personal to them.³³ In this instance, the index aims to avoid representing the experts’ view of what is valuable, presenting those of the person interacting with the dashboard back at them.

This is all well and good, but how does this impact on change for social good? Are the OECD listening/watching/recording these interactions, and how might it change the way they value what is important? While some analysis has been done on people’s interactions and values (OECD 2018), and this dashboard implies democratic engagement or participatory decision-making to a degree, there is no commitment to this. People are also only able to interact with the pre-defined categories: were something of importance to you not there, there is no way to include this in the dashboard or tell anyone it should be included.

The terminology, processes and decisions behind what are used for objective well-being data, and how they are used together—as an objective list of indicators—are complex. I have tried to cover specific examples and drill down into the processes of why things happen in certain ways and to explain some of the terminology. We are going to look at one index in greater detail in the next section. This is to help those who wish to understand what goes on underneath the hood of a well-being index and to have a better understanding of what decisions are made about what good data practices might be for well-being data.

3.5 The OECD as a Case Study of What Lies Behind Objective Well-being Data

Measuring well-being and progress has been and will continue to be a key priority for the OECD, in line with its founding tradition to promote policies designed to achieve the highest living standards for all. (OECD 2011a, 4)

The OECD have been key to the ‘second wave’ of framing well-being as important to measure (Bache and Reardon 2013). National well-being initiatives have tended to be in OECD or EU countries, and it is thought that the OECD had a hand in the process of the influential Sarkozy commission (Bache 2012, 26, 30). The OECD Framework for Measuring Well-Being and Progress is said to be based on the recommendations from the commission (OECD n.d.b). We are going to peer under the bonnet of how the OECD devised its well-being indicators to reveal the decisions and care that go into such a programme.

The OECD claim that:

the ultimate objective of this work is not just measurement per se, but to strengthen the evidence-base for policy making. Better measures of well-being can improve our understanding of the factors driving societal progress. Better assessments of countries’ comparative performance in various fields can lead to better strategies to tackle deficiencies. (OECD 2011a, 4)

The OECD wanted to understand well-being in a way that can both offer comparisons across nations and potentially inform policy evaluations. They decided the qualities that best represented well-being; made objective lists; researched appropriate proxy indicators using existing data that can answer the dimensions of well-being. They tested the indicators that they have used to meet the demands of their well-being framework and ensured that they meet additional quality criteria; they sought expert advice on these moving parts and then offered a caveat on the experimental and evolutionary nature of these metrics: they will change and they are not perfect. This level of detail is not always readily available when research is published. So, we are going to look at the decisions made in the devising of the index in order to understand what lies behind these well-being data.

The OECD devised a list of criteria of what would be good to measure. Crucially, they also undertook a review of the data available from member countries (who, of course, may not be measuring the same thing). Prior to finalising the index, a compendium was released, which contained the framework on which decisions were made regarding which well-being indicators they might use (OECD 2011a). This was the criteria they published in the compendium:

• the well-being of people in each country, rather than on the macro-economic conditions of economies; hence, many standard indicators of macro-economic performance (e.g. GDP , productivity, innovation) are not included in this Compendium.

• the well-being of different groups of the population, in addition to average conditions. Measures of inequalities in people’s conditions will figure prominently in the “How’s Life?” report but are only discussed briefly in this Compendium.

• well-being achievements, measured by outcome indicators, as opposed to well-being drivers measured by input or output indicators.

• objective and subjective aspects of people’s well-being as both living conditions and their appreciation by individuals are important to understand people’s well-being. (OECD 2011a, 5)

The OECD were also keen that their framework distinguished between current material living conditions and quality of life, on the one hand, and the conditions required to ensure their sustainability over time, on the other. Notably ‘material living conditions’ do not always mean economic, and often the term elsewhere incorporates quality of life dimensions, as discussed above.

• Material living conditions (or ‘economic well-being’) determine people’s consumption possibilities and their command over resources. While this is shaped by GDP , the latter also includes activities that do not contribute to people’s well-being (e.g. activities aimed at offsetting some of the regrettable consequences of economic development) while it excludes non-market activities that expand people’s consumption possibilities.

• Quality of life, defined as the set of non-monetary attributes of individuals, shapes their opportunities and life chances, and has intrinsic value under different cultures and contexts.

• The sustainability of the socio-economic and natural systems where people live and work is critical for well-being to last over time. Sustainability depends on how current human activities impact on the stocks of different types of capital (natural, economic, human and social). However, suitable indicators for describing the evolution of these stocks are still lacking in many fields. For this reason, indicators of sustainability are not included in this Compendium, although some of them will feature in ‘How’s Life?’ (OECD 2011a, 5).

The OECD claim that the framework reproduced above ‘underlies the selection of indicators in each dimension of well-being’ that work within two additional quality criteria:

• conceptual soundness (i.e. relevance in terms of measuring and monitoring well-being across the population in the perspective of informing policies)

• data of high quality (i.e. based on well-established standards and codes of practice). The selection of indicators has been made following extensive consultation with National Statistical Offices and experts from various OECD directorates. (OECD 2011a, 5)

It is within the tension between conceptual soundness and the quality of data that the sustainability indicators sit: they would be what we would ideally be measuring if we want to capture well-being; remembering that the principle of an objective list is that the indicators included (BetterEvaluation 2012) are all vital to well-being. It is interesting that the OECD consulted with individual statistics offices on which indicators to select. The UK’s ONS also state they consulted the OECD to decide their well-being metrics (Oman 2017).³⁴ Therefore, despite apparently separate investigations, the same experts were informing different indices. Sharing expertise is undoubtedly a good thing, especially when it comes to methodological rigour, but it might arguably limit the possibility for independence or innovation in how countries measure the well-being of their citizens. Notably, despite the fact that Bhutan’s measures of Gross National Happiness are often cited as inspiring the OECD, Sarkozy commission, and so on, expertise from Bhutan is not very evident on these advisory groups. We return to this in Chap. 6, but who the experts are, are always important questions to ask.

Another important thing to note about the OECD’s contribution to well-being data are the caveats that were presented alongside these domains, namely that the indicators are:

• experimental, in that the proposed selection of indicators has not yet reached the stage of meeting all agreed standards;

• evolutionary, as the indicators proposed in this Compendium are, in many cases, only proxies of a broader underlying outcomes, for which ideal measures are currently lacking. (OECD 2011a, 7)

The report also notes that the selection of indicators will change in the future as better measures are developed, and as member countries reach agreement on indicators that are more appropriate to summarising conditions in the various dimensions of people’s lives (OECD 2011a). So, whilst these national indicators tend to be presented as absolute, or fixed, in some way, like other forms of science and social science, they are invented to respond to developments and improvements. This is rarely acknowledged when objective indicators are presented in official reports and briefings.

So, what might these indicators look like?

The description ‘bewildering array’ (Scott 2012, 36) may come to mind when looking at Fig. 3.1. As a result, Table 3.4 shows only the domains and indicators in 2010. There are 21 indicators across the 11 domains, with a row for each member country. This is how the indicators were presented in 2010. Some of these have now changed, perhaps imperceptibly to most. It can be difficult to establish exactly what is meant by or what has changed about an indicator, why, and when that change happened, because this information is not readily available.

Table 3.4 Summary of the OECD indicators in 2010

To explain what I mean here, we are going to zone in on the ‘domain’ of ‘Personal Security’, in our case study. Personal Security has two indicators in our 2010 visualisation: intentional homicides and self-reported victimisation. So, one question might be, ‘why not just say crime, if you mean crime?’ If you look at all the domain names, they are all positive in their inflection: environmental quality might read as pollution, or litter, for example. What is also interesting about the idea of personal security is that it does not necessarily mean crime, really. It could possibly include financial security to most people: do you have a pension; do you own your own home, and so on?

Another question is why, then, has the domain changed in the current 2020 version of the index? The Personal Security domain name is now called ‘safety’. The OECD explain this domain as follows: ‘Personal security is a core element for the well-being of individuals, and includes the risks of people being physically assaulted or falling victim to other types of crime’ (OECD website/topics/safety). Therefore, the headline domain name has shifted from ‘personal security’ to ‘safety’, but has retained the credibility of the original measures.

Not only has the domain name changed. The indicators themselves have shifted: ‘homicide rate’ remains the same, but ‘self-reported victimisation’ has been replaced with ‘feeling safe walking home at night’. Thus, an objective indicator has been replaced with a subjective indicator, as the data were collected by surveying how someone feels, rather than the administrative data from reporting crimes.

There are methodological reasons why this is a sensible change. In some places people do not report crimes, as they happen, so as a chosen proxy measure of a domain well-being, this is not necessarily the best indicator of the relationship between crime and quality of life. Secondly, it could be argued that it is in the ‘feeling safe’, rather than the reporting of crime that we experience well-being. This is why more subjective measures—even on an objective list—can be a better way of capturing what it is about well-being that we need to know.

In the previous section we encountered what objective indicators are, and this section has presented a lot of detail on one well-being index, as it is not always clear where such official-looking data come from. We focussed on some of the decision-making aspects of devising an index. This also revealed their methodological complexity—even without the quantitative modelling involved in statistics. We have also questioned the nature of the data assembled in objective lists and what is implied by their naming as objective. We have learnt that they are, in fact, shifting rather than fixed sets of measures. They evolve and respond to reflections on their limitations and how they could be done better. As we continue to use these sorts of data as objective facts, we lose these qualities, which are not considered important. Yet, the contexts of these data practices are both valuable and credible. It is a disservice to statistics and people who wish to understand them, that they remain obscured.

3.6 Conclusion

Understanding whether data are ‘good data’, as in good quality—or whether they are data for good (and thus good for well-being) requires us to look at context. We have to consider whether international indicators appeal to certain standards, and if so, how so, or to whose standard? Data are often used as if they are neutral and context-less, yet they have rich context that is rarely acknowledged. Understanding the expertise, reflections and decisions involved in these ‘objective data’ makes them appear richer and therefore could be argued to demonstrate, rather than decrease the appearance of rigour.

This chapter has aimed to offer an overview of different contexts that dictate both what and how good well-being data are. These environments have varied from local parks to international statistics forums; from a youth club in Derry five years ago, to a presidential candidacy speech in Kansas over half a century ago. Across qualitative and quantitative data; primary, secondary and tertiary data; proxy data, administrative data, survey data and ethnographic data. Data collected from talking to people can be harder to imagine as data, because we usually think of data as numbers. However, the contexts of these data—how they are collected and analysed—are also often easier for most people to imagine than those of international statistics. This is because it is easier for most people to picture themselves being the person speaking to people, either asking questions or answering them.

Most of us don’t spend much time thinking about how data experts work. Why should we? But then how statisticians and data experts work are not transparent, or often discussed. This is, in fact, a barrier to understanding how their statistics and data work. This is not a textbook, and so looking at all these different types of data may not make you a statistician, but in reading this chapter, you may have improved your understanding of well-being data and their diversity. Looking at these data in context should also better enable you to better appreciate these data when you next see them in the media or in another government briefing (hopefully not about COVID-19).

We start this chapter by contextualising a political quote that is used a lot to justify why well-being data are good. We also look at a collection of attempts to define well-being for data across some policy documents over time that coincides with the recent rise of well-being data. The reflections on this political speech and policy documents treat these texts as data, enabling us to contextualise policy, politics and data with well-being.

The chapter then reflects on a number of different situations in which well-being data are generated, interpreted, analysed and applied. A hypothetical scenario of a well-being at work survey, a questionnaire outside a concert and real-life examples of well-being data that are relevant to social and cultural policy are shared to show the variety and accessibility of some approaches to well-being data collection, but the need for caution, consideration of others and the foregrounding of context in these matters. How you affect the data and the participants by collecting and using data is crucial to all research on society, especially that which supposedly improves it. This is not only a moral and ethical issue, but one that can limit the claims that can be made using these sorts of well-being data, should the wrong decisions be made, or should they not be explained. Therefore, the consequences of well-being data are crucial contexts, as well.

The HDI and the OECD well-being measures evolved from working within professional codes to innovate and generate the indices. It is not always obvious that this is a long process of organising and interpreting by experts before final decisions are made. In presenting good practice and contextualising how these things work, these sections hoped to improve your capability and confidence (which we identified as data issues in Chap. 1) if you are less familiar with these data. The objective lists that feature in these new well-being indices are often made of data that have been long collected. Once this context is understood, they seem less revolutionary than the politics sometimes implies. It is actually the newer subjective well-being measures that were being developed over the 2000s that were the more novel aspects of these well-being indicators, and we come to the limits of these claims in the next chapter.

The very name ‘objective indicator’ suggests it is that: objective, but often the data does not measure what they say it measures, instead being a proxy for what would ideally be measured, were there a measure for it. You may have found yourself reading the section on quality of life indicators, thinking how these indicators would pick up on the negative well-being impacts of the bedroom tax that Bogue’s research uncovered. The answer is they are very unlikely to at national population or international population level, and were not designed to do that.

Well-being data are not all one thing. They have different purposes, pros and cons. Qualitative data are able to get closer to the meaning of well-being and the experiences of ill-being in some cases, but are often unable to generalise and are criticised for the subjective nature of the associated processes and the limits to claims of causation. We will look at how asking people how they really feel in surveys attempts to address some of these circumstantial issues of capturing the human experience in the next chapter where we put ‘the new science’ of happiness into context.