Introduction

Resilience has rapidly risen to the top of the development agenda (Burnard and Bhamra 2011; Frankenberger et al. 2014; Bahadur et al. 2013; Béné et al. 2012). The term is seen by development actors as a valuable conceptual tool in understanding how people respond and adapt to the many changing shocks and stresses that affect livelihood outcomes (Manyena 2006; Miller et al. 2010; Nelson et al. 2007). Inevitably, a push for resilience-building within the development and humanitarian communities has led to increased demand for ways of measuring levels of resilience amongst people and communities (Brooks et al. 2011). In theory, more accurate measurement and tracking of resilience can help to ensure that resilience-related policies and programmes are supporting the right activities and targeting the right people (Oddsdóttir et al. 2013).

Unfortunately, the assessment of resilience is fraught with complexity: both the definition of resilience and the methodologies used to measure it are heavily contested (Cumming et al. 2005). Confounding factors, such as what mix of indicators to choose, which systems and scale of analysis to apply, and how to recognise the context-specific nature of resilience each muddy the waters (Béné et al. 2015). Indeed, despite growing global interest in supporting resilience-building activities, existing approaches to the measurement and tracking of resilience have generally not been able to the deliver the desired policy support (Levine 2014).

A large number of frameworks and approaches have been proposed for quantifying household resilience (Bahadur et al. 2015). Most concentrate on ‘objective’ indicators by identifying key socio-economic variables and other capitals that support people’s livelihoods. The selection of these variables is often value-laden and contested (Carpenter et al. 2001; Bahadur et al. 2015). However, a complementary means of assessing resilience has largely been overlooked: ‘subjective’ household resilience. Subjective household resilience stems from the premise that people have a legitimate understanding of their own capacities, capabilities and limits. The measurement of perceived resilience is therefore about how people rate their own resilience, and the resilience of the wider community of which they form part.

In this paper, we call for the tracking and measurement of subjective resilience at the household level. We argue that efforts to measure resilience should take into account people’s perceptions of their own capabilities and capacities, either in combination with, or separate to, objective forms of resilience measurements. We then put forward different options for the design and delivery of survey questions on subjective resilience at the household level.

In order to narrow the context and provide illustrative examples of question format options, we have chosen disaster resilience as the entry point for this paper—more specifically, the resilience of households to weather and climate extremes. However, the same principles are likely to apply equally to other aspects of resilience, such as livelihood, community or social resilience, all of which possess many of the same characteristics.

Understanding resilience

As a concept, resilience has a wide variety of meanings and definitions. Although references to resilience can be found across the arts, literature, law, psychology and engineering (Alexander 2013), the use of the term within the ecological sciences has been particularly influential, where ‘resilience’ is used to understand and explain the different trajectories of ecological systems as they seek equilibrium (Walker et al. 1969; Odum 1985; Alexander 2013). Ecological conceptualisations of resilience largely focus on the capacity of a system to absorb changes but still maintain its core function (Nguyen and James 2013). Holling (1973:14) describes resilience as ‘a measure of the persistence of systems and of their ability to absorb change and disturbance and still maintain the same relationships between populations or state variables.’ The term has been widely adopted as a way of framing the complex dynamics between linked social-ecological systems and their ability to respond to disturbance (Carpenter et al. 2001; Folke et al. 2002). Seen through this lens, social or livelihood resilience is used to describe ‘the capacity of all people across generations to sustain and improve their livelihood opportunities and well-being despite environmental, economic, social and political disturbances’ (Tanner et al. 2015:p23; see also Eakin et al. 2012).

These disciplinary transitions have also challenged traditional framings of resilience, namely that systems may not necessarily return back to the same function or existence after a perturbation (Olsson et al. 2015). Cannon and Müller-Mahn (2010) argue that because the systems theory approach inherent in the term resilience stems from understanding of natural systems, the concept can be problematic when transferred uncritically to human systems that are regulated by ‘irrational’ power relations. Evidence from climate change and development policy discourses further suggest that resilience is largely used to support the status quo and promote ‘business as usual’ (Leach 2008; Brown 2012). More recent conceptualisations of resilience—mostly with regard to human systems—give greater recognition to the potential need of a system to adapt and change its core structures and functions, with some cases even requiring complete transformation (Pelling 2010; Aldunce et al. 2015). As a more radical agenda, resilience can thereby be recast from a concept that it is focused on returning a system to its original state, towards one that questions the underlying root causes of vulnerability and resilience (Pelling 2010; O’Brien 2012). Therefore, the resilience of a human system can be thought to comprise a range of different capacities and components, including, but by no means limited to: the capacity to absorb change (Nelson et al. 2007); preparedness and contingency (Twigg 2009); innovation and learning (Adger 2000); and renewal, reorganization and development (Folke 2006).

Resilience’s definitional and conceptual evolution has in turn made it difficult to agree on what constitutes a resilient human system. Knowing which components constitute the resilience of a system depends on the nature of the threat(s) (resilience to what?), the unit of analysis (resilience for whom?) and the context of the internal social dynamics of the system. The abstract and malleable nature of the term, the lack of conceptual clarity and strong overlaps with related concepts each make the process of conceptualising resilience difficult (Aldunce et al. 2015; Nelson et al. 2007). With regard to the latter issue, uncertainty about the relationship between resilience and similar properties such as adaptive, coping and transformative capacities (Béné et al. 2015) remains a source of contention. Indeed, the terms are often referred to interchangeably across the academic literature (Bahadur et al. 2015). This is the case even for specific sub-fields of resilience:

In disaster management, [resilience] refers to multiple aspects ranging from absorbing and recovering from, to resisting, the effects of a hazard, as well as preserving and restoring “essential basic structures and functions”. Such wide meanings may end up being contradictory as in the notion of restoring equilibrium and getting away from it by moving to a new system state. (Olsson et al. 2015:22)

A lack of clarity in how to apply resilience in practical terms and no clear consensus on what should and should not constitute resilience further muddy the waters, particularly when it comes to designing tools for the measurement of resilience (Djalante and Thomalla 2011). Despite these challenges, the development and humanitarian communities have shown great interest in using the term to help guide operational activities and create greater cross-disciplinary linkages (DFID 2014; Béné et al. 2015; Schipper and Langston 2015), and there are growing quantities of international finance allocated towards ‘resilience-building activities’ (Peduzzi et al. 2009; DFID 2014). With increasingly ambitious commitments and large international programmes dedicated to resilience-building, there is a clear need to measure impact and track resilience on the ground. In spite of the conceptual ambiguities and inconsistencies, the race is on to determine the best ways of measuring resilience.

Rationale and foundation of a subjective approach

The primary focus of this article is on household-level assessment of resilience. Reasons for this are manifold, recognising the centrality of the household unit in governing responses to external stimuli (Toole et al. 2016)—alongside other widely applied scales of analysis including individual, local and national-levels. Indeed, many of the assets, capacities and functions required to respond to climate risk derive from are dictated by household-level dynamics (Frankenberger and McCaston 1998; Barrett et al. 2001). More widely, assessments of household resilience offer challenges to national and community level assessments that dominate much of the resilience literature: ‘household-scale analyses show that assumed capacities and vulnerabilities may end up being quite different to those imagined or measured at a macro-scale’ (Toole et al. 2016). Crucially, household-level assessments also offer value in capturing the interactions of individual-level decisions and traits with wider social norms, behaviour and institutions that collectively affect responses to climate hazards (Adger 2000). With this in mind, a thorough understanding of individual-level characteristics and measurement biases—such as psychological resilience, risk tolerance and personality traits—is required to be able to disentangle any such individual influences on household-level assessments (see “Learning from subjective indicators in related disciplines” section). However, it is important to note that the theoretical underpinnings of the methods discussed can apply equally to subjective assessments of individuals, organisations, communities or nations if tailored accordingly.

Similar to definitions of resilience, numerous different approaches, methods and frameworks for measuring resilience at the household level exist. In spite of such diversity, many quantitative measurement frameworks follow the same core steps, though not all approaches tackle them in the same way (Bahadur et al. 2015). The first step is usually to identify suitable characteristics of resilience through the observation of a particular system and drawing on the wider literature (Twigg 2009). Relevant ‘objective’ indicators are then assigned as proxies for each characteristic, typically drawing on the available socio-economic data. Lastly, in the case of single-item measures, these characteristics and indicators are amalgamated into a composite index, often with indicators being weighted differently (Constas and Barrett 2013; FAO 2014; Elasha et al. 2005; USAID 2013). While these assessments may have drawn on locally collected data, they often rely on statistical relationships at an aggregate scale.

While such approaches have operational benefits, they are not without weaknesses. For one, it is extremely difficult to identify all the relevant traits and indicators—from economic to sociocultural and political factors—that influence a household or community’s resilience (Cutter et al. 2008). Approximations have to be made, and this places considerable weight on the choice of framework and characteristics used. The context-specific (and scale-specific) nature of resilience also means that identifying the right indicators is challenging: what contributes to resilience in one community may not have the same effect in another (Engle 2011). Measures determined from the top down may favour more structural determinants at the expense of those based on human agency, which may be harder to understand and measure (Tanner et al. 2015). In addition, the range of different data sources and inputs needed in compiling such indices means that large household surveys are usually required, which are often costly and time-consuming (Constas et al. 2014). Crucially, such traditional approaches speak little to how people evaluate their own lives, and often require value judgements to simplify the complex nature of resilience across so many different contexts (Diener et al. 2002).

Limited attention has been given to date to exploring approaches to measuring subjective aspects of resilience. Understanding subjective resilience is imperative to understand the less visible but potentially crucial aspects of what makes households resilient, and what resilience really means to different people. Subjective household resilience can relate to two important (and overlapping) factors. Firstly, it relates to the notion that a household’s resilience is comprised not only of tangible objective elements, such as the availability of various livelihood assets, but also wider social, cultural and psychological elements (Adger et al. 2013). The subjective elements of resilience are associated with a range of issues such as perception of risk, sense of place, beliefs and culture, social norms, social cohesion, power and marginalisation, and cultural identity (Grothmann and Patt 2005; Adger et al. 2009; Clayton et al. 2015). Despite the difficulty of assessing many of these factors, they are nonetheless crucial to household and community resilience. Current assessments and conceptualisations of resilience seldom capture these more subjective elements (Brown and Westaway 2011). Factoring them into evaluative frameworks is therefore key to gaining a more holistic understanding of resilience, particularly at the household and community levels.

The second factor relates to the subjective assessment of an individual’s own resilience or the resilience of others and other conditions around them, whether at the personal, household or community level (Marshall 2010). This topic has been far less studied and is the primary focus of this paper. We define subjective household resilience in terms of people’s perceived level of household resilience to specific external shocks and stresses. It relates to a person’s cognitive and affective valuation of their own capacity to anticipate, buffer and adapt their livelihoods to disturbance and change.

The relationship between these two factors of subjective resilience is complex. On the one hand, psychological and cultural elements will inevitably affect how a person rates their household’s ability to respond to disasters (Jones and Boyd 2011; Graber et al. 2015). For example, two members of the same household—perhaps one with the personality traits associated with overt optimism, the other with pessimism—may well rate their household very differently. Thus, in many ways subjective elements can act as a significant bias to subjective self-assessments of a household’s capacities. On the other hand, these same psychological and cultural elements also have a profound influence on household resilience. For example, cultural norms such as ethnic marginalisation will impact the ability of certain social groups to respond to disasters, perhaps through restricted access to key resources or economic marginalisation (Burton and Cutter 2008). Individual subjective traits, such as risk aversion or risk-taking, may also affect how a household chooses to respond to disaster risk and therefore influence their household’s overall resilience. Any self-assessment of household capacities, therefore, has to be mindful of the distinctions between these two potentially opposing traits, and seek ways of recognising and accounting for relevant biases.

A useful way to illustrate the differences between subjective and objective methods of resilience assessment is to compare the approaches taken by Nguyen and James (2013) and FAO (2014). Under FAO’s objective Resilience Index Measurement and Analysis (RIMA) tool, household resilience is broken down into five key dimension of productive assets, access to basic services, social safety nets, sensitivity, and adaptive capacity, each with a large number of associated variables. Adaptive capacity, for example, is calculated using five key objective variables, including the number of different sources of household income; the ratio between employed people and labour force in the household; the total number of years of education for the household head; the number of literature people in the household; and a coping strategies index. Factor loadings are then applied to each of the five dimensions and used to calculate an overall score. This differs markedly from the more subjective approach taken by Nguyen and James (2013) that calculates household resilience to flood risk by assessing individual responses to ten questions marked on a five-point likert scale. Questions include: ‘I am confident that my household has enough rice to eat during the flood season’; ‘I am confident that my household can find a safe place to evacuate to if there is an extreme flood event in the future’; and ‘I am confident that my house will not collapse or be swept away by the highest floods in the last 20 years’ (Nguyen and James 2013:17). While both approaches aim to assess the same property, neither can be considered a true reflection of a household’s overall resilience; both have their relative strengths and weakness that need to be considered when choosing which approach to apply in any given context (see “Advantages of a subjective resilience approach to measurement” section).

Importantly, assessments of subjective resilience are not an alternative to more ‘objective’ definitions of indicators of resilience, but rather can provide both ground-truthing and an indication of the causal relationships with a wide range of socio-economic, psychological and institutional factors that contribute to greater or weakened levels of resilience. If care is taken to design suitable methodologies for data sampling and collection, then a household’s subjective resilience can, in theory, be readily quantified and used as a complementary approach to objective resilience measurement. Importantly, assessments of subjective resilience are also subject to response bias, affected by context and difficult to translate into different languages. However, they offer the opportunity to complement and significantly enhance current resilience measurement practices.

Learning from subjective indicators in related disciplines

The idea of subjectively defined resilience has many parallels with the conceptual and practical challenges of related fields across the social sciences. In particular, work on perceived adaptive capacity, subjective well-being, and psychological resilience is salient.

A number of climate change adaptation studies have explored subjective elements at the individual and household- levels (Adger et al. 2009; Brown and Westaway 2011; O’Brien 2009; Nguyen and James 2013). For example, Grothmann and Patt’s (2005) conceptual framework seeks to understand ‘perceived adaptive capacity’ based on sub-components of perceived adaptation efficacy, perceived self-efficacy and perceived adaptation costs. Their qualitative case studies in Germany and Zimbabwe were expanded by Frank et al.’s (2011) study of coffee farmers in Chiapas, Mexico, which identifies social identity as an important additional component of an individual’s perceived risk and adaptive capacity. While more quantitative studies have been limited to date, some of these have questioned the suitability of identified components from previous qualitative frameworks (Blennow et al. 2012; Blennow and Johannes 2009).

Survey methods to investigate levels of ‘social resilience’ have also attempted to determine sets of subjective explanatory variables, including perceived levels of risk and perceived capacities to cope, plan, learn and organise (Marshall and Marshall 2007; Marshall 2010; Seara 2014). Similarly, Lockwood et al.’s (2015) psychometric approach to adaptive capacity and personal resilience study is notable for its systematic process of identifying base survey questions. However, most studies to date are case study based and the development of standardised questions to assess subjective resilience is a key future challenge. Indeed, few studies have taken the next step and sought to use subjective approaches to help guide resilience-building initiatives and policies (Marshall 2010).

Perhaps the field most closely related to subjective resilience is well-being. Buoyed by the recognition that a country’s progress and development should be measured not just by its GDP but wider measures of economic, social and environmental impact (Costanza et al. 2009), the assessment of well-being has received considerable attention both from the research and policy communities (Diener 2000; OECD 2013). Well-being is commonly measured in two ways: either through objective or subjective indicators. While objective well-being is determined by a predefined list of requirements deemed to contribute to a ‘good life’ (Guillen-Royo and Velazco 2006), subjective well-being can be thought of as people’s multidimensional self-evaluation of people’s own lives, including cognitive judgments of life satisfaction as well as affective evaluations of moods and emotions (Frey and Stutzer 2002; McGillivray and Clarke 2006). Subjective well-being is also playing an increasing role in international development and livelihoods research, recognising that the need to ‘move away from outsider categories towards an actor-oriented focus which emphasises ‘strengths’ rather than ‘needs’, and to recognise the multiplicity and integrity of people’s lives forged in a complex mix of priorities, strategies, influences, activities and therefore outcomes’ (White 2010:3; White and Pettit 2004).

The discrepancies between subjective and objective measures of well-being, including where increased income does not correlate with increased self-reported well-being (Guillen-Royo and Velazco 2006), make a strong case for the investigation of similar properties in the context of resilience. Given that it is likely that perceptions, norms and behaviours play a strong role in shaping a household’s resilience (McIvor and Paton 2007; IFRC 2014), greater understanding of the relationship between objective and subjective resilience will undoubtedly add considerable value to this emerging field of research. It may also help to galvanise policy interest on looking beyond a reliance on objective measures of resilience, both internationally and nationally.

There are also lessons for subjective resilience from research on psychological resilience, which seeks to understand the ability of individuals to cope with and adapt positively in the face of loss, hardship or adversity (Singh and Yu 2010). This research has examined a wide range of determinants of personal resilience including epigenetic, developmental, psychosocial, and neurochemical factors (Wu et al. 2013). Others have sought to situate these individual responses within the wider contexts of social and physical ecologies that link individual risk, social organisation and culture (Ungar 2011).

Methods of measuring psychological resilience vary widely, some focusing on clinically robust quantitative methods, including longitudinal cohort studies, cross-sectional thematic qualitative studies, and randomised control trials (Graber et al. 2015). Windle et al. (2011) divide these into those that use a self-evaluation of prior experiences of successfully overcoming stressful events and positive changes, and those that measure subjective factors deemed to be determinants of resilience, such as personal competence or social resources. Both are relevant and applicable in the context of subjective resilience.

Potentially transferable lessons for subjective resilience are also presented by the Connor Davidson Resilience Scale, a self-administered scale of 25 (later shortened to 10) questions testing psychometric properties that cover five factors corresponding to: personal competence, high standards, and tenacity; trust in one’s instincts, tolerance of negative affect, and strengthening effects of stress; positive acceptance of change and secure relationships with others; control; and spiritual influences (Connor and Davidson 2003; Singh and Yu 2010). A person’s psychological resilience will inevitably have a strong influence on how resilient they perceive their household or community to be. Factoring this into any assessment of subjective resilience will therefore be key, particularly in acknowledging and correcting for any biases.

Advantages of a subjective resilience approach to measurement

There are many reasons why subjective household resilience can add value to objective methods of measurement. Perhaps the most important factor is that it recognises that people have a good understanding of their capabilities and capacities to deal with disturbance and change (Nguyen and James 2013). They are also often aware of many of the factors that enable or constrain the resilience of their livelihoods (Marshall 2010). A subjective approach to measurement challenges the notion that experts are best placed to evaluate other people’s lives, and have a better understanding of the factors that contribute to people’s own resilience (Diener et al. 2002). Thus, in some ways, the assessment of subjective resilience is more of a bottom-up process. It relies on people to self-assess and consider what characteristics are most important to their own livelihoods. While agent-based assessments are not without weakness or bias (see Box 1), they offer valuable insights that should be considered alongside traditional objective measures of resilience.

Box 1 Examples that demonstrate the strengths and weaknesses of using a single-item question to evaluate subjective disaster resilience at the household level
Full size table

Resilience is heavily shaped by sociocultural and psychological factors such as risk perception, cognitive barriers and personal or cultural values, which can each play a key role in determining whether adaptation is sought, or whether people have access to vital resources in times of need (Kuruppu and Liverman 2011; Jones and Boyd 2011). Given that the point of view rests with the individual directly, subjective indicators allow for many of these ‘softer’ aspects of resilience—often difficult to capture through objective means—to be better factored in. In turn, this also brings limitations with it, as cultural factors can present an inherent bias to self-reported score. Subjective household resilience would face similar challenges of having to account for ‘cultural measurement bias’ and the effects of emotions and norms as seen in the measurement of subjective well-being (Suh et al. 1998). For example, in collectivistic societies, such as Japan, people will tend to present themselves as ‘average’ citizens, scoring themselves as less happy than they are (Iijima 1982). Could it also be the case that resilience is culturally relative? Though these present significant methodological obstacles, they have shown not to be limiting factors and do not lead to significant cross-national differences in scores in other related fields such as subjective well-being and happiness (Veenhoven 1990, 2012).

Options for the assessment of subjective resilience at the household level

What would a question, or set of questions relating to subjective resilience look like in practice? Although the process of asking people questions about their perceived levels of resilience may at first seem straightforward, it is anything but. There is a multitude of ways of asking questions relating to subjective resilience, each with its own methodological challenges and biases. Careful thought is needed in designing and delivering questions to ensure the robustness and utility of subjective information. Below we briefly describe a number of options, and associated strengths and limitations, in designing questions related to subjective resilience. Given that this is a relatively new area of practice, with few existing tools, we aim simply to provide and introductory guide to the sorts of tradeoffs and decisions that need to be taken into account when seeking to collect subjective resilience information in practice—it is by no means meant to be exhaustive, and many further considerations will need to be considered (see OECD 2013).

To begin with, there are many different types of ‘resilience’ referred to in the literature. These include: personal resilience, psychological resilience, livelihood resilience, community resilience, social resilience, economic resilience and disaster resilience, to name but a few. While there are many overlaps between them, each is focused on the characteristics that make their respective systems resilient to particular threats. Each is also applied at a specific geographical scale and unit of analysis. Thus, the characteristics and properties of an individual’s psychological resilience may not be the same as those that make up a country’s economic resilience. The first step in designing an assessment of subjective resilience is therefore to decide on the type and scale of resilience one wishes to investigate.

The example we use in this paper to illustrate the potential for subjective assessments is a subset of disaster resilience. Specifically, we are interested in the resilience of households to respond to weather and climate-related extremes. We define this as the ability of households to manage change by maintaining or transforming living standards in the face of shocks related to weather or climate events—such as droughts, floods or the delayed onset of rainfall seasons—without compromising their long-term prospects (adapted from DFID 2011). This focus on disaster resilience can either relate to a single hazard or an aggregate of multiple hazards. A subjective assessment of any of the different types of resilience listed above is entirely feasible, but though would require a different set of questions and wording.

The assessment of subjective resilience can be undertaken using many different evaluative survey techniques. Given the multifaceted nature of resilience, perhaps the most robust manner of collecting information is through open-ended questions, whereby a series of semi-structured (or structured) questions are administered, allowing people to freely reflect on how resilient they perceive their household or livelihood to be. This method allows for rich qualitative data to be collected without prescribing responses. However, open-ended questions and surveys are often difficult to quantify. They also require considerable human and technical resources in collecting relevant data at scale.

The most practical and useful means of collecting information on subjective resilience may therefore be through the delivery of structured surveys. Here, a fixed list of questions and answers that limit the respondents to pre-selected answers from which respondent are requested to choose are administered. The advantage of such an approach is that surveys can be administered quickly, are easier to code and interpret, and standardised. Most importantly, they are more readily quantified. Typically, this type of approach is accompanied by either dichotomous (two-point), multiple choice or scaled questions (such as those reliant on Likert scale responses). However, they can also lend themselves to visual analogue scales or even be combined with open-ended responses.

Before delving into the specifics, it is first important to consider the options for formulating a single close-ended question relating to subjective resilience. Small differences in the way a question is constructed can have large implications for respondent comprehension, reporting and the comparability of data collected (see Table 1). Questions that are easy to understand, low in ambiguity and do not burden the respondents should be sought (OECD 2013). With the assessment of household resilience to weather and climate extremes in mind, one of the first challenges is to specify the threat that is being assessed. Two options exist: a question could either relate to the ability of households to respond to the impacts of a singular stressor, such as drought (see Q1 in Box 1); or it could relate to the collective impact of weather-related extremes (Q2)—this would imply the full range of weather-related extreme events that may affect that particular household, such as floods, droughts and more variable rainfall events.

Table 1 Factors thought to influence the likelihood of error, response biases and heuristics of subjective survey questions
Full size table

The former is specific, easier to comprehend and therefore likely to provide answers that are more robust and tailored to a particular threat. While the latter is more vague in its construction and prone to ambiguity—a household may be very resilient to flood events but not at all resilient to drought—its generalisability allows for it to be applied across a wider range of contexts and derive useful information in relation to the many weather-related threats that affect household disaster resilience. This is critical when considering resilience as a wider approach to securing development in the face of a range of shocks and stresses. Choosing between the two approaches is therefore dependent on the research aims and objectives. While there is no right or wrong approach, users should be aware of the merits and limitations of each.

A second, related challenge is deciding on the structure of the question. Precise wording is key, particularly when there are ambiguities with regards to definitions. For example, Q1 in Box 1 presents a simple and direct way of formulating a resilience-related question. However, the term ‘resilience’ means different things to different people. Another option is to omit the word ‘resilience’ in the question and allude to its characteristics. For example, Q4 instead refers instead to the ability of a household to cope and adapt to climate extremes. However, it is very difficult to cover the multifaceted nature of resilience in a single question without sacrificing the validity and utility of the information gleaned from the question. In addition, any singular question that refers to two separate capabilities may elicit different responses and confuse respondents, i.e. referring to Q4, my ability to cope with increased flood risk may be different to my ability to completely adapt my livelihood in response to continued flood risk.

There may also be difficulties in translating questions effectively across languages. Issues of translation affect any cross-cultural survey, whether quantitative or quantitative. Yet, subjective surveys are likely to require particular care in ensuring robust translation given the heavy emphasis on intangible properties, capacities and assets. Some reassurance can, however, be taken from past experiences in translation of surveys of subjective well-being, where studies have documented similar scores across language groups and bilingual individuals in a number of country contexts (Diener and Suh 2000).

Another consideration is the time period of assessment. This is particularly relevant to resilience, as it is comprised of both short-term (e.g. absorptive/coping capacity) and long-term (adaptive capacity) components. Thus, it is important to make reference to the specific time period (and capacity) within the structuring of all relevant questions. For example, questions Q3, Q4 and Q5 each ask respondents to sum up their experiences over a given reference period—either in relation to the present time or in comparison with a stated period. Alternatively, leaving out reference to a specific time period will likely imply that respondents indicate their views at the present moment while drawing on their experiences from the close (and potentially distant) past.

Equally challenging is deciding on the format of response options. Researchers need to consider how many responses to offer, how to label them as well as the scale of intervals. More importantly, they have to decide on whether questions regarding subjective resilience should be measured on a bipolar scale (e.g. agree/disagree) or a unipolar scale (e.g. not at all—completely), and whether respondents should be asked for a judgement involving frequency (how often do you feel…?) or intensity (how resilient do you feel…?) (OECD 2013). Examples of different types of response items, and the various pro and cons associated with each are presented in Box 1.

As with many of the choices described above, each method of designing response options should be tailored to the needs of the user. Some may choose to prioritise concise and short responses (see Q1 and Q4) to limit ambiguity and make cross-country comparison or longitudinal analysis easier. Yet, this will reduce the level of detail that can be extracted from the answers (particularly in the case of binary answers) (Cummins 2003). Note that in the context of subjective resilience, single question answers are likely to be unipolar (running from low resilience to high resilience) rather than bipolar (between two opposing constructs—resilient/not resilient). Others may choose to allow for a greater number of response options to allow for such detail. However, increasing numbers beyond the optimal length can result in information loss, increased error and reduced motivation (ibid.). Five- and seven-point scales remain the most common options within the context of most life evaluation surveys, though there is an increasing number of surveys using higher point scales (typically 11 points). Choosing meaningful labels that are easy communicable, translatable and adequately reflect each of the gradients on the point scale are an equally important consideration.

Drawing on experiences from related fields, it is likely that questions administered to assess subjective resilience to weather-related extreme events (or any other types of resilience) would consist of two main delivery options. The first is to have a simple standalone single-item question (see Fordyce 1988). This approach has long been used in assessments of subjective well-being (SWB). Examples of stand-alone SWB questions include: “All things considered, how satisfied are you with your life as a whole these days?” or “Taken all together, would you say that you are very happy, pretty happy, or not too happy?” These questions aim to elicit an easily replicable global evaluation of one’s life (Krueger and Schkade 2008). They also seek to be as universally applicable as possible in order for comparison (both with other geographic contexts and across time). A similar approach could no-doubt be adopted for the assessment of subjective household resilience. The aim being to design a question that could, to the best possible extent and recognising the limitations associated with it, give an accurate account of a person’s perceived level of household resilience with a single question. With this in mind, each of the examples presented in Box 2 showcases the types of questions that could be applied as a single question to assess subjective disaster resilience at the household level (note that the design of each question is meant to highlight the strengths and weaknesses of different approaches, and is not a proposition for an effective question).

Box 2 Examples of a set of questions used to evaluate subjective resilience
Full size table

The weaknesses in a single-question approach becomes quickly apparent. Primary amongst them is the difficulty in condensing the different components of resilience into a single concise question. To counter some of these methodological challenges, a second approach would be to ask a series of questions related to aspects known to affect disaster resilience (see Box 2). Each question would probe a different aspect of disaster resilience, aiming to provide a more holistic response. We would consider this to be a far more appropriate way of measuring subjective household resilience. For example, a similar approach is taken by the widely used Satisfaction with Life Scale (SWLS), identifying five related questions that are then used as global measure (Diener et al. 1985). Typically, these questions are then grouped or consolidated to form a composite index. A number of different statistical techniques (such as principal component analysis or various regression-based approaches) can be applied to either identify a small set of questions from a larger subset (that account for much of the variance), or to assign relevant weightage to each question.

As with a single-item question, multiple questions and composite indexes also have their methodological challenges. To begin with, agreeing on which (and how many) questions to include is inevitably difficult and subjective. Indeed, it is possible for numerous different combinations to arise. For example, in the case of psychological resilience, Windle et al. (2011) identify 19 different methods of assessment in the academic literature, each with their own way of questioning, classifying and weighting within their respective resilience scales. One approach would be to start with a clean slate and use bottom-up qualitative research to identify questions that people and communities themselves consider as best representing the characteristics of a resilient household—indeed, questions identified under the first approach may be ‘ground-truthed’ by the latter. This would help avoid expert-led bias, but require extensive initial pilot surveying in order to develop the subset of question areas.

Another option would be to isolate particular characteristics of resilience and assign a small number of questions that relate to each characteristic. These questions could be drawn from the wider literature and would then be grouped and weighted accordingly. For example, given that resilience is often broken down into three interrelated capacities (Folke et al. 2002)—the capacity to cope; the capacity to adapt; and the capacity to transform—questions could quite easily be identified to suit each. See Q5 and Q6 that probe different capacities associated with resilience. The five livelihood capitals (Scoones 1998) are also closely associated with household resilience in many objective frameworks for resilience assessment (Eakin and Wehbe 2009) and could be used as the basis for understanding and probing subjective assessments of resilience—see questions Q8 and Q9. In addition, resilience is often characterised as being comprised of various different processes and functions, such as the iterative learning, accessing knowledge and information or promoting innovation (Jones and Boyd 2011)—see questions Q10 and Q11.

Each of the different frameworks presents a viable way of assessing subjective disaster resilience at the household level. Part of the problem, however, is that there are so many different existing frameworks, many often tailored to specific contexts (Bahadur et al. 2015). Choosing from amongst them inevitably injects some degree of bias, requiring careful thought and transparency. Indeed, while this method offers a useful way of standardising subjective questions relating to common characteristics, it inevitably draws heavily on expert judgement, similar criticisms of traditional objective methods.

It is important to consider that any weighting of the different questions is likely to be subject to various assumptions and methodological weaknesses. Assigning weights can either be done though simplistic and naïve means (such as assuming that each question or category of questions is equally important) or more empirically (such as the use of various statistical analysis to decide on weighting of each question). A number of studies have also adopted hybrid approaches such as engaging local communities to identify and rank the characteristic most relevant to their own resilience (often through participatory rural appraisal methods). These are then used to weight subsequent surveys delivered to households within the community and nearby (Choptiany et al. 2015). No approach is perfect, and judgement calls are required in deciding which methods are best suited to the objectives of any research programme.

A further consideration relates to context. Self-assessment of an individual’s perceived level of climate risk will inevitably be affected by past experience. Thus, an understanding of climate risk (or even listing responses to flood and drought) in a rural setting, where climate hazards are often felt more directly, will not be the same as in an urban setting, where climate hazards tend to be comparatively indirect and mediated through wider socio-economic factors (Da Silva et al. 2012). Accordingly, subjective questions—particularly with regards to the urban contexts—need to be conscious of the interactions between climate and non-climate drivers and be factored into the design of targeted question. For example, a focus on the impacts of climate hazards on well-being or the importance of critical social safety nets during times of hardship may provide a useful entry point to communication and capturing such interactions.

Perhaps the best way of ensuring accurate assessment of subjective resilience is to build on the growing number of approaches and frameworks (see Marshall and Marshall 2007; Marshall 2010; Choptiany et al. 2015; Nguyen and James 2013; Grothmann and Patt 2005; Seara 2014; Lockwood et al.’s 2015), as well as those from wider related fields, and ensure that the lessons learned from their applications are shared, taken forward and further refined. Above all, maintaining the diversity of methods and approaches that range in complexity, scope and focus will be important in gaining a more holistic understanding of resilience.

Discussion

The collection of information related to subjective resilience can have a number of important practical uses. For a start, it can offer a quick, efficient and cost-effective tool for M&E of resilience-building initiatives. The assessment of subjective resilience at various stages of project implementation—prior, during and subsequent—allows valuable insights to be gained on how and where activities have influenced people’s perceived disaster resilience over time. It allows for inferences to be made with regard to the effectiveness of resilience-building initiatives—an issue of considerable interest to international donors, multilateral development agencies, governments and NGOs given the current scale of investments. Any attribution would, however, have to carefully consider the type of assessment and design of survey delivery (such as the use of Randomised Control Trials) in making any such claims. While measurement of the impact of interventions on subjective resilience can never provide a complete account of objective resilience (an intervention can lead to a person feeling more resilient while unwittingly placing them at greater risk to an unforeseen or underprepared risk), it can complement other information in evaluating and attributing the impact of external interventions from a recipient and bottom-up perspective.

Any subjective assessment approaches may need to account for bias due to tactical reporting. For example, in areas that receive considerable development or humanitarian assistance in meeting people’s basic livelihood needs, it is possible that respondents may choose to respond in their own self-interest, i.e. claiming to be more vulnerable than they actually are in the hope of securing sustained or increased levels of assistance. The opposite may equally be true, whereby people do not want to be considered as having low levels of resilience—perhaps due to the social stigma attached with the label—and deliberately claim that their household has a higher level of resilience than in reality. This is where a thorough understanding of the context and political economy of the surveyed area can be of immense value. Clear and neutral wording can also be important.

At a higher level, the same tools may feasibly be applied to the evaluation of national or international resilience-building initiatives, although this has so far proven difficult. If international policy commitments such as the Sendai Framework Disaster Risk Reduction (SFDRR) or the Sustainable Development Goals (SDGs) are working effectively, then it is only reasonable to expect a marked difference in how resilient local people perceive themselves to be. Indeed, similar approaches have been proposed for the evaluation of national social and economic policies by collecting well-being and life evaluation data (Dolan and Metcalfe 2012; Diener 2000). Such a tool may therefore allow a way of holding NGOs, businesses and governments to account through a bottom-up method that captures the collective voice of beneficiaries and those most affected by disaster events.

Crucially, subjective assessments can help to reveal the underlying causes of vulnerability and resilience that might otherwise not be visible to traditional survey or statistically based techniques. Shedding light on the structural root factors rather than only proximate factors, as well as from the ground perspectives on how human agency can challenge these factors, is vital to enabling a more transformational approach to building resilience.

Lastly, information on subjective resilience can allow us to gain a more holistic and bottom-up perspective on our understanding of resilience at household and other scales. It can help to elaborate the relationship between subjective assessments of a household’s disaster resilience and psychological and cultural factors such as attitudes, emotions, personality traits, beliefs and norms (Graber et al. 2015; Krüger et al. 2015). In addition, a more comprehensive understanding of household resilience allows us to better identify what factors contribute to increased (and decreased) resilience, including the intangible factors that may not be picked up in objective assessments. In turn, this can feed into improved targeting of resilience-building activities at all levels of governance. By comparing objective and subjective assessments, further research should be able to indicate whether people who rate themselves as highly resilient also score high on objective measures of resilience, and vice versa. Conversely, it is highly likely that there will be areas where objective and subjective assessments differ. Understanding the drivers (and biases) for such disparities could point to different interpretations of resilience on the ground, as well as the effectiveness of resilience-building activities, and may point to different policy options.

Conclusion

In this paper, we outline the rationale for assessing subjective disaster resilience at the household level. While it is clear that any approach to subjective assessment will face significant methodological and conceptual challenges, we show these to be far from insurmountable. Most importantly, measuring subjective resilience offers a valuable opportunity to capture the perspectives of those who know most about their own resilience and the factors that contribute to it: the people themselves. Moreover, this type of information has a number of unique practical applications, such as helping to improve our understanding of what works and does not work with regards to resilience-building activities; enhanced targeting of resilience-related programmes and resources; as well as providing a useful bottom-up tool for capturing the voice of beneficiaries and local communities.

Establishing the feasibility and methodological robustness of a subjective approach to measuring disaster resilience will inevitably take time. However, a tremendous amount of knowledge can already be drawn from current understandings of household disaster resilience, as well as insights gained through gathering subjective information in related fields, such as subjective well-being and psychological resilience. Care should nonetheless be taken in examining the merits and limitations of various different approaches to measuring subjective resilience. It is likely that a range of methods, surveying tools and applications will be required to satisfy the diversity of user needs and resources available.

Ultimately, the aim should not be to entirely replace traditional methods of resilience measurement. On the contrary, objective measures are a vital component of the measurement process. Rather, if shown to be effective, we argue that bottom-up subjective methods should be used alongside objective methods, helping to capture many of the components of resilience that are difficult to observe and allowing people’s perspectives to be heard in a systematic manner. Getting the process right will be an important step forward in gaining a more holistic understanding of what it takes for a household to be resilient to disaster risk.