1 Introduction

This book matches two concepts, one substantive, and the other methodological: ‘vulnerability’ and ‘survey quality’. It concerns the challenges involved in conducting high quality social research using survey methods in non-standard contexts, to learn about vulnerable people and their experiences of vulnerability. The volume brings together nine contributions to a major Swiss research infrastructure named ‘LIVES – Overcoming Vulnerability: Life Course Perspectives’, each of which touches on the tension between the pursuit of elaborate, often delicate substantive research aims, and the demand for methodological rigour. The LIVES research programme has been funded by the Swiss Confederation through the National Science Foundation’s National Centres of Competence in Research (NCCR) scheme, which is designed to promote long-term research networks in areas of strategic importance for Swiss science, the Swiss economy and Swiss society. As the name ‘LIVES’ suggests, this multifaceted project aims to address human vulnerabilities across life trajectories, from the cradle to the grave. Yet while LIVES’s defining feature and its raison d’être is its ambitious substantive focus, much of the research being carried out by the NCCR is equally driven and characterised by two specific epistemological features: the multidisciplinary nature of the work being undertaken, and a strong emphasis on quantitative methods and survey data collection. These features, along with the thematic focus of the research, present a double challenge to the researchers involved, the first of which is simply how best to obtain “meaningful” data about vulnerabilities and populations at risk, and the second of which is whether ‘quality’ is best achieved by applying standard, widely-accepted techniques for gathering new data, or by adapting these methods to the specific populations of interest, or research conditions.

The pertinence of this challenge – the methodological aspects of which have long troubled comparative researchers (Harkness et al. 2010; Davidov et al. 2010) – extends beyond LIVES and beyond Switzerland. In this sense the nine chapters of this volume contribute to crucial scientific debates in survey methodology, as well as in both psychological and social sciences. They provide detailed presentations of concrete experiences of conducting surveys and using other innovative approaches to study vulnerabilities and vulnerable individuals (belonging or not to populations considered to be vulnerable). Each one highlights the importance of a proper integration of theory, concepts, questionnaire content and design, procedures of data collection, analytical methods, and interpretations, to develop original perspectives in an area of research, which continues to gain in popularity. Indeed, the topic of ‘vulnerability’ has seen an explosion of interest in the scientific world over the course of the past 20 years, partly in response to demands from stakeholders and citizens, in spite of – or maybe because of – the fact that the concept of vulnerability is not clearly defined. Its many connotations and uses have both general and specific implications for the methods used to investigate the phenomena to which it pertains, and notably, for the various decisions involved in the design of quantitative surveys.

This book tackles the challenge of using survey research methodology – either on its own or in combination with other innovative approaches (such as social networks and observation) – to collect data from vulnerable population subgroups, to investigate their experiences of vulnerability and the resources available to them for surmounting ‘vulnerabilising’ contexts, events and life transitions. While at the same time aspiring to uphold the scientific standards of survey research methodology, researchers interested in vulnerability face a number of distinctive challenges. Many are extensions of existing problems affecting the quality with which any survey can be carried out.

Apart of being helpful for people working with data produced within the LIVES project, this book is mainly addressed to researchers specialized on topics around vulnerability, survey practitioners and survey methodologists who aim at questioning their routines, as well as to young researchers looking for examples of honest and innovating social science making.

After this introductory chapter, which discusses the contribution of the following chapters, the book is organized as follows: the first six chapters are grouped around the surveyed topics and populations: elder people for chapters “Representation of Vulnerability and the Elderly. A Total Survey Error Perspective on the VLV Survey” and “Adapting Quantitative Survey Procedures: The Price for Assessing Vulnerability? Lessons from a Large-Scale Survey on Aging and Migration in Switzerland”, harsh events (such as widowing and cancer) during middle age for chapters “Vulnerability Following a Critical Life Event: Temporary Crisis or Chronic Distress? A Psychological Controversy, Methodological Considerations, and Empirical Evidence” and “A Survey of Couples Facing Breast Cancer in Women”, working careers and disruptions for chapters “Career Pathways and Professional Transitions: Preliminary Results from the First Wave of a 7-Year Longitudinal Study and “How to Survey Displaced Workers in Switzerland: Ways of Addressing Sources of Bias”. The last three chapters are more focused on a specific survey tool: life calendars for chapter “Using Life History Calendars to Survey Vulnerability”, online social networks for chapter “Studying Youth Transitions Through a Social Network: First Impressions” and panels for chapter “Attrition in the Swiss Household Panel: Are Vulnerable Groups More Affected than Others?”.

1.1 Background to This Volume

At the origin of this collection is an international workshop organized by LIVES called ‘Methodological and Substantive Challenges in Measuring vulnerability across the Life Course’, held at the University of Lausanne, Switzerland, in June, 2012. The aim of the workshop was to address some of the methodological challenges involved in measuring vulnerability and resilience using social surveys. The workshop brought together substantive specialists in researching these topics from within LIVES, and invited international experts in survey methodology to shed light on some of the shared methodological challenges being faced by the thematic research projects that make up LIVES. The challenges discussed at the workshop included (1) sampling and surveying ‘hard-to-reach’ populations; (2) the risk of low and differential rates of participation across important sub-groups, and attrition for surveys with a longitudinal design; (3) difficulties associated with using traditional data collection methods, including telephone under-coverage and the high cost of face-to-face interviewing, which have increased the demand for mixed mode data collection; and (4) measurement challenges including the collection of retrospective data (e.g. event histories and retrospective evaluations of personal wellbeing across different points of the life course), and the potential for interviewer effects on data quality. Understanding the nature of such challenges and the implications they have for the substantive aims of the LIVES project was one of the primary objectives of the workshop. A secondary, and relatively seldom pursued aim, was to forge a dialogue between substantive and methodological specialists, to try to seek solutions that could help to ensure the best possible synergy between the substantive goals of the LIVES research, and the methods used to gather new data. With this latter aim in mind, the program for the conference was based around three substantive areas of central interest to the LIVES project: (a) specific vulnerable populations – notably, national minorities, and younger and older people; (b) transitions in and out of employment, and (c) vulnerability in family life, with each session incorporating presentations from the methodological experts and the substantive specialists conducting research within the NCCR.

Though the specific focus of this volume is on the Swiss context (as for the workshop), the issues addressed have a broader significance in the social sciences, and provide lessons for quantitative researchers working in other European and North American contexts, as well as elsewhere. As in many countries, survey data in Switzerland play a central role in the social sciences, with more and more researchers carrying out secondary analysis of large-scale datasets available through national data archives, as well as conducting their own original data collection. In LIVES, major national studies of the general population (including notably, the Swiss Labour Force Survey (SLFS), and the Swiss Household Panel Survey (SHP)) have been supplemented by several purpose-designed quantitative surveys focused on different aspects of vulnerability and resilience among special subpopulations living in Switzerland. The chapters in this book have been contributed by the researchers responsible for designing and carrying out these new surveys and broadly reflect the three areas of focus addressed by in the workshop.

The LIVES surveys include (1) a study of a cohort sample of young adults, including an over-sample of second generation immigrants from Albanian-speaking countries in the former Yugoslavia being surveyed alongside the third sample of the SHP (Gomensoro and Bolzman 2015), and an associated pilot survey that was designed to field test a suitable sampling strategy (Elcheroth et al. 2011); (2) two studies investigating pathways out of unemployment, including a survey looking at the impact of mass redundancy among ex-employees of five firms that closed down between 2009 and 2010 (chapter “How to Survey Displaced Workers in Switzerland: Ways of Addressing Sources of Bias” in this volume), and a survey of the newly unemployed in the canton of Vaud (chapter “Studying Youth Transitions Through a Social Network: First Impressions”); (3) a seven-wave longitudinal survey of workers and the impact of individual characteristics and resources on professional trajectories (chapter “Career Pathways and Professional Transitions: Preliminary Results from the First Wave of a 7-Year Longitudinal Study”); (4) a two-wave extension of an existing panel survey of married and unmarried couples looking at changing family configurations in response to critical events (Widmer et al. 2013); (5) a two-wave panel study of the role of the couple relationship as a source of support for women facing breast cancer (chapter “A Survey of Couples Facing Breast Cancer in Women”); (6) a longitudinal survey of divorcees and widows investigating the effects of losing an intimate partner in the second half of life (chapter “Vulnerability Following a Critical Life Event: Temporary Crisis or Chronic Distress? A Psychological Controversy, Methodological Considerations, and Empirical Evidence”); (7) a survey of older adults aged 65 and over (called Vivre, Leben, Vivere, or VLV for short) investigating inequalities during old age (chapters “Representation of Vulnerability and the Elderly. A Total Survey Error Perspective on the VLV Survey” and “Adapting Quantitative Survey Procedures: The Price for Assessing Vulnerability? Lessons from a Large-Scale Survey on Aging and Migration in Switzerland”); and (8) a retrospective complement to the SHP (chapter “Attrition in the Swiss Household Panel: Are Vulnerable Groups More Affected than Others?”) using a life history calendar (chapter “Using Life History Calendars to Survey Vulnerability”).

In the next section, we discuss the concept of vulnerability, providing some definitions to help clarify the substantive ambitions of the authors of this volume. Then, we discuss issues related to survey quality, to provide a context for understanding the various methodological challenges related to surveying vulnerable individuals and populations, and measuring vulnerabilities in surveys. Along the way, we refer to the various contributions to the volume, touching on their substantive aims and their methodological concerns, in an effort to piece together the various pieces of the jigsaw. We conclude with a reflection on lessons learned, relating in particular to the need for close collaboration between substantive and methodological experts when producing new data, and the importance of transparency in interdisciplinary research.

2 Vulnerability

2.1 The Ambiguous Success of the Concept “vulnerability”

A simple search in English for the term “vulnerability” on Google Scholar shows its almost inexistence in the academic literature until the mid-1980s, its marginal presence in the 1990s and its sudden explosion from 2000, confirmed and amplified in more recent years. It seems that the original concept appeared in risk research when questions were raised about the naturalness of natural disasters. It was used at a macro and meso level to disentangle the physical hazard and the resulting disaster, and to take into account the various capabilities of structures and institutions to prevent or attenuate the consequences of a disaster, resulting in differential impact on and responses from different subpopulations (Henke 2015). The widespread appeal of the concept developed later, however, when vulnerability was adopted in many fields of research that have in common a micro – individual-level – approach to inequalities in human development, which has largely been promoted in recent decades by the life course/life span perspective (Mayer 2009; Oris et al. 2009). Hanappi and her colleagues (2015) have recently provided a statistical account of this association based on a content analysis of some 10,632 abstracts of articles published since 2000. They highlight the centrality of the concept of vulnerability across a variety of disciplines, from psychology to sociology and demography, and from youth studies to gerontology, covering more than 30 specific fields of study, including personality disorders and single motherhood, to take just two examples in a long list.

On the basis of this statistical assessment but also of a more qualitative literature review, Spini and colleagues (2013) conclude that the concept of vulnerability has a terrific advantage when you have interdisciplinary ambitions, which is its disciplinary neutrality.Footnote 1 However, at least a part of its success is related to the absence of an accepted definition, authorizing a range of uses of the term based on implicit disciplinary assumptions (see more in Perrig-Chiello et al. chapter “Vulnerability Following a Critical Life Event: Temporary Crisis or Chronic Distress? A Psychological Controversy, Methodological Considerations, and Empirical Evidence” of this volume). This field of research continues to grow tremendously but remains like the many pieces of still not unified puzzle. We do not have the arrogance to impose here the ultimate definition of vulnerability that could be recognized in the same way by all the many researchers concerned with the concept, but in the following we come up with one possible proposal and we make explicit how the contributors to this volume use the term, to then understand the questions they asked, to whom and how.

In LIVES, we see vulnerability “as a lack of resources, which in specific contexts, place individuals or groups at a major risk of experiencing (1) negative consequences related to sources of stress; (2) the inability to cope effectively with stressors; and (3) the inability to recover from the stressor or to take advantage of opportunities by a given deadline” (Spini et al. 2013, 19). Resources and stress are key elements in this definition that was inspired by the seminal works of Leonard Pearlin (1989) on stress, that consider the coping strategies and other resources individuals have to manage it. This definition of vulnerability, however, also reintegrates the notion of risk, and takes into account the context, the opportunities, as well as their temporality. Moreover, resources, stressors, and individual responses are all interrelated in a dynamic system that includes feedback, as well as spill-over effects across different life domains like family and work (Spini et al. 2013).

Generally speaking, vulnerability is considered as being particularly present in post-industrial societies, which can be characterized by growing uncertainty. According to this perspective, recent decades are contrasted with the Thirty Glorious Years (1945–1975) and the simultaneous apex of the welfare states after the Second World War. For Martin Kohli (2007), this period saw the peak of the life course institutionalisation with age-fixed related roles: being a child and studying, being a working adult and raising a family, and being retired. The life trajectories were highly standardized, the components of a transition like the one to adulthood were properly ordered and rapidly traversed. The whole resulted in a relatively normative but very transparent system, and provided a general sense of security about the life course. Then, with the globalization of the economy and increasing demands for flexibility in more competitive environments (Rudisill et al. 2010), with the myriad of changes affecting family and the multiplication and growing complexity of living arrangements (Fokkema and Liefbroer 2008), with the expansion of individualistic values (Giddens 1991; de Singly and Martucelli 2009), and the (albeit incomplete) gender revolution (Esping-Andersen 2009; Levy and Widmer 2013), life courses became de-standardized, desynchronized, more heterogeneous or disordered, more turbulent, less predictable, and increasingly insecure.

This double embedding of changes in history and in the individual life course makes up the dominant perspective in life course research. It is moreover completely coherent with the recent explosion of interest for the topic of ‘vulnerability’. Although this vision is far from undisputed (see for example, Widmer and Ritschard 2009), its prevalence explains why two sets of theories are often contrasted with one another in the literature, the first one tending to be associated with the past, the second one with more recent dynamics: the social stratification versus biographization approaches. Given that our aim here is not to initiate a long theoretical discussion but to try to be concrete, we will mainly refer to the case studies that are discussed more in-depth in the remainder of this book.

2.2 The Social Stratification Perspective on Vulnerabilities

Social stratification is an old and prestigious area of research (Grusky 2001; Tillmann nd Voorpostel 2012) that found an echo in the early 1970s in the seminal works of Matilda Riley (see Dannefer et al. 2005) on the age stratification of society. In this perspective individuals are vulnerable if they are located in the inferior (dominated) classes or strata, as well as when they are unable to fulfil age-related duties. This is the case for young adults when they do not manage to cope with the education system and succeed in their transition to a stable job. This group is the focus of chapter “Studying Youth Transitions Through a Social Network: First Impressions” in this volume, by Eicher and her colleagues, who confirm what is well-known by life course specialists, i.e. that transitional phases of life are moments of increasing exposure to risks (Levy et al. 2005). At the opposite end of the age spectrum are the older adults aged 65 and over studied in chapter “Representation of Vulnerability and the Elderly. A Total Survey Error Perspective on the VLV Survey”. Increasing longevity and the associated improvements in health and living conditions have been so great during the last few decades that we can no longer assume that the elderly are a vulnerable population just because of their age. However, rising life expectancies also implies that a growing proportion of people with limited resources, leading difficult lives, is now able to reach not only the age of retirement, but also very old age. In this sense, progress has resulted in growing inequality among the older population (Gabriel et al. 2015). A typical illustration are the old immigrants for whom vulnerabilities have been institutionally constructed through rules that have changed over time, and affected by other important determinants like their socioeconomic and family trajectories. They are at the heart of Laure Kaeser’s chapter “Adapting Quantitative Survey Procedures: The Price for Assessing Vulnerability? Lessons from a Large-Scale Survey on Aging and Migration in Switzerland”.

More generally, social stratification is often associated with the theory of cumulative advantages and disadvantages (di Prete and Eirich 2006), which has gained great popularity in life course research and associated research designs (Elder et al. 2015). In this perspective, even modest initial differences in young age widen across life, little by little, leading to a maximal heterogeneity among the so-called “young old”, until differential mortality reduces this diversity (Oris and Lerch 2009).

This heterogeneity entails related scientific challenges in terms of identifying the diversity of vulnerabilities (psychological or physical health, socioeconomic conditions, isolation, etc.) and their degree or intensity. Identifying vulnerable individuals within a given population or subgroup is an issue for the elderly in contemporary developed societies, but this is true for all potentially vulnerable groups since any a priori categorization has to be challenged. The classical dialectic of inter- versus intra-variance remains crucial for any empirical assessment of vulnerability. Chapter “Vulnerability Following a Critical Life Event: Temporary Crisis or Chronic Distress? A Psychological Controversy, Methodological Considerations, and Empirical Evidence” (by Perrig-Chiello, Hutchinson and Knöpfli) provides an excellent illustration of this difficulty since the authors not only compare people who experienced the breakup of a long marital relationship with people who are still married, but also, within the divorced group, the most vulnerable with the least, regarding, among others, the circumstances surrounding the divorce. This discussion is related to the fact that vulnerability can not only be viewed as a state, but also as a process. This has at least two methodological consequences. First, it entails a longitudinal perspective that considers how vulnerabilities are constructed across the life course, based either on a retrospective approach using life calendars, or on a prospective construction with panel data. Both pose specific methodological challenges but both are part of the analytical strategy (discussed in more detail in the next section). Second, this longitudinal approach puts more attention on transitions or events in one sense, or on sequences, on the order or disorder of life, on the structure of the life course (Ritschard and Oris 2005). These are also central concerns in the second most important body of literature related to vulnerability, discussed next.

2.3 The Biographization Approach to Vulnerabilities

The second main approach to inequality and vulnerability across the life course, biographization, can be seen as a by-product of the recent changes at the macro and micro levels that were briefly discussed above. In this perspective, lives nowadays are more turbulent, with more incidents. The norm of self-realization imposes on the individual a responsibility to face the trials of his or her plural life (professional, familial, etc.) and to be accountable for his/her successes, but consequently also for his/her failures. It is obviously a source of stress that can also result from uncertain or insecure social interactions in a “liquid” (Baumann 2006) and competitive world (Rudisill et al. 2010). Biographization is mainly concerned with event-based vulnerabilities associated with health problems, family discontinuities, and professional uncertainty. Typical examples addressed in this book are those who experienced the break-up of a long-term partnership (as mentioned, chapter “Vulnerability Following a Critical Life Event: Temporary Crisis or Chronic Distress? A Psychological Controversy, Methodological Considerations, and Empirical Evidence” by Pasqualina Perrig-Chiello and colleagues), couples where the woman has been diagnosed with breast cancer (chapter “A Survey of Couples Facing Breast Cancer in Women” by Linda Charvoz et al.), and displaced workers following the closure of an industrial enterprise (chapter “How to Survey Displaced Workers in Switzerland: Ways of Addressing Sources of Bias” of Isabel Baumann and colleagues).

In these kinds of contexts, the challenge for researchers is to capture the heterogeneity within the populations created by the event, not only associated with inequalities in personal and social resources, but also with the diversity resulting from the nature, intensity, and predictability of the stressor(s) (see here chapter “Using Life History Calendars to Survey Vulnerability” by Maggiori and his colleagues). Even more specifically, a crucial aim for these studies is to reveal the variety of responses to the event, which may include temporary crisis, chronic strain, adaptation, or even personal growth. The biographization theory tends to view more frequent life events as less harmful, provoking less permanent states and becoming part of more turbulent lives through a succession of limited failure and resilience phases (Vandecasteele 2010). However, some authors identify the long-term scarring effects of events like a dismissal (Antonini and Bühlmann 2015; Chauvel 2010). A similar debate contrasts psychological theories (see chapter “Vulnerability Following a Critical Life Event: Temporary Crisis or Chronic Distress? A Psychological Controversy, Methodological Considerations, and Empirical Evidence” by Pasqualina Perrig-Chiello and colleagues for an overview).

Of course, both from a social stratification and a biographization perspective, individuals are interconnected. Linked lives, significant others, social convoy and other forms of social support are not only significant resources, but they can also be a source of stress in case of inadaptive responses. Charvoz and her colleagues make this point very clearly in chapter “A Survey of Couples Facing Breast Cancer in Women” of this book, where they look at dyadic coping, at the reactions and interactions of couples where the woman is affected by breast cancer. Conversely Eicher and her co-authors (chapter “Studying Youth Transitions Through a Social Network: First Impressions”) highlight the distinction between bridging and bonding social relations, their differential protective effects for individuals as well the potential importance of group identity for dealing with vulnerabilities. However, theories of post-modernity assume a decrease in formal, obliged, institutional ties, and an increase in elective relationships grouped in configurations that an individual has to animate using personal resources to manage this social capital (Widmer 2010).

Along the same lines, institutions other than family are also supposed to become less important, at least in their capacity to frame people’s life courses and impose age- (and gender-) related norms. There is a consensus that formal recognition of vulnerability is a categorization process that often has negative drawbacks in terms of loss of self-esteem for the person or group being categorized, who become exposed to social stigmatization (Ruof 2004; Thomas 2008). There is also a general agreement that individualization implies an increasing role for the state and its various welfare components, which should be important in preventing vulnerability, and are particularly important in responses to so-called “social risks”, old or new, and consequently in the public management of vulnerable situations and individuals (Leisering 2003; Ranci 2010). We can cite the education system, unemployment offices, hospitals, and institutions caring for the elderly as examples considered in this volume. They all tend to be affected by the “activation” paradigm which is coherent with the values of individualization, and it is now recognized that like family configurations, institutions can be a source of support but also of stress (Bonvin et al. 2014).

Stress is also present in the two main theoretical approaches discussed previously. Pearlin (1989) explicitly associated a lower position in the socioeconomic structure with greater exposure to stress. More recently, as we discussed briefly earlier, post-modern or post-industrial societies have also been seen as structural producers of stress through their demands for flexibility and competition (Rudisill et al. 2010) and, more generally, through the ambivalence of the ideal of self-realization (Giddens 1991), the burden of being the actor of his or her life (Ehrenberg 1998). Pressure, insecurity, chronic strain, daily hassles, incivilities are obvious sources of vulnerability when their effects accumulate over time. Their impact is maximized by contexts of discrimination, xenophobia, or other attitudes that challenge fairness. These negative associations are specifically discussed in chapter “Career Pathways and Professional Transitions: Preliminary Results from the First Wave of a 7-Year Longitudinal Study” by Christian Maggiori and colleagues in the context of professional life.

From a survey research perspective, the challenge for researchers interested in measuring vulnerability is to properly locate individuals in a web of social interactions that is more or less dense, with different actors that have to be identified. It is also to analyze those interactions from an individual’s subjective point of view, in terms of perceived social support or discrimination (chapters “A Survey of Couples Facing Breast Cancer in Women” and “Career Pathways and Professional Transitions: Preliminary Results from the First Wave of a 7-Year Longitudinal Study”). It is finally to identify and measure chronic strain and daily hassles. Psychologists are of course the main contributors in relation to the latter (see chapters “A Survey of Couples Facing Breast Cancer in Women”, “Career Pathways and Professional Transitions: Preliminary Results from the First Wave of a 7-Year Longitudinal Study, and “Studying Youth Transitions Through a Social Network: First Impressions”) but all social scientists recognize the importance of subjective perception since another widely accepted definition of vulnerability is simply low levels of well-being (see chapter “Representation of Vulnerability and the Elderly. A Total Survey Error Perspective on the VLV Survey”).

Finally, if social stratification and biographization are often contrasted, for example, in research on poverty, the preceding discussion has shown that they have a lot of associated concepts in common, although with different modulations. Indeed, these two theoretical bodies should not be seen as incompatible as their integration promises much. An illustration of this potential is provided by Isabel Baumann and her colleagues who, in chapter “How to Survey Displaced Workers in Switzerland: Ways of Addressing Sources of Bias”, study a population defined by both its status and its experience of a vulnerabilising event: the workers of five factories which closed down, resulting in collective dismissals. They simultaneously lost their position and acquired an inferior one, as unemployed, because of an external shock that provoked a turning point in their professional lives and no doubt affected other domains, such as their family life.

3 Survey Quality

Survey research is at the center of the LIVES project, with over one hundred collaborators drawing on the new data sources described (as well as others) to develop new knowledge about the phenomenon of vulnerability and the resources people draw upon to overcome it, with a view to contributing to the development of innovative social policy measures informed by the findings of their research. The quality of the data collected is integral to the reliability and validity of these conclusions (Groves and Lyberg 2010), and the effectiveness of any recommendations derived from them. Yet the quality of all survey data is inevitably compromised by trade-offs made in the survey design process – trade-offs that even in general population surveys are becoming increasingly complex as a result of growing challenges associated with carrying out surveys using traditional tried-and-tested methods of data collection.

In particular, the LIVES studies face a unique set of challenges associated with sampling, and achieving an adequate representation of their chosen populations, many of which can be described as either ‘hard-to-reach’ or ‘hard-to-survey’, either because there are no suitable listings available for sampling purposes, or because they are notoriously hard to contact or reluctant to participate in surveys (see Riandey and Quaglia 2009; Tourangeau et al. 2014). Added to this, the substantive focus of the LIVES research poses quite particular, and often complex measurement challenges, such as how to ask about subjective phenomena likely to be perceived as sensitive by respondents, how best to capture life event histories (see chapter “Using Life History Calendars to Survey Vulnerability” by Davide Morselli and colleagues) or the configuration of social networks. To complicate matters, many of the LIVES surveys have a longitudinal design, incurring an additional threat to data quality due to the risk of selective sample attrition over the lifespan of the study (an issue addressed in chapter “Attrition in the Swiss Household Panel: Are Vulnerable Groups More Affected than Others?” by Rothenbühler and Voorpostel in relation to the SHP). Each of these challenges and the way in which they are managed in the survey design process has implications for the ‘quality’ (typically defined as accuracy) of the estimates derived from the data.

The accuracy of survey estimates is determined by the degree to which they are affected by different sources of error. In addition to sampling error, which can be controlled through the sampling design, these include a number of non-sampling errors: coverage error (associated with the failure to provide all eligible population members with a known and non-zero probability of being selected to participate in the survey); nonresponse error (resulting from nonparticipation among particular subgroups, and differences between the responding and nonresponding samples); and measurement error (resulting e.g. from problems with the design of the questionnaire, or the way in which respondents formulate their answers to the questions). Other sources of error may be present, such as data input errors, coding errors, processing errors (see Groves (1989) for a detailed discussion), but we focus on the principal sources here. The impact of survey errors on accuracy can be understood in terms of how each one affects the twin inferential processes of representation and measurement on which the accuracy of surveys depend (Groves et al. 2004). While the former concerns the process by which inferences to the population can be made on the basis of estimates derived from a sample (the success of which depends on the minimisation of coverage errors, sampling error, and nonresponse errors), the latter concerns estimate validity – the process by which inferences are drawn from survey statistics based on questions designed to tap the theoretical constructs of interest (for which it is essential to mitigate errors of measurement and processing errors and maximise reliability and validity).

This ‘Total Survey Error’ (Groves 1989) paradigm for thinking about survey quality continues to dominate the survey methodology literature and provides a helpful framework for thinking about the various challenges present in the LIVES empirical programme and comprehending the nature of the trade-offs that have been necessary in the survey designs. Nevertheless, it could be argued that surveying vulnerable populations presents such specific research challenges, that special rules in survey methodology should be applicable. Tourangeau et al. (2014), however, argues that the difficulties involved specifically in hard-to-survey populations can be classified in terms of the survey operations they affect – sampling, making contact, persuading people to participate, and measurement. In the following, we discuss how the different projects have negotiated these different features of the survey design in the context of surveying vulnerabilities and vulnerable populations.

3.1 Sampling and Contacting Vulnerable Populations

The populations studied in this volume are highly diverse, requiring diverse sampling strategies. For example, the entire Swiss population (resident in private households) is sampled by the SHP (chapter “Attrition in the Swiss Household Panel: Are Vulnerable Groups More Affected than Others?”); older adults, simply defined as all people aged 65 and over, are targeted by VLV, and those aged 40–89 by the study on the experience of divorce and widowhood (chapters “Representation of Vulnerability and the Elderly. A Total Survey Error Perspective on the VLV Survey” and “Vulnerability Following a Critical Life Event: Temporary Crisis or Chronic Distress? A Psychological Controversy, Methodological Considerations, and Empirical Evidence”); while the working population of middle-aged adults (aged 25–55) is considered in chapter “Career Pathways and Professional Transitions: Preliminary Results from the First Wave of a 7-Year Longitudinal Study”. For most of these studies, the sampling frame was the population register and random probability samples were drawn for the research teams, in most cases by the Swiss Federal Statistical Office (SFSO), but also sometimes by the cantonal administrations. Within each of these populations specific challenges appeared that were directly related to situations of vulnerability. For example, the unemployed form part of the active population but identifying them required the use of a different source, the national register of the unemployed held by the State Secretariat of Economy, as well as a screening procedure (chapter “Career Pathways and Professional Transitions: Preliminary Results from the First Wave of a 7-Year Longitudinal Study”). Elderly people with cognitive impairments also needed to be identified for VLV, but this was a post-identification by the interviewer.

Elderly people ageing in a country where they arrived as migrant workers illustrate another approach to sampling in vulnerability research: the selection of specific subpopulations assumed or at least suspected to be vulnerable. Older migrants or the unemployed are typically categorized in a social stratification perspective. Alternatively, event-based vulnerability is reflected in the choice of studying divorcees and widows or widowers or the couples where the woman faces breast cancer. To identify the former Perrig-Chiello and her colleagues used a large SFSO sample, which they decided to supplement with a convenience sample recruited through calls in the media (chapter “Vulnerability Following a Critical Life Event: Temporary Crisis or Chronic Distress? A Psychological Controversy, Methodological Considerations, and Empirical Evidence”). For the latter, Charvoz and her coauthors established a partnership with a university hospital (chapter “A Survey of Couples Facing Breast Cancer in Women”). Similarly, Eicher and her coauthors have obtained targeted samples of young adults from institutions. As noted above, chapter “How to Survey Displaced Workers in Switzerland: Ways of Addressing Sources of Bias” about the displaced workers uses both social stratification and event-based perspectives. In terms of defining the population to be studied, individually displaced workers are a selective group since the characteristics that provoked their layoff are highly likely to affect their probability of remaining unemployed. That is why Isabel Baumann and her colleagues provide a nice illustration of the “sociological imagination”. Using data from plant closures where all workers were displaced they identify a group of job seekers, of some 1200 persons who worked in manufacturing plants, which closed their doors between January 2009 and August 2010.

Once a suitable sampling strategy has been implemented, finding and contacting the potential participants is the next step and an additional source of trouble for researchers. In the surveys considered in this volume, between 7.5 and 15 % of the postal addresses were no longer valid for the persons listed on the frame. Reasons include errors in the original registers, confusion between de jure and de facto populations (for fiscal or other reasons), death and mobility. The latter is especially important for some event-based vulnerabilities (divorce usually produces at least one change of residence) and for transitional life periods (like the transition to adulthood). This is one of the reasons why Véronique Eicher and her colleagues used an alternative approach involving an online social network. However, even there, from the 380 people initially sampled, some 23 % provided an invalid email address (see chapter “Studying Youth Transitions Through a Social Network: First Impressions”), radically reducing the size of the available sample with which further contact could be made.

3.2 Modes of Data Collection and Vulnerability

The potential participants who are successfully contacted may be confronted with various modes of data collection selected by the researchers. For example, the VLV survey of the elderly included two separate questionnaires deployed in a classical sequence with first a self-administered paper-pencil questionnaire, then a computer-assisted interview (CAPI) administered by a trained interviewer in a face-to-face interaction (chapters “Representation of Vulnerability and the Elderly. A Total Survey Error Perspective on the VLV Survey” and “Adapting Quantitative Survey Procedures: The Price for Assessing Vulnerability? Lessons from a Large-Scale Survey on Aging and Migration in Switzerland”). The study of marital break-up and that of displaced workers adopted an alternative, less expansive approach since the participants could choose between paper-pencil or online questionnaires (chapters “Vulnerability Following a Critical Life Event: Temporary Crisis or Chronic Distress? A Psychological Controversy, Methodological Considerations, and Empirical Evidence” and “Career Pathways and Professional Transitions: Preliminary Results from the First Wave of a 7-Year Longitudinal Study”). The research on the working population offered even more choices for their two-stage data collection design: a full online version, a phone interview (CATI) plus an online questionnaire, or a CATI plus a paper-pencil questionnaire. Estimations of the cost reduction associated with using the web-based instrument range from 15 to 20 % (chapter “Career Pathways and Professional Transitions: Preliminary Results from the First Wave of a 7-Year Longitudinal Study”). A shared rationale for the use of different modes is that sensitive questions are located in the self-administered questionnaires to give better comfort to the respondents who feel more reassured about their anonymity.

If mixed-mode approaches have obvious advantages, they can also have drawbacks. Measurement differences can appear when not all participants use the same response format since the stimulus of a question can vary across modes (de Leeuw 2005; Dillman and Messer 2010; and in-depth discussions in chapters “Career Pathways and Professional Transitions: Preliminary Results from the First Wave of a 7-Year Longitudinal Study” and “How to Survey Displaced Workers in Switzerland: Ways of Addressing Sources of Bias”). Linda Charvoz and her co-authors report different results relating to social support depending on whether self-administered questionnaires or interviews are used, the “leniency effect” varying according to the mode (chapter “A Survey of Couples Facing Breast Cancer in Women”). By contrast, in Maggiori and his colleagues’ survey of the labour force, differences between the respondents answering in each mode exist but they are not very large (chapter “Career Pathways and Professional Transitions: Preliminary Results from the First Wave of a 7-Year Longitudinal Study”). The decision to mix modes is a question of searching for the best trade-offs (de Leeuw 2005), also because mixing survey modes is one of the available options for increasing the rate of participation to the surveys. This means that substantive researchers preoccupied by achieving a particular target response rate may be willing to accept a reduction in measurement quality.

3.3 Persuading Vulnerable Populations to Participate

Obtaining the cooperation of the potential participants is another crucial challenge and an increasing source of worry for survey researchers. In most developed countries, refusals to participate in surveys are growing, compounding increased difficulties associated with making contact. Nationwide, the field interviewers were confronted with refusals given without reasons, opposition to any form of survey, no interest in the topic, lack of time, language barriers, and frustration of people living a situation of vulnerability they do not want to speak about. The authors of chapter “Representation of Vulnerability and the Elderly. A Total Survey Error Perspective on the VLV Survey” emphasize the negative impact of aggressive phone calls for commercial purposes and the resulting importance of clearly communicating the scientific objectives of the research to the potential respondents. However, this may not be enough for some vulnerable populations since universities themselves are sometimes assimilated to the authorities. Martina Rothenbühler and Marieke Voorpostel (chapter “Attrition in the Swiss Household Panel: Are Vulnerable Groups More Affected than Others?”) evoke the “cynicism against institutions” that even took an aggressive form among the older immigrants (chapter “Adapting Quantitative Survey Procedures: The Price for Assessing Vulnerability? Lessons from a Large-Scale Survey on Aging and Migration in Switzerland”).

In the end, participation rates across the board were generally low, ranging from 32 to 40 % with however an exceptional 62 % in Baumann and colleagues survey of displaced workers. Different tactics were used to deal with this problem. In Maggiori et al.’s research on the working population, the private company hired by the researchers sent a first reminder by mail, then a second by phone. At the end of this process, a conversion-strategy was implemented and realized by specially-trained interviewers who tried to persuade those who initially refused to participate (see chapter “Career Pathways and Professional Transitions: Preliminary Results from the First Wave of a 7-Year Longitudinal Study”, and a similar approach in chapter “How to Survey Displaced Workers in Switzerland: Ways of Addressing Sources of Bias”). Only the team working on the elderly (the VLV survey) took an ideological position and forewent conversion efforts, though they did invest considerable effort in making contact with the sample members and permitted the use of proxies to obtain a completed interview (chapters “Representation of Vulnerability and the Elderly. A Total Survey Error Perspective on the VLV Survey” and “Adapting Quantitative Survey Procedures: The Price for Assessing Vulnerability? Lessons from a Large-Scale Survey on Aging and Migration in Switzerland”). Another answer to low participation rates is the use of incentives, which is discussed in depth in chapters “Career Pathways and Professional Transitions: Preliminary Results from the First Wave of a 7-Year Longitudinal Study” (Maggiori et al.), “How to Survey Displaced Workers in Switzerland: Ways of Addressing Sources of Bias” (Baumann et al.) and “Attrition in the Swiss Household Panel: Are Vulnerable Groups More Affected than Others?” (Rothenbühler and Voorpostel). Additionally, a big frustration is when a participant starts, then renounces, a pattern which seems to be more common in the online mode of data collection where a simple click is all that is needed to drop out of the survey (see chapter “Career Pathways and Professional Transitions: Preliminary Results from the First Wave of a 7-Year Longitudinal Study”).

The crucial issue is not so much the proportion of those sampled that participate but whether those that participate differ from those that do not on the key variables of interest. The principal concern is that those least likely to participate will also be those who are most vulnerable, resulting in bias in estimates based on the less vulnerable responding sample. People with lower levels of education, and especially those with literacy challenges are known to be less willing to participle in data collections that may be particularly cognitively demanding for them – particularly if the questionnaire is to be self-administered. A less nice way to say the same is that the survey could confront them with their ignorance and marginality (see chapter “Attrition in the Swiss Household Panel: Are Vulnerable Groups More Affected than Others?” by Rothenbühler and Voorpostel). This was found to be true among the older immigrants, who often cumulated limited or inexistent education opportunities in their native country together with a partial linguistic integration in their country of migration (see chapters “Adapting Quantitative Survey Procedures: The Price for Assessing Vulnerability? Lessons from a Large-Scale Survey on Aging and Migration in Switzerland” and “Attrition in the Swiss Household Panel: Are Vulnerable Groups More Affected than Others?”; and also Lipps et al. 2011, as well as many references in the survey literature).

Repeated contact attempts to improve response rates carry the risk of bringing more participants of the same type into the sample, while harder-to-reach “marginal” subpopulations that are more reluctant to contribute to surveys remain nonrespondents (Groves and Peytcheva 2008). Indeed, among displaced workers, Baumann and her colleagues found that repeated contact attempts increased the response rate but did not reduce nonresponse bias. Mode diversification, on the other hand, i.e. telephone interviews in addition to paper questionnaires, improved the participation of underrepresented subgroups. Remaining differences between respondents and non-respondents concerned age, education and occupationalstructures, but none were found on key variables such as the re-employment prospects of the respondents (see chapter “How to Survey Displaced Workers in Switzerland: Ways of Addressing Sources of Bias”).

In chapter “Adapting Quantitative Survey Procedures: The Price for Assessing Vulnerability? Lessons from a Large-Scale Survey on Aging and Migration in Switzerland”, Laure Kaeser discusses how the standard fieldwork procedures in use on the VLV survey had to be adapted to tackle the problem of nonparticipation among migrant elders. The survey included a number of oversamples of elderly migrants from Italy, Spain, and former Yugoslavia, but non-response due to both non-contact and refusal threatened to undermine the success of the project. Different strategies were employed by the fieldwork supervisors and interviewers to try to mitigate the problems, with varying degrees of success. Once again researchers cannot hope for a perfect solution but have to find the best trade-offs when they have to make a choice between adapting the procedures for a specific sub-population with the risk of impacting comparisons with other groups surveyed under different protocols, or strictly applying the same rules to not pre-construct differences associated with the risk of under-representing vulnerable persons.

3.4 Dealing with Time: Prospective and Retrospective Longitudinal Approaches

Of course, for panel surveys, which are so important when a the life-course perspective is adopted, these difficulties are even more important, as the challenge is not only to contact and obtain answers from sample members, but to keep them involved in the survey. The combination is complex between the first survey that has to be as complete as possible and the follow up, keeping as many sample members as possible in subsequent waves. Many of the LIVES surveys involve a longitudinal component, but at the time of writing most were still in their first wave of data collection. To retain participants and maintain their commitment to the survey goals, all the teams use tactics like a small gift, a yearly newsletter and access to the project website. In their research on the labour-market population Maggiori and his colleagues asked their respondents directly about their intention to participate in the next wave. Logistic regressions showed expected but also unexpected biases: young respondents, wealthier respondents, with a higher score on openness, a lower score on neuroticism, but also people with a high score on stress measures were more inclined to maintain their participation (chapter “Career Pathways and Professional Transitions: Preliminary Results from the First Wave of a 7-Year Longitudinal Study”). Of course, these authors were only looking at intention to participate in these analyses, while in chapter “Attrition in the Swiss Household Panel: Are Vulnerable Groups More Affected than Others?” Martina Rothenbühler and Marieke Voorpostel analyze the actual experiences of attrition from the SHP over a period of 14 years. They show a clear effect of education with a higher drop-out rate for people with lower levels. This is also the case for men and young single adults, the latter because of their increased mobility. Indeed, the authors are able to disentangle the reasons for drop-out between individuals who are not eligible anymore, people who left the household, the cases of missing contact information, the refusals, but also event-based vulnerabilities like having health and/or family-related problems. The authors ultimately show that in 80 % of 802 tested variables, the means and frequencies do not appear to be affected by attrition. The use of weights corrects the effect in 10.6 % of the cases but is unable to correct the biases in 8 % and even decreases the accuracy of the estimates in 1.1 %. The delicate issue of post-survey adjustments by means of weighting is also discussed in chapter “How to Survey Displaced Workers in Switzerland: Ways of Addressing Sources of Bias”.

To deal with time and the interweaving of vulnerabilities into the life course – that is, to study vulnerability as a dynamic condition – researchers can be imaginative. Pasqualina Perrig-Chiello and her colleagues distinguish four marital disruption groups according to the time since the breakup or loss. Baumann and her co-authors surveyed displaced workers who lost their job 2 years ago to see whether and how they escaped from unemployment during this period. Basically, unlike the prospective approach that usually characterizes longitudinal research designs, these researchers use a retrospective perspective, which is discussed in-depth in chapter “Using Life History Calendars to Survey Vulnerability” by Davide Morselli and his co-authors. On the basis of a large literature review that crosses psychological and sociological perspectives on human memory, and using as an empirical basis three studies associated with the LIVES program, they discuss the advantages and limitations of these two approaches (among the latter being memory biases, omission or misreporting of life events). Research into solutions to minimize memory errors led to the development of event history calendars, a method that has been incorporated into the SHP, as well as in VLV. The calendar tool appears to be ideal for testing the theory of cumulative (dis)advantages, and the construction of vulnerabilities across the life course and across life domains. Moreover, the authors demonstrate its potential for confronting factual events with subjective interpretations. Adding to the literature results from cognitive interviews realized within LIVES, Morselli and his colleagues convincingly defend the provocative view that recall errors can, in fact, be a highly significant source of information.

Needless to say, research challenges are not restricted to those discussed above, which are only the most present in the contributions to this book. Issues like interviewer effects, respondent-interviewer social interactions, measurement errors, questionnaire design, unclear questions, misunderstandings, etc. are also touched on but only here and there. Furthermore, surveying vulnerable populations raises a number of important ethical challenges. Ethical considerations are already associated with identification (stigmatization) of vulnerable populations, with carrying out repeated contact attempts and handling refusals, with the effect our questions could have on the well-being of the respondents, and with the duty to include vulnerable people in data collections that are the foundations for scientific research likely to document political action. And this list is of course far from being exhaustive. Once again, this is a question about making trade-offs between upholding methodological ideals and negotiating the complex reality of pursuing substantive research aims in vulnerable contexts.

4 Surveying Vulnerabilities: Lessons Learned

In the preceding discussion, we have highlighted some of the challenges involved in conducting quantitative research into vulnerability across the life course. Drawing on theoretical perspectives from the literature on survey methodology that emphasize the need to mitigate potential sources of error, we discussed the various risks to quality present in the LIVES surveys, and the solutions and compromises to manage these risks reached by the researchers responsible for collecting new data. For each of the studies represented in the book, this involved finding a balance between the need to optimize measurement accuracy against the need to adapt to and respect the idiosyncrasies of particular subpopulations and research contexts. In this concluding section we reflect on the key lessons that have been learned as a result of these experiences, and the take-home conclusions of this book. The first concerns the significance of methodology and the collaboration between substantive researchers and methodologists. The second concerns the challenges of interdisciplinarity and its concomitant need for transparency in methodological matters.

4.1 Methodology Matters

The first take-home message concerns the central role of methodology in the pursuit of substantive research interests and the importance of dialogue between substantive researchers and methodologists when designing new research. Put simply, methodology matters. Firstly, data quality – meaning, in a conventional sense, the accuracy of estimates produced by a survey, or more generally, the reliability and validity of research findings – can be enhanced by seeking input from and acting on the empirically-based recommendations of methodologists. Given the popularity of research into vulnerability and its potential influence on policy development, quality is critical and fostering this collaboration should be a priority for future research. The beneficiaries of such collaboration are not only subject specialists, however, but also methodologists. Survey research methodology must be able to respond to the needs of life course researchers interested in vulnerability, which means methodologists involved in researching this field need to constantly interrogate common practices and dominant quality paradigms, to ensure they can adequately accommodate and adapt to the particular demands posed by studies of vulnerable people. As the preceding discussion testifies, conducting surveys of vulnerable sub-populations raises important questions about the extent to which conventional tools and procedures should be retained or adapted depending on the research context. This concerns all aspects of a research design from the methods used to ensure adequate representation of a particular target population, the tools developed for measuring pertinent theoretical constructs, to the procedures used to analyze the data.

Alongside the various challenges involved in data collection described earlier, vulnerability research presents numerous measurement and analytical challenges to which methodologists must be able to respond. Vulnerability is multidimensional, which means that we have to care about measurement quality and not rely on single indicators. Statistical methodology also has to take into account this multidimensionality, by using adequate analytical strategies. Equally, as we have seen, vulnerability can be seen as the result of tensions between stressors and resources. The evaluation of social resources can refer to different types of capitals: economic and cultural, but is also linked to a web of social relations. In this sense, research needs to take into account ‘lives in context’, that is how individuals fit within their social networks and the characteristics of the social and institutional contexts within which these are embedded. All these features of the research domain present unique challenges for future collaborations between substantive and methodological experts.

The rise in popularity of quantitative research into vulnerability has occurred at a time when survey methodologists are having to adapt the methods traditionally used to carry out surveys of the general population. This has been due to the deterioration of traditional sampling frames (particularly for telephone surveys), declining rates of participation, and rapid growth in information and communication technologies (Groves 2011). Survey research is changing. Paradoxically this revolution in how surveys are carried out comes at a time when survey methodology as a science has never been more knowledgeable or better equipped to both pre-empt and correct threats to data quality. Yet much of what has been learned so far has been derived from methodological research conducted alongside surveys of the general population. The empirical base is much more sparse for guidance about how best to survey those at the periphery of society, or how best to capture the dynamics of how individuals respond to major life transitions (whether they be normative or non-normative). In the context of turmoil surrounding mainstream survey methodology, building up specialized areas of expertise around specific substantive research objectives, topics and populations of interest will be of paramount importance. Longitudinal surveys, such as the household panel surveys, are playing a key role in the vulnerability domain – particularly where funding is explicitly made available for methodological research (e.g. in the case of the Dutch LISS Panel, the UK Household Longitudinal Survey). Via this book, the LIVES projects, along with the Swiss Household Panel Survey, are similarly making essential contributions to the methodology of vulnerability research.

A second argument for the essentialness of methodology concerns the use of the data themselves. The results of a survey are not simply “data” in the etymological sense of the wordFootnote 2 but are also “produced” by the system that was used for observation and the context in which it operates (see Desrosières 2001; Héran 1984). This is not a limitation in the validity of surveys themselves, just an important point to keep in mind and to document before using data, and particularly when they have been collected in a complex, non-standard context. In other words, to take into account the characteristics of the data used and the way they were produced is a basic condition for their proper scientific use. It also means that it would be dangerous to separate the roles of methodologists and social scientists, as they are profoundly interdependent. This book as a whole demonstrates that the union of methodological and substantive issues is a prerequisite for a good scientific work.

4.2 Interdisciplinarity and the Need for Transparency

At the start of this chapter we noted two defining features of the LIVES research programme besides the shared substantive goal of investigating the experience of vulnerability at different stages and key transitions of the life course. These features were its interdisciplinarity – both within and across the various individual projects involved – and a predominant emphasis on quantitative methods. Yet while the overriding methodological approach of the LIVES projects relies on survey data collection, variations in implementation can be observed as a result of deeply entrenched, unique methodological traditions of the different disciplines involved. Each discipline has its own toolbox, which is not purely technical but also acts as a lens through which to look at social phenomena, and to produce, manipulate and interpret data. Such “traditions” are apparent when reading different journals,Footnote 3 but most of the time they manifest themselves as strong but implicit rules governing the conduct of research within a given scientific field. In this book, as in the LIVES project more generally, there has been no attempt to obviate this reality. As a result of varied disciplinary priorities, the various projects described have inevitably paid more or less attention to different sources of error in their data, or emphasised different notions of quality over others.

One example of this concerns measurement, where it is not just that the questions asked of respondents in different disciplines can differ, but also that the priorities, in terms of design, can vary: for example, psychologists tend to privilege “validated” scales, often long, while sociologists tend to favour shorter multi-item measures. Similarly, the way in which household income is considered will often be different in economic models where exact values are seen as important, while in other disciplines a more basic ranking (e.g. into deciles) will offer sufficient precision. A second example concerns what is considered as “ethical”, which can similarly vary across disciplines: a practical exercise of walking in a room with elderly people is seen as routine by medical doctors, while a name generator network measure is seen as intrusive. For a sociologist, less at ease with the first case than the second one, the reverse is true.Footnote 4 In other words some crucial elements of design can vary between disciplines and one of the challenges is to initiate from the outset a space for multidisciplinary discussion, organised around the necessity of a common methodology.

This has implications not only at a practical level in terms of the day-to-day negotiations that must take place between the various stakeholders in multidisciplinary research, but also at a managerial level. Multidisciplinary work imposes more documentation requirements and a greater demand for transparency, particularly in relation to research methods. One reason for this is the need to make explicit the implicit methodological patterns at play, but it is also good research practice more generally. An interdisciplinary context should encourage the scientific use and exchange of data, but entails a need among participants to be prepared to accept and share the best practices of different disciplines, and to document the idiosyncrasies of different fields. One aim of this book, therefore, is to achieve this by recording not only the design decisions that have been made in executing the substantive research agenda of each project, but also reflections on the reasons behind decisions, for the sake of transparency and ultimately, to facilitate data sharing. This is of course important for every scientific program and is quite a general challenge.

5 Conclusion

In this introduction, we have emphasised the ‘total survey error’ framework as a theoretical basis for understanding the challenges involved in producing good quality data. However, we do so in recognition of the fact that central to this approach is a notion of quality, which may be at odds with the varied priorities of different disciplines. The ultimate goal in survey methodology is to find the optimal balance between errors and costs in survey design, while prioritising the accuracy of the estimates produced. But the goal of accuracy is often not the driving aim of researchers designing surveys, particularly where the survey concerns vulnerable populations, and the measurement of vulnerabilities. This raises the question of whether we should appeal to notions of quality other than statistical accuracy, given that in the real world, trade-offs and compromises are inevitable (even in studies of the general population). This question is especially pertinent at a time when survey research organisations worldwide are adapting the methods they have conventionally relied upon, and with them, the quality framework that has governed best practices. Alternatives emphasise other standards besides estimator quality, such as the credibility, relevance and usability of a survey’s results (their ‘fitness for purpose’), as well as constraints such as ethical considerations and timeliness (Groves and Lyberg 2010; Weisberg 2005). Many of the chapters here underline the inherent tension involved in trying to apply standards of accuracy in hard-to-survey research contexts, and in many respects its publication is timely, providing an alternative perspective on the realities of implementing academically-led surveys in contemporary society.

Ultimately, the purpose is not to catalogue the difficulties, weaknesses, limits and sources of potential misinterpretation. On the contrary, the various contributions demonstrate that a reflexive attitude and a dialogue between methodological specialists and topic experts are the best means to develop intelligent research designs, avoid blind routines, make proper use of the data, and hopefully, also improve future survey research in this domain. It goes without saying that we cannot hope to completely eliminate all sources of error that affect the quality of the data we collect, but we can strive to make more transparent the various decisions involved in achieving the optimal trade-off between them.