This book started with a reference to literature dealing with transparency and replicability of research. Such literature for the most part highlights issues related to the transparent documentation of methodological proceedings (Damian et al. 2019) and the replicability of existing results (Munafò et al. 2017; Breznau et al. 2019). This book relates in so far to these contributions, as it shows that achieving a transparent, comparable and ultimately replicable result not only means disclosing how an empirical analysis is designed but also spelling out the theoretical conceptualisations that are later included in an empirical test.

The operationalisation of the welfare state as an independent variable proved to be a suitable example to show why it is highly advisable to look in detail at the conceptual aspects behind the operationalisations. It relates to both issues—transparency and replicability: if it is not clearly stated why one operationalisation is preferred over another, the process is not transparent; at the same time, replication of results is not possible if the selection of measurements is not based on comparable conceptual premises. While the operationalisation of the welfare state seems to be particularly affected by these issues, greater attention to the conceptual premises behind the independent variables is likely to be beneficial in other research areas as well.

7.1 Summary and Main Findings

The objective of this book was twofold: to evaluate common approaches to including the welfare state as an independent variable and to develop a proposal for standardising measurement. Two research questions guided this endeavour:

How comparable are the results that emerge from different approaches to operationalising the welfare state as an independent variable?

How can we derive a more standardised, transparent and comparable approach to operationalising the welfare state as an independent variable?

In order to answer these questions, I provided an overview of how the welfare state is measured in existing research, and why—from a more theoretical point of view—it is not only an important dependent variable but also an essential independent variable in comparative analyses of various objects of research on the individual level. Both aspects are discussed in the second chapter of this book. In the brief introduction to comparative welfare state research and the functions welfare states fulfil—intentionally or unintentionally—the high relevance of the indicator is confirmed.

In the third chapter of this book, this led to the question of how the welfare state is operationalised in current research that treats it as an independent variable and thus to the exploration of the first research question. A review of existing approaches reveals considerable heterogeneity in the existing operationalisations. This heterogeneity is reflected in the results of a brief empirical comparison, which confirms that empirical results are very susceptible even to minor changes in the operationalisation.

Based on this, the answer to the first research questions can be stated very clearly. Comparability of results is frequently impaired on several levels if different operationalisations are treated as interchangeable options. First, the type of selected operationalisation matters. The single indicator approach, the regime approach and the composite index approach manifest entirely different perspectives on welfare stateness. They conceptualise based on different premises and neither of the approaches can substitute another. Second, within each approach the choice of distinct indicators is crucial. Whether they highlight, e.g. social rights, welfare effort, or social investment represents a commitment to only one perspective that cannot be compared unconditionally to others. Therefore, combining approaches is problematic. Third, multidimensional operationalisations (regime typologies and composite indices) are highly dependent on the respective country sample. The extent of a possible bias is most evident in the buffet approach in case of regime typologies where small changes between typological approaches due to mixing or extending classifications already produce severely differing results (cf. section 3.2). This realisation represents an important finding of this contribution: it suggests that the very frequent practice of using welfare regime typologies as independent variables is highly problematic and should be abandoned in the majority of cases. Fourth, comparability can also be distorted by subtler issues such as differing data sources for similar indicators.

All of these issues have in common that they appear almost trivial. At the same time, they are neglected frequently as soon as welfare stateness is treated as an explanatory variable. A lack of available macro-level data, a lack of discussion of how and why welfare stateness is conceptualised in a specific way and the lack of a guideline clearly spelling out possible proceedings for specific research questions are among the explanations for this deficit. All of this raises the importance of discussing not only a dependent variable problem but also an independent variable problem, which arises when operationalising features of welfare states in cross-cultural multilevel analyses. Furthermore, this independent variable problem requires its own distinct debate. While we can borrow some insights from macro-sociological comparative welfare state research (such as the distinction between a focus on social rights and expenditure), the mechanisms at work when social policies shape individual-level outcomes represent unique puzzles and require their own reflection.

This answer to the first research question leads directly to asking the second question (how can we derive a more standardised, transparent and comparable approach to operationalising the welfare state as an independent variable?). Clearly, what is needed in order to increase comparability is a guideline specifically targeting research endeavours in which welfare stateness serves as an explanatory factor. If existing approaches to including the welfare state as an independent variable are too heterogeneous, produce too deviating results and are not comparable enough—how can we solve this problem? Fathoming this question, I took a step back in the fourth chapter and looked at the research questions we are actually dealing with: what are frequently examined outcomes assumed to be impacted by social policies? What are the reasons for including the welfare state as an independent variable? And, more specifically: which mechanisms are underlying popular hypotheses and how do these mechanisms relate to the functions of welfare states, which warrant their inclusion as an independent variable in the first place? This review resulted in a list of mechanisms and explanations that are either implicitly or explicitly addressed in theoretical argumentations and hypotheses. More importantly, it also revealed two distinct perspectives on the welfare state as an explanatory factor. The first is a top-down perspective tied to mechanisms assuming that explanations originate from the factual nature of welfare states (provision of security, redistribution, stratification, activation, and socialisation). The second—a bottom-up perspective—captures mechanisms and explanations tying welfare states to outcomes based on the individual perception of social policies—regardless of the accuracy of such perception (evaluation and self-interest). I have discussed in detail whether this list of mechanisms and perspectives is exhaustive or not. In the end, I reach the conclusion that this question is not important for the purpose that they serve in this contribution. This purpose is to be a starting point for a differentiation of analytical perspectives on the welfare state that are embedded in popular research objectives.

This differentiation was the focal point of the fifth chapter where distinct conceptualisations of welfare stateness were explored in more detail. Within the top-down perspective, this revealed at least three systematised concepts: The Responsive Welfare State, the Enabling Welfare State, and the Normative Welfare State. In addition, the Assessed Welfare State captures the bottom-up perspective. In the course of the chapter, several suitable sets of indicators for each systematised concept of welfare stateness were proposed. The selection was made on the basis of three criteria (clarity, comparability, and availability) and it highlights single indicators rather than composite measures, as this is the only way one can be sure what is actually captured by an indicator. I argued that distinguishing such conceptualisations should considerably help to achieve operationalisations that are more comparable. Instead of trying to capture “the welfare state” in an empirical measurement, it narrows down the operational choices to indicators that represent a distinct concept of welfare stateness. Ultimately, this presents a possibility to standardise the operationalisation in a theoretically meaningful way.

The usefulness of distinguishing between such conceptualisations of welfare stateness was put to the test in the sixth chapter. Here, two popular objects of research—risk of poverty among the working-age population and attitudes towards the welfare state—served as exemplary dependent variables. Indeed, the different systematised concepts proved useful for the theoretical deduction of relevant analytical perspectives on the welfare state and the interpretation of results. They narrow down the broad spectrum of relevant facets of welfare stateness and their functions and thus serve as guidelines for the conceptual discussion. However, it remains somewhat unclear, which indicators capture these concepts best.

The performed analyses revealed some helpful insights. First, some conceptualisations could be captured more successfully than others. Especially the Enabling Welfare State could be differentiated clearly from other conceptualisations. Second, the importance of distinguishing the top-down perspective from the bottom-up perspective was clearly confirmed. The performed analyses suggest that in the field of attitude formation the latter—represented by the Assessed Welfare State—is of particular importance. Third, capturing the Responsive Welfare State and the Normative Welfare State proved to be more difficult. In case of the former, this was because almost all indicators of social policy-making potentially represent responsiveness. Still, only benefit coverage allowed a clear interpretation in the sense of securing against risks, while the rest of the indicators (benefit duration, contribution period) failed to produce results that could be allocated clearly to the Responsive Welfare State in either of the two analyses. Regarding the Normative Welfare State, the operationalisation was complicated because measuring a socialising impact of social policy-making is difficult, may require a more nuanced approach and possibly another temporal perspective (for instance, by including features of welfare stateness during formative years of a respondent).

The encountered difficulties in pinning down a definitive list of indicators per conceptualisation should by no means discourage from pursuing the proposed approach. At its very core, this contribution showed that taking a step back to narrow down the concepts behind hypotheses about the impact of social policies on individual-level outcomes is a fruitful and highly valuable endeavour. Against the backdrop of the very problematic inconsistency of approaches in the literature, a major goal must be to achieve better comparability and standardisation. I believe the proposal of using systematised concepts of welfare stateness as an intermediary step before the empirical operationalisation, is able to increase such comparability and standardisation substantially. In short, this is because we need to know which areas within the welfare state are highlighted in our theoretical arguments and hypotheses before turning to any kind of measurement. Using a shared conceptual framework will reduce the operational choices to those that are eligible for a specific concept. Moreover, it will force to elaborate in more detail the reasons for choosing one operationalisation over another and it will present a shared reference for discussions reviewing previous research on a given explanandum.

While all of these concluding remarks are promising and worth more attention in the future, they remain fuzzy when it comes to specific operationalisations and concrete practical recommendations. The open questions emanating from the empirical analyses in the last chapter are in so far unsatisfactory, as they still leave us without a clear recipe spelling out which specific indicators to select for which research question. While such a recipe would have been attractive, it may also be premature if only based on the two analyses performed in this book. In order to agree on a distinct set of indicators for specific research questions and hypotheses, much more research is needed. Similarly, the proposed framework including the four deduced conceptualisations of welfare stateness is anything but set in stone. Here, as well, further critical research is needed.

Finally, embedded in the issue discussed in this contribution is a much bigger question: what does the welfare state do? This question is not meant in the sense of mere arrangements of policies, the manner of resource allocation and similar organisational details. It aims at the functions of welfare states, which go well beyond those intended when social policies were first introduced in the late nineteenth century and which are not necessarily intentional at all. Behind the proposed conceptualisations is the implicit assumption that today’s welfare states shape individuals in much more complex and subtle ways than they did in the past. It suggests that relying on institutional aspects alone does not fully cover the functions contemporary welfare states serve for individuals—the evidence for the importance of the Assessed Welfare State supports this quite well. In this contribution, I treated this issue as being relevant to empirical choices. It may, however, also be a more general theoretical problem, which deserves much more attention. It is my hope that the discussion in this book inspires more research on the matter.

7.2 Implications for Further Research

This contribution highlights a problematic issue, which has not received detailed and systematic attention before. As argued in the second and third chapters of this book, empirical operationalisations of welfare stateness are discussed in comparative welfare state research—but usually not with an emphasis on the consequences of different methodological choices if features of the welfare state are analysed as independent variables. Because this specific issue presents a blackbox in many ways, even the more detailed discussion in this contribution leaves some aspects unanswered.

From a conceptual perspective, the proposed framework and especially the four conceptualisations of welfare stateness have to be explored in more detail. Throughout this book, I always reduced the discussion to exemplary research questions, dependent variables and hypotheses. In many cases, it seems likely that arguments can be translated to other subjects as well, but this remains a postulate until explicitly explored in more detail. Moreover, the addition of other conceptualisations, besides the four discussed here, seems quite possible. From a more general perspective, the universal usefulness of the proposed framework and the intermediary step of narrowing down discussions to distinct conceptualisations of welfare stateness needs to be tested. The mere fact that they proved useful in the examples chosen in this contribution does not yet mean that this is always the case. In the preceding section of this chapter, I have compared the proposed approach to a typology of analytical perspectives that may serve as templates during the research process. Such templates can be very positive and fruitful but they can also present a limitation for research processes if they narrow down proceedings too much. Only further research can show, which of these two options proves to be true.

Room for further research exists especially from an empirical perspective. The proposed indicators have to be tested on other data sources, perhaps using other country samples and other analytical approaches. More importantly, however, other indicators representing further targeted operationalisations of specific concepts should be explored. As this contribution shows, this will require looking beyond the more established indicators. Lastly, the proposed framework has to be tested empirically on other outcomes—for instance, behaviour or physical well-being (as discussed in chapter 4). Only if the proposed conceptualisations are translated successfully into empirical operationalisations that fit all research endeavours highlighting similar mechanisms, can they amount to a universally applicable tool.

Besides such points of departure for further research, this contribution also produced several implications in the sense of clear recommendations. The first is an appeal: discussing operationalisations of welfare stateness is not just a dependent variable problem. Instead, it is equally relevant in the case of independent variables—especially because welfare states are assumed to be responsible for outcomes of great socio-political relevance. However, the assumption that the same issues discussed in literature on the dependent variable problem are also relevant in the case of operationalisations as independent variable is wrong and potentially dangerous for the reliability of results and their comparability. This is because established operationalisations—such as typologies—are often unsuitable as independent variables and because the role of social policies in explanations of individual-level outcomes as diverse as well-being, risks, behaviours, attitudes, and more is entirely different compared to the welfare state in comparative welfare state research.

The second implication is one that may be relevant for other research objects besides the welfare state as well: the operationalisation of central independent variables is as important as that of the dependent variable. Justifying the selection of a measurement only by its previous use in the current state of research is not sufficiently accurate and it may lead to severe bias in results—again, using welfare regime typologies and the buffet-approach comes to mind.

The third implication addresses the general approach towards the welfare state as an independent variable. Evidently, instances in which the entirety of social policy-making is emphasised when explaining why outcomes differ between countries are very rare. Hypotheses highlight specific perspectives on the welfare state (e.g. top-down versus bottom-up), specific functions of welfare states (e.g. activation versus socialisation) and specific areas of policy-making (e.g. unemployment versus family policies). In light of this, it should be obvious that there cannot be one universal operationalisation and that it is almost never “the welfare state” as a whole that is relevant for testing hypotheses. Regardless if it is the framework proposed in this book or another—we must narrow down which parts of social policy-making are relevant in hypotheses and model these specific parts. Only then can empirical analyses be comparable between studies exploring similar dependent variables.

The fourth implication is a very specific one: only those operationalisations seem suitable as independent variables that are clear, comparable, and available. Considering these criteria (in addition to the various other problematic issues), typologies appear to be very problematic candidates for independent variables. Other composite measures are similarly problematic as testing distinct explanations and capturing specific mechanism is almost impossible as long as effects cannot be attributed to clearly definable features of welfare states. Some single indicators—such as social expenditure—are partly affected by similar problems. Overall, single indicators present the most recommendable choice as they allow narrowing down results to specific aspects.

This book approached an issue, which has received too little attention so far and this oversight is indeed problematic. The welfare state is an important independent variable for the explanation of a multitude of objects of research. Such research is only comparable if it follows a transparent and replicable operationalisation of welfare stateness as an explanatory factor. After it became evident that such a standardised approach could not be found in existing methodological contributions within comparable welfare state research, this contribution addressed the issue from an entirely different perspective. By focussing in much detail on the conceptualisation of welfare stateness as an explanatory concept, it proposes a new intermediary step, which serves as a tool for narrowing down distinct perspectives on welfare stateness. This intermediary step entails a strong focus on pinpointing distinct conceptualisations of welfare stateness before an operationalisation takes place. Explicitly committing to a conceptualisation and explicitly modelling it empirically will increase the transparency of approaches as well as the comparability of results. While this contribution leaves open several questions and raises several more, it still shows the importance of discussions on this issue. Such discussions will hopefully continue in further research, including a critical assessment of the framework proposed in this book and fruitful additions to the list of empirical choices for the operationalisation of distinct conceptualisations of welfare stateness.