This guide offers readers a concise overview of the elements of social science research, from topic to design to data and analysis. By incorporating examples of research in Arab societies and including exercises using data from the Arab Barometer survey project, the guide situates social scientific concepts and methods in the social and political environment of the Arab world. It will, therefore, be of particular relevance to social scientists who study the Arab region, including, and perhaps especially, Arab scholars who conduct social science research.

The guide does not go into as much depth as do textbooks on the theory and methodology of social science inquiry, but it provides much more information than would an annotated checklist or glossary. Accordingly, with a particular audience and application in mind, the guide seeks to find the middle ground between too much information and not enough information. This introductory chapter discusses the following topics: the origins of this guide and the context in which it has been prepared; what it does and does not cover and the audience for which it is most likely to prove useful; the approach to social science research that has guided decisions about what to cover and what to emphasize; the Arab Barometer, a 16-year old multi-country survey research project with which this guide has a special connection; and a brief summary of the five substantive chapters and the four appendices.

1.1 Origins and Institutional Connections

The idea for this guide to social science research began to take shape when our Arab Barometer team was invited by the Carnegie Corporation of New York, hereafter CCNY, to develop and submit a proposal for the advancement of social science research in the Arab world. In consultation with the very knowledgeable and engaged CCNY program officers, Hillary Weisner and Nehal Amer, a plan was developed to recruit young Arab scholars based at universities or other research institutions and to work with these men and women to develop research and capacity-building opportunities. The proposed project received generous Carnegie Corporation funding through grants to be administered by the University of Michigan and Princeton University. The project’s title reflects both its ambitious goals and the intended beneficiaries: “Understanding Marginalized Communities in the Arab World through Social Science Research: Gaining Insight, Enhancing Capacity, and Building Collaborations to Impact the Region.”

The young Arab social scientists participating in the Marginalized Communities project came from Egypt, Iraq, Jordan, Kuwait, Lebanon, Morocco, Palestine, Sudan and Tunisia. These women and men participated in a series of workshops devoted to the theory and method of social science research and to the development by each participant of an original and publishable research project. Examples of the topics on which participants produced publishable, and published, papers are “Quality of Life among Older Syrian Refugees in Jordan: A Quantitative Study,” “Perceptions of Sub-Saharan Immigrants about Morocco,” and “Are More Educated Women Less at Risk of Experiencing Intimate Partner Violence? Evidence from Tunisia.” The preparation of this guide to social science research was also one of the deliverables that the Arab Barometer team happily agreed to provide.

This was by no means the first time that the Carnegie Corporation of New York had supported and helped to initiate projects focused on social science research in the Arab world. On the contrary, for the last two decades, if not more, CCNY has been encouraging and supporting the conduct of social science research both in the Arab world and by scholars from this region. Capacity-building activities have been an important part of many of these projects. An example, beginning in 2013, is the provision of two-year grants to American universities for “mobility fellowships in support of social scientists from the Arab region.” Fellowship recipients spent a semester or year at a U.S. research university, where they worked on research projects and collaborated with local faculty members. For example, from fall 2014 to spring 2018, the University of Michigan, one of the participating universities, hosted six Arab social scientists from, respectively, Morocco, Tunisia, Palestine, and Oman. Two were political scientists, and the other four were from the fields, respectively, of sociology, history, women’s studies, and development studies.

One of the most important Carnegie-supported projects devoted to building social science research capacity in the Arab world has been developed and led by the American Political Science Association. Designed for Arab doctoral students in Political Science and other social science disciplines and known informally as the APSA-MENA Workshops Program, the project enables teams of Arab and U.S.-based scholars to plan a pair of workshops devoted to research on an issue of both academic and policy significance. Following a call for applications, the team selects about two dozen participants, three or four of whom may be doctoral students at U.S. universities, and then plans the agenda for the workshops.

For example, workshops devoted to research on petroleum rich countries in the Arab world and elsewhere were planned by social scientists from Kuwait, Qatar, and the U.S. in 2017. Workshops were held in Qatar in May and Kuwait in December, and sessions were divided between those devoted to reading and discussing relevant scholarly literature and those devoted the design and development of an original research project by each participant. In 2016, to give another example, a team of social scientists from Egypt and the U.S. organized a pair of workshops devoted to research on associational life and civil society. The workshops were held in Beirut and Cairo.

These few examples, and there are many others, give a sense of the important investment that the Carnegie Corporation has made, and continues to make, in providing opportunities and building capacity to strengthen social science research in the Arab world.

The Arab Barometer has made a different kind of investment in social science research in the Arab world. The Arab Barometer is a nonpartisan research network that conducts public opinion surveys in the Arab region in order to provide insight into the attitudes, values, and concerns of ordinary citizens. Since its establishment in 2006, the Arab Barometer has conducted 68 large, probability-based, and nationally representative surveys organized into seven temporal waves. There are 16 Arab countries in which at least one survey has been conducted; in three of these countries, a survey has been conducted in all seven waves; there are another three in which a survey has been conducted in six of the seven waves; and there are three more in which there has been a survey in five of the seven waves. Table 1.1 shows, as of 2022, where and when Arab Barometer surveys have been carried out. With the exception of Wave VI, when the coronavirus pandemic required phone surveys, all Arab Barometer surveys have been conducted through face-to-face interviews. Slightly more than 100,000 men and women have been interviewed at the time of this writing.

Table 1.1 Seven Waves of Arab Barometer Surveys
Full size table

The following topics are generally covered, some in greater depth than others, in Arab Barometer surveys: corruption, economic concerns, extremism, gender issues, governance, international relations, political institutions, religion, social justice, and youth. The Arab Barometer survey instrument is thoroughly evaluated and revised at the start of each new wave of surveys. The topic and format of many questions asked in previous surveys are retained in order to maximize comparability over time. Questions on topics that are no longer relevant are deleted, and new questions on important topics not covered previously are added. In Wave IV, for example, a battery of items designed to measure attitudes toward the Islamic State was added, and in Wave V a battery on sexual harassment and intimate partner violence was added. Beginning in Wave IV, survey experiments were added to the Arab Barometer interview schedule. Finally, when needed, small changes are made in the instrument to be used in specific countries. Questions may be added to assess attitudes toward important local concerns. Questions that are not relevant or too sensitive may be removed.

Arab Barometer surveys are subjected to extensive assessment and quality control procedures, including both before and after a survey has been carried out. The Barometer’s multi-country Steering Committee, and representatives of in-country partner institutions in some instances, oversee construction of the interview schedule and selection of the questions to be asked, with the primary version in Arabic and a secondary version in English. Thereafter, interviewer training is carried out in participating countries and, as is standard procedure in serious survey research, the survey instrument is pretested and if necessary refined. Following conduct of the surveys, the data are reviewed and cleaned by members of the Arab Barometer leadership team. Information provided by enumerators about each interview, such as whether or not other persons were present during the interview, is considered to assess the likelihood that distortions were introduced. Reliability and validity are assessed, particularly for questions about topics that might be sensitive, and tests are run to identify any interviews that might have been fabricated by an enumerator. Following this review, undertaken at the completion of each survey, a final dataset is prepared and placed on the Arab Barometer website for examination or downloading by others, and Barometer datasets have in fact been downloaded hundreds of times by individuals in many different countries.

This makes the Arab Barometer the largest repository of reliable and publicly available data on the views of men and women in the Arab world. Although designed primarily for use in scholarly or policy-relevant research, and increasingly for instructional purposes as well, Arab Barometer surveys also perform the very important function of giving voice to the needs, concerns, and aspirations of ordinary citizens in Arab countries.

Arab Barometer data can also be used by interested persons who do not wish to download the wave-specific or country-specific data, possibly because they do not have the necessary software or perhaps have not had sufficient training in the analysis of quantitative social science data. These individuals can make use of the Online Analysis Tool, which is available in both English and Arabic. The tool will show the distribution of responses to any question or questions, separately, if desired, for either all respondents or subsets of respondents with different demographic profiles—showing the response distributions separately for women and men, for example. The online analysis tool will show these distributions for any specified wave or waves and for any specified country or countries. The image below shows the output of the Online Analysis Tool when asked to provide the Tunisian Wave III responses to the question, “How would you evaluate the current economic situation in your country?”

figure a

In addition to the data, the survey instrument, and the Online Analysis Tool, the Arab Barometer website contains a great deal of additional information that will be useful to persons interested in the views of Arab publics. Some of the information is technical or methodological. This may not be of general interest, but persons using Arab Barometer data should be familiar with it. The information is also instructive for persons interested in designing new surveys. The Arab Barometer website also contains numerous reports, short articles, and blog posts. Most of these are available in both Arabic and English. They are organized by topic, making them easy to find.

1.2 The Difficult to Define Middle Ground

With support and encouragement from the Carnegie Corporation of New York and with direction and substance provided by the Arab Barometer leadership team, the present guide to the design and implementation of social science research, with particular reference to the Arab world, has been prepared in the hope that it will prove to be a useful supplement to existing textbooks on the theory and methodology of social science inquiry.Footnote 1 A direct and substantive discussion of the elements of social science research begins in the next chapter. Before turning to this chapter, however, it will be useful for readers to know something of the assumptions and choices that have shaped the preparation of this guide to social science research in the Arab world and beyond. Accordingly, we turn now to a brief account of some of these assumptions and choices: the question of depth, the question of breadth, and the intended audience.

The Question of Depth

This guide to the design and conduct of social science research seeks to situate itself at an appropriate and helpful middle ground between textbooks, on the one hand, for which these chapters are by no means a substitute, and on the other, study guides that are little more than checklists of key concepts and terms or glossaries with two- or three-sentence definitions. Readers should approach this guide to social science research with the sense of a continuum ranging from a detailed and very thorough approach on the one hand to a limited and very introductory approach on the other. The present volume seeks to make its contribution at an intermediate point on this continuum, a point that is distant from each of the two poles.

Specifying the degree of detail that defines this intermediate space is to at least some degree an exercise in decision-making that is unavoidably subjective, and in the end perhaps also arbitrary. There is also a significant degree of subjectivity and attendant variation associated with the concepts, methods and applications about which the guide offers information. The guide does not provide the amount of information about any of these concepts, methods and applications that is comparable to that available in textbooks on social science research, of which there are many. So, it definitely is not a textbook, or even what might be called a textbook light, which would be distinguished primarily by the presence of illustrative examples drawn from social or political research in Arab societies. Even with respect to the subjects discussed in relative detail, it will often be necessary to consult a textbook in order to arrive at a fuller and more complete understanding of the meaning, utility, advantages, disadvantages, and assumptions of a particular concept or method. Turning to a textbook will, of course, be even more necessary for a full understanding of concepts, methods and applications discussed in relatively less detail.

Although turning to a textbook is often necessary for an in-depth understanding of a concept or method, the guide is, nonetheless, much more than a checklist or glossary. It provides a valuable foundation for thinking about tasks and operations, conceptual as well as methodological, that are necessary elements of systematic theory-driven and evidence-based social science research. Further, the guide offers not only an introduction to the elements that must be considered in the design and implementation of social science research, but also provides an understanding of how these elements fit together to define the stages and give analytical coherence to a research project.

The Question of Breadth

The challenge of finding a helpful and instructive middle ground between a thorough and in-depth account and a largely superficial account of social science research theory and method pertains not only to the amount of detail to be provided when discussing a particular topic or subtopic; it also pertains to the number of topics and subtopics to be discussed. The goal here is to cover enough ground to give a meaningful understanding of the whole as well as the important parts of the research process in social science, and to do so without taking numerous side-trips into the finer points, what some would call the minutia, of the research process. Thus, with respect to scope as well as depth, the guide seeks a kind of equilibrium between too much and too little—and for those familiar with the reference, it seeks an equilibrium that is the “Goldilocks” point.

The Audience

This middle ground approach to content helps to define the audience for which the guide will be most useful. It will not provide enough detail for individuals whose professional aspirations give an important place to the design and conduct of social science research. To those with such aspirations who are at an early stage in their professional journey, it offers a well-informed point of departure and a helpful guide for selecting the topics and subtopics to which they should devote more attention. But these individuals will soon enough need to engage with and master details and specificities that are below the depth and beyond the scope of the present text.

For others, those who seek a meaningful understanding of social science research but have no interest or need for the kind of mastery required of professional researchers, the guide provides a clear and coherent overview and synthesis of relevant concepts and procedures. Among those who fit this description, those who are the primary audience for which this guide is intended, might be:

  • university students taking an introductory or middle level class on the theory and method of social science research, with the present text assigned along with a textbook on the theory and method of social science research;

  • other university students majoring in a social science or a relevant professional school discipline, the latter to include public health, public policy, and education, among others;

  • men and women who would be part of a social science research team in which others take the lead and have primary responsibility for analytical and methodological details; and

  • those who by interest or profession regularly consume, and perhaps must also utilize in their own work, publications and other papers that report the findings of research projects.

These latter categories suggest that beyond the university setting, which is where most of those for whom this guide is intended are to be found, there are many professions for which an understanding of social science research will be helpful and perhaps even necessary.

There is an additional and very important consideration pertaining to the audience for this guide. Although it will be of value to others, this guide is intended in particular for social scientists who conduct research in the Arab world and, perhaps especially, for social scientists from the Arab world. Discussions of concepts, methods, and applications are regularly accompanied by examples from theory-driven and evidence-based social science research in Arab societies on topics of concern to Arab publics. These examples demonstrate the relevance and value of this approach to social science research; they show that the fruits of such research can be meaningful insights and deeper understandings.

Many of the research reports discussed for purposes of illustration have been published in prominent and high impact Western scholarly journals. This demonstrates that insights derived from the study of Arab society are often of value to social scientists and others whose own research does not have an Arab world focus. This shows, in other words, that there should not be, and is not, an epistemological barrier between Arab social science and social science elsewhere. Each draws upon, and contributes to, the framing, findings, and insights of the other. This guide exposes and clarifies the recursive relationship between Arab social science research and social science research more generally.

It remains the case, however, that many of the published research reports presented in this guide to illustrate various concepts, methods, and applications have been authored by Western social scientists. Many of these are based on in-depth fieldwork, frequently involving collaboration with local counterparts. Some of the studies discussed for purposes of illustration are the work of Arab social scientists, but most are not, and this identifies a potentially non-trivial limitation in the degree to which Arab scholars, investigators, and students will find these illustrations to be entirely instructive. The addition of examples based on research projects designed and led by Arab investigators will make this guide more instructive and valuable, and readers are encouraged to seek out and consider such examples. Finding and discussing such examples might be a particularly useful assignment in courses that ask students to read the present volume.

1.3 Options and Choices

In addition to making clear that this guide to the design and conduct of social science research self-consciously situates itself in the middle ground between textbooks, on the one hand, and annotated checklists and glossaries, on the other, the elements of social science research discussed in the chapters to follow also reflect options and choices of a different kind. These choices pertain to the general approach to social science research that is emphasized, to the primacy that is given to particular units and levels of analysis, and to the chosen balance between quantitative research and qualitative research. It will be helpful for readers to be familiar with these options and choices, and with the reasons particular choices have been made, as they turn to the chapters that follow.

Additionally, these are precisely the kind of decisions about approach and emphasis that an investigator must make when selecting and framing a research question, as well as designing a program of research addressed to this question. Important research questions can often be meaningfully studied from a variety of approaches and perspectives, only one of which is the positivist approach discussed below and in the following chapters. Similarly, it is often the case that there is not only one “correct” unit and level of analysis and not only one “correct” decision about whether the investigation should collect and analyze quantitative data or qualitative data or some combination of the two. The nature of the research question and the investigator’s own interests and objectives are, of course, paramount, as they should be. Indeed, neophyte researchers are routinely advised to select a topic about which they are passionate. Nevertheless, the concerns discussed here represent common choices that investigators must make, whether or not they are aware of doing so.

The Approach

The approach to research discussed in this volume is often described as “positivist.” Positivism refers to investigations that rely on empirical evidence, that is to say factual knowledge acquired through direct observation, measurement, or experiments. Positivism does not refer only to factual knowledge that is quantified, or quantifiable, and thus suitable for statistical analysis. It also refers to factual knowledge, that is to say data based on observation, that is qualitative.

The kind of social science research that is the focus of the present guide is also sometimes described by the term “causal inference.” The goal in such research is to identify and evaluate cause and effect relationships that help to explain not only what the world looks like but also how and why it behaves the way it does; the goal, in other words, is to move from description and prediction to explanation. A search for insights and understandings that have explanatory power references first the way that research questions are selected and framed, and only thereafter the kind of data that are sought and the way in which these data are used and analyzed. This usually involves the specification of one or more conceptually-defined causal relationships and the mechanisms that tell how and why the presumed cause impacts the presumed effect. This may be described as a “causal story,” and it should be made explicit when formulating research questions that ask about cause and effect.

Also to be noted with respect to causal inference is the importance and meaning of the term “inference.” The term references the fact that causation usually cannot be directly observed and must instead be inferred. The analysis is designed to build as strong a case as possible that a relationship presumed to be causal actually is causal, that it does, therefore, have explanatory power. But, with the partial exception of experiments, causation usually cannot be proved, and so it must be inferred and then evaluated in light of the strengths and weaknesses of the case for causation that a researcher is able to build. The evidence with which the case is constructed includes the findings that analysis of the data has produced. It also includes the degree to which the causal story makes sense and is persuasive.

Social science research projects characterized by both positivism and causal inference are also sometimes described as theory-driven and data-based. It is not necessary to be concerned with the differences between these terms. They sometimes overlap in discussions about the approach of a particular research project or set of projects, and the logic of the terms themselves will become more familiar and clear in the chapters ahead. The point to be retained here is that there are various approaches to social science research, all of them valid and frequently used, and that theory-driven evidence-based inquiry is only one among a number of approaches, albeit the one that probably is most widely used today. Other approaches, which are beyond the scope of this volume, include studies that are normative and do not seek empirical grounding; projects that develop innovative methods of observation but seek only thick description and do not lay a foundation for causal inference and explanation; and lines of inquiry that construct formal theoretical models that are developed deductively and are not necessarily evaluated with reference to factual knowledge and empirical evidence.

Units and Levels of Analysis

Unit of analysis refers to the entity whose actions or attributes a research project seeks to describe and explain. Level of analysis just describes research about that unit. If investigators are interested in voting behavior, for example, they can interview individuals, men and women, to learn whether they did or did not vote in the last election. The unit of analysis is the individual, the person who either did or did not vote, and a study that interviewed individuals about their voting behavior would be described as an investigation at the individual level of analysis.

The unit and level of analysis may be determined by the questions a research project seeks to answer. Given the topic, there may be only one right and obvious unit and level. But many subjects can be studied at any one of several levels of analysis. Rather than investigate whether, and then why, each member of a population of individuals did or did not vote, researchers may deem it more productive to describe and explain voter turnout across a number of countries. In this case, country is the unit of analysis, and the study itself is being conducted at the country level of analysis. Still other units and levels of analysis may also be relevant for a study of voting, and the researcher will have to decide which one or ones to pursue. Further, the choice will be based not only on the investigator’s particular and idiosyncratic interests but also on an understanding of the level at which the phenomenon to be studied actually operates.

The conceptual and methodological considerations discussed in the chapters to follow give examples in which sometimes the individual and sometimes the country is the unit and level of analysis. More frequent, however, are individual-level examples, and the reason for this, in part, is because of the availability and convenience of using Arab Barometer survey data to illustrate important points, Arab Barometer data being based, of course, on interviews with individuals. But the principles and methods, as the chapters will make clear, apply in the same way to different units and levels of analysis. Indeed, Arab Barometer survey data can be aggregated to capture variance in which country is the unit of analysis, thereby laying a foundation for comparison across countries and/or over time.

Quantitative and Qualitative Analysis

Much of the discussion and a disproportionate number of examples in the chapters to follow will involve quantitative analysis. This is partly because of the availability and convenience of using Arab Barometer survey data for illustrations and exercises. But this is also, and even more, because data management and analysis methods are not the same for quantitative data and qualitative data, and familiarity with the former is necessary for the investigation of many topics, or at least for the consumption of a significant proportion of the social science research that is being carried out at the present time and made available through publications or other dissemination vehicles.

The present volume’s emphasis on quantitative data and analysis is not to imply that working with qualitative data is less valuable. Studies based on qualitative data can also be theory-driven and evidence-based, just as a study based on quantitative data can eschew explanation and limit itself to description. The conceptual and analytical aspects of research that provide a foundation for causal inference, that come into play when moving beyond description and proposing relationships that may have explanatory power, are just as available and relevant in qualitative research as in quantitative research. The important point, then, is that theory-driven and evidence-based research, research seeking to make the case for causal inference, is not confined to investigations that gather and analyze quantitative data. Research that gathers and analyzes qualitative data may also be theory-driven and evidence-based, and a few examples of such studies are presented and discussed in Chap. 2.

1.4 The Arab Dimension

What, if anything, makes the information presented in this guide of special relevance for social scientists who study society and politics in the Arab world? Put differently, and perhaps better, how can this information be introduced and discussed in ways that make it especially relevant for social scientists conducting research in the Arab world, or who are themselves Arab social scientists? And this despite a possible limitation noted earlier: that many of the research projects presented as examples in this guide were carried out in Arab societies by Western investigators.

This is a non-trivial question that requires a meaningful response. At the same time, the elements of theory-driven and evidence-based social science inquiry that are described and discussed in the chapters to follow are not relevant only for certain political environments, be these Arab or Western or other. On the contrary, they are relevant for social science research in general, and hence in many, and potentially almost all, social, political, and economic environments. Accordingly, then, there is not a specifically Arab social science research methodology. Nor is there one specifically for Muslims or the Middle East or, equally important, for the United States or Europe or Africa or other.

Political conditions in some Arab countries may for a time limit the ability to carry out research and apply the elements of this broadly-applicable approach and methodology, and conditions on the ground may make some research designs more useful than others. And of course, a theory-driven and evidence-based investigation may not be the approach best suited to the questions and objectives of a particular research project. But if the political conditions in an Arab country make it difficult or even impossible to implement the research design that best suits the questions being investigated, the barriers to feasibility are not the result of the country’s Arab character; they are the result, rather, of the political conditions. Investigators in a non-Arab country characterized by the same political conditions would face the same barriers. Similarly, in the absence of these or other limiting conditions, the opportunities for conducting research would be the same in Arab and non-Arab countries.

It is thus with facts on the ground that an Arab dimension begins to appear. It is in their application that the concepts and methods discussed in this guide take on the flavor of a society. Accordingly, the chapters to follow illustrate many of the concepts and methods they introduce with examples of real-world research projects in the Arab region, projects in which the selection and application of particular conceptual considerations and methodological techniques can easily be seen. It is in this way, here through exposure and explanation and elsewhere through the design and conduct of real and lived research experiences, that a connection is made between the elements of social science research in general and in the abstract and the on-the-ground reality of a society, in this case an Arab society—or Arab societies, actually, since there are large societal, political, and economic differences across the Arab world, and indeed, within Arab countries. Notwithstanding the need for more examples based on research projects designed and carried out by Arab social scientists, the illustrations provided in the chapters that follow demonstrate the applicability to Arab milieu, as to other milieu, of the approach to social science research that is the focus of this guide.

Even here there are limitations in the degree to which this becomes an Arab story. On the one hand, there really is not an “Arab” story. There are very important differences among the societies and political communities to be found in the Arab world, differences possibly large enough to outweigh any commonalities that might exist and to call into question the existence of observations or knowledge to which the differentiating term “Arab” can be legitimately applied. On the other hand, research questions and research findings that are relevant to the Arab world, or to a substantial part of it, do not cease to be interesting or relevant beyond the borders of the region. So, in a certain sense, the search for a uniquely Arab research agenda, let alone an Arab research approach or methodology, is for the most part illusory.

These considerations notwithstanding, genuine and deep familiarity with the elements of social science research is acquired not by reading about them and thinking about them in the abstract, unrelated to their application in an actual research endeavor. On the contrary, it is exactly by their application in real-world research endeavors that this deep familiarity is acquired, and such experience must be sought and obtained by the investigator herself; its provision is beyond the scope of the present guide. But by including examples drawn from actual research in Arab societies, the chapters in this guide encourage readers to put themselves in the place of the investigator. In this way, the elements of social science research are to be considered and understood by readers from the perspective of their application in a real or imagined research project addressed to important on-the-ground concerns in one or more Arab societies.

Thinking about real-world applications will increase the perceived relevance for readers of the concepts and methods being discussed, thus contributing to learning about them and understanding more fully their place in the research process. In addition, however, this will bring an Arab sensibility to their thinking about social science research. As they reflect on the places and issues in the Arab region that are central to the research exemplars described in the text, and as they perhaps also reflect on a project they would themselves design and conduct, their thinking about facts on the ground will strengthen the sense of a connection between the principles of social science research and the Arab arena, or arenas, in which these principles will be applied.

There is one additional way, and an important way, in which this guide seeks to develop and communicate an Arab sensibility. This is by drawing on the Arab Barometer in a number of instances in the chapters that follow. The nature and value of the Arab Barometer survey project were discussed earlier, and the data it provides are an invaluable resource for examining the nature and determinants of the attitudes, values, concerns, and preferences of Arab publics. The chapters that follow build a connection to the Arab world by using Arab Barometer data to illustrate procedures, give examples, and suggest exercises. The latter can easily be done using the Online Analysis Tool on the Arab Barometer website. As readers consider possibilities and make decisions about the countries and time periods from which to take data to complete a suggested exercise, the sense of a connection between the principles of social science research, on the one hand, and the reality of the Arab world, on the other, will again be strengthened. Readers may also imagine their own exercises, looking into the nature and distribution of attitudes toward selected issues or just exploring response distributions without any previously selected research question. Here again, and even more fully, readers will immerse themselves in the world of Arab publics, guided by their own questions but also by the principles of social science research.

1.5 The Chapters

There are five chapters in this guide. They are based on and ordered by the number of variables to be considered. Chapter 2 deals with variables one at a time, and hence is devoted to univariate analysis. Chapter 3 deals with variables two at a time and the possible relationships between two variables. This, of course, is bivariate analysis. Chapter 4 considers the different ways that analyses can be enriched if they involve three or more variables. This can be described as multivariate possibilities.

As discussed briefly below, the three chapters may also be described and named by the principal conceptualizations and methodologies they discuss. Chapter 2 focuses on variance, a concept that is central to theory-driven and evidence-based research. It is, therefore, labelled Variance, Variables, Data, and Measures. Chapter 3 builds on the notion of variance and asks either why something varies, on what does it depend, or what difference does it make. The chapter is appropriately named Associations, Hypotheses, and Causal Stories. Chapter 4 discusses the many ways that the analysis of data can be enriched and made both more rigorous and more informative by the addition of more variables. This ranges from testing hypotheses and evaluating causal stories with models that include control variables to specifying in conceptual terms the conditions under which confirmed hypotheses do and do not apply. Consistent with these concerns, the chapter is named Causation, Control, and Conditionality.

Variance, Variables, Data, and Measures

Chapter 2 is devoted to univariate analysis, which is concerned with description. The phenomena in which social science is interested for research purposes differ in either degree or kind, hence the variance. Also essential in this connection is unit of analysis, which is the entity that varies and the level at which that variance occurs. Considering research at the individual level of analysis, for example, meaning that the unit of analysis is the individual, there is an almost infinite number of ways in which an attitude or value, hence a normative orientation, or a pattern of behavior, can vary across a set of individuals. Some think and/or behave in one way, whereas others are different. This applies equally to a set of countries. Some countries are characterized by one set of attributes or have had one kind of experience, whereas other countries are characterized by different sets of attributes or have had different kinds of experiences. And of course, there is a vast array of other units of analysis, including provinces, political parties, charitable organizations, religious movements, and many, many more.

The objective of univariate analysis, as will be discussed in Chap. 2, is description. It is to determine and describe the distribution of ways that units—read individuals, countries, or some other unit—can and do differ with respect to the phenomenon under study, the phenomenon being a variable since it does in fact vary. For example, suppose again that the unit of analysis is the individual, that a researcher is working with a set of 1000 individuals, and that the variable of interest is interpersonal trust, which is the belief that most people are honest and trustworthy. In this case, the objectives of univariate analysis would be to determine, first, the range of ways that interpersonal trust can vary across the available set of individuals, presumably ranging from very high trust to little or no trust; next, to gather data that will permit each of the 1000 individuals to be assigned a value that records his or her particular level of interpersonal trust; and finally, to present a frequency or percentage distribution of the interpersonal trust values. This information can be presented graphically, as well as in tabular form.

This distribution, sometimes described as a mapping of the variance on a variable of interest, or on several variables of interest with each mapped separately, provides descriptive information. The intent of the researcher may be to build on this description and proceed to bivariate or multivariate analysis. Or description alone may be the objective, with the researcher thus using one or several of the standard graphical and/or statistical ways to display and communicate findings about the distribution of interpersonal trust ratings. The discussion in Chap. 2 includes a summary of the most common ways to display and communicate univariate and descriptive findings.

Two other topics covered in the chapter are data collection and measurement. The former includes a discussion of different methods of data collection and the implications this has for the concepts a researcher wants to study. The latter gives particular attention to the measurement of concepts that cannot be directly observed. This includes a discussion of the criteria by which the quality and suitability of a measure can be evaluated: particularly validity (whether the measure does in fact measure the concept it purports to measure) and reliability (whether the measure is replicable, which in social science often means agreement among a number of presumably valid indicators of the concept).

The term “operationalization” refers to the procedures an investigator uses to measure a concept that is abstract and cannot be directly observed. These measurement procedures are often described as an “operational definition” of the concept. Continuing to focus on interpersonal trust, with individual the unit of analysis and data to be collected by a public opinion survey, the investigator would begin by including in the survey instrument a number of questions that appear to be indicators of interpersonal trust. These might be questions that have been shown in previous surveys to be reliable and valid indicators of the concept, or they might be original questions devised by the researcher, or some of both.

Once the investigator’s survey has been completed and the data are ready for evaluation and analysis, she can use any one of several standard statistical or other tests to determine whether each question is reliable and valid in her particular study. It is possible that indicators of a concept that are reliable and valid in some settings are not reliable and valid in other settings. Finally, the investigator will use, singly or in combination, those items that her tests have shown to be, or likely to be, reliable and valid. She might also choose to combine these items and construct a scale or index of interpersonal trust, using one of the available scaling techniques. The questions or multi-question scale that the investigator will then use to rate or categorize each individual with respect to interpersonal trust constitute, as stated, an operational definition of the concept.

Associations, Hypotheses, and Causal Stories

In theory-driven and evidence-based social science research, the variable or variables that have been the focus of univariate analysis are often the point of departure for bivariate analysis. The goal of bivariate analysis may simply be to describe the way two variables are related to each other. But in the context considered here, and as discussed in Chap. 3, the objective is to move from description to explanation. Toward this end, the researcher asks why the variance occurs—in other words, what determines or accounts for the variance, and/or what difference does it make. Prior to data collection and analysis, a researcher should propose one or more answers to these questions, along with the reasoning that makes them persuasive, or at least plausible. Later, once data have been collected and are ready for analysis, the propositions can be evaluated in order to determine their accuracy.

The proposed answers, that is to say the propositions, are most frequently offered in the form of “if…then” statements, or cause and effect statements, which are usually called hypotheses. Simple and straightforward bivariate hypotheses are composed of a dependent variable and an independent variable. The independent variable is the cause, or the “if” in an “if…then” statement, and the dependent variable is the effect, or the “then” in an “if…then” statement; and a proposed hypothesis, meaning an “if…then” statement, will usually specify how as well as whether the independent and the dependent variable are related.

To continue using interpersonal trust for purposes of illustration, the interest of a researcher in this variable may be either as a dependent variable or as an independent variable, or possibly both. Recalling that the unit of analysis is the individual and that interpersonal trust varies among a population or sample of individuals from a very high level to a very low level, a hypothesis that considers interpersonal trust as a dependent variable will specify what other variable or variables have, or tend to have, an impact on it, either raising it or lowering it or changing it in some other way. If one such variable is level of education, for example, and if the hypothesis posits a direct and positive linear relationship, then the hypothesis simply states that an increase in a person’s level of education causes, or disproportionately frequently causes, her level of interpersonal trust to go up. In other words, if an individual’s level of education increases, then her level of interpersonal trust will also generally increase. If the hypothesis is confirmed, it may be said that the variable education helps to account for the variance on interpersonal trust.

The researcher may, of course, posit a hypothesis in which interpersonal trust is an independent variable, and so it posits an answer, or partial answer, to a question about the consequences of a change in interpersonal trust. Perhaps support for gender equality is one of the variables affected by interpersonal trust, such that, if the hypothesis again posits a direct and positive linear relationship, then the hypothesis states that an increase in interpersonal trust tends to bring about, or frequently brings about, an increase in support for gender equality, Once again as an “if…then” statement, if an individual’s level of interpersonal trust increases, then her level of support for gender equality will generally increase as well.

A research project often has more than one hypothesis, and the direct and positive linear relationship in these examples is only one possibility among many. Chapter 3 discusses the way that variables and variable relationships are selected—the process of hypothesis formation—and it introduces and illustrates different kinds of bivariate relationships. In addition, Chap. 3 shows how bivariate relationships can be displayed in charts and graphs. It also discusses the use of bivariate statistics to determine whether an observed bivariate relationship differs significantly from the relationship that would be expected if the variables in the relationship were, in fact, unrelated.

Causation, Control, and Conditionality

Chapter 4 provides an overview of the reasons and ways that one or more additional variables might be included in the analysis of a hypothesized bivariate causal relationship. One of these involves the inclusion of control variables, variables that are related to both the independent and dependent variables and might lead to an erroneous conclusion about the hypothesis if not held constant by including them in a multivariate analysis. Without appropriate control variables, the analysis might confirm the hypothesized relationship when in fact this conclusion is not justified. The erroneous and unjustified confirmation of a research hypothesis constitutes a false positive. It is called a Type I error. There is also a Type II error, as this suggests. A Type II error refers to a false negative, when a research hypothesis is rejected, or unconfirmed, when this is in fact an erroneous conclusion.

For example, the hypothesis of a positive linear relationship between a person’s level of education and her level of interpersonal trust might appear to be confirmed in a bivariate analysis. It might be the case that if the person’s education increases, then her level of interpersonal trust does indeed increase as well. But it might be erroneous to attribute causality to this bivariate association since one or more variables related to both the dependent variable and the independent variable, to both education and interpersonal trust, have not been controlled. Salary, or salary level, is such a variable; and to avoid the risk of making a Type I error, it would be necessary to carry out a multivariate analysis in which salary is included in the analysis. If the relationship between education and interpersonal trust remains strong when salary is included as a control variable, the case for inferring causality will be strengthened. If the hypothesized relationship does not remain strong when the control variable is included, it may be concluded that the bivariate relationship is not a causal relationship.

There are many ways in which the inclusion of one or more additional variables can also make it possible to offer a more refined and complex causal story, and/or to make the story more precise by specifying when, that is to say under what conditions, a particular variable does and does not have explanatory power. One of the possibilities discussed in Chap. 4 is the distinction between a direct and an indirect bivariate relationship. The distinction is discussed and explained in a section of the chapter entitled “Third Variable Possibilities,” as is the way that data can be analyzed to determine whether the relationship between an independent variable and a dependent variable is direct or indirect. Determining whether a variable relationship is direct or indirect has the potential to significantly enrich the causal story on which a hypothesis is based.

Additional variables may also constitute conditionalities, which specify in terms of concepts when the findings about a relationship do and do not apply. Returning to the hypothesis that education accounts for variance in interpersonal trust, an analysis of the available data may confirm the relationship and justify risking a Type I error. But if the data come from one particular kind of location, say for example, wealthy MENA countries, the researcher and people interested in her findings will probably want to know whether the relationship is also confirmed and has explanatory power in less affluent MENA countries. Attributes that specify when a finding does and does not apply are called scope conditions. Should further study by the researcher or others gather data from less affluent countries and find that an analysis of these data do not confirm the hypothesis, national wealth will have been established as a scope condition; the hypothesis is confirmed and has explanatory power in countries with one level of national wealth but not in countries with a different level of national wealth.

Chapter 4 also discusses conditional relationships within a set of scope conditions. Testing for this sort of possibility involves disaggregation, which refers to testing a hypothesis separately among different subsets of the units about which the researcher has data. If the researcher expects a relationship to be different for men and women, for example, she might do the test for men and women separately. And if it is found that the hypothesis is confirmed for one sex but not the other, sex will have been found to be a conditionality. As noted earlier, this information will enable the researcher to tell a more complex and potentially instructive causal story.

Chapter 4 will very briefly discuss some of the multivariate statistical methods that are used to test hypotheses and increase confidence in a causal story. A detailed account of multivariate statistics is beyond the scope of Chap. 4, however. The chapter will also briefly discuss the use of multivariate analyses for purposes other than testing hypotheses or identifying conditionalities.

Appendices

Four appendices follow Chap. 5. One deals with fieldwork and its challenges, including fieldwork in one country or society carried out by a researcher from a different country or society. A second deals with ethical considerations in social science research, including and particularly, international and cross-cultural social science research. A third appendix gives a somewhat fuller description of survey research methodology. Although surveys are only one of the many ways of collecting or generating social science data, the topic is given more attention here due to the connection of this guide to the Arab Barometer survey project. A final appendix very briefly describes the most widely used software packages for the management and analysis of social science data.