The idea for this special issue emerged after a myriad of lively and provocative debates on the Association for the Social Scientific Study of Jewry (ASSJ) list-serve about the methods used in the social scientific study of Jewry. The methods scholars use largely determine the kind of data they are able to collect and the claims they can make. And yet, sometimes we see scholars get so bogged down in their methods that they lose sight of the big picture question that they are trying to answer. At other times, we see scholars make unfounded claims given their methods and empirical findings. Based on the debates that take place on the ASSJ, it is clear that ASSJ members have varying opinions about the validity and utility of different research methods.

Regardless of scholars’ positions on which methods they prefer, we should all be adhering to current methodological standards. We were excited to co-edit a special issue focused on methods because we want to raise the bar for what counts as rigorous social scientific research. In fact, we agree with Charles Kadushin who pointed out that using the term “scientific” to describe the ASSJ is perhaps superfluous if we are adhering to the highest methodological standards. As Kadushin (personal communication, December 2017) explains:

The word “scientific” study added to “social scientific study of Jews” is superfluous. If it means anything at all it is simply that the studies should conform to the current best methodological practices of social research. With a few notable exceptions, there is no such thing as Jewish methods or Jewish research. There is research that represents the best methodological standards and there is research that falls short of those standards. Sadly, much current Jewish research whether published or in unpublished reports to clients and Jewish stakeholders fails to meet current methodological standards.

We believe that by improving the methodological rigor of research about Jews, scholars can better understand the complexities of Jewish lives and Jewish experiences. We hope that the articles in this issue help readers improve the questions they ask and the methods they apply to their research. As you will see, the articles in this issue are methodological in content rather than reports of empirical research. This special issue make four contributions, which we describe below.

First, this special issue shows how important it is to broaden our studies to include populations that tend to be written off as “too difficult to access.” Sivan Zakai brings the voices of American Jewish children—a population that has been generally absent from the literature on American Jews—to illuminate the discourse about contemporary Jewish relationships to Israel. Using data from The Children’s Learning About Israel Project, Zakai describes three methods that are crafted to shed light on children’s unique ways of understanding Israel and its impact on American Jewish childhood: interviews, elicitation exercises, and storytelling exercises. Her methodological approach shifts the focus of understanding away from a “deficit model” that measures participants’ knowledge against an existing ledger, and towards an “inventory model” that takes stock of participants’ cognitive and emotional warehouses. Zakai’s study shows that despite the challenges of recruiting children for research (including extra IRBFootnote 1 precautions), and the challenges of tailoring questions and prompts to capture the complex thinking, it is both possible and productive to study this important population.

Ilana Horwitz also brings new voices into the study of American Jewish life through an ethnographic study of families who are trying to decide whether their child will continue Hebrew School after (s)he becomes a bar/bat mitzvah. As with young children, studying families requires flexibility to accommodate families’ schedules and relationship-building to gain access to people’s homes. As Horwitz shows, this effort is worthwhile. By placing the family as the unit of analysis rather than individual students or parents, Horwitz illustrates the complex negotiations and conflicts that family members experience. By foregrounding the family, Horwitz identifies two issues that do not surface when families are omitted or relegated to the background. First, negotiations about Hebrew school can have potentially deleterious effects on the family system, including strained marital and parent–child relationships. Second, decisions around Hebrew school may actually reflect parenting styles rather than the extent to which parents value Jewish education. Horwitz’s study shows us the value of taking a family systems perspective and using an ethnographic approach to understand families’ decisions about Hebrew school and Jewish commitments more broadly.

Second, this issue showcases statistical tools that can be used to better understand the nuance of Jewish engagement and identification. Janet Krasner Aronson, Leonard Saxe, Charles Kadushin, Matthew Boxer, and Matthew Brookner present a statistical method—Latent Class Analysis (LCA)—to measure Jewish engagement using a multivariate approach. LCA allows researchers to identify classes (subsets of respondents) with distinctive response patterns across survey items measuring Jewish engagement. Since LCA yields classifications of cases or respondents, it is different from approaches that use a low-high continuum or that develop classifications based on variables (e.g., factor analysis, similarity structure analysis). By identifying subsets of respondents with distinctive response patterns, Aronson et al. find distinct patterns of engagement with Judaism: (1) minimally involved, (2) familial, (3) affiliated, (4) cultural, and (5) immersed. The “cultural” category is noteworthy because it identifies respondents who do not regularly practice Jewish rituals or affiliate with synagogues but do feel strong connections to the Jewish community. In a typical analysis of Jewish engagement, the strong attachments of this group might be overlooked or underestimated. The “affiliated” category is also noteworthy because it identifies a category of Jews who join synagogues and support institutions but do not feel strong emotional ties to the Jewish world. Other analytical approaches, based solely on institutional involvement, might overstate the Jewish engagement of this group. Aronson et al.’s study illustrates how LCA can be used to identify a more complex and nuanced view of Jewish engagement.

Sergio DellaPergola, Ariela Keysar, and Shlomit Levy are also interested in capturing the complex nature of Jewish identification, which they do by comparing measures of Jewish identification between the Jews in the U.S. and in Israel. DellaPergola et al. use Similarity Structure Analysis (SSA) to compare identical questions from two recent Pew Research Center surveys, one fielded in the U.S. and one fielded in Israel. Unlike LCA, which categorizes Jews into different “types” (latent classes) based on the different ways they engage in Jewish life, SSA proposes a holistic approach focusing in particular on the overall configuration of a large number of relevant variables in order to clarify the conceptual structure of the overall identification space and to better assess the meaning of each variable as a function of its position versus all other variables. Using SSA, DellaPergola et al. find the strongest similarity between American and Israeli Jews in the oldest cohort, those aged 70 and over. There is significant divergence starting with the baby boomers in the two countries. Among the youngest, aged 18–21, the authors find a higher degree of religiosity in both countries—which may point the way toward convergence in the future.

Third, this issue considers how to deal with the “rarity problem”—the fact that there are usually too few Jews in national survey data (across all diasporic countries) for statistical analyses. Daniel Staetsky examines whether convenience samples—which rely on membership and subscription lists from Jewish organizations—accurately represent the experiences of the broader population. Researchers have been skeptical of convenience samples, fearing that they represent the more communally involved segment of the Jewish population, and that they don’t represent the socio-demographic characteristics of Jewish populations. Using data from a major multinational European survey of Jews about their perceptions and experiences of antisemitism, and about their engagement in a set of Jewish practices, Staetsky finds that convenience sampling yielded a reliable picture of the perceptions and experiences of antisemitism amongst Jews. However, with regard to engaging in Jewish practices, the convenience sample is reliable only if understood as reflective of the Jewish communal realities. Staetsky’s study shows that while probability-based sampling is preferred, convenience sampling can still be usable under certain conditions.

Fourth, this issue highlights a current debate about what counts as “Jewish philanthropy.” Hanna Shaul Bar Nissim and Matthew Brookner offer a new way to identify Jewish philanthropic organizations by counting an organization as Jewish if it was formed by an individual, family, or group that identifies as Jewish or has a Jewish background. Based on this definition, Bar Nissim and Brookner collect and analyze data of U.S. Jewish-founded grant-making organizations between 2000 and 2015. Bar Nissim and Brookner’s study shows the complexity of defining what counts as “Jewish philanthropy”—a debate which we highlight in this issue by publishing a critique of their study offered by Paul Burstein, and a subsequent response by Bar Nissim and Brookner. While such debates between an author and a reviewer are usually cordoned off to the blind review process, we thought it was an important tension to highlight, especially given the methodological focus of this issue. We thank Paul Burstein, Hanna Bar Nissim and Matthew Brookner (whose identities were concealed throughout the process) for being willing to engage in this debate.

Finally, we want to highlight three types of research designs that we were not able to feature in this special issue but that we believe can greatly improve what we know about Jews. We advocate scholars to consider using these research designs, and to offer more courses so that graduate students can be better prepared to conduct rigorous quantitative research.

  1. 1.

    Natural Experiments are empirical studies in which individuals (or clusters of individuals) are exposed to experimental and control conditions that are determined by nature or by other factors outside the control of the investigators. The process that determines whether individuals fall into the experimental or control condition is as good as random assignment. Two types of natural experiment designs are regression discontinuity (RD) and difference-in-differences (DD). RD designs are often used in educational settings where students receive treatment based on scores, or in policy settings where people receive treatment based on some sort of eligibility criteria. For example, Dee & Penner (2017) use RD to estimate the causal effects of an ethnic studies curriculum piloted in San Francisco high schools. In this case, several schools assigned students with eighth-grade GPAs below 2.0 to take the ethnic course in ninth grade. By comparing outcomes among students whose GPAs fall just below and just above 2.0, we can determine if the ethnic studies course improved academic performance (which it did). RD designs may be particularly helpful to scholars examining the effects of Jewish educational programs, or the effects of financial aid programs on camp attendance or day school enrollment. Meanwhile, DD study designs can be particularly useful for teasing out the effects of policies or programs that might differentially affect different groups of Jews. For example, Shores and Steinberg (2017) use a DD strategy to estimate the impact of the Great Recession on the math and English language arts (ELA) achievement of all grade 3–8 students in the United States. Their DD strategy leverages both cross-district variation in the economic shock of the recession and within-district, cross-cohort variation in school-age years of exposure to the recession. They find that the onset of the Great Recession significantly reduced student math and ELA achievement, and that the recessionary effect on student achievement was concentrated among school districts serving more economically disadvantaged and minority students. This indicates that the adverse effects of the recession were not distributed equally among the population of U.S. students.

  2. 2.

    Quantitative descriptive analysis characterizes the world or a phenomenon—identifying patterns in data to answer questions about who, what, where, when, and to what extent. For an excellent guide to conducting descriptive analysis, please see Description: Why and how descriptive analysis informs education decisions (Loeb et al. 2017). As this report explains, good descriptive analysis presents what we know about capacities, needs, methods, practices, policies, populations, and settings in a manner that is relevant to a specific research or policy question. Thus, data alone are not descriptive research because data are not purposeful: data dumps, all-purpose data dashboards, and generic tables of summary statistics may be useful for some purposes, but do not qualify as descriptive analysis. We need descriptive analysis to clarify our basic understanding of the key aspects of the new phenomenon. Descriptive research can be particularly valuable in the age of large datasets in which the volume of information may otherwise obscure recognition of basic relationships. For example, Arnold et al. (2009) were the first to identify the “summer melt” phenomenon following descriptive analysis of counselor records, exit surveys from graduating seniors, counselor/student interviews, and reports of actual college enrollments for students. The authors reported that with slight variations across schools, 95 to 100 percent of students were accepted into college but, even under best-case scenarios, one-third of the students reconsidered their college plans over the summer after graduation and at least one in five decided not to begin college at all—the “summer melt”. This type of descriptive study can go a long way in helping stakeholders understand that there is a problem and, subsequently, target and test interventions for the population in need.

  3. 3.

    Longitudinal studies, which can be quantitative and qualitative, involve repeated observations of the same variables (such as people) over short or long periods of time, sometimes lasting many years. Longitudinal studies can allow researchers to detect developments or changes in the characteristics of the target population at both the group and the individual level. Since longitudinal studies extend beyond a single moment in time and can establish a sequence of events, they are more likely to suggest cause-and-effect relationships than cross-sectional studies (which are one point in time). Longitudinal studies of Jews are rare because they are expensive and take a long time to complete. In one of the few studies that exist, Keysar and Kosmin (2004) have been following Jews who were bar/bat mitzvah-ed in Conservative synagogues in 1994/1995 (and are now in their late 30s). Although the Jews in this study started in a seemingly similar place, they have gone on to make vastly different choices about their Jewish commitments over the past 20 years. Having data on their lives across multiple time points allows scholars to tease out cause-and-effect relationships that are impossible to identify with cross-sectional studies. Pomson and Schnoor (2018), which follows families whose children were originally enrolled in the same Jewish day school, is an excellent example of a qualitative longitudinal study. Longitudinal study designs can be a particularly effective way to understand the lives of Jews because religious and cultural commitments tend to change over the life course.

In sum, we hope that reading these papers will stimulate senior and junior scholars to think about new and more rigorous ways to approach their work.

Ilana Horwitz & Ariela Keysar