1 Introduction

Qualitative researchers sometimes talk about objectivity in relation to qualitative data sets. For example, Kathleen DeWalt and Billie DeWalt maintain that a qualitative data set in the form of a “record of observations (field notes) should be as accurate, complete, detailed, and objective as possible (DeWalt and DeWalt 2011: p. 166—my italics). Similarly, Matthew Miles, A. Michael Huberman, and Johnny Saldana state that qualitative data in the sense of “[t]the words we choose to document what we see and hear in the field can never truly be ‘objective’; they can only be our interpretations of what we experience” (Miles et al. 2020: p. 7—my italics of “objective”).

In this paper, I take up the idea that qualitative data sets may be more or less objective. More precisely, I develop an account according to which a qualitative data set is objective to the extent that it, in conjunction with true assumptions, possesses a combination of good-making features (epistemic values, epistemic virtues). In virtue of these features, the data set is suited to serve as evidence base for a satisfying answer to the research question under study. I recommend this account on the ground that it may serve as a useful and reachable guiding ideal in qualitative data generation.

The qualitative researchers quoted a second ago hold that the notion of objectivity applies to qualitative research. Yet there are also theorists who maintain that the notion should be completely dropped in the context of qualitative research (see Hammersley 2012). By offering an account of objective qualitative data sets, I obviously part company with this latter group of qualitative researchers. In my view, the notion of objectivity is, to borrow Sandra Harding’s formulation, too powerful to be abandoned (Harding 2015: p. ix). “Objectivity” is a term of epistemic praise—a stamp of epistemic quality. Insofar as qualitative researchers renounce on its application to qualitative research, it may seem as if they do not regard their own research as being epistemically praiseworthy. I think that qualitative researchers should counter this impression: properly conducted qualitative research is worthy of epistemic praise. Thus, rather than give up on the notion of objectivity, it is better to promote an understanding of objectivity which may serve as a reachable—and useful—guiding ideal in qualitative research. I believe that such an ideal is currently lacking and the account I present is meant partly to fill out this lacuna.

In order to develop an account of objective qualitative data sets, I draw on the current, very lively, philosophical debate on objectivity. This debate has paid little attention to questions of objectivity pertaining to qualitative research and the social sciences more generally. Instead, it has mainly revolved around objectivity in the context of the natural sciences. Moreover, the debate has not been concerned with “objectivity” as predicated of data. This latter point is nicely illustrated by Helen Longino’s discussion of objectivity (Longino 1990). Here, she acknowledges that “[o]ften scientists speak of the objectivity of data” yet she also makes it clear that her analysis does not deal with this topic (ibid. p. 63). In view of this situation, my ideal of an objective qualitative data set does not take off from any existing accounts to this effect. Rather, it takes its inspiration from an account of objective knowledge claims recently put forward by Alison Wylie (Wylie 2004; Chapman and Wylie 2016). To be more specific, my ideal takes over the basic idea informing the notion of objective knowledge claims and develops it into an account of objective qualitative data sets.

The paper is organized as follows. In Sect. 2, I offer an introduction to qualitative methods of data generation. In Sect. 3, I briefly present Wylie’s account of objective knowledge claims. On that basis, I move on, in Sect. 4, to lay out the ideal of objective qualitative data sets. In Sects. 5 and 6, I defend this ideal against two possible lines of objection and then wrap up in Sect. 7.

2 Qualitative methods of data generation

Qualitative research is widespread in the social sciences, humanities, and health sciences. In qualitative research, data are generated by way of methods such as participant observation, qualitative interviewing, focus group interviewing, and the collection of documents. In order to focus the subsequent discussion, I concentrate on what are likely the two most frequently used methods, namely participant observation and qualitative interviewing (Bryman 2012: p. 493). Thus, I have in mind data sets generated by one or both of these methods, when talking about qualitative data sets in the following.

The method of participant observation has two components. The participatory component requires the researcher to take part in the research participants’ ways of life over an extended period of time. She may participate to different degrees. For example, she may participate in the weaker sense of merely hanging around or in the stronger sense of engaging actively in the research participants’ activities. Whatever the extent of her participation, she should try to be as nonintrusive as possible in the ways of life she studies, as her immediate aim is not to alter, but to learn about them. The observational component has it that, while participating, the researcher should observe, in the sense of notice, what goes on.

The researcher’s observations form the basis for her field data, i.e. field notes. These detail when and where the researcher made her observations, the people present in the situation she observed, their physical locations, their nonverbal and verbal behavior, their equipment, and the like. Furthermore, field notes are phrased “in words that are as close as possible to the words used by the participants. Indeed, verbatim quotes should be included to the extent to which the researcher has jotted them down or can accurately remember them. Specific words, special language, terms, and vocabulary should be recorded” (DeWalt and DeWalt 2011: pp. 165–166).Footnote 1

The method of qualitative interviewing requires the researcher to pose questions to a research participant about (some aspect of) her way of life. In her replies, the research participant is permitted, or even encouraged, to exemplify her points, to expand on her answers, to digress, to introduce her own concerns, and the like. Qualitative interviews may be either semi-structured, unstructured, or somewhere in between. In a semi-structured interview, the researcher has a list of questions that she goes through. She may pose them in whatever order seems natural during the interview just as she may add questions. In an unstructured interview, the researcher has at most a list of topics that she wants to cover. She does not introduce these in any pre-fixed order and formulates her questions as she goes along. As a result, the interaction comes close to an ordinary conversation.Footnote 2 In both cases, the researcher should make an effort not to influence what people say on the issues that come up during the interview. Moreover, as far as possible, she should conduct the interviews in the settings in which the research participants live their lives.

The qualitative interview makes possible the generation of interview data, that is, interview notes. These consist in transcriptions of audio or video recordings of interviews, or in descriptions of what was said during interviews as based on the researcher’s observations and scratch notes. In addition, interview notes detail other aspects of the interview situation such as the time of the interview, the setting in which it took place, and the research participant’s facial expressions and nonverbal behavior during the interview.

As should be clear by now, I identify the data generated by way of participant observation and qualitative interviewing with the descriptive claims that make up a researcher’s field and interview notes. Accordingly, it is an account of the objectivity of qualitative data thus understood that I present below. In discussions of qualitative research, this conception of data is common (see, e.g. DeWalt and DeWalt 2011: p. 157ff; Miles et al. 2020: p. 7; Schwartz-Shea and Yanow 2002: p. 460). Yet, there are also other ones. While their adoption would require some rewording of the subsequent discussion, they would not change its substance.

For instance, field and interview data are sometimes identified with what transpired while the researcher carried out participant observation or conducted her qualitative interviews. Accordingly, field and interview notes are seen as descriptions of data. From this perspective, the subsequent discussion may be rephrased as dealing with the objectivity of these descriptions. Other times, field and interview data are taken to comprise not only the researcher’s field and interview notes, but also her audio and video recordings of observed and interview situations. Insofar as these recordings are transcribed and hereby turned into field- or interview notes, the following analysis may be reformulated as being exclusively concerned with the objectivity of the subset of data constituted by the field or interview notes. However, it may also happen that the recordings are used directly as basis for analysis (see Tessier 2012). When this is the case, the ensuing discussion may be re-described as being about the objectivity of the researcher’s understandings or mental representations of the recorded doings, sayings, etc.

In this fashion, nothing hinges on my identification of field and interview data with field and interview notes respectively. That said, it should be emphasized that I do not mean to suggest that data in general, that is, however generated, should be equated with descriptive claims.Footnote 3 All I maintain is that relative to field and interview data, it is common, and for the present purposes most simple, to understand these as the descriptive claims that make up a researcher’s field and interview notes.

3 Wylie’s account of objective knowledge claims

In philosophy of science discussions of objectivity, there has been little concern with the objectivity of data sets, let alone qualitative data sets. In contrast, objectivity as a property of theories, hypotheses, knowledge claims, and the like is a recurrent topic in the debate. In what follows, I briefly present Wylie’s account of objective knowledge claims and propose that its key idea may serve as a basis for the development of an ideal of objective qualitative data sets.

Wylie’s account is best introduced via a brief detour to Kuhn’s work. In his classic paper, “Objectivity, Value Judgement, and Theory Choice,” Kuhn asks what the characteristics of a good scientific theory are (Kuhn 1977: p. 321). In response, he states that these include accuracy, internal consistency, external consistency, broad scope, simplicity, and fruitfulness (ibid. pp. 321–322). Moreover, he proposes to refer to these good-making features as values because they influence, rather than dictate, scientists’ decisions about whether a given theory is better than its competitors (ibid. pp. 330–331).

In the current debate, Kuhn’s reflections have become mainstream. It is widely held that theories have good-making features though there are different views as to what exactly these features are. Likewise, the features are often regarded as values or they are alternatively referred to as virtues, standards, norms, criteria, and the like. In any case, it is now standard to specify that the values, virtues, etc. in question are epistemic ones: they are good-making features in the sense of contributing to the attainment of truth or knowledge (see, e.g., Elliott 2017: p. 12; McMulllin 1983: p. 18; Steel 2015: p. 161).

Wylie’s ideal of objective knowledge claims builds upon this Kuhnian tradition. She contends that knowledge claims, which encompass theories, hypotheses, and similar research products, are objective to the extent that they realize a combination of epistemic virtues. Here is how she puts it: “[a]s a property of knowledge claims, objectivity seems to designate a loosely defined family of epistemic virtues that we expect will be maximized, in some combination, by the claims we authorize as knowledge” (Wylie 2004: p. 345). According to Wylie’s list, the epistemic virtues include empirical adequacy, internal coherence, inferential robustness, external consistency, and explanatory power. In combination and on their own, these virtues are, to repeat, constitutive of the objectivity of knowledge claims; they are objectivity-making characteristics.

The basic idea informing Wylie’s account, viz. that a combination of good-making features are constitutive of the objectivity of knowledge claims, may be extended and adapted to qualitative data sets. To motivate this proposal, note that it makes perfect sense to ask, similarly to Kuhn, what the characteristics of a good qualitative data set are. Moreover, a reasonable response to this query is to offer a list of good-making features in the sense of features that somehow contribute to the attainment of truth or knowledge. The same point may also be reached via a different route. It may be registered with Dan Steel that not only theories, but also methods, social practices, community structures, and the like, may manifest epistemic values (Steel 2015: p. 163). This granted there is no obvious reason not to add data sets to the list of items that may manifest epistemic values (see Zahle 2019). In any case, it is only a small step to continue by maintaining that a qualitative data set is objective to the extent that it realizes a combination of epistemic values, virtues, etc. The good-making features of a qualitative data set are, in combination and on their own, constitutive of the objectivity of a qualitative data set; they are objectivity-making characteristics and all there is to a qualitative data set being objective is that it has a combination of such features. As it stands, this ideal is very crude. In the remainder of the paper, I develop and defend it.

4 The ideal of an objective qualitative data set

In this section, I start by saying something more about what to understand by a good qualitative data set and by a good-making feature of a qualitative data set. Next, I present in some detail a list of good- or objectivity-making features of a qualitative data set. On that basis, I present a more precise formulation of the ideal of an objective qualitative data set.

A good qualitative data set, I shall take it, is one that is fit to serve as evidence base for a satisfying answer to the research question under study. By implication, a good-making feature of a qualitative data set is one which contributes to making the data set suited as evidence base for a satisfying answer to the question under study.

In this formulation, “the research question under study” is in need of some unpacking. In qualitative research, the research question is frequently changed during data generation. Often it is rendered more precise as the generation of data progresses, but it may also be modified in other ways or even discarded in favor of a different research question. Obviously, the most important is that a qualitative data set is suited to serve as evidence base for a satisfying answer to the final research question, that is, the research question that the study ends up aiming to answer. This being the case, the research question under study should be taken to refer to the final research question.

The notion of a satisfying answer is in need of clarification too. An answer to a qualitative research question usually takes the form of a research report or account describing the findings of a study. For the present purposes, a satisfying answer may be identified with an answer that is not only true, i.e. it contains no false or incorrect claims, but also complete, i.e. unbiased or nonpartial (see Anderson 1995: p. 39). No doubt, there is more to a satisfying answer as also testified by the lively discussions of this issue in the qualitative research literature.Footnote 4 Still, from the perspective of the objectivity-making features I discuss below, there is no need to work with a more elaborate understanding of a satisfying answer: the features are ones that a data set must have in order for it to be possible to arrive at a true and complete answer on its basis. Or differently put, the features contribute to the attainment of a true and complete answer to the research question under study—an answer that qualifies as knowledge.

What are then the objectivity-making features of a qualitative data set that renders it suited as evidence base for a satisfying answer to the research question under study? In the following, I outline six such features, namely descriptive adequacy, reactivity transparency, deception transparency, relevance, balance, and sufficiency. This list is not meant to be exhaustive: there might be further objectivity-making features that I have not yet come across and I return to this issue in Sect. 5. For now, it may be registered that the list displays what may be regarded as key objectivity-making features. In discussions of qualitative research, these features are often invoked, implicitly or explicitly, as characteristics of a good qualitative data set. However, their treatment is scattered and mostly cursory. My examination provides a more systematic and detailed account of them. To anticipate, their scrutiny will bring out that some of them are only possessed by a data set in conjunction with true assumptions and that some of them are only had by a qualitative data set relative to a given research question. I now consider each feature in turn.Footnote 5

A first objectivity-making feature of a qualitative data set is that it contains descriptively adequate data only, that is, data that accurately describe what transpired when the researcher carried out participant observation or conducted her interviews. As noted above, both field and interview notes should do so in words that approximate as much as possible those the research participants would use or did in fact use. An example of descriptively adequate field data are ones that correctly report that the newlyweds cut the wedding cake together. Similarly, interview notes are descriptively adequate if they, say, correctly cite an interviewee for saying that she is against Christian wedding rituals. Evidently, if a researcher mis-describes what she observed or people told her, a satisfying answer to her research question may not be built upon these notes.

Whether field or interview data are descriptively adequate is sometimes relative to a research question. To see this, consider an audio-recorded interview with a research participant who speaks in a dialect. For the purposes of answering some research questions, it does not matter whether the researcher transcribes the interview using standard spelling or spelling that conveys the dialect. In other situations where the focus of the study is, say, local identity, it may be important that the dialect is written out (see Bailey 2008: p. 129). To the extent that this is the case, it is reasonable to maintain, the transcript is only descriptively adequate if it conveys the dialect.

A second objectivity-making feature of a qualitative data set is that its data are reactivity transparent. In qualitative research, reactivity refers to the researcher’s influence on what the research participants do and say during data generation as exemplified by their diverging from their routines when the researcher is around or by their telling the researcher what they think she wants to hear. Various features ascribed to a researcher may have this effect including her role as researcher, her gender, her age, marital status, ethnicity, or social role(s) in the community under study.

The traditional view is that in order for data to be indicative of the ways individuals act, talk, feel, think, etc. when they are not under study, the data must describe situations without any reactivity. In contrast, the currently dominant position contends that data about situations with reactivity may be employed too. The data are informative of social life independently of its being studied as long the researcher is clear on the nature of the reactivity so that she may take it into account when drawing inferences from her data (Hammersley and Atkinson 2007: p. 102, Maxwell 2005: p. 109).

This latter idea may be further elaborated by stating that data should be reactivity transparent: the reactivity, if any, in the various situations described by field and interview data should be clear. To this end, data should be supplemented by true reactivity assumptions. These assumptions concern whether or not the researcher was ascribed some feature that might plausibly cause reactivity and, if so, whether and how this feature shaped the research participants’ sayings and doings. In this fashion, field and interview data are not, to repeat, reactivity transparent on their own, but only in conjunction with true assumptions of the sort just mentioned. Evidently, if the researcher wrongly thinks that she did—or didn’t—influence the research participants in some situation, she is not in position correctly to interpret her data about it.

Tea Bengtsson’s study of young offenders in secure care institutions may serve to exemplify the feature of reactivity transparency (Bengtsson 2014). Bengtsson conducted interviews with the young offenders and the data she generated were reactivity transparent in combination with two assumptions: that the young boys regarded her “as an adult and a representative of ‘society’ and mainstream values and morals” (ibid.: p. 739) and that this shaped the way they chose to represent their past, their aspirations for the future, and so on, in the interviews. In light of these assumptions, Bengtsson saw her data as being informative about the boys’ ideas about the right way to represent their history and ideas about the future to an adult representative of mainstream society.

A third objectivity-making feature of a qualitative data set is that its data are deception transparent. Deception occurs when research participants mis-describe a situation, omit information, change their behavior, and the like, in order to delude the researcher. While data that describe non-deceiving research participants are more straightforward to interpret, data about situations with deception are also telling as long as the researcher is aware of deception. In other words, what matters is that any deception is clear to the researcher. Accordingly, data should be supplemented with true deception assumptions about whether or not the research participants deceived the researcher and if they did, how and possibly why so. If the researcher wrongly takes it that the research participants did, or didn’t, deceive her on some occasion, she will draw wrong conclusions from her data about this situation.

The idea of deception transparency may be illustrated by a study of massage parlors by Jack Douglas and Paul Rasmussen (Douglas 1976: p. 143ff). Rasmussen’s role was, among other things, to hang out in a massage parlor and talk to the women working there. During the first months of the study, the women consistently denied that sex, too, could be bought at the parlor. Rasmussen’s data to this effect were deception transparent in conjunction with the assumptions that they were lying to him and this because they had placed him in the category of potential and actual boyfriends (and so ones to be shielded from the facts) rather than in the category of potential customers. Rasmussen learned this well into his fieldwork when one of the women accidentally let slip a comment about sex, which he was able to follow up on.

A fourth objectivity-making feature of a qualitative data set is that its data are relevant from the perspective of providing a satisfying answer to the research question under study. Typically, data only have this feature in conjunction with true assumptions that link whatever is described by the data to the focus of the research: it is in virtue of these linking assumptions that the doings, sayings, etc. described by the data may be seen as relevant to providing an answer to the research question. Without relevant data, it goes without saying, it is impossible to provide a satisfying answer to a research question.

By way of example, consider Charlotte Davies’ research on the transition to adulthood by young people with learning disabilities (Davies 2007). Davies generated a lot of field and interview data about the young people’s food preferences, what they usually had to eat, and when they didn’t themselves decide what to eat. At first sight, it may perhaps seem puzzling that these data should be relevant to her study. Yet, the data linked up to the issue of transitioning to adulthood via the assumptions that the young people and their surroundings associated different food preferences with children and adults, and regarded the social status of adulthood as going together with more control over what one eats.

A fifth objectivity-making feature of a qualitative data set is that it is balanced. This means that the data set as a whole describe all the relevant (types of) people, activities, settings, viewpoints, events, etc. necessary to provide a satisfying answer to the research question. In the absence of a balanced data set, the researcher is unable to offer a satisfying answer to her research question in the sense that her answer will fail to be complete.

These points may be exemplified by Greg Guest, Emily Namey, and Marilyn Mittchell’s discussion of one of the authors’ study of contraceptive decision making in India (Guest et al. 2013: p. 45). At the beginning of this study, the plan was to conduct qualitative interviews with married men, married women, as well as healthcare providers. However, during the first interviews, it became clear that both mothers and mothers-in-law have a say on their children’s choice of contraception. Consequently, interviews with mothers and mothers-in-laws were set up too. In case the researcher had failed to interview them, the data set would have lacked their perspective on contraceptive decision making and their role in these decisions. In short, the data set would have lacked balance.

A sixth objectivity-making feature of a qualitative data set is that it is sufficient size-wise: it contains enough relevant (types of) data describing people, activities, settings, events, etc. to provide a satisfying answer to the research question under study. For instance, a researcher may need to observe a certain type of activity many times, before her field notes describe the activity in sufficient detail or allow her to infer the routine elements in the activity. A more concrete example may be provided by elaborating on the study of contraceptive decision making mentioned a second ago. Assume that the researcher had to conduct several long interviews with most of the research participants in order to exhaust what they had to say on the topic. In this situation, a single brief interview with each research participant is unlikely to have resulted in a sufficiently large data set to provide a satisfying answer to the research question.

In light of these considerations, a more elaborate formulation of the ideal of an objective qualitative data set may now be set forth: a qualitative data set is objective to the extent that it, in conjunction with true assumptions, has a combination of features including descriptive adequacy, reactivity transparency, deception transparency, relevance, balance, and sufficiency in virtue of which the data set is suited to serve as evidence base for a satisfying answer to the research question under study.

It is worth noticing that, typically, this ideal only pertains to a subset of the data that a qualitative researcher generates during a study. That is, a qualitative researcher standardly generates more data than ends up forming part of the evidence base for a satisfying answer to her final research question. For example, this is sometimes the case because the qualitative researcher changes her research question during data generation. As a result, some of the data generated prior to the modification of her research question may be irrelevant from the perspective of answering the research question she has now settled on. Similarly, the researcher may realize that data she thought would be relevant from the perspective of answering her research question are not so after all. In both these cases, then, the data that lack relevance fall short of this objectivity-making feature and hence they should not be regarded as part of the subset of data that serves as evidence base for a satisfying answer to the research question under study.

The ideal of an objective qualitative data set is recommendable on two grounds. By providing a list of the features that a qualitative data set should ideally end up having, it may serve as a useful guiding ideal in the generation and evaluation of qualitative data sets. Moreover, it represents a reachable ideal: qualitative researchers may produce qualitative data sets with the key objectivity-making features and do so when at their best. This is also the view informing their discussion in the qualitative research literature. For lack of better term, I shall henceforth refer to the ideal as the reconstructed notion of an objective qualitative data set.

5 First line of objections: replace the listed objectivity-making features

One obvious way in which to oppose the reconstructed ideal is by arguing that while it rightly maintains that the objectivity of a qualitative data set is a matter of its possession of some feature(s), it goes wrong with respect to the features it picks out. All, or at least some, of these features should be replaced by different ones. I now consider—and reject—various ways in which to push this line of objection.

In discussions of qualitative research, the traditional view of objectivity is sometimes identified with the position that social research is objective to the extent that it has the features of validity and reliability (Dewalt and Dewalt 2011: p. 112; Kirk and Miller 1986: p. 19). Applied to qualitative data sets, this translates into the claim that a data set is objective to the extent that its data are valid, i.e. they accurately describe what transpired during data collection, and reliable, i.e. they are reproducible. The latter means that either the same or a different researcher could generate similar qualitative data within the same period of time (say, within the same day or week) or at a later point in time (DeWalt and Dewalt 2011: p. 112ff). It might be argued that this traditional conception of objective qualitative data sets is preferable to the reconstructed notion.

In response, note first that the feature of validity is equivalent to that of descriptive adequacy. The reason I prefer to talk about descriptive adequacy is that, differently from validity, it is not a term with multiple and contested meanings. This observation, though, does not change much: the adoption of the traditional conception would still mean that the other features, listed by the reconstructed notion, should be dismissed as objectivity-making characteristics. Yet, by the lights of the traditional conception, validity/descriptive adequacy and reliability are desirable features of qualitative data sets. As shown in the last section, so are the features of reactivity transparency, deception transparency, relevance, balance, and sufficiency. This being the case, why not admit the latter features as objectivity-making features too? Why insist that validity/descriptive adequacy and reliability alone are objectivity-making features? In the absence of an answer to these questions, the traditional conception must be found wanting.

This line of reasoning does not dispute the supposition, made by the traditional conception, that reliability is an objectivity-making feature. Accordingly, it may be argued that the reconstructed ideal should include reliability on its list of objectivity-making features. Though the addition would not constitute an objection to the ideal—the list makes no claim to being exhaustive—it would still be an important finding. Hence, it is worth briefly to pause and examine this issue.

Following the traditional conception, reliability is a desirable feature of qualitative data sets. However, a data set being reproducible is not desirable in and of itself, but only insofar as this is suggestive of the qualitative data set being in some respect(s) fit to serve as evidence base for a satisfying answer to the research question under study. This means that to claim that a particular qualitative data set is reproducible is tantamount to saying that the data set likely has one or several of the key objectivity-making features listed by the reconstructed ideal. Consequently, it is redundant to add reliability to the reconstructed ideal: what makes reliability a desirable feature is already covered by the list of key objectivity-making features.

This clarified, consider another notion of objectivity that is sometimes mentioned in the context of qualitative research. It states that a qualitative data set is objective to the extent that it contains true aperspectival descriptions, that is, true descriptions which do not reflect any interpretation of (perspective on) what transpired during data collection (Miles et al. 2020: p. 7; Kirk and Miller 1986: p. 14).Footnote 6 It might also be maintained that this ideal is preferable to the reconstructed notion.

In discussions of qualitative research, it is widely held that the aperspectival ideal, as it may be called, is unreachable. For instance, in the passage quoted in the introduction, Miles, Huberman, and Saldana contend that, relative to this notion of objectivity, data “can never be truly ‘objective’; they can only be our interpretations of what we experience” (Miles et al. 2020: p. 7—italics in original). Moreover, it is not clear that it even makes sense to try to approach the ideal. Qualitative data describe research participants’ doings, saying, etc. in words approximating as much as possible those the research participants use. It may be wondered what more or less aperspectival field and interview notes look like and, relatedly, why it is recommendable to strive for notes that are as aperspectival as possible. Until these queries are answered satisfyingly, the aperspectival ideal should be dismissed too.

So far I have discussed how the reconstructed notion may be challenged by pointing to specific alternative objectivity-making features. Another option is to question the notion by maintaining that its features should be replaced by a different type of features. An argument of this sort might draw its inspiration from the debate on the proper criteria of evaluation in regard to qualitative research reports (answers to qualitative research questions). Most notably, the dispute turns on whether these criteria should be the same as, or different from, those applied to quantitative research results. In his 2014, Justin Lee argues against the employment of identical criteria on the ground that:

(s]uch criteria are not technically inappropriate, just not discriminating enough. This is akin to recommending that soccer players be evaluated by the criterion of ‘fitness,’ which is no doubt valid. However, it is a criterion that is too encompassing since physical fitness is relevant for assessing almost all types of athletes. Therefore, it cannot tell us what makes soccer players good as soccer players (Lee 2014: p. 318).

Lee’s line of reasoning is easily extended to qualitative data sets. It might be asserted that the key objectivity-making features are not distinct to qualitative research. Intuitively, data generated by way of quantitative methods, like structured interviewing or structured observation, should also be descriptively adequate, reactivity transparent, and so on. As a result, the key objectivity-making features are not discriminating enough: just as the feature of fitness does not pick out what makes soccer players good as soccer players, the features fail to capture what makes qualitative data sets good as qualitative data sets. Hence, the key objectivity-making features should be substituted by ones that are distinct to qualitative data sets.

The main problem with this argument is that it assumes that an ideal of objective qualitative data sets should list features that set apart good qualitative data sets from good quantitative ones. Yet, that is not the purpose of the reconstructed ideal: it is rather to single out (some of) the features in virtue of which a qualitative data set is suited to serve as evidence base for a satisfying answer to the research under study. From this perspective, all that matters is the extent to which the ideal succeeds in pointing to such features. It is simply irrelevant whether the features are distinct to qualitative data sets and this issue may be left open.

These reflections conclude the examination of what I consider to be the most significant and likely objections to the objectivity-making features mentioned by the reconstructed ideal. I have demonstrated that the arguments fail to establish that all, or at least some, of the features should be replaced by alternative ones. It is important to keep in mind that this point does not add up to the claim that the ideal is complete as it stands. As noted several times already, the list of objectivity-making features does not pretend to be exhaustive. The reconstructed ideal points to features that are objectivity-making in all contexts of qualitative data generation, i.e. relative to all qualitative data sets. I do not want to rule out the possibility that further features of this sort should be included into the ideal (though at this point I cannot think of any more to add). Also, there might be features, which are only objectivity-making in some contexts of qualitative data generation. For example, maybe some features are objectivity-making relative to the adoption of a phenomenological approach, whereas others are objectivity-making relative to the espousal of an interpretive approach, and so on. The reconstructed ideal is compatible with the addition of such features too. Future research will have to determine whether the reconstructed ideal of qualitative data sets should be expanded in these ways.

6 Second line of objections: focus on process instead

The reconstructed ideal and the objections discussed in the previous section have in common that they see one or several features of a qualitative data set as being constitutive of its objectivity. I shall put this by saying that they agree that the ideal of an objective qualitative data set should be feature-focused. Another option is to hold that the ideal should be process-oriented: the fact that a qualitative data set was generated or evaluated in a certain way should be regarded as constitutive of its objectivity. Longino gives voice to this view in relation to data in general, when she comments that scientists who talk about the objectivity of data “seem to mean that the information upon which their theories and hypotheses rest has been obtained in such a way as to justify their reliance upon it” (Longino 1990: p. 63). In this section, I rebut the claim that process-oriented ideals of objective qualitative data sets are preferable to the reconstructed ideal. Moreover, I propose that process-oriented ideals should be reformulated such that they are rendered compatible with the reconstructed notion.

It is possible to distinguish between two kinds of process-oriented ideals. The first contends that qualitative data sets are objective to the extent that individual qualitative researchers generated or evaluated them in a certain way. The notion of objectivity as value freedom—sometimes mentioned in the qualitative research literature—may serve to illustrate this approach (see, e.g., Bryman 2012: p. 39, Lincoln and Guba 1985: p. 300). Adapted to the present discussion, it states that a qualitative data set is objective to the extent that the researcher’s nonepistemic values (such as her moral, social, and political values) did not influence any of her data generation decisions. Another notion—likewise mentioned in discussions of qualitative research—is the notion of objectivity as emotional detachment (see, e.g. Paul 1953: p. 441, Vidich 1969: p. 82).Footnote 7 Adjusted to the present concerns, it maintains that a qualitative data set is objective to the extent that the researcher was emotionally detached during its generation.

Both these ideals are, to repeat, just examples of process-oriented notions of objective qualitative data sets. As such, there is no need to go into the question of whether qualitative researchers are well advised to proceed in the manners they recommend. What matters is that they bring into view the appeal of ideals of this sort: these ideals provide guidance as to what qualitative researchers should do; they tell qualitative researchers how they should proceed. By comparison, the reconstructed ideal is completely silent on this matter: it merely lists the features that a qualitative data set should ideally end up having. This being the case, it might be argued, a process-oriented ideal is more useful than—and hence preferable to—the reconstructed notion.

In response, consider that a process-oriented ideal of this kind may always be rendered compatible with the reconstructed ideal. All it takes is to maintain that the fact that individual qualitative researchers generated or evaluated their data in certain ways is constitutive of the objectivity of qualitative research practices rather than qualitative data sets. Thus, assume that qualitative researchers were equipped with both the reconstructed ideal and an ideal of objective qualitative research practices. In that case, qualitative researchers would know not only what features their data set should ideally end up having but also how they should, ideally, go about the generation or evaluation of their data set. These reflections show that the reconstructed ideal should not be replaced by a process-oriented ideal of objective qualitative data sets. It would be much more useful to supplement the reconstructed notion with an ideal of objective qualitative research practices.Footnote 8

From the perspective of deciding on, or working out, the latter ideal, it may be noted, it is helpful to be able to turn to the reconstructed ideal. It is reasonable to think that an ideal of objective qualitative research practices should state how qualitative researchers should generate or evaluate their data sets such that they likely end up with an objective qualitative data set. Consequently, it is also useful to be in possession of an ideal to this effect, viz. the reconstructed notion. It makes it possible to determine more precisely what constitutes effective research practices to achieve this goal and to put forward a more systematic and comprehensive ideal of objective qualitative research practices.

The second kind of process-oriented ideals does not focus on individual ways of proceeding but on practices at the level of the scientific community. Ideals of this sort may naturally draw on Longino’s work (Longino 1990). In her view, a scientific community should have recognized avenues for criticism and shared standards of theory assessment. Further, there should be uptake of criticism and equal sharing of intellectual authority (ibid. p. 76). Thus organized, a scientific community should engage in the critical scrutiny of theories with the aim of detecting and rejecting theories that reflect individual researchers’ idiosyncratic values and biases. This account may easily be modified so that it deals with qualitative data sets rather than theories. Accordingly, it states that evidence—qualitative data sets—invoked in support of qualitative research reports should be critically scrutinized by a research community with the aforementioned features: this will likely result in the detection and rejection of data sets that reflect idiosyncratic values and biases. Further, it might be elaborated, a qualitative data set is objective to the extent that it has gone through this process of critical scrutiny without having been dismissed.Footnote 9

A process-oriented ideal of this kind might also be recommended on the ground of its usefulness: it offers guidance as to how a properly organized scientific community should proceed in order to weed out qualitative data sets that reflect individual qualitative researchers’ idiosyncratic values and biases. Because the reconstructed notion offers no similar advice, it should be rejected in favor of the community-level ideal of objective qualitative data sets. Or so it may be argued.

Similarly to above, the appropriate response here is to maintain that the process-oriented ideal should be slightly amended. It should maintain that the critical scrutiny of qualitative data sets by a properly structured scientific community is constitutive of the objectivity of scientific community practices rather than of objective qualitative data sets. In this case, the reconstructed ideal of an objective qualitative data set might be complemented not only by an account of objective individual qualitative research practices, but also by an ideal of objective communal practices vis-à-vis qualitative data sets. Clearly, a tripartite account of this sort is more useful—it offers better guidance—than the community-level ideal of objective qualitative data sets on its own. Therefore, the reconstructed ideal should also be supplemented by an ideal of objective scientific community practices.

The reconstructed ideal of an objective qualitative data set also comes in handy from the perspective of further cashing out such an ideal that bases itself on Longino’s account. In her characterization of an ideally organized scientific community, Longino specifies that the community should have shared standards of theory assessment á la Kuhn’s good-making features of theories (Longino 1990: p. 77). When it comes to an ideally structured scientific community that critically examines qualitative data sets, its shared standards may likewise be explicated as standards of evaluation for qualitative data sets. Since the reconstructed ideal provides a list of (some of) the good-making features of qualitative data sets, it presents itself as a ready account of what (some of) the shared standards of evaluation should be for qualitative data sets. In this fashion, it may be employed to spell out one important aspect of the ideal of objective scientific community practices.

To recapitulate, the reconstructed ideal should not be replaced by a process-oriented ideal of objective qualitative data sets. Rather, it should be complemented by ideals of objective qualitative research practices and objective community practices vis-à-vis qualitative data sets. From the perspective of working out both these ideals, it is going to be helpful to be able to draw on the notion of objective qualitative data sets. Taking on this task, though, is beyond the scope of the present paper.

7 Conclusion

In the first part of this paper, I presented a reconstructed ideal of objective qualitative data sets. According to it, a qualitative data set is objective to the extent that it, in conjunction with true assumptions, has a combination of features including descriptive adequacy, reactivity transparency, deception transparency, relevance, balance, and sufficiency in virtue of which the data set is suited to serve as evidence base for a satisfying answer to the research question under study. I recommended this ideal on the ground that it may serve as a useful and reachable guiding ideal in qualitative data generation.

In the second part, I defended the reconstructed ideal against two lines of objections. In this connection, I stressed that the ideal does not claim to offer an exhaustive list of objectivity-making features: it may be that further features should be added to the list. Moreover, I observed that the ideal may usefully be supplemented by ideals of objective qualitative research practices and objective community practices vis-à-vis qualitative data sets. This being the case, the reconstructed ideal of objective qualitative data sets should be seen as laying the foundation for further work on objectivity in relation to qualitative data generation.