Keywords

Introduction

After gathering and analysing the empiricaldata from your study of values or values work, how can you ensure the trustworthiness of your study? Trustworthiness is important for you as a researcher, for the informants who have contributed to your study and for the reader. The technical terms often used to describe this are validity, reliability and generalisability (Denzin & Lincoln, 2018). In qualitative research, where the boundaries between the researcher and the researched are unclear, Denzin and Lincoln (2018; see also Krefting, 1991) recommend using credibility, dependability and transferability as equivalent terms. Different measures, like extended periods of fieldwork and triangulation of methods and sources of data, can be used to strengthen the credibility of a study. Participant validation, or member checking (the terms are here used interchangeably), is another strategy to strengthen the credibility of data and results (Lincoln & Guba, 1985; Merriam & Tisdell, 2015). Participant validation implies that you as a researcher in one way or another present the data material or the preliminary analysis to the informants to validate and assess interpretations. The purpose is to ensure the trustworthiness of your study from the perspective of the researcher, the informant and the reader (Carlson, 2010). With participant validation you are transparent about how your informants are represented, and it allows you to correct misunderstandings and document the research process.

This chapter describes how participant validation can be incorporated in the research design of valueswork studies. It is a strategy to address ethical concerns in a study, for example, related to transparency and power, but it also raises new ethical concerns. To decide how to incorporate participant validation in your study, it is useful to explore and develop a broad understanding of the ethical dilemmas involved. This chapter thus addresses the following questions: how can participant validation be incorporated into a study of values or values work, and how does participant validation respond to and generate ethical concerns? The chapter first reviews existing literature on participant validation and then uses a case study of cultural diversity and inclusion as an example of how participant validation can be incorporated into the research process. For researchers studying valueswork, the example demonstrates how participant validation may be an opportunity for valueswork in and of itself, generating valuable data that can be incorporated into a study.

Former Studies on Participant Validation

The most referenced text on participant validation, or member checking, is Lincoln and Guba’s (1985) book on naturalistic inquiry. Naturalistic inquiry is the study of a social phenomenon or people’s actions in their specific context or natural environment. In this type of research, the boundaries between you as a researcher and the subjects being researched are fuzzy (See Chap. 12). The ontological and epistemological foundation of naturalistic inquiry is that the realities you study are socially constructed. In the research process, the researcher and the researched interact and cocreate understandings and interpretations. Participant validation is one strategy for cocreation in research, and Lincoln and Guba (1985) suggest how it can be incorporated at different stages of the research process. Most studies claiming to have used participant validation refer to sharing interviewtranscripts or quotations with the informants. While that may be a way to correct misunderstandings and errors, it does not involve informants in the analysis of the data, and it does not reap the full benefits of member checking as an approach. Other researchers have demonstrated how participant validation can be incorporated in the research design and have applied it in a more extensive way. Three studies are presented here: Buchbinder’s (2011) review of experiences with validation interviews, Birt et al.’s (2016) elaboration of a synthesised member checking method and Slettebø’s (2020) use of participant validation in an action research project. The three studies highlight different aspects of the use of participant validation and illustrate different ways of applying it in studies of values or values work.

The first study analyses experiences with validation interviews. Buchbinder (2011) interviewed social work students who had used individual validation interviews in their study of more experienced social workers. The students first interviewed the social workers, transcribed the interviews and identified core themes. In the validation interview, the preliminary analysis was presented to the social worker, offering him or her an opportunity to confirm, modify or reject the analysis. Buchbinder’s study surfaced various ethical concerns: the legitimacy of offering interpretations going beyond the interviewees’ own understanding of their narratives, handling relationships and roles and the use and abuse of power in the validation process. The validation interviews challenged the students’ handling of the boundaries between interviewer and interviewee. As social work students, they were younger and less experienced than the social workers they interviewed. The interviews generated feelings of uneasiness when the students presented their interpretations of what had been said in the first interview. The feelings of uneasiness varied with how close or distant the interviewer and interviewee were prior to the interview. During the research process, the students experienced several shifts of power. In the initial interview, the experienced social workers had more power in determining what was said but were simultaneously vulnerable when sharing personal information. In the validation interview, the students assumed a more powerful position, offering interpretations of the first interview. At the same time, they felt vulnerable as their interpretations were being assessed by a senior person. In summary, Buchbinder presents validation interviews as one way of incorporating participant validation into a study. Buchbinder demonstrates how validation interviews address the ethical concerns of interpretations and power differences by offering informants an opportunity to correct the researcher’s interpretation. On the other hand, the validation interviews generated new ethical concerns related to roles, boundaries and power.

The second study offers an example and a model for how participant validation can be incorporated in studies with larger samples of informants, using written communication between the researcher and the informants instead of face-to-face validation interviews. Birt et al. (2016) developed a five-step ‘synthesized member checking’ (SMC) process and tested it out in a health research study. The first step of the model is to prepare a synthesised summary of emerging themes from the total sample of interviews using illustrative, anonymised quotes from the different interviews. In the second step, the informants’ eligibility for participating in the member checking process is considered to ensure that the research process will not inflict unnecessary harm on the informants. In the third step, the synthesised report is sent to the selected informants with an invitation to make corrections and add comments. The responses are collected and added to the data material in the fourth step. Finally, the new data are integrated and coded. In addition to developing a model for member checking, Birt et al.’s study addresses two central ethical concerns. First, by offering the informants an analysis of the total sample, the information from the interview is placed in a broader context, which gives the informants a better understanding of how their responses have been interpreted in relation to others. This relates to the ethical responsibility of ensuring that informants understand how the information they have provided is used. Even if the informants have received information about the purpose of the study before the interview, this form of member checking enhances a more comprehensive understanding of the research process. Second, an ethical concern in social research is that the study should be as little harmful to the researched subjects as possible. The second step in the SMC model addresses the ethical issue of the harmful effects of the research on the informants. For research on sensitive issues, participant validation may represent an additional burden and harm to the informants and thus generate an ethical concern. By evaluating whom to include in the member checking process, the possible negative effect is reduced. The fourth and fifth steps of the model illustrate how participant validation is used to generate new data for the study.

The third study highlights the empowering effect of participant validation and demonstrates how the process may modify and generate new and relevant data. The study presents an action research project involving parents who had involuntarily had a child placed in care (Slettebø, 2020). Throughout the research project, the parents participated in focus groups with the aim of developing new types of services for parents in their situation. At the end of the project, a preliminary report was elaborated and shared with the parents. This use of participant validation was aligned with the empowering purpose of the action research project. About a quarter of the participants received a 70-page hard copy version of the report, and after three weeks, comments from the parents were collected through telephone interviews. The comments were incorporated into the text and analysed as additional data. Participant validation contributed to the final report by complementing the researcher’s first draft, adjusting the analysis, and refining the use of theoretical concepts. Beyond generating additional data for the study, the process encouraged revisions of the use of concepts and methods for future studies. In this study, participant validation helped maintaining the proactive role of the parents throughout the process—a central ethical concern in action research. Slettebø discusses how the academic jargon of the research report represented a barrier as well as an empowering conceptual tool for the parents to handle their experiences, thus demonstrating how participant validation in this study both addressed and generated new ethical concerns.

In the three studies reviewed here, informants were not only invited to review the transcripts of the interviews they gave but were provided with an opportunity to respond the researchers’ interpretations of the data material at different stages of the process. In Buchbinder’s (2011) study, informants were presented with thematic analyses of their own interviews, whereas Birt et al. offered participants a synthesised preliminary analysis of the whole sample. In Slettebø’s (2020) study, the participants received copies of a preliminary report on the whole research project. The three studies illustrate how comments from participants may be collected through face-to-face interviews, in writing, or through telephone interviews. Inviting the informants to respond to and engage with the researcher’s interpretation of the data material disrupts the inherent power relations of the research process, but it also generates additional ethical concerns.

Participant Validation in a Study of Cultural Diversity and Inclusion

A case study of cultural diversity and inclusion in three nursing homes will here be used as an example of how participant validation can be incorporated at different stages of the research process. The case study combined different methods and sources of information to generate empirical data. In the nursing home units, I observed the interaction between employees and residents and participated in their different meetings and activities. Six unit managers were shadowed for a full shift each. During the shadowing, the unit managers’ activities were recorded in a format indicating how much time was spent on the activity, the location, the participants and who initiated the activity (see Chap. 8). After observation and shadowing, 27 interviews with managers and employees were conducted.

In the following, three different uses of participant validation are described. The examples illustrate how to incorporate participant validation in a study and how this strategy both addresses and raises ethical concerns. In addition, the examples demonstrate how participant validation provides opportunities for valueswork when the informants assess their own work and the management of their units.

Validation of Shadowing Reports

The unit managers received transcripts of the shadowing report before the interview, and in the interview, they validated my understanding of their working day. The unit managers could then correct mistakes in the shadowing report and comment on how representative this day was in comparison to other working days. In the interview, they further explained and interpreted what happened during the day I shadowed them. In general, the managers found it interesting to get this report of their day. Some of them had felt it awkward to be shadowed, and they were uncertain and curious about what information had been recorded about them. When they read the shadowing report, I sensed a sigh of relief, and one of them expressed that it was not as bad as she had thought it would be. Sharing the shadowing report with the unit managers thus responded to an ethical concern for transparency with informants in the research process.

Validating the shadowing report was an opportunity for the unit mangers to assess their own role and work. The following two quotes demonstrate the unit managers’ responses to the report:

It was very exciting to read. I was really happy when I read it, so shared it with my partner at home and said: “See! I have never written down what I have done at work, but now you can see what I do when I go to work!” (laughing). But I am encouraged by what I see. From this I see that I am not sitting so much by the computer to cover shifts, and that is good, because that is what I prioritise the least. (…) I spend more time on my employees, in conversations, listening to what they want, what we can change, having time for employees and procedures in the unit. (Dragan, unit manager)

First and foremost, I thought about how much and how varied [my day was], and how much could actually have been done without me—I think. I thought that right away. I am going to share this with Hege [the CEO]. It is a supervising tool for us. (…) [My] lack of structure is quite evident in the report. (Jonathan, unit manager)

These two quotes highlight the unit managers’ priorities at work and what they consider to be important. Dragan was proud of how the shadowing report confirmed his priorities, showing that he spent more time engaging with employees than doing administrative tasks. Jonathan was less satisfied. The report showed that he spent time on things he should not have done, and he suggested discussing the report with his supervisor. As such, participant validation generated reflections on priorities and subsequent initiatives to make changes. The unit managers’ evaluations and adjustments represent valueswork as a result of the validation process.

Validation of Observation in Interviews

Participant validation was also applied in the other interviews with employees in the units. The interviews took place after observation in the units, so incidents from these days were presented and discussed in the interviews, giving the informants an opportunity to offer their points of view or to explain further what had happened. As such, validation in the interviews adjusted my interpretation of the observational data.

Validation of observations that involved other informants generated new understanding and dilemmas. In one of the units, I had followed the unit manager closely and was in many ways impressed with what I saw. When I interviewed one of the employees about the unit manager’s leadership, more critical observations surfaced:

She is a bit direct. And it is not everybody who likes that. You feel that you are treated very hard sometimes. Nobody likes to be treated badly. Everybody does their best, and still, they get “pepper”. (…) And then we have heard she is the best to save money. So, it means that she doesn’t spend money on calling in substitutes. (Zahra, nurse)

At first, these comments were surprising, but in the following interviews with other employees in the unit, Zahra’s comments were confirmed. When employees talked about the unit manager’s leadership in the interview, they also engaged in reflections around the issue. Milan, another nurse in the unit expressed it this way:

She can be experienced as strict, and maybe unfair. But I think she is a good leader. I know that in our unit there is a general discontent with her. And I understand that the others can get upset or feel that she is condescending in the way she talks to them. (…) If it had been a male manager who had behaved the same way, there would have been fewer employees reacting. Because if a man is very direct and strict and so on, he’s ambitious, he wants things done. If it is a woman, then, well, well, she’s a bitch, she’s strict, you know. That’s how people think.

When the unit manager was interviewed at the end, the questions were revised based on the information from the employees. The unit manager then shared about the ongoing conflict in the unit and how she was handling the situation (Lindheim, 2020). In this example, participant validation elicited discussions of central leadershipvalues and generated further valueswork. On the other hand, participant validation generated ethical concerns related to how information should be shared and used with other informants (see Røthing, 2002 for further discussion).

Validation of Preliminary Analysis in Focus Groups

After a preliminary analysis of the data material from observation and interviews, validation meetings were held with a selected group of managers in two of the nursing homes.Footnote 1 A central finding of the study concerned the employment situation of immigrant employees without formal healthcare credentials (Lindheim, 2021). Tables that displayed the numbers and percentages of employees in different categories of healthcare positions and the size of their employment contracts were presented in the validation meetings. The participants could then compare the information from their nursing home with the information from the other two nursing homes. They were informed that the three nursing homes had different operating structures (one run by the municipality, one run by a non-profit entity and one run by a for-profit entity), but the identities of the nursing homes were kept anonymous. The comparison of the three nursing homes revealed that employment policies were applied differently, and informants from a nursing home with one operating structure justified their way of doing it and criticised the others:

We, too, follow the Working Environment Act in that you are entitled to a permanent position [when you have worked for three years]. It is exploitation of the staff not to give them extra shifts to avoid [them claiming] a permanent position. (Excerpt from validation meeting)

The validation meetings stirred up discussions among the participants about the identity and values of the nursing homes and evolved into what is here understood as valueswork. The validation meetings thus generated new data material that was incorporated into the study. The arguments and interpretations that emerged would not have been accessed without participant validation of the analysis of the data material. The validation meeting also generated concerns related to how informants’ reactions should be handled. How should I balance ethical responsibility and analytical freedom (Røthing, 2002)? Should I accept their responses at face value and incorporate their feedback directly as new data, or could I further interpret their reactions as potential justifications and defence mechanisms?

Participant Validation—Ethical Concerns and Values Work

Participant validation is a strategy to strengthen the trustworthiness of a study. The review of the literature and the examples from the case study highlight three further contributions of participant validation when it is incorporated in the research process: it addresses and raises ethical concerns; it generates new data that can be incorporated into the study and it functions as a site and instantiation of values work.

Addressing and Generating Ethical Concerns

Participant validation addresses ethical concerns in the research process. Core issues in this regard are transparency and trust in the research process and the unequal power relation between the researcher and the researched (Buchbinder, 2011; Fangen, 2010; Slettebø, 2020). In the case study described above, by sharing the shadowing reports with the unit managers, the informants trusted that their work situation and everyday challenges were understood. A side effect of trust in the research process was that it improved the quality of the interviews that followed. When trust and rapport were established, the unit managers shared information more openly in the interviews. The case study also illustrates that transparency and power are interrelated. Sharing instead of withholding data, like the shadowing reports, modified the experience of power imbalance between researcher and informants, which in turn increased trust.

In the validation meetings in the nursing homes, the informants were invited to respond and react to the analysis of the data material from all three nursing homes, addressing again the ethical concern of transparency in the research process. The opportunity to compare findings from their own nursing home with other nursing homes also modified the power relation between the researcher and the researched (Birt et al., 2016). The interpretation and outside perspective offered in the validation meetings had an empowering potential (Slettebø, 2020), which could further reduce the power imbalance in the research process.

However, participant validation also generated a new set of ethical concerns. Of the examples presented above, the situation with the manager who had conflictful relationships with her employees elicited the most ethical concerns and feelings of uneasiness (Buchbinder, 2011). The discrepancy between the manager’s perspective and the employees’ perspective in the interviews surfaced questions around handling the issue of anonymity, protecting both managers and employees from harmful effects of the research process. In the information provided prior to the study, informants were ensured anonymity. In publications from the study, informants and nursing homes are anonymised. However, the informants in the study had knowledge of the other persons involved from their nursing home, in particular the other interviewees from their units. Røthing (2002) discusses the dilemma of external versus internal anonymity. In her study of couples, the partners were interviewed individually, while the data material from both parties was analysed together. If the couples read the analyses, the partners’ perspectives would be revealed. My solution to the challenge in the case study presented here was to examine even more carefully which quotes from the informants to use. I wanted to shed light on the tension between the manager and the employees’ perspective without causing further conflicts and placing the informants in a vulnerable position. By choosing quotes that contained information that was already known to both parties, I sought to safeguard both concerns.

The validation process also raised questions of representation of informants in the articles published from the study. How should the information and feedback received from one informant or from one validation meeting be balanced with information from other informants and my own interpretation. (Birt et al., 2016)? Would they feel betrayed if they read the publication afterwards (Røthing, 2002; Slettebø, 2020)? In the writing process, this question was troubling, and the papers written for publication were revised yet again to ensure that the presentation stayed true to the data material. These questions reflect the challenge of balancing the impetus to conduct research that sheds light on injustice in organisations with concerns for avoiding bias and partiality.

The validation meeting with the managers surfaced yet another ethical concern. Who should participate in the validation meeting? Was it right to have this meeting only with managers? What about the informants in subordinate positions? In hindsight I would have preferred a more representative validation meeting. The selection of participants was a pragmatic solution, which is often the case in research. It was easier to gather a smaller group of managers who had more flexibility in their work schedules than to organise a larger gathering for which employees had to leave their daily duties in the units at the nursing homes.

Generating New Data

In line with Slettebø’s (2020) findings, the experience from the case study discussed here was that participant validation generated new data that were incorporated into the study. The clearest example was the discussions generated during the validation meetings. When the informants examined the statistics on employee categories and employment contracts, they offered new information of how the system regulated these issues in the nursing homes, and they argued for their positions and priorities with reference to the other nursing homes. The tendency for employees without formal healthcare credentials in the nursing homes to remain in precarious employment (Lindheim, 2021) was an issue that stood out more clearly after the analysis of the data material. The validation meetings thus offered an opportunity to probe further into this issue, which had not been as evident during observations and interviews.

Focus groups are not frequently used in participant validation (Birt et al., 2016). However, the use of focus groups or validation meetings with multiple informants has the potential to generate discussions at a different level than what individual validation interviews or written feedback can do.

Participant Validation as a Site and Opportunity for Values Work

The examples from the case study presented above illustrate how participant validation may represent a site and an opportunity for values work. Beyond researching values work as a topic, incorporating participant validation into the research process may generate processes of values work, which offers an opportunity to study values workin situ and in vivo (Zilber, 2020). This was evident when the unit mangers assessed and evaluated their management practices in light of the shadowing format. Another example was the validation meetings, which generated opportunities to discuss the identity and values of the nursing home when the managers compared their nursing home with the others included in the study. This finding resonates with Slettebø’s (2020) experience with validation interviews in his study.

Concluding Remarks

Why should you incorporate participant validation in the research process when you study values work in an organisation? A first answer to that question is that it is a strategy to ensure the trustworthiness of the data and results of your study, and, second, it is a way to address ethical concerns of transparency and power imbalance in the research process. In addition, the validation process may itself result in values work. You may use participant validation when you collect different sources of data and data from different informants early in the process. To reap the benefits of this strategy I would encourage you to also include participant validation at a later stage in the research process, inviting the informants to validate and discuss your analysis and interpretation of data. This way, participant validation have a further empowering potential and may add valuable data to your study of values and values work.