Qualitative Study Design and Data Collection

Abstract

While the prior chapter set the stage for an understanding of the nature of qualitative evaluation, this chapter will offer strategies for planning a study and making decisions about how to gather data. The process is depicted as an iterative looping through steps beginning with idea generation to dissemination of results. It is critical that strategies for rigor be incorporated throughout the process. This chapter outlines methods for data collection utilizing interviews, focus groups, observation, and naturally occurring data, and then it also describes combinations often used together, which constitute toolkits of complementary techniques.

Answers to Self-Tests

Self-Test 15.1

Which of the strategies to ensure study rigor is primarily employed in the qualitative study scenarios below:

• 1. Data from interviews about the usability of a resource are analyzed thematically. The evaluation study team looks to see if and how similar themes have arisen in earlier meetings of the team.

Audit trail

• 2. A member of the study team, who has recently participated in another study of a similar kind of resource, becomes concerned that that person’s views about the current study are being shaped by that previous experience. That person sits with another member of the study team to share that person’s concerns and put them in perspective.

Reflexivity

• 3. At a “town hall” meeting called to present the results of a qualitative study, the sponsor of the study raises deep and serious questions about the validity of the findings. The study team returns to notes from their team meetings to review how and based on what data they came to this conclusion.

Member checking

• 4. During an evaluation project team meeting, one of the study team members finds themselves deeply repelled by off-color comments made by one of the project staff. The team member makes a note of this personal response as part of field notes.

Reflexivity

• 5. After interviewing 10 patients participating in a study, a study team member perceives that they are hearing the same points raised by all interviewees. The team member requests a study team meeting to consider reducing the total number of interviews from 20, as previously planned, to 12.

Data saturation

• 6. A study team member “corners” a participant in a system development effort following a meeting and asks for the participant’s impressions on what transpired in the meeting.

Member checking

Self-Test 15.2

Label each of the following interview scenarios, conducted as part of a qualitative study, as representing the fully structured, semi-structured or unstructured approach.

• 1. A study team member “corners” a participant in a system development project following a meeting and asks for that person’s impressions on what transpired in the meeting.

Open-ended

• 2. A study team member schedules time with a patient who is using an information resource to acquire specific information about the patient’s medical history.

Likely fully structured, though it could generate discussion, in which case it could veer towards semi-structured.

• 3. A study team member works with partners on the study team to develop a set of questions to be asked to all interviewees. Each question is to be followed up with the question: “Why do you think this is the case?”. At the end of the interview, subjects will be asked: “What else would you like to tell us to shed light on these matters?”

Semi-structured

• 4. An interview begins with the statement: “In general, what has been your experience using this EHR?” The remaining questions depend on how the interviewee answers this opening question.

Unstructured

• 5. A set of specific questions are read verbatim from an interview guide. No other questions are asked. The interviewees’ responses are recorded.

Fully structured