Introduction

Type 2 diabetes, a highly burdensome chronic illness affects 3.7 African Americans (AAs) in the United States and leads to a greater risk of death and disability compared to non-Hispanic whites [1]. AAs are also likely to develop diabetes-related complications such as heart disease, stroke, and amputations that eventually lead to death, compared to non-Hispanic whites [2]. A prominent contributor to these diabetes health disparities is poor medication adherence [3]. Medication adherence is defined by the World Health Organization as the extent to which patients take their medication in correspondence with the agreed recommendations from a health care provider [4]. Poor medication adherence causes 50% of treatment failures, leading to 125,000 deaths and 69% of medication-related admissions [2, 5,6,7,8]. While there are many reasons for poor medication adherence, compared to patient sociodemographic and clinical factors, patient beliefs about their illness has one of the largest influences on medication adherence [9, 10].

Poor medication adherence among AAs may be partly due to the lack of personalized and culturally-adapted interventions that account for their unique illness beliefs which may be different compared to other racial groups [3, 11]. Also, the differences in diabetes complications between AAs and whites may result from these overlooked illness beliefs. For example, compared to whites, some AAs have religious beliefs about the control of diabetes including a belief in God’s ability to cure diabetes [11]. One of the important steps in improving medication adherence among AAs with diabetes is to assess their beliefs about diabetes in a valid and reliable way. The only validated and reliable measure available to assess illness beliefs is the Illness Perception Questionnaire (IPQ). The components of the survey provide a framework for patients to make sense of their symptoms, assess their health risk, and direct their actions and coping skills. However, this survey has only been validated in a Western European population [12, 13] and has significant reliability and validity measurement problems when used with AAs [14,15,16,17]. For example, in a study by Ward et al. [15], five of the revised Illness Perception Questionnaire sub-scales had consistency/accuracy issues when used among AAs with mental illness in relation to their health-seeking behavior [16]. The measure did not address unique AA cultural beliefs that defined their coherent view of their mental illness. The challenges in measuring beliefs about mental illness might be similar when measuring beliefs about diabetes. For example, AA diabetes-related cultural beliefs such as familial relationships, their racial identity and/or religious beliefs are not accounted for in the IPQ. Thus, there is an urgent need for a valid and reliable culturally-adapted survey to assess illness beliefs for AAs. Importantly, to develop and implement culturally appropriate diabetes medication adherence interventions which account for AAs illness beliefs, a measure that appropriately assesses their culturally informed beliefs is needed.

In evaluating and making changes to a survey tool, it is important to understand if added or revised items measure what the researchers intend and respondents understand and correctly interpret the items [18]. For this study, subsequent to the adaptation of the Illness Perception Questionnaire for AAs with diabetes, it was important to ensure the adapted questionnaire had content (survey items) that represented illness perceptions and was understandable to AAs. Hence, cognitive interviewing (a psychologically oriented method for empirically studying how individuals mentally process and respond to survey questionnaires) [19] was used as a means to pre-test the culturally adapted IPQ to explore if the new survey items measured what was intended and whether respondents understood and could respond to the items.

Objective

The study aimed to utilize qualitative findings from prior focus groups [20, 21] to adapt the established Illness Perception Questionnaire [13, 22] for use in AAs with diabetes, and to use cognitive interviews to explore patients’ interpretation and understanding of the survey. This study describes the cognitive interviewing process used to pretest the survey. The cognitive interviews were an important pre-survey evaluation step in the questionnaire development process because it identifies response errors, and can inform early identification of questionnaire problems [23]. We discuss the methods that went into refining and validating the questionnaire in this early development process. Cognitive interviewing identifies potential response problems that may compromise the response rate and the quality of the data and are used in creating culturally appropriate questionnaires for diverse populations. The items selected for evaluation were significantly different in context and topic compared to the original items.

Main text

Methods

Study design

Given the focus of this study, a qualitative approach with the use of cognitive interviewing seemed the best fit. Cognitive interviewing identifies sources of response error in survey questionnaires by focusing on the cognitive process respondents use to answer questions [18, 19, 24, 25].

To prepare for the cognitive interviews, we conducted the following:

  1. 1.

    Based on the revised IPQ measure [22], an adapted survey with culturally-embedded characterizations of illness beliefs was developed using themes generated from six focus groups of 40 AAs with diabetes [20, 21].

  2. 2.

    Forty-four new items were added to the 39-item revised IPQ, resulting in a culturally adapted tool with 83 survey items (Additional file 1: Appendix).

Two experts on illness perceptions (EW and CB), as well as an expert on survey design and psychometric methods (DB), examined the survey for content areas needed to measure the illness beliefs important to AAs medication adherence, as well as survey item structure and wording. Then, the cognitive interviews were completed.

Sample

Six women and four men English-speaking AA age 45–60 years were recruited using purposive sampling. The inclusion criteria were self-identified AA/Blacks who spoke English, self-reported diagnosis of type 2 diabetes by a health care provider at least 1 year prior to the time of the study and reported taking at least one diabetes medication by mouth. The cognitive interviews were conducted 2 weeks apart in March 2018. The study was approved by the University of Wisconsin-Madison Health Sciences Institutional Review Board (IRB number: 2018-0281).

Recruitment

Study participants were recruited from different AA community locations in a Midwestern City in the United States. Flyers were distributed at AA churches, non-profit organizations, public libraries, and community centers. Interested participants then contacted the study recruiter about their intention to participate. Informed consent was obtained prior to the interviews.

Measures and data collection procedures

Interviews

Interviews were conducted in private locations at the convenience of participants. Two iterative interview processes were conducted 2 weeks apart, with 5 respondents in each phase. All interviews were audio-recorded. The survey was administered by an AA Ph.D. trained health-equity researcher and an AA project assistant trained in facilitating cognitive interviews. At the start of the interview, standardized instructions were read aloud to each participant. Respondents were told that the essence of the interview was not to test respondents for right or wrong answers. Rather, it was to understand their thought process and understanding of questions and to explore any suggestions or comments they had about the questions.

Probing methods

Verbal probing (a form of data collection in which the cognitive interviewer administers a series of probe questions specifically designed to elicit detailed information beyond that normally provided by respondents) was used [26]. The interviewers guided the interaction more proactively by asking additional, direct questions about the basis of respondents’ responses. Respondents were offered a copy of the Likert-scale and asked to identify their response choice following each question by the interviewer. Sample questions included, “There is no cure for your diabetes. Do you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree?” The interviewer further probed the respondents of their answer choices. Probing questions were tailored to the respondents’ replies. For example, “Tell me more about why you answered [ANSWER] for this question.” If a participant chose “neither agree nor disagree” they were further probed to decide if they had to choose between “agree” or “disagree,” which option would they have selected and the reason for that choice. Salient words in the questions were selected and further probed. For example, the interviewer asked, “What did the word “cure” mean to you in this question?” For each question, respondents were encouraged to share any suggestions or concerns regarding the questions. To ensure that respondents only focused on each separate survey question per time, only the pertinent question was visible to the respondents at each time.

Think aloud methods (which requests that the survey respondents actively verbalize their thoughts as they attempt to answer the survey questions) were also used as a cognitive process for exploring participants’ understanding [27]. The respondents used this think-aloud technique to talk through the process of their answer selection. Basic demographic characteristics were collected prior to the interviews. Participants were given $50 cash upon completion of the interviews.

Data analysis

All interviews were transcribed by a certified transcriptionist. Thematic analysis was used to analyze the interviews by combining the notes pertaining to each evaluated survey item, exploring and aggregating common themes across interviews, and identifying key findings that may indicate differences between the intended interpretation and that of respondents [28]. The analysis was completed by two individuals trained in qualitative data analysis. Data analysis was conducted using Nvivo 11 (QSR International-Melbourne) and continued until the researchers could not identify newer dimensions within the data (data saturation). After all analyses were completed independently, the coders met to identify similarities and differences between themes and discuss any divergences. Data analysis was completed upon data saturation.

Results

Ten AA men and women participated. Most respondents were female (n = 6, 60.0%) with a mean age of 57.5 (± 4.5) years. Each participant evaluated each survey item and response option presented to them. The experts on illness perceptions, survey design, and psychometric methods determined that each new survey item was important in the measurement of AA beliefs about diabetes. Using face value, they evaluated each new item for how important it was in the conceptualization of AAs’ illness beliefs. If the survey item did not seem to represent AAs beliefs, they indicated this in their evaluation. Hence, no items were deleted.

Cognitive interviews

After the interviews, an item-by-item review of the 29 new survey items identified problems with five items related to AA’s comprehension, reference/applicability, and wording/tone. These items were related to AA’s understanding of a cure for diabetes, perception of how food influences their diabetes, how their identity as AA influences diabetes control, and the dialogue about diabetes within their families and/or community (Table 1). For the problem types, comprehension of question was related to their understanding of what the question meant, unclear reference focused on their ability to connect the question to their life and answer appropriately, wording or tone referred to their sensitivity to the language/word used. Five of the items were revised to make them clearer and no questions were deleted (Table 2).

Table 1 Questionnaire items tested in cognitive interviews and found to be confusing or misinterpreted
Full size table
Table 2 Changes made to new survey questions based on participant feedback
Full size table

Discussion

Our findings suggest that to create a culturally adapted survey that is usable by AAs, focusing on the target population perception of the meaning of the questions prior to survey administration is fundamental in the design of the survey. Listening to and probing respondents as they verbalized their thought processes allowed for the refinement of the survey. Out of 29 newly developed survey items that were culturally specific to AAs with diabetes and tested using cognitive interviews, five questions were identified as problematic and revised accordingly.

The study results show that by utilizing cognitive interviewing, the comprehension of questions for a culturally adapted illness perception questionnaire can be improved and made culturally appropriate for underserved populations, and questions can be easily interpreted when they are more applicable to the daily life of respondents. By better addressing respondents’ concern about the new survey items in the culturally adapted IPQ, more reliable and valid data on AAs illness beliefs about type 2 diabetes can be collected and utilized for clinical and research purposes. For example, the adapted questionnaire can be offered to AA patients with poor control of diabetes and/or medication nonadherence to understand which illness beliefs should be addressed in an intervention.

Although the Illness Perception Questionnaire has been pretested in different populations, no intensive cognitive interviewing process had been conducted before prior use in an AA population. This study’s results further support that cognitive interviews are an effective technique for improving health surveys and modifying them to be culturally relevant to diverse populations [19, 26]. The finalized version of the culturally adapted Illness Perception Questionnaire for AAs with diabetes will be further tested for reliability and validity among a larger group of AAs with type 2 diabetes.

Conclusion

The study findings support the newly developed illness perception questions as culturally specific to AAs with diabetes after being tested for content validity and participants’ understanding using cognitive interviews.

Limitations

The study had some limitations. General probes were administered before specific probes for most questions. It is possible that the response to the specific probes was enhanced because participants had been primed from the general probes. The respondents were selected from a single city in a Midwestern, United States town. AAs from more diverse locations may have a different interpretation of the survey questions. Future studies should conduct further cognitive testing among AAs from different parts of the United States. Due to the time constraints of the cognitive interview process, only 29 survey items out of the 44 new items were tested.