Abstract
Objective
To develop an analysis scheme capturing the cognitive processes underlying QoL assessment to increase our understanding on how to interpret responses to QoL items. Tourangeau et al.’s (The psychology of survey response, 2000) and Rapkin and Schwartz’ (Health Qual Life Outcomes 2:14, 2004) cognitive process models form the basis for this analysis scheme.
Methods
We conducted think aloud interviews with six cancer patients prior to and following radiotherapy to elicit the cognitive processes underlying the assessment of 7 EORTC QLQ-C30 items. Content analysis was carried out by two to four researchers independently. Eighty text fragments were analyzed inductively and combined in an iterative process with deductive analyses based on both models.
Results
We have developed a comprehensive analysis scheme feasible for analyzing the cognitive processes underlying QoL assessment qualitatively. All cognitive components of both models could be distinguished in our data. The cognitive component ‘reporting and response selection’ needed extension to fully capture the cognitive processes used.
Conclusion
The two models combined are useful in describing the cognitive processes cancer patients use in answering QoL items, and as such facilitate insight into patients’ self-reported QoL assessments. Interestingly, the content of the cognitive processes not only differed between patients but also between items within patients and over time.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Since the 1980s, the number of quality of life (QoL) studies has increased dramatically [1] and QoL is becoming an increasingly important and integral outcome in clinical trials [2–4]. To better interpret responses to QoL questionnaires, we need to examine the underlying cognitive processes [5], which in turn will increase the insight into the validity of QoL questionnaires. However, studies that systematically investigate these cognitive processes are scarce. In this study, we will examine the cognitive processes underlying cancer patients’ responses to QoL items by means of two theoretical models.
In the area of survey research, several models have been developed to describe the answering process to survey questions [6–9]. We will use the model of Tourangeau and colleagues [9], because it largely resembles earlier survey models and excels in the elaborate description of each process. Second, to specifically address the cognitive processes underlying QoL assessment, we will also use the Rapkin and Schwartz’ QoL appraisal model [10]. Table 1 provides the cognitive components of both models. As can be seen, these models show great resemblance, as they share three cognitive processes and add one each. An important distinction is that the model of Tourangeau et al. [9] aims to explain the cognitive processes used for answering individual questionnaire items whereas the Rapkin and Schwartz’ model [10] focuses on the cognitive processes used in answering an entire QoL questionnaire.
To the best of our knowledge, only Wyrwich and Tardino [11] have used the Rapkin and Schwartz model [10] in analyzing qualitative data on the cognitive processes underlying health-related quality of life (HRQoL) transition items. In this study, 41 chronically ill patients were interviewed by phone to identify their thought processes in answering transition items completed at an earlier occasion. A limitation of this study is that the transition assessments and the cognitive interview were not administered simultaneously. It is questionable whether respondents were able to recall their thought processes or rather reconstructed these processes during the interview.
To increase our understanding on how to interpret responses to QoL items, this study’s aim is to develop an analysis scheme capturing the cognitive processes underlying QoL assessment, based on the models of Tourangeau et al. [9] and Rapkin and Schwartz [10]. To elicit these cognitive processes, we will ask cancer patients to think aloud while completing QoL items prior to and following radiotherapy. In contrast to Wyrwich and Tardino [11], we will probe the patients directly after the completion of QoL items to reveal more information on their thought processes.
The specific objectives of this study are to examine whether (1) patients’ answers can be categorized according to the distinct cognitive components of the models, (2) a text fragment can be categorized exclusively in one cognitive component, and (3) the proposed models are exhaustive in capturing the cognitive processes underlying responses to QoL items or need to be extended.
Pilot study
The interview consisted of seven items selected from the 30-item EORTC QLQ-C30 [12], since it is too burdensome for patients to be cognitively interviewed about all items. Therefore, we first conducted a pilot study aimed at selecting the questionnaire items, the interview procedure and interview probes.
Material and methods
Sample
We selected four individuals with different health problems (mononucleosis, migraine, pelvis injury, repetitive strain injury) and with a social science background so that they were able to provide useful comments on the cognitive interview. These individuals were not part of the research team. The sample consisted of three women and one man (mean age 38.8 years, SD 12.4, range 25–55).
Questionnaire items
The seven items derived from the EORTC QLQ-C30 [12] were selected such that they covered global as well as specific content, and physical as well as emotional dimensions.
We thus included two items from the physical functioning scale (Do you have any trouble taking a short walk outside of the house?, Were you limited in doing either your work or other daily activities?), two items covering the most common symptoms (Have you had pain?, Were you tired?), one emotional functioning item (Did you feel depressed?), and two global items (How would you rate your overall health during the past week?, How would you rate your overall quality of life during the past week?). All items employed a one week time frame.
Cognitive interview
We used the Three-Step Test-Interview (TSTI) [13], combining the two subtypes of cognitive interview methods (think aloud interviewing and verbal probing techniques) [14]. In the first step, the respondents are instructed to think aloud while completing questionnaire items. In the second step, the interviewer poses focused questions aimed at remedying gaps in the data collected in the first step, e.g., pauses in which the respondent did not think aloud. The third step consists of semi-structured probes aimed at eliciting more information about the cognitive processes used. The first two subjects were interviewed retrospectively (after administering all seven items), the other two subjects were interviewed while probing concurrently (immediately after the think aloud response to each QoL item).
Interview probes
The interview probes were based on the models of Tourangeau et al. [9] and Rapkin and Schwartz [10]. Example probes within each cognitive component are illustrated in Table 2. To clarify patients’ responses we additionally posed non-leading probes like “Could you tell me more about that?” and “Could you explain that to me?”.
Results
Questionnaire items
After answering the first item covering physical functioning ‘Do you have any trouble taking a short walk outside of the house?’, the second physical item ‘Were you limited in doing either your work or other daily activities?’ did not add new information about the cognitive processes underlying patients’ assessment of their physical functioning. To better reflect the multidimensional character of the concept QoL, we replaced this item by the social functioning item ‘Has your physical condition or medical treatment interfered with your social activities?’. Second, since respondents did not endorse the item ‘Did you feel depressed?’ we came to realize that this item reflects a too extreme emotional state. To elicit more elaborate cognitive processes we replaced it by the more prevalent state “Did you worry?”.
Interview procedure
In probing retrospectively, the respondents indicated that they could not remember what they were thinking previously when answering the questionnaire items. This resulted in respondents reconstructing their answering process. Therefore we chose to probe concurrently. The disadvantage of this approach is that the think aloud process can be influenced by the semi-structured probing of the preceding item. Although this effect was not apparent in the pilot study, it cannot be ruled out [14].
Interview probes
The interview probes appeared to elicit the cognitive processes used in answering the QoL items, since the respondents understood the probes and answered them accordingly.
However, when respondents assessed their current functioning, they were often not aware of the fact that they used their prior functioning as a comparison. Direct probing about their standard of comparison proved troublesome in these cases. Since respondents mentioned their used standards spontaneously, direct probing about standards of comparison was often not needed.
Main study
Material and methods
Sample
At the Department of Radiotherapy of the Academic Medical Center (AMC) in Amsterdam, four radiotherapists consecutively recruited newly diagnosed cancer patients undergoing radiotherapy and matching our inclusion criteria: a minimum age of 18 years, fluent command of Dutch language and absence of cognitive impairments. Data saturation was reached based on 80 response processes of six patients (mean age 64.7 years, SD 9.0, range 49–75) (for further explanation see the Discussion). Four patients completed all seven QoL items at baseline and follow-up. The other two patients provided interpretable data for six items at both interviews. Overall, the interviews yielded 80 response processes useful for qualitative analysis ((four patients × seven items × two interviews) + (two patients × six items × two interviews)). All patients consented to participate. Four patients were men, two of them were diagnosed with prostatic cancer and two had esophageal cancer. One female patient was diagnosed with a gynecological tumor, the other with bladder cancer. One patient had elementary education, two patients had lower general secondary education, and three patients had lower or intermediate vocational education. The mean time between diagnosis and baseline interview was 62 days (SD 23.9, range 28–91).
Design
Patients were interviewed using the questionnaire items, TSTI procedure and probes resulting from the pilot study. Interviews were conducted prior to and following radiotherapy. Baseline interviews were administered on the day the patient had an appointment for the simulator, a medical procedure to plan the actual treatment. The follow-up interviews were either held at the last day of radiotherapy or in combination with an appointment with the patient’s treating radiotherapist, varying from 1 to 2 weeks after completion of radiotherapy. The mean duration of the interviews was 41 minutes (SD 11.3, range 30–71). The interviews were held at the AMC with only the patient and the interviewer being present and were conducted by the same interviewer (EB).Footnote 1 During the interview, the interviewer kept the distinct cognitive processes of both models at the back of her mind and only probed for those cognitive components that did not emerge in the think aloud answers as provided by the patient. All interviews were tape-recorded and transcribed verbatim. After the interviews, the interviewer (EB) made field notes describing contextual information.
Analysis
Since the cognitive process models of Tourangeau et al. [9] and Rapkin and Schwartz [10] have not been specifically designed for the analysis of qualitative data, we had to develop a detailed plan of analysis. Content analysis of the interviews was carried out by at least two out of four researchers (EB, MK, FvZ, MS) independently and started directly after a patient had completed both interviews. To provide an open and unbiased account of the cognitive processes used in answering QoL items, we started all analyses inductively (data driven) by summarizing the salient content of each interview. Subsequently, all 80 text fragments were analyzed deductively (theory driven) according to a preliminary analysis scheme based on a combination of the two models using MAXqda software. The inductive and deductive analyses were compared in an iterative process. Since knowledge of the models might influence the inductive analyses, we actively searched for information that would not fit in or run counter to these models. After both interviews of each patient had been analyzed, all researchers involved discussed their findings. Frequently, this discussion yielded new insights that resulted in adaptations of the preliminary analysis scheme. Consequently, these amendments resulted in renewed analyses by all researchers of all text fragments and a renewed discussion of the findings. Finally, this iterative process resulted in a comprehensive analysis scheme based on mutual consensus. This analysis scheme is provided in Appendix 1, presenting all possible codes within each cognitive component and a description of each code. Please note that not all codes within each cognitive component will be applicable in analyzing the response process underlying each QoL item, which is illustrated in Appendix 2.
Results
Categorization of text fragments according to the cognitive models
(A) Comprehension/frame of reference The patients indicated what they understood the target construct to mean in 77 response processes. In 51 response processes the patients additionally mentioned what they considered to be the opposite of the target construct. For example:
How would you rate your overall quality of life during the past week? (range 1 (very poor) – 7 (excellent))
“I would say a ‘3’. (…)Footnote 2 Quality of life is being healthy and to be able to do everything you like. (…) I feel limited however, I can’t do a thing. I’m no longer self-reliant, I am dependent on others.”[Female (F), 71 years, gynecological cancer]
(B) Retrieval/sampling strategy In retrieving relevant information within the frame of reference, subjects can either recall positive or negative experiences. Sampled experiences could be distinguished in all 80 response processes. The patients retrieved far more negative (227) than positive (141) samples in their QoL assessments.
The period of time the patients considered in arriving at their answers could be identified in all response processes. In 11 response processes patients referred to two to three different periods, e.g., the time prior to cancer diagnosis and the past week. In 34 response processes, the patients referred to the past week, in agreement with the time-frame as employed by the EORTC QLQ-C30. Other coded periods are the period since cancer diagnosis (N = 27), period of radiotherapy (N = 18, follow-up), time prior to cancer diagnosis (N = 12), time since cancer treatment other than radiotherapy (N = 7), time since radiotherapy (N = 4, follow-up) and other periods, e.g., the future (N = 8).
The following excerpt exemplifies this cognitive component; in assessing his pain, the patient retrieves a negative sample (injection as part of hormonal therapy) by thinking about his last painful experience 2 days before (period).
Have you had pain?
“Well, what is pain? When I get an injection in my stomach, with a very thick needle, like two days before. That hurts.”
You immediately mentioned the injection, was that the first thing that came to your mind?
“Yes, since that’s the last painful thing I have had this week.” [Male (M), 66 years, prostatic cancer]
(C) Standards of comparison To arrive at an answer, the patients spontaneously referred to a reference group in 77 response processes, of which 33 contained two to four different reference groups, e.g., self prior to cancer diagnosis and an expectation about the course of the disease. In the majority of the response processes, the patients made a comparison with themselves (N = 68). Most frequently (N = 47) the patients compared their current functioning with their own functioning prior to cancer diagnosis and treatment. In 15 response processes, other (cancer) patients were used as comparison, for example:
Did you worry?
“(…) I have buried my brother, he also had prostatic cancer. But I know, all patients are different and he had metastases in his bones, and I haven’t”. [M-2, 66 years, prostatic cancer]
Additionally, patients compared themselves with an expectation (N = 19), an ideal (N = 5), other people the same age (N = 2) and other reference groups (N = 13), e.g., mentally ill patients.
(D) Judgment/combinatory algorithm This cognitive component is of particular relevance when respondents consider both positive and negative samples (N = 54). In arriving at a response to the QoL item, the respondents were found to emphasize the positive (N = 15) or negative samples (N = 10), or find a balance between both (N = 29). The following excerpt shows an example of a respondent emphasizing the positive sample.
Has your physical condition or medical treatment interfered with your social activities?
“Not at all. Well, at the moment I am on sick leave. (…) I really enjoy my work, I have friends there. I consider my work to be a social activity. (…) The place where you have the majority of your social contacts during the day. (…) But I also have my social contacts at home. I can’t say I have no social contacts left. (…) Now there are other people I can visit, people I do not get around to normally.” [M, 49 years, esophageal cancer]
(E) Reporting and response selection In 67 response processes patients explained how they arrived at their answer. For example:
“I did worry a lot, now I worry to a lesser degree. So my answer would be ‘quite a bit’ or ‘a little’ (…) I’m convinced that everything will be all right when I’m being operated, so I’ll leave it at ‘a little’” [M, 71 years, esophageal cancer].
According to Tourangeau et al. [9] respondents may edit their response for consistency, acceptability or other criteria, e.g., the wish to avoid embarrassment or politeness to the interviewer. This way of editing a response was found in only two of the 80 response processes. In the following excerpt, the patient edits his initial QoL rating to make it consistent with his response to the preceding question in which he rated his overall health:
“(…) Since I’ve just rated my overall health as ‘very poor’, I have to do the same for my quality of life, don’t I?” [M, 71 years, esophageal cancer].
Exclusivity of categories
Systematic overlap was found between three cognitive components. First, between the components retrieval/sampling strategy and standards of comparison. This is partly in line with Rapkin and Schwartz’s model [10] according to which each sampled experience is judged against subjective standards of comparison. However, from the 368 samples mentioned, only 193 samples were judged against standards of comparison and thereby coded in both cognitive components. The remaining 175 samples were not compared to any reference group. Second, overlap was found between the components retrieval/sampling strategy and judgment/combinatory algorithm. This overlap is suggested by both models in that all sampled experiences need to be combined in a particular way to arrive at an answer. The following excerpt shows the overlap between these cognitive components:
“[Trouble taking a short walk] means that you dread walking or that you simply just can’t manage to get it done. I have experienced that once when I was operated on for my back2,3. And now I am not experiencing that kind of trouble at all1. I even prefer taking a longer walk than a short one4.” [M, 66 years, prostatic cancer]
1 Positive sample (Retrieval/Sampling strategy)
2 Negative sample (Retrieval/Sampling strategy)
3 Self – other [medical treatment in the past] (Standards of comparison)
4 Emphasis within sampling – positive (Judgment/Combinatory algorithm)
Extension of models
Sixty-five response processes reflected editing processes for criteria other than those proposed by Tourangeau et al. [9]. These editing processes are interpreted as mitigating the initial response to make it acceptable for the respondent him/herself instead of making it socially acceptable. Therefore, we extended the cognitive component ‘reporting and response selection’ with three editing processes; self-protection (N = 34), self-presentation (N = 17) and normalization (N = 14). For example, in the following excerpt the patient edits his initial answer for self-protection in taking into account that his situation might deteriorate.
How would you rate your overall quality of life during the past week?
“Actually I would rate my quality of life with a ‘7’ [excellent]. But life is full of surprises, they might find a metastasis in the future. So I can’t choose a ‘7’, I have to rate my quality of life with a ‘6’.” [M, 66 years, prostatic cancer]
The wish to present one-self in a specific way was also found to be a motive for respondents to edit the initial answer. For example:
Were you tired?
“Yes, I suppose radiation treatment and chemotherapy caused that. In the beginning it didn’t cause that much trouble, but lately I do notice something’s going on. I get tired very easily, I haven’t got any energy left. I just want to sit on the couch the entire day. (…) I’ll say I’m tired ‘quite a bit’, ‘very much’ doesn’t suit me.” [M, 49 years, esophageal cancer]
In answering the QoL items, the patients referred to their ageing to normalize their deteriorated health and functioning. This process of editing for normalization resulted in reporting a mere absence of symptoms. For example:
Has your physical condition or medical treatment interfered with your social activities?
“No, not at all. (…) Five weeks ago I’ve painted four houses from the outside, when I already knew about me having cancer. Now I don’t feel up to it anymore (…) Well, you need to close certain periods of your life. But that’s simply a part of getting older.” [M, 71 years, esophageal cancer]
Discussion
We have developed a comprehensive analysis scheme feasible for the qualitative analysis of the cognitive processes underlying responses to QoL items. Our patients were capable of verbalizing their cognitive processes used in answering the QoL items. The models of Tourangeau et al. [9] and Rapkin and Schwartz [10] combined proved applicable in the subsequent analysis of these cognitive processes since all cognitive components of both models could be discerned in our patients’ responses.
The fact that the interview probes we used to elicit more information about the cognitive processes were based on both models might account for the finding that all cognitive components as purported in the models could be distinguished in our data. However, in their think aloud answers, the patients frequently passed through the distinct cognitive components spontaneously, making further probing redundant. When we did use interview probes we formulated them as open and non-directive as possible.
In retrieving relevant information, the patients more often used negative samples than positive ones (sampling strategy). Considering the negative content of the QoL items (e.g., worry, pain), this can be considered a logical finding.
In accordance with the models, the only systematic overlap was found between the cognitive components retrieval/sampling strategy, standards of comparison and judgment/combinatory algorithm.
Although all cognitive components could be identified, the process of editing a response for consistency or acceptability (reporting and response selection) could only be discerned in two response processes. This can be explained by the fact that our patients might not consider the items intrusive or sensitive in a way that they felt the need to edit their response in order to provide a socially desirable response, as proposed by Tourangeau et al. [9]. However, our patients might consider the selected QoL items to be sensitive as they confront them with their deteriorating health. The use of processes to edit their responses for self-protection, self-presentation and normalization might reflect this sensitivity. In our analysis scheme, we extended the cognitive component ‘reporting and response selection’ with these editing processes to fully capture the cognitive processes underlying responses to QoL items.
Since this study, we continued using our analysis scheme to examine the cognitive processes underlying 84 QoL response processes of six other cancer patients. In analyzing their thought processes, our analysis scheme did not need revision, thereby confirming data saturation, which makes us feel confident about the scheme’s adequacy.
A remarkable finding merits attention. Rapkin and Schwartz’ model [10] describes the cognitive processes respondents use in answering an entire QoL questionnaire. It thereby assumes that respondents use the same cognitive processes in answering all individual questionnaire items. However, we found that the content of the cognitive components did not only differ per patient, but also per item within patients and over time. For example, patients compared themselves with other patients in one item, and referred to their own functioning prior to cancer diagnosis in another. Likewise, patients differed per item in the period of time they considered, the way they combined positive and negative samples and so forth. We therefore could use the response processes underlying each QoL item as unit of analysis rather than the individual patient. Consequently, we achieved data saturation based on 80 response processes instead of six cancer patients. Although this finding needs to be confirmed in more patients, it has implications for our insight into the cognitive processes underlying QoL items and for the design of qualitative QoL studies.
A range of studies have addressed the first component of the model of Tourangeau et al. [9]. In these studies, patients, elderly or healthy respondents were asked to complete a global item regarding QoL or health and were subsequently asked how they comprehended that question [16–21]. Only Groves et al. [17] and Kaplan and Baron-Epel [21] additionally addressed the comparison category of Rapkin and Schwartz’s model [10], by asking respondents whether they used subjective standards of comparison. To the best of our knowledge, this study is the first to combine both models in a comprehensive analysis scheme.
The practical implications of our findings are twofold. Firstly, our findings may point to possible flaws or unintended responses which may help to improve questionnaires, e.g., in assessing their QoL patients do not refer to the period of time as employed by the questionnaire. Secondly, in examining the cognitive processes underlying baseline and follow-up assessments, we will be able to determine whether patients’ QoL evaluation processes remain stable or rather change over time. Such findings will increase the insight into the interpretation and validity of self-reported change in QoL.
Notes
The COREQ 32-item checklist for interviews [15] is used for an explicit report of this study. Regarding the domain research team and reflexivity, we state that the interviewer (EB) is a PhD student at the Department of Medical Psychology, AMC. She was trained in qualitative research methodology at Utrecht University, The Netherlands and Kwalon (platform of qualitative research). She is experienced in administering interviews with cancer patients after conducting her Master’s thesis at the Helen Dowling Institute, Center for Psycho-Oncology. Prior to this study’s commencement, no relationship between EB and the participants was established. The domains study design and analysis and findings are accounted for in the body of this paper.
Omitted quotes are depicted by means of (…).
Abbreviations
- QoL:
-
Quality of life
- HRQoL:
-
Health-related quality of life
- TSTI:
-
Three-Step Test-Interview
- EORTC QLQ-C30:
-
European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire-Core 30
- SD:
-
Standard deviation
- AMC:
-
Academic Medical Center
- F:
-
Female
- M:
-
Male
References
Gill, T. M., & Feinstein, A. R. (1994). A critical appraisal of the quality of quality-of-life measurements. Journal of the American Medical Association, 272(8), 619–626. doi:10.1001/jama.272.8.619.
Revicki, D. A. (2007). Regulatory issues and patient-reported outcomes task force for the international society for quality of life research FDA draft guidance and health-outcomes research. Lancet, 369, 540–542. doi:10.1016/S0140-6736(07)60250-5.
Bottomley, A., Vanvoorden, V., Flechtner, H., Therasse, P.: EORTC Quality of Life Group EORTC Data Center. (2003). The challenges and achievements involved in implementing Quality of Life research in cancer clinical trails. European Journal of Cancer, 39, 275–285. doi:10.1016/S0959-8049(02)00729-3.
Osoba, D. (1992). The Quality of Life Committee of the Clinical Trials Group of the National Cancer Institute of Canada: Organization and functions. Quality of Life Research, 1, 211–218. doi:10.1007/BF00635620.
Barofsky, I. (2003). Cognitive approaches to summary measurement: Its application to the measurement of diversity in health-related quality of life assessments. Quality of Life Research, 12, 251–260. doi:10.1023/A:1023284807340.
Cannell, D., Miller, P., & Oksenberg, L. (1981). Research on interviewing techniques. In S. Leinhardt (Ed.), Sociological methodology (pp. 389–437). San Francisco: Jossey-Bass.
Krosnick, J. A., & Alwin, D. (1987). An evaluation of a cognitive theory of response-order effects in survey measurement. Public Opinion Quarterly, 51, 201–219. doi:10.1086/269029.
Strack, F., & Martin, L. (1987). Thinking, judging and communicating: A process account of context effects in attitude surveys. In H. Hippler, N. Schwartz, & S. Sudman (Eds.), Social information processing and survey methodology (pp. 123–148). New York: Springer.
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. New York: Cambridge University Press.
Rapkin, B. D., & Schwartz, C. E. (2004). Toward a theoretical model of quality-of-life appraisal: implications of findings from studies of response shift. Health and Quality of Life Outcomes, 2, 14. doi:10.1186/1477-7525-2-14.
Wyrwich, K. W., & Tardino, V. M. (2006). Understanding global transition assessments. Quality of Life Research, 15, 995–1004. doi:10.1007/s11136-006-0050-8.
Aaronson, N. K., Ahmedzai, S. A., & Bergman, B. (1993). A quality of life instrument for use in international clinical trials in oncology. Journal of the National Cancer Institute, 85(5), 365–376. doi:10.1093/jnci/85.5.365.
Hak, T., van der Veer, K., & Jansen, H. (2002). The Three-Step Test-Interview (TSTI): An observational instrument for pre-testing self-completion questionnaires. Paper for the International Conference on Questionnaire Development, Evaluation and Testing Methods (QDET).
Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. California: Thousand Oaks.
Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19(6), 349–357. doi:10.1093/intqhc/mzm042.
Padilla, G. V., Ferrell, B., Grant, M. M., & Rhiner, M. (1990). Defining the content domain of quality of life for cancer patients with pain. Cancer Nursing, 13(2), 108–115. doi:10.1097/00002820-199004000-00006.
Groves, R. M., Fultz, N. H., & Martin, E. (1994). Direct questioning about comprehension in a survey setting. In J. M. Tanur (Ed.), Questions about questions (pp. 49–61). New York: Russell Sage Foundation.
Borawski, E. A., Kinney, J. M., & Kahana, E. (1996). The meaning of older adults’ health appraisals: Congruence with health status and determinant of mortality. Journal of Gerontology: Social Sciences, 51B, S157–S170.
Montazeri, A., Milroy, R., Gillis, C. R., & McEwen, J. (1996). Quality of life: Perception of lung cancer patients. European Journal of Cancer, 32(13), 2284–2289. doi:10.1016/S0959-8049(96)00288-2.
Archenholtz, B., Burckhardt, C. S., & Segesten, K. (1999). Quality of life of women with systemic lupus erythematosus or rheumatoid arthritis: Domains of importance and dissatisfaction. Quality of Life Research, 8, 411–416. doi:10.1023/A:1008915115370.
Kaplan, G., & Baron-Epel, O. (2003). What lies behind the subjective evaluation of health status? Social Science and Medicine, 56, 1669–1676. doi:10.1016/S0277-9536(02)00179-X.
Acknowledgements
This study was funded by the Dutch Cancer Society (UVA 2005-3197). We thank H. J. G. D. van den Bongard, E. G. Geijsen, M. C. C. M. Hulshof and B. R. Pieters from the Department of Radiotherapy of the Academic Medical Center (AMC) in Amsterdam for their help in patient accrual. We additionally thank C. Tishelman for helpful discussions about the design of the study.
Open Access
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix 1: Analysis scheme
Cognitive component | Code | Description code |
---|---|---|
Comprehension/frame of reference | Definition | Meaning of target construct |
Definition opposite | Opposite of target construct | |
Meaning | Personal interpretation/underlying construct of the concept | |
Comprehension | Separate terms and/or overall question is/is not interpreted as meant | |
Retrieval/sampling strategy | Positive sample | Retrieval of positive sample |
Negative sample | Retrieval of negative sample | |
Comment physician | Referral to comment of physician regarding cancer and/or treatment | |
Comment other subjects | Referral to comment of other subjects regarding cancer and/or treatment | |
Period—prior to cancer diagnosis | Referral to period prior to cancer diagnosis | |
Period—past week | Referral to past week | |
Period—since cancer diagnosis | Referral to period since cancer diagnosis | |
Period—radiotherapy (follow-up) | Referral to period of radiation treatment | |
Period—since radiotherapy (follow-up) | Referral to period since radiation treatment | |
Period—since other cancer treatment | Referral to period since other cancer treatment | |
Period—other | Referral to other period | |
Focus—prior to cancer diagnosis | Basis for appraisal is time prior to cancer diagnosis | |
Focus—cancer | Basis for appraisal is having cancer | |
Focus—radiotherapy | Basis for appraisal is radiation treatment | |
Focus—other cancer treatment | Basis for appraisal is cancer treatment other than radiotherapy | |
Focus—other | Other basis for appraisal | |
Standards of comparison | Self—prior to cancer diagnosis | Comparison with own functioning prior to cancer diagnosis |
Self—period between diagnosis and start radiotherapy | Comparison with own functioning in the period between diagnosis and start radiotherapy | |
Self—during treatment with radiotherapy | Comparison with own functioning during radiotherapy | |
Self during medical treatment other than radiotherapy | Comparison with own functioning during medical treatment other than radiotherapy | |
Self—future | Comparison with own future functioning | |
Self—other | Comparison with own functioning in another period | |
Other cancer patients | Comparison with other cancer patients | |
Other patients | Comparison with other patients not diagnosed with cancer | |
Healthy subjects | Comparison with healthy subjects | |
Other subjects the same age | Comparison with subjects the same age | |
Comment physician | Comparison with comment of physician regarding cancer and/or treatment | |
Comment other people | Comparison with comment of other subjects regarding cancer and/or treatment | |
Expectation | Comparison with an expectation about one’s functioning | |
Ideal | Comparison with an ideal | |
Other reference group | Comparison with another reference group | |
Judgment/combinatory algorithm | Emphasis—positive | Emphasis within retrieved samples is one positive sample(s) |
Emphasis—negative | Emphasis within retrieved samples is one negative sample(s) | |
Emphasis—balance | Balance between positive and negative sample(s) | |
Reporting and response selection | Response selection | Clarifying the selection of the given response |
Consideration of other response categories | Consideration of other response categories | |
Editing response—consistency | Editing of initial response to be consistent with preceding questionnaire items | |
Editing response–acceptability | Editing of initial response to be social desirable | |
Editing response—self-protection | Editing of initial response for self-protection deteriorating health state | |
Editing response—self-presentation | Editing of initial response for the wish to present oneself in a specific way | |
Editing response—normalization | Editing of initial response in order to normalize the deteriorated health status and functioning |
Appendix 2: Application of the analysis scheme (example)
Were you tired?
-
1. “Yes, a little. I have the impression that I’ve less strength than normally.
-
2. (…) I’m more tired than I used to be.”
When you read this question, what did you think of?
-
3. “Well, last week I went for a ride on my bike and damn, they passed me!
-
4. And then I thought, normally that wouldn’t happen.
-
5. (…) I used to cycle faster, so there must be something wrong.” (…)
-
6. “But it isn’t so that I can’t do a thing.
-
7. Two weeks ago we were on holiday and we did cycle and walk a lot. And in the morning a fresh rise.
-
8. (…) You actually do notice that you are losing strength, but well, I’m getting older too.” (…)
-
9. So I’ll pick a ‘2’ [response category ‘a little’] because it isn’t the case that I can’t do a thing.
-
10. So it isn’t that bad.”
[M, 66 years, prostatic cancer]
Cognitive component | Code | Description code | Text fragment |
---|---|---|---|
Comprehension/frame of reference | Definition | Meaning of target construct | Less strength (1.) Losing strength (8.) |
Retrieval/sampling strategy | Positive sample | Retrieval of positive sample | Two weeks ago we were on holiday and we did cycle and walk a lot. And in the morning a fresh rise. (7.) |
Negative sample | Retrieval of negative sample | I have the impression that I’ve less strength than normally (1.) I’m more tired than I used to be (2.) Last week I went for a ride on my bike and damn, they passed me! (3.) You actually do notice that you are losing strength (8.) | |
Period—past week | Referral to past week | Last week (3.) | |
Period—other | Referral to other period | Two weeks ago (7.) | |
Focus—cancer | Basis for appraisal is having cancer | So there must be something wrong (5.) | |
Standards of comparison | Self—prior to cancer diagnosis | Comparison with own functioning prior to cancer diagnosis | I have the impression that I’ve less strength than normally (1.) I’m more tired than I used to be (2.) Normally that wouldn’t happen (…) I used to cycle faster (4–5.) |
Judgment/combinatory algorithm | Emphasis—balance | Balance between positive and negative sample(s) | But it isn’t so that I can’t do a thing (6.) |
Reporting and response selection | Response selection | Clarifying the selection of the given response | So I’ll pick a ‘2’ (response category ‘a little’) because it isn’t the case that I can’t do a thing. So it isn’t that bad (9–10.) |
Extension of models | Normalization | Appraisal is affected by normalization | You actually do notice that you are losing strength, but well, I’m getting older too (8.) |
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Bloem, E.F., van Zuuren, F.J., Koeneman, M.A. et al. Clarifying quality of life assessment: do theoretical models capture the underlying cognitive processes?. Qual Life Res 17, 1093–1102 (2008). https://doi.org/10.1007/s11136-008-9380-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11136-008-9380-z