FormalPara Key Points for Decision Makers

Existing instruments for the evaluation of varicose veins have limitations pertaining to symptom assessment from the patient perspective.

Symptoms that patients deem most important were identified, i.e., heaviness, achiness, swelling, throbbing, and itching (HASTI™ symptoms).

A new patient-reported outcome daily diary instrument, the VVSymQ® instrument, was developed, and its content validity was established. It can be used to capture the patient experience of HASTI™ symptoms related to superficial venous insufficiency (varicose veins) in clinical trials using a 6-point duration-based scale.

1 Introduction

Varicose veins affect up to 73 % of women and up to 56 % of men [1]. Varicose veins in the lower extremities can be associated with heavy, aching, and restless legs, swelling and night cramps, and burning and tingling sensations [2, 3]. In the early stages, varicose veins may present a variably painful problem; progression typically leads to severe and largely irreversible problems of chronic venous insufficiency (CVI). A severe and debilitating outcome of CVI in the lower limbs is lower extremity ulceration [4].

To understand the effects of treatment, it is important to evaluate the patient’s experience of symptoms and symptom impact on functional health and well-being. Symptoms vary among individuals, and some treatment effects are not directly observable by clinicians, thus, symptoms are best measured using patient-reported outcome (PRO) measures, as recommended in the US FDA Guidance for Industry: Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims (FDA PRO guidance) [5]. Currently, a lack of consensus regarding the best way to evaluate symptoms of varicose veins and chronic venous disease is evidenced by the wide range of measures used in previous studies [3]. Available instruments (e.g., Aberdeen Varicose Veins Questionnaire [AVVQ] [6], Chronic Venous Insufficiency quality of life Questionnaire [CIVIQ-20] [7]) do not exclusively measure patient-reported symptoms associated with varicose veins and were not developed to meet regulatory expectations to support labeling claims [5, 8]. Widely used measures such as the Venous Insufficiency Epidemiological and Economic Study instrument (VEINES-QOL/Sym), used for evaluating the broad range of chronic venous disorders of the leg (CVDL) [3], are not specific enough or adequately responsive to changes in symptoms reported by patients treated for superficial venous insufficiency [9]. Additionally, whereas initial draft versions of the VEINES-QOL/Sym were pilot tested with patients, no patients, and none with varicose veins, were involved in the initial item-generation process [3].

To address the need for a PRO measure for assessing symptoms of superficial venous insufficiency (varicose veins) in clinical trials that are conducted to support labeling claims for symptom improvement, research was conducted to develop a new instrument that focuses exclusively on symptoms. This research followed FDA guidelines for measure development, including an appropriate recall period and patient input for the generation of questions (FDA PRO guidance) [5]. The initial development work for this new measure was based on the existing VEINES-QOL/Sym [3], and three phases of qualitative patient interview work were conducted to result in the final, electronic version of a new measure called the VVSymQ® instrument. The current report delineates the evolution of and the establishment of the content validity of the VVSymQ® instrument.

2 VEINES-QOL/Sym

The research to develop the VVSymQ® instrument built on previous work conducted to develop the VEINES-QOL/Sym questionnaire [3]. The 26-item VEINES-QOL/Sym includes two content domains: (1) QOL (VEINES-QOL) and (2) symptoms (VEINES-Sym). The symptoms domain consists of ten items that provided the basis for the development of the new PRO instrument.

The generation of potential symptom items for the VEINES-Sym domain was based on expert clinical opinion about the problems commonly reported by patients with CVDL and literature reviews of PROs and existing outcome measures in CVDL [3], which supported the conclusion that symptoms are important to patients. From these sources, potential items were generated through consensus discussions with a multidisciplinary group of clinicians and methodologists with expertise in CVDL, questionnaire design, psychometrics, and epidemiology. The resultant symptoms assessed in the VEINES-Sym domain were heavy legs, aching, swelling, night cramps, restlessness, heat or burning sensation, throbbing, itching, tingling sensation, and pain, and the authors based the format of the items and 4-week recall period on the Medical Outcomes Study 36-Item Short-Form Health Survey (SF-36) [10].

3 Modification of the VEINES-QOL/Sym

To begin the development of the new PRO instrument that met FDA standards, the VEINES-QOL/Sym was first adapted by shortening the recall period to 1 week and by changing the response options to a 6-point scale ranging from “None of the Time” to “All of the Time” to reflect this shorter recall period. The modified (m)-VEINES-QOL/Sym instrument, administered on paper, provided the starting point for new qualitative research with patients.

Study protocols were Institutional Review Board approved, and patients provided written informed consent. In all studies, investigators identified potential patients from their existing medical records; patients were recruited by letter or email (or from a waiting room flyer or newsletter posting), and screened for study eligibility either in person or via telephone by clinic staff.

4 Qualitative Patient Interviews: Phase 1 (Modified VEINES-QOL/Sym)

4.1 Phase 1 Methods

Initial qualitative research in 2008 was conducted via three concept-elicitation focus groups and individual cognitive interviews with additional patients who were recruited from two clinical sites in Fort Myers, Florida, and Panama City, Florida, USA. The following objectives were included: obtain open-ended input from patients regarding their symptoms, evaluate patients’ ability to understand and use the symptom domain of the m-VEINES-QOL/Sym, evaluate the 1-week recall period and 6-point duration-based scale, and ensure that all relevant symptoms of varicose veins were captured.

Patients were eligible for the study if they were aged 18–60 years, had been either evaluated or treated for severe varicose veins within the previous 6 months, and had a physician’s diagnosis of saphenous vein incompetence, severe varicose veins, and a Clinical, Etiology, Anatomy, Pathophysiology (CEAP classification for lower extremity disorders) of 3 (edema), 4 (skin changes without ulceration), or 5 (skin changes with healed ulceration). Patients in this phase of study were not evaluated for current symptoms of varicose veins at screening.

The interviewers were qualitative research scientists with extensive experience in qualitative interviewing with patients across a broad range of therapeutic indications. Each focus group session was a one-time group interview, and the primary discussions centered on patients’ spontaneous description of their varicose vein symptoms. Following the open-ended discussion, patients completed the modified (m)-VEINES-QOL/Sym, and took part in a group debriefing session to explore patients’ understanding of symptom items and probe specifically on the acceptability of the 6-point scale and one-week recall period.

For the individual cognitive debriefing interviews, a separate set of patients completed the m-VEINES-QOL/Sym. Using a semi-structured discussion guide, interviewers explored the m-VEINES-QOL/Sym items with respect to relevance, understanding, and acceptance of the modified recall period and response scale.

Sessions were audio-recorded and data were transcribed. Descriptive statistics were used to characterize the samples in terms of sociodemographic and clinical characteristics. To evaluate the qualitative data, ATLAS.ti qualitative analysis software [11] was used to assist with content analysis, and in-depth review of transcripts was performed with transcription of key points, which served as the primary data source for the analyses. Saturation of concept analysis was conducted as assessment of the focus group data quality.

Saturation of concept is defined as the point at which no new relevant information is likely to be gained from conducting additional research (e.g., focus group sessions or interviews) [5]. In this research, saturation of concept was evidenced by no new concepts appearing in the last transcript group.

4.2 Phase 1 Results

Three concept-elicitation focus groups (five to eight patients each) were conducted with a total of 20 patients, and individual cognitive interviews were conducted with an additional 11 patients. See Table 1 for demographic and clinical characteristics.

Table 1 Sociodemographic characteristics of phase 1 focus group and cognitive interview patients
Full size table

4.2.1 Phase 1 Concept-Elicitation Focus Group Results

The key results of the focus groups were the patients’ reports of symptoms relevant to their experience of varicose veins. The open-ended discussions elicited a comprehensive list of symptoms experienced by the patients, and all of the symptoms listed in the m-VEINES-Sym were discussed by many of the focus group patients, although ‘itching’ was reported by only one. Table 2 shows the number of patients who reported having experienced each symptom and sample patient quotes illustrating the relevance of these symptoms to the patients. The only additional symptoms elicited in the focus groups were numbness and bruising, both mentioned by only one patient with no other endorsement. Saturation of symptom concept was achieved in the second focus group, as no novel concepts were reported in the third group.

Table 2 Example phase 1 patient quotes demonstrating relevance of VEINES-Sym concepts
Full size table

Additional exploration during the focus groups centered on patients’ impressions of the m-VEINES-Sym 6-point response scale and various recall periods. The majority of the patients interpreted the items to be asking about the frequency with which they experienced symptoms over the past week, and all said that they felt confident in their ability to think back over the past week and respond accurately about their symptoms. All patients were able to use the 6-point scale to respond to the items without issue, and 88 % agreed that no changes should be made to the scale.

The discussions with patients with varicose veins established that concepts covered in the m-VEINES-Sym resonated with patients and that the language used was understood accurately. During the open-ended discussions, every symptom in the m-VEINES-Sym was mentioned by at least one patient, and patients used the same words to describe symptoms as those used in the m-VEINES-Sym. The discussions also verified that patients could comprehend, preferred, and in their judgment, accurately respond when using a 1-week recall period.

4.2.2 Phase 1 Cognitive Interview Results

The individual cognitive interviews confirmed patients’ understanding of the m-VEINES-Sym items, 1-week recall period and 6-point response scale. Most important to the VVSymQ® instrument development work were the patients’ reports of recognition and understanding of the VEINES-Sym symptom concepts. Data revealed that although the majority of patients had heard of and understood each of the symptoms, many of them reported that they had not personally experienced some of the symptoms, or had experienced some symptoms in the past prior to treatment, but not recently. Further, patients offered no other symptoms (i.e., symptoms not listed in the VEINES-Sym) in relation to their varicose vein experience. Table 2 shows the number of patients who reported knowing of or having experienced each symptom and sample patient quotes.

4.2.3 Phase 1 Summary

Overall, the results of the phase 1 work confirmed acceptance of the 6-point response scale and the suitability of the 1-week recall period of the m-VEINES-Sym. Further, the symptoms listed in the m-VEINES-Sym covered the breadth of symptoms among the patients of both studies, with only two patients in the focus groups both reporting one symptom that was not contained in the measure.

While all of the symptoms were known to most of the patients, some were not known to many, and other symptoms had not been experienced recently since beginning treatment. Therefore, it was decided to conduct subsequent research with patients confirmed to be experiencing symptoms at screening.

5 Qualitative Patient Interviews: Phase 2 (m-VEINES-Sym)

5.1 Phase 2 Methods

In 2009, three concept-elicitation focus groups and individual cognitive debriefing interviews (novel patients) were conducted with patients who had varicose veins and were symptomatic. The objective was to evaluate whether the m-VEINES-Sym item content aligned with language used by symptomatic patients in describing their disorder, as well as to elicit the best response options and recall period to capture these symptoms in symptomatic patients. An additional objective was to determine the importance of the frequency and severity of varicose vein symptoms to symptomatic patients. Patients were recruited from three clinical sites in the USA (Vestavia Hills, AL; Hunt Valley, MD; and Charlotte, NC) based on study inclusion/exclusion criteria and additional clinical characteristics that were similar to those for patients with moderate to severe varicose veins who would complete a symptom instrument (the final version of which was called the VVSymQ® instrument) in future clinical trials. Patients were eligible if they were aged between 19 (the age of majority in Alabama) and 75 years and had a current physician diagnosis of great saphenous vein (GSV) system incompetence, specifically demonstrating incompetence of the saphenofemoral junction (reflux >0.5 s on duplex ultrasonography) associated with incompetence of the GSV or other major accessory vein. Patients must have had a current diagnosis of moderate to severe varicose veins and a CEAP clinical classification C2–C6 (C6 indicates active venous ulcer). Patients had to be experiencing superficial venous disease manifested by both symptoms and visible varicosities, and had to have been evaluated or treated for the disorder within the previous 6 months.

To ensure patients were symptomatic and had experienced at least four of the nine m-VEINES-Sym symptoms during the past week, a list of 18 symptoms was included in the screening script. Nine of these symptoms were from the m-VEINES-Sym; the remaining nine symptoms were unrelated and included so as not to bias the patients to subsequently report and discuss only the symptoms reported during screening.

Each focus group session was a one-time group interview and, as with the phase 1 work, the discussion centered on the patients’ spontaneous description of their symptoms. After patients had described all symptoms, the moderator probed the group about any of the nine symptoms of the m-VEINES-Sym that were not mentioned in the discussion. Patients were also asked whether they preferred to report each symptom in terms of frequency or severity to confirm the appropriateness of the response scale.

For the individual cognitive debriefing interviews, patients reviewed the nine symptoms of the m-VEINES-Sym and were asked to identify any symptom(s) they had not experienced as a result of their varicose veins. Thereafter, patients completed the m-VEINES-QOL/Sym and were asked a series of questions about the clarity of instructions, recall period, and response options.

Data analyses followed the work described for phase 1.

5.2 Phase 2 Results

Three focus groups (four to eight patients each) were conducted with 19 patients, and individual cognitive interviews were conducted with an additional ten patients. See Table 3 for demographic and clinical characteristics.

Table 3 Sociodemographic characteristics of phase 2 focus group and cognitive interview patients
Full size table

5.2.1 Phase 2 Concept-Elicitation Focus Group Results

The key results of the focus groups were the patients’ reports of symptoms relevant to their experience of varicose veins. The open-ended discussions elicited a comprehensive list of symptoms experienced by the patients and, as with the phase 1 focus groups, all of the symptoms listed in the m-VEINES-Sym were discussed by many of the patients. Table 4 shows the number of patients who reported having experienced each symptom and sample patient quotes. Only three additional symptoms were elicited in the phase 2 focus groups. Two of those, “a feeling of ‘stuff/bugs crawling’ on the legs,” and “cramping during the day (not only at night)” were discussed as not being a direct result of the condition, but rather, to removal of compression stockings and diet and activity, respectively. The third novel symptom was the feeling of blood or fluid rushing into the legs upon standing, which was mentioned in only one focus group and was associated with swelling as the main symptom. Saturation of symptom concept was achieved in the second focus group, as no novel concepts were reported in the third group. When asked specifically about the 1-week recall period in the m-VEINES-Sym, patients in all focus groups reported that they were able to answer questions about their varicose veins symptoms when thinking about the past week. Focus group patients understood and were able to use the 6-point response scale to respond to each of the nine VEINES-Sym items. Finally, almost all patients in each focus group reported thinking of symptoms in terms of both frequency and severity. However, when asked, patients generally preferred reporting symptoms based on frequency, noting that it was more informative and applicable to their symptoms.

Table 4 Example phase 2 patient quotes demonstrating relevance of VEINES-Sym concepts
Full size table

5.2.2 Phase 2 Cognitive Interview Results

The individual cognitive interviews confirmed patients’ understanding and comprehension of the m-VEINES-Sym items, 1-week recall period, and 6-point response scale. Most important to the development of the future VVSymQ® instrument were the patients’ reports of recognition and understanding of the m-VEINES-Sym symptom concepts. While the majority of patients indicated having experienced most of the symptoms in the m-VEINES-Sym, tingling and restless legs were experienced by far fewer. Table 4 shows the number of patients who reported having experienced each symptom and sample patient quotes.

5.2.3 Phase 2 Summary

Overall, the results of the phase 2 work confirmed acceptance and usability of the 6-point response scale, including the assessment of symptoms based on frequency, as well as the suitability of the 1-week recall period. Further, the symptoms listed in the m-VEINES-Sym cover the breadth of symptoms among the patients who confirmed experiencing symptoms of varicose veins at the time of screening.

Across all sources in the phase 2 data, five symptoms were consistently part of the patient experience of varicose veins: heavy legs, aching legs, swelling, throbbing, and itching (HASTI™ symptoms). Moreover, after exclusion of data from one cognitive debriefing patient who confirmed only three symptoms during the interview despite having met all screening criteria, 89–100 % of the cognitive debriefing patients identified the five symptoms as relevant to their experiences with varicose veins. This level of endorsement (89 %) was also seen with night cramps, but it was excluded from the next version of the instrument because it was not spontaneously offered in one of the focus groups and, even with probing, was not a key symptom emphasized in any focus group discussions. No other symptoms related to varicose veins were discussed extensively in the focus groups or cognitive debriefing interviews. Almost all focus group patients reported thinking of symptoms in terms of both frequency and severity, but they considered frequency of symptoms more informative and crucial to assessing their symptom experiences.

5.2.4 Emergence of a New Instrument

These activities resulted in a new instrument, administered on paper, with the HASTI™ symptoms as the five items queried (Fig. 1), a 6-point response scale ranging from “None of the time” to “All of the time,” and a 7-day recall period. This paper instrument was successfully implemented in a study of the treatment of patients with varicose veins [12]. Although the paper-based version of the new five-item instrument with the 7-day recall period was well received by patients with varicose veins [12], it was later acknowledged as having potential shortcomings, including cumbersome paper implementation (and possibly lowered compliance rates), lack of verification of timing of assessments, potentially inaccurate patient recall of symptoms over the past 7 days, and the potential for data entry transcription errors, all of which could negatively impact data quality.

Fig. 1
figure 1

Conceptual framework: the VVSymQ® instrument (initial version) as a measure of important varicose veins symptoms

Full size image

Thus, paper was replaced with an electronic diary, the recall period was shortened to 24 h, and this new measure querying the five HASTI™ symptoms was named the VVSymQ® instrument. To better match the descriptor in the electronic version of questions, “heavy legs” and “aching legs” were changed to “heaviness” and “achiness,” respectively. The new measure was imbedded in a longer electronic daily diary that was implemented on a personal digital assistant (PDA). The electronic system included a user-friendly screen (Fig. 2) and an alarm schedule that prompted the user to complete the assessment at scheduled times, per protocol, and facilitated automatic data transfers each night when the PDA was attached to the charger, as well as back-up data transfers at clinical sites that may be required per protocol.

Fig. 2
figure 2

Screen image of the VVSymQ® instrument in an electronic diary

Full size image

6 Qualitative Patient Interviews: Phase 3 (VVSymQ® Instrument in a Daily Diary)

A qualitative study was undertaken to evaluate the VVSymQ® instrument in patients with characteristics similar to those who would use the instrument in future clinical trials for superficial venous incompetence.

6.1 Phase 3 Methods

Cognitive debriefing interviews were conducted in four waves with patients recruited from five clinical sites in the USA (Vestavia Hills, AL; Hunt Valley, MD; Charlotte, NC; Bellevue, WA; and Seattle, WA). Key recruitment criteria were similar to those described for the phase 2 focus groups and cognitive interviews, with the exception that the current patients were required to have at least three of the five HASTI™ symptoms in the week prior to screening, and a CEAP clinical classification of C2–C5. Data-driven modifications were made to the items between waves, as necessary.

Patients were asked to complete the electronic daily diary containing the VVSymQ® instrument, and were then asked a series of questions about the instructions, item stem, and response options. Patients were asked about the meaning and relevance of the item, the fit and adequacy of the response scale, the language used, and lack of clarity in terminology or sentence structure [13].

Interviews were audio recorded and transcribed, and quotations from the patients’ responses to each question asked in the cognitive interview guide were summarized to show patient interpretation and understanding of the instructions and of each item and response option, and to identify any difficulties that arose with the proper understanding of the content. Evaluation of these data focused on (1) assessment of understanding of the draft items, and (2) identification of potentially problematic terms or phrases that prevent comprehension of the measure.

6.2 Phase 3 Results

Cognitive debriefing interviews were conducted in four waves with 29 patients. Table 5 shows the demographics and clinical characteristics for this sample.

Table 5 Sociodemographic characteristics of phase 3 cognitive interview patients
Full size table

The results of the phase 3 cognitive interviews indicated that the electronic version of the VVSymQ® instrument was well accepted by the patients and that they generally understood and could use the items and response options as intended. However, some revisions to the recall period were made between the first and second waves. After the first wave of cognitive interviews with 11 patients, the recall period was revised from “In the past 24 hours” to “Since waking today” in all items, based on patients’ reports of not experiencing some symptoms during sleep. The recall period was further revised to “Since waking up today” after wave 2 testing with eight patients, because many patients misread “waking” as “walking.”

The wave 3 and 4 cognitive interviews were each conducted with five patients. Results indicated that version 3 of the electronic daily diary, which included the final five-item VVSymQ® instrument (HASTI™ symptoms assessed on a 6-point scale), was confirmed to be well understood and suitable for use as intended by patients with GSV system incompetence.

After psychometric validation in a quantitative study [14], the five-item VVSymQ® instrument was implemented as part of an electronic daily diary in phase 3 clinical trials of treatment of superficial venous incompetence of the GSV system [15, 16].

7 Discussion

Symptoms are not observable by clinicians and can be evaluated only by patients’ self-report. This research aimed to develop and establish the content validity of a PRO measure relevant to patients with moderate to severe varicose veins that could serve to support a clinical trial endpoint to assess the efficacy of treatments for the symptoms of superficial venous incompetence of the GSV system. The VVSymQ® instrument was developed based on qualitative research with patients with varicose veins, and its development followed FDA guidelines and recommendations for PRO instruments [5].

Concept elicitation focus groups and individual interviews indicated that the five HASTI™ symptoms (heaviness, achiness, swelling, throbbing, and itching) are the most important and relevant symptoms for patients with varicose veins. These five symptoms comprising the VVSymQ® instrument were imbedded in an electronic daily diary that was tested in the final waves of the cognitive interviews. Cognitive testing indicated that the instructions, items, and response options were well understood by patients with varicose veins, the concepts were relevant to patients’ experiences, and patients found the electronic format and functionality of the device easy to use. The appropriate population for this instrument is patients with symptoms. Psychometric results demonstrating the reliability, validity, and responsiveness of the VVSymQ® instrument have been presented [14].

8 Conclusion

Results from this research support the content validity of the VVSymQ® instrument. A combination of focus groups and individual interviews demonstrated that the five items of the VVSymQ® instrument are the most relevant and important to patients and appropriately reflect the patient’s symptom experience. Further, this qualitative work demonstrated that patients understand and can respond to the individual VVSymQ® items.