Breast cancer treatment is the most common cause of upper extremity lymphedema in Western countries.1 Risk factors for the development of arm lymphedema are axillary lymph node dissection (ALND), sentinel lymph node biopsy (SLNB), and radiation therapy of the axilla, or a combination of these therapeutic methods.

The overall incidence of breast cancer-related arm lymphedema has ranged between 14 and 21%.1,2,3 Findings from a prospective cohort study of 2171 women investigating time course and incidence of breast cancer-related lymphedema identified that patients receiving ALND with radiation therapy were at a greater risk for the development of lymphedema (31.2 %) than those with ALND alone (24.6 %) or with SLNB plus radiation therapy (12.2 %).2 Furthermore, early onset of lymphedema at less than 12 months postoperatively was associated with having ALND, whereas onset at 12 months or later was associated with having radiation therapy.2

Arm lymphedema is a debilitating diagnosis that may impose a significant detriment to a patient’s health-related quality of life (HRQOL) due to symptoms (e.g., swelling, pain, infection) and reduced arm function.4,5 The rapidly emerging field of lymphedema research has had little consensus on the most suitable metric for measuring outcomes. Clinicians commonly use limb volume and circumference, but these metrics do not capture the HRQOL burden of arm lymphedema from the patient perspective.6,7

To better understand and measure outcomes that matter to patients with arm lymphedema, a valid and reliable patient-reported outcome measure (PROM) is needed. Given that arm lymphedema affects how patients function and feel, information captured through the use of a PROM may be a better indicator of disease than traditional clinical metrics.6

Currently, 14 lymphedema-specific PROMs have been used to measure upper extremity lymphedema outcomes from the patient perspective.8 A systematic literature review identified that 13 of the 14 PROMs were developed with limited input from patients.8 Patient involvement in the development of a PROM is considered crucial to ensuring that its content is comprehensive and relevant to patients (i.e., content validity).9,10 The systematic literature review also identified that the quality of each PROM was low to moderate in terms of meeting the criteria for reliability and validity as set out in the COnsensus‐based Standards for the selection of health Measurement Instruments (COSMIN).11

To address the shortcomings in existing PROMs for upper extremity lymphedema, our team developed the LYMPH-Q Upper Extremity Module. This PROM was developed to complement the BREAST-Q that we previously developed to measure HRQOL and patient satisfaction among women with breast surgery.12 This study aimed to describe the development and psychometric validation of the LYMPH-Q Upper Extremity Module.

Methods

Best practice guidelines for the development of a PROM were used to guide this multi-phased mixed-methods study.10,13,14,15,16,17 Phase 1 involved qualitative patient interviews to elicit concepts. Interpretive description was used to inform the qualitative approach.18 Subsequently, cognitive interviews with patients and expert input were used to refine the new scales content. In phase 2 (quantitative), a field-test was performed and Rasch measurement theory (RMT)19,20 analysis was used for item reduction and to examine the psychometric properties of each scale.

Research Ethics

For phase 1, approval was obtained from the Research Ethics Boards at McMaster University (Hamilton, ON, Canada), Toronto General Hospital (TGH) (Toronto, ON, Canada), Memorial Sloan Kettering Cancer Center (MSK) (New York, NY, USA), and Brigham and Women’s Hospital (BWH) (Boston, MA, USA). In Denmark, the study was reported to and approved by the Region of Southern Denmark and included on the list of Health Research for data protection safety.

Written consent was obtained from all the participants before each qualitative and cognitive interview. The participants in Canada and the United States were sent a $50 (CAD, USD) gift card to thank them for their participation.

For phase 2, in the United States, approval was obtained from BWH and the Scientific Advisory Committee of the Love Research Army (LRA; formerly known as the Army of Women), an online non-profit community started by the Dr. Susan Love Research Foundation in 2008 that connects breast cancer researchers to women with and without breast cancer.21 An email describing the study aims was sent to LRA members. Completion of the study questionnaire implied consent.

For Denmark, phase 2 of the study was approved by the Region of Southern Denmark and included on the list of Health Research for data protection safety. Ethics approval from the Regional Committee on Health Research Ethics was not required because the study involved completion of a questionnaire. An email invitation was sent to the electronic secure mailbox (Eboks) of potential participants. Informed consent to take part in the study was obtained electronically in REDCap.

Phase 1: Qualitative Interview

Sample and Recruitment

Women who were 18 years of age or older with a breast cancer diagnosis and fluent in English were invited to participate in a qualitative interview as part of a larger study to develop new scales for the BREAST-Q. A purposive sampling approach was used to ensure that participants varied by age and breast cancer stage (stages 0–4), as well as by surgical (i.e., breast-conserving therapy, mastectomy with/without reconstruction) and nonsurgical (i.e., adjuvant or neoadjuvant) breast cancer treatments. Data from the subset of women in the sample who reported arm lymphedema were used to develop the LYMPH-Q Upper Extremity Module.

Health care professionals described the study to potential participants in clinics or by telephone. Permission was obtained to share contact information with the research team. Interviews were scheduled and took place by phone or face-to-face at a time that was convenient to each participant.

Concept Elicitation

Interviews were performed by experienced qualitative researchers who followed a semi-structured interview guide with open-ended questions. The participants were asked to discuss how lymphedema and its treatment influenced their physical, psychological, and social well-being, as well as their overall HRQOL. The interviews were audio-recorded and transcribed verbatim.

To establish rigor, data collection and analysis took place concurrently so that new concepts elicited from participants could be added to the interview guide. Furthermore, the interviews were coded by two coders independently, and regular team meetings were used to review coding. Multiple levels of codes (top-level domains and themes) were applied to the text. Codes were created inductively through the generation of new codes and deductively through the application of relevant codes from the BREAST-Q conceptual framework.12 Recruitment continued until redundancy of concepts elicited through the interviews was achieved.

Participant quotes and associated codes were transferred from Microsoft Office Word to Excel for further refinement of themes and subthemes using constant comparison. In Excel, an item pool was developed for use in scale development. Scales covered key concepts elicited from the participants. Each scale was given instructions, a time frame for answering, and a set of response options.

Scale Development and Refinement

Patient and expert input was used to establish content validity of the LYMPH-Q Upper Extremity Module. A semi-structured cognitive interview guide was used with questions and probes to determine whether the content of each scale (i.e., instructions, recall period, item set, response options) was comprehensive, relevant, and comprehensible.15 Participants were asked to suggest missing concepts. Women with breast cancer-related lymphedema from Canada, the United States, and Denmark 18 years of age or older who could speak and read in English or Danish, were invited to participate in the cognitive interviews, which used the think-aloud approach.22,23,24 Interviews and analyses were performed by skilled qualitative researchers and took place in rounds to enable the refinement of scale content between rounds.

Experts known to our team who treat patients with arm lymphedema were invited to provide feedback on the comprehensiveness and relevance of the LYMPH-Q Upper Extremity Module content. An email invitation was sent by a member of the research team with an attached PDF copy of the lymphedema scales. A reminder email was sent after 1-week. Experts were asked to provide written feedback via email and to add missing concepts. The input from the experts was analyzed descriptively by two researchers with the results used to refine the scales.

Elsewhere we describe the methods and results of a linguistic validation study to translate the LYMPH-Q Upper Extremity Module into Danish.25 The scales were translated into Danish according to the International Society for Pharmacoeconomics and Outcomes Research26 and World Health Organization27 guidelines. Feedback from patients and experts provided additional evidence of the scales’ content validity.

Phase 2: Field-Test Study

Sample and Recruitment

The analysis included data from two samples as follows:

  1. 1.

    Love Research Army

The LRA study was performed as part of a larger study to develop new scales for the BREAST-Q. The study was open to women 18 years of age or older with a diagnosis of breast cancer who could read English. The LRA members were sent an electronic recruitment email (e-blast) containing a description of the study and the eligibility criteria. Women who agreed to participate were directed to a REDCap survey28 designed by our team and hosted at BWH.

The REDCap survey included demographic and clinical questions, new BREAST-Q scales and the LYMPH-Q Upper Extremity Module. Targeted clinical questions and branching logic were used to ensure that only women with arm lymphedema completed the LYMPH-Q Upper Extremity scales. The LRA participants were invited to take part in a test-retest (TRT) study. Those who provided their email were automatically sent a URL link 3 weeks after the initial survey, with one reminder sent after 2 days.

  1. 2.

    Danish National Health Data Authority

In Denmark, a list of all patients 18 years of age or older with a diagnosis of both breast cancer and arm lymphedema in the past 12 years was obtained from the Danish National Health Data Authority. An invitation with written information about the study and a REDCap public link to the survey was sent to patients’ Eboks. The REDCap database was hosted by the Open Patient Data Explorative Network.29 Patients were invited to use the URL provided to access and complete clinical and demographic questions as well as the LYMPH-Q Upper Extremity Module. Two reminders were sent 7 and 14 days after the initial invitation.

Data Analysis

In this study, RMT analysis was performed using RUMM2030 software and the unrestricted Rasch model for polytomous data (RUMM version 2030; RUMM Laboratory Pty Ltd., Duncraig, Western Australia, 1998-14). The RMT analysis involved a series of diagnostic tests, described in detail elsewhere.30 Briefly, a set of statistical and graphic tests, were used to identify items and scales that did not work as hypothesized.19 Scales that work have a set of items that line up to map out a single continuum. RMT analysis uses the chi-square statistic to examine both item fit and the overall model fit. Because this test is highly sensitive to sample size, we adjusted the sample to 500. We also applied Bonferroni corrections to account for multiple comparisons.

Ordering of Item Thresholds

Threshold maps were examined to determine whether response options (e.g., very dissatisfied, somewhat dissatisfied, somewhat satisfied, very satisfied) were used appropriately. Disordered thresholds indicate problems in comprehension, or that the response options do not work as intended.

Item Fit

We examined individual item fit and overall fit of the data to the Rasch model.31,32 Indicators of fit were inspected and interpreted together. Item fit was evaluated statistically by whether fit residuals were within – 2.5 and + 2.5 and had nonsignificant chi-square values after Bonferroni adjustment. Fit residuals also were inspected graphically to determine whether item characteristic curves showed agreement between observed and expected scores.31,32

Local Dependency

We examined residual correlations to identify their influence on the Person Separation Index (PSI, reliability). Any pairs of items with a residual correlation of 0.30 or higher were included in a subtest to determine their impact on scale reliability.33,34,35

Targeting

Scales should measure the construct as experienced by the sample. We examined graphic displays (person-item threshold distributions) of item and person spread to determine whether these overlapped. We also computed the proportion of the sample that scored outside the range of measurement.

Differential Item Function (DIF)

We examined whether subgroups in the sample responded differently to items in a scale despite having similar level of the construct measured. DIF was examined for dataset (USA and Danish) and age group (18 to 49, 50 to 59, 60 to 69, ≥ 70 years). For each variable, we performed DIF three times with random samples drawn to match the smallest subgroup. The DIF then was performed with and without adjustment of the overall sample in the analysis to 500. Any items that evidenced significant DIF in the unadjusted analysis were split on the sample characteristic. The person locations based on the original and split analyses were correlated to determine whether the DIF had any impact on scale scoring.32

Reliability

The PSI and Cronbach alpha36 reliability coefficients were determined within RUMM2030. Values of 0.70 or higher were considered acceptable.37

From the final models for each scale, the Rasch scores were obtained and transformed from 0 (worst) to 100 (best). The following statistical tests were performed in SPSS version 26.0 (IBM Corporation, Armonk NY, USA for Windows/Apple Mac).

1. Test-Retest Reliability. We used the transformed scores of 0 to 100 to examine test-retest reliability. The participants were asked if anything had changed with their health or in their life since they completed the questionnaire (response options: yes, no). Those who said yes were excluded from the TRT analyses. We computed two-way random Intraclass Correlation Coefficients (ICCs) with the test-retest data. ICC values greater than 0.70 were considered acceptable.38

2. Data Quality. We examined scale-level missing data and the proportion of patients to score at the floor and ceiling.

3. Construct Validity. We examined the normality of the data by examining kurtosis and skewness.39 Data that exceeded ± 2.0 were examined using nonparametric statistics.39 We tested the following hypotheses. First, we expected that correlations between the scales measuring similar, related but dissimilar, and unrelated constructs would meet the COSMIN guidelines for construct validity (i.e., correlations should be ≥0.50 for similar constructs, 0.30 to 0.50 for related but dissimilar constructs, and < 0.30 for unrelated constructs).11 Second, we expected that participants’ scale scores would be incrementally associated with severity (none, mild, moderate, severe) of self-reported arm swelling. Third, we expected that participants’ scale scores would be incrementally associated with having a self-reported arm problem (none, minor, major) as a result of breast cancer, treatment, or both. Finally, we expected that scale scores would be lower for women who reported that they wore a compression sleeve in the past 12 months compared with those who did not.

Results

Phase 1: Qualitative Phase

Data collection took place between January 2017 and June 2018. Qualitative interviews were performed with 58 patients as part of the larger BREAST-Q study. Data from 15 participants with arm lymphedema were used to develop the LYMPH-Q Upper Extremity Module scales. Those with arm lymphedema were mainly 40 to 74 years of age. The participants included 13 white patients and 10 married patients. Most of the participants had a mastectomy (n = 10) and a history of combination treatment with chemotherapy, radiotherapy, or endocrine therapy (n = 7).

Analysis resulted in the development of a framework of concepts important in arm lymphedema. The framework included top-level domains with two or more of the following major themes: arm appearance (body image, characteristic, clothing), physical (function, symptoms), psychological (distress, impact), social (support, function, relationships), and experience of care (lymphedema information, arm sleeve). The item pool was used to develop content for five LYMPH-Q Upper Extremity Module scales as follows: function, symptoms, appearance, life impact, and information. Each scale was assigned instructions, a time frame for responding, and four response options that measured severity (symptoms, life impact), bother (appearance), difficulty (function), and satisfaction (information).

To establish content validity further, we performed 16 cognitive interviews of patients with breast cancer who had arm lymphedema. Interviews took place in three rounds between October and December 2018. Round 1 included two U.S. participants; round 2 included 10 Danish participants; and round 3 included four U.S. participants. The sample included women 38 to 74 years of age who were mainly white (n = 16) and married (n = 11). Most of the participants had a mastectomy (n = 10), ALND (n = 14) and a history of a combination of chemotherapy, radiotherapy, and endocrine therapy (n = 11).

Feedback was obtained from 12 of 22 invited multi-disciplinary experts after round 2 (response rate, 55 %). The experts represented four countries (Canada, Denmark, Poland, United Kingdom) and included eight plastic surgeons, two breast surgeons, a medical oncologist, and a nurse practitioner.

In round 1, the participants reviewed 57 items in five scales (symptoms, function, appearance, life impact, and information). Two new scales (psychological, arm sleeves) were added after round 1 participant feedback. The psychological and arm sleeve scales measured whether lymphedema affects how participants feel (response options: always, often, sometimes, never) and satisfaction with the arm sleeve (response options: very dissatisfied, somewhat dissatisfied, somewhat satisfied, very satisfied), respectively. The final set of items was tested in round 3, which resulted in a total of 110 items in the following scales finalized for the field-test: symptoms (n = 20 items), function (n = 19 items), appearance (n = 14 items), life impact (n =11 items), psychological (n = 19 items), information (n = 13 items), and arm sleeve (n = 14 items).

The LYMPH-Q Upper Extremity Module was translated into Danish.25 The scales were reviewed by an expert panel consisting of the primary investigator leading the translation process, two professional bilingual translators, a breast surgeon/plastic surgeon, a physiotherapist specialized in lymphedema treatment, a medical doctor specialized in lymphedema research, and a medical doctor specialized in PROM research. This was followed by cognitive debriefing interviews with 10 women who had arm lymphedema. The feedback received by the patients and the expert panel confirmed that the scales were comprehensive and comprehensible and included highly relevant questions.

Phase 2: Quantitative Phase

Field-test data were collected between October 2019 and January 2020. A total of 1717 LRA members opened the REDCap link and self-selected themselves to be eligible for the study. Among these members, 364 had a diagnosis of lymphedema and completed at least one of the lymphedema scales. Of the 364 participants, 79 also provided data for the TRT.

In Denmark, 8139 women with breast cancer and arm lymphedema were identified. Of these women, 6850 used Eboks and were invited to participate. Responses were obtained from 3945 women (57.6 %). Of these women, 1087 were excluded from the study (426 declined to participate, 298 did not have lymphedema, 363 completed only the clinical/demographic information). After these exclusions, 2858 Danish participants were included in the analysis. Sample characteristics for the combined sample of 3222 participants are presented in Table 1.

Table 1 Characteristics of the 3222 participants in the field-test sample
Full size table

The RMT analysis led to a reduction of items from 110 to 68. Items were dropped due to either poor fit to the Rasch model or redundant content. All 68 items had properly ordered thresholds (Appendix 1) and nonsignificant chi-square p values after Bonferroni adjustment. Data fit the Rasch model for all six scales, with nonsignificant p values (Appendix 2). Item fit was within ± 2.5 for 27 of the 68 items. Scale level findings for the six scales that formed the item-reduced version of the LYMPH-Q Upper Extremity Module are shown in Appendix 2, and item fit statistics are shown in Appendix 3. The proportion of the sample that scored on each scale is shown in Appendix 2. All but one scale (psychological) had at least 80 % of participants’ scores within the scale’s measurement. Targeting can be seen graphically in Appendix 1, which shows the person measurement and item locations for each scale.

Differential item function was evident for 31 items in the unadjusted analysis that compared the Danish and U.S. datasets and for 14 items by age group (Appendix 3). In the adjusted analysis, DIF was evident for 26 items by dataset and for 3 items by age group (Appendix 3). When the items that evidenced DIF in the unadjusted analysis were split by the relevant participant characteristics, Spearman correlations between the original and split-person locations indicated that DIF had a negligible impact (r ≥ 0.991 for all correlations).

The PSI values were 0.80 or higher (with and without extremes), and the Cronbach alpha values were 0.89 or higher (with and without extremes) (Appendix 2). One pair of items in each of the symptoms (swelling, heavy), function (hold phone, hold book), and appearance (photos, noticeable) scales had residuals that correlated greater than 0.30. A subtest performed on these three item pairs showed the impact on the PSI values to be marginal, with a maximum drop in PSI of less than 0.01 with and without extremes.

Test-retest data were provided by 79 of the participants. Five of the participants reported a change in their health or life since completing the scales and were excluded. Appendix 2 shows the ICC values with 95 % confidence intervals. The ICC values for the six scales was 0.92 or higher. The scale-level missing data value was low (≤ 1.4 %, see Appendix 2). Floor effects were low (≤ 4.3 %), and ceiling effects ranged from 4.1 % (symptoms) to 22.7 % (psychological) (Appendix 2). The mean grade reading levels for the items in each scale were between 2.5 (symptoms, sleeve) and 15.6 (psychological), and the grade reading levels for the instructions ranged from 3.7 (psychological) to 14.1 (information).

Table 2 shows the Pearson correlations between the LYMPH-Q Upper Extremity Module scales. As hypothesized, the correlations between the scores on the four outcome scales were stronger with each other than with the two satisfaction scales. The correlations between the four outcome scales all met the level of >0.50 for related measures. The only correlations not in accordance with our hypothesized values, as per the COSMIN guidelines for construct validity, were the correlations between the arm sleeves scale and the symptoms, function, and psychological scales, which were higher than predicted.

Table 2 Pearson correlations between the LYMPH-Q upper extremity module scales
Full size table

Consistent with our hypotheses, increased severity of arm swelling (Fig. 1), reporting of an arm problem caused by cancer or cancer treatments (Fig. 2), and wearing of a compression sleeve to reduce or prevent swelling in the past 12 months (Fig. 3) all were meaningfully associated with worse outcomes in all six LYMPH-Q Upper Extremity Module scales. Differences between scale scores by subgroups were statistically significant (p < 0.001) for all the scales in these three hypotheses. The characteristics of the subgroups for these tests of construct validity can be found in Appendix 4a–c.

Fig. 1
figure 1

Mean scores for LYMPH-Q scales based on self-reported severity of arm swelling

Full size image
Fig. 2
figure 2

Mean scores for LYMPH-Q scales based on having a problem with the arm(s) as a result of breast cancer and/or its treatment

Full size image
Fig. 3
figure 3

Mean scores for LYMPH-Q scales based on whether the participant wore a compression sleeve in the past 12 months

Full size image

Discussion

The LYMPH-Q Upper Extremity Module provides clinicians and researchers with a rigorously developed PROM that can be used to measure outcomes of breast cancer-related arm lymphedema. Given the high prevalence of arm lymphedema and its significant impact on HRQOL, this new PROM represents an important addition to the literature.

The lack of an upper extremity lymphedema PROM developed with patient input has impeded advancements in the field of lymphedema research and treatment. Whereas an increasing number of studies have used HRQOL as the primary outcome in lymphedema research,7,40,41,42 recent literature reviews by Coriddi et al.42 and Beelen et al.8 have highlighted the frequent use of ad hoc instruments and generic PROMs to assess HRQOL. Ad hoc questionnaires are surveys, often composed for a specific study, with unknown psychometric properties. Generic PROMs are those designed for use with any patient population, which therefore do not ask about lymphedema-specific concerns. Although generic PROMs can facilitate comparison of outcomes across disease groups, such PROMs may not detect clinically important change after treatment for specific patient groups.43,44

For patients with lymphedema, no single objective measure adequately reflects the totality of a patient’s disability. Limb volume is a commonly used measure, but it can fluctuate throughout the day and can be manipulated with physiotherapy. In addition, this measurement does not account for other important concerns, such as recurrent cellulitis, physical disability, and psychological distress. Patient-reported outcomes in lymphedema used in conjunction with objective measures would provide a more complete picture of whether a therapeutic intervention has helped or not.

Due to the complex nature of lymphedema, greater fidelity in patient-reported outcome measurement is needed, which prompted the development of the LYMPH-Q Upper Extremity Module. The LYMPH-Q Upper Extremity Module was developed using a modern psychometric approach. Patient input ensured that the concepts most important to patients with lymphedema were identified and used to form the scales. The use of RMT analysis ensured that the scales provide interval-level measurement and are well-suited for use in individual patient care settings. The module can be used to evaluate the impact of new medical and surgical interventions for arm lymphedema, such as lymphovenous anastomosis and vascularized lymph node transplantation.

When choosing a PROM, high content validity, largely established through qualitative input from patients who have the condition of interest, is vital to measurement of change after an intervention. An important strength of our study was the careful qualitative research performed to ensure that the LYMPH-Q Upper Extremity Module had content validity for patients in three countries and was validated in two languages. Furthermore, evidence from the field-test study showed that the LYMPH-Q Upper Extremity Module worked the same by language and by age in the DIF analysis. These findings are important because they mean that the LYMPH-Q Upper Extremity Module provides a common metric with comparable scoring that can help to facilitate international research in lymphedema treatments.

Our study had several limitations. The initial qualitative sample did not include Danish participants, and the field-test sample did not include Canadians. However, for the Danish participants, we were able to ensure that the scales had content validity by including 10 women with arm lymphedema in the cognitive interviews performed to refine the scales and 10 additional women in the review of the Danish translation. Future research is needed to test the scales in a Canadian population.

To collect a large sample of data, we used an online survey, which can provide a large sample quickly at a low cost. Online surveys, however, do not reach participants with no Internet access and those who have access but are not active online. The majority of our participants were Danish and white, which limits applicability. Further validation studies could include a more diverse sample recruited from other countries.

Finally, we discovered a problem with the branching logic in the longitudinal setup within LRA REDCap for the test-retest reliability portion of the study. As a result, the LYMPH-Q Upper Extremity Module was sent 3 weeks after treatment instead of the planned 1 week after treatment. However, we believe 3 weeks still are a valid time for assessment of test-retest reliability.45 Because our research used a cross-sectional study design, testing the responsiveness of the scales was beyond the scope of this study. Future research is needed to examine the ability of the LYMPH-Q Upper Extremity Module scales to measure change and establish a minimal important difference.

Conclusion

The LYMPH-Q Upper Extremity Module was created through a rigorous development process with an emphasis on qualitative input from patients and experts. It addresses an unmet need in the literature by providing a PROM for use in upper extremity lymphedema care and outcomes research with strong content and construct validity.