BACKGROUND/INTRODUCTION

In the last decade, the use of patient-reported outcomes (PROs) has become more widespread and accepted in health care settings for a variety of patient populations and conditions.1,2,3,4,5 It is critical to use PROs to evaluate new health care treatments or ongoing strategies to improve health and well-being from the perspective of the patient; however, measures, to date, have been limited by lack of precision, standardization, and comparability of scores across studies, diseases, and cohorts. Patient-Reported Outcomes Measurement Information System (PROMIS®) provides item banks that offer the potential for efficient (minimizes number of items without compromising reliability), flexible (enables optional use of interchangeable items), and precise (has minimal error in estimate) measurement of commonly studied PROs.6

PROMIS® includes instruments that measure physical and psychological health and quality of life that can be used with cohorts of individuals with a variety of health conditions. It includes item banks that can be administered as computerized tests (CATs) or as paper short forms of various lengths.7 One adaption is the PROMIS-29® Profile, which includes seven domains (physical function, anxiety, depression, fatigue, sleep disturbance, satisfaction with participation in social roles, pain interference) of 4-item short-form assessments and an additional pain intensity assessment using a 0–10 numeric rating scale.7 The PROMIS-29® Profile has been used to measure health-related quality of life in the US general adult population,8, 9 in addition to individuals with a variety of health conditions, for example, neuroendocrine tumors,1 idiopathic pulmonary fibrosis,2 chronic rhinosinusitis,10 systemic sclerosis,11 chiropractic care needs,12 human immunodeficiency virus,13 and chronic musculoskeletal pain.14 Although PROs are vital indicators of health, PROMIS® items are not widely or systematically used within Veterans Health Administration (VA).

The limited information reported on use of PROMIS® items in a Veteran population represents a gap, given that the VA is the largest integrated health care system in the USA, serving over 8.3 million Veterans each year at 152 medical centers and approximately 1400 outpatient clinics and community living centers.15 In recent years, VA has championed the delivery of patient-centered care16; an important approach to evaluating and measuring patient-centered care quality improvement efforts should include the use of PROs, such as the PROMIS® measures, to gather Veterans’ perspectives and feedback. PROs are powerful tools that may be used to inform Veterans and health care providers about their symptoms and perceived state of health and are especially important to understand patient experiences among persons with chronic diseases. Among Veterans, many of whom have multiple morbidities and complex health conditions; such patient-reported insights can guide the development of strategies to enhance care in a manner that aligns with the needs of Veterans.

Recently, there has been some use of select PROMIS® scales and item banks in research studies conducted with condition-specific Veteran samples to measure constructs of physical and mental health and quality of life. Such studies have examined PROMIS® constructs in Veteran cohorts17 with various diagnoses, such as those with a lower limb amputation,18 cancer,19, 20 and chronic pain.21, 22 Yet, examination of the full PROMIS-29® Profile in a general sample of Veteran patients receiving VA health care has not been previously published. The overall study goals were to (1) examine the concordance of electronic medical record clinical diagnoses and PROMIS-29® domain scores (anxiety, depression, fatigue/sleep, and pain) in a sample of US Veterans and (2) compare PROMIS-29® scores of Veterans relative to US population normative scores.

METHODS

Design

Cross-sectional surveys were mailed to Veterans in mid-2014, along with a cover letter explaining the study, and a postage paid return envelope. Approximately 4–6 weeks later, non-respondents received a follow-up mailing with a request to participate to facilitate response, along with another copy of the survey. This multi-site project was conducted as part of a VHA quality improvement effort to evaluate and understand patient-centered care delivery to Veterans across a large integrated health care system.

Sample

Participants included Veterans (n = 3221) from eight geographically dispersed VA health care facilities who had at least one inpatient or outpatient visit in the 6 months prior to the end of FY2012 (April–September 2012). We used SAS 9.3 Proc SurveySelect to conduct systematic random sampling. This method of systematic random sampling selects units at a fixed interval throughout the sampling frame after a random start. For our survey sample, we chose a 10% random sample from the sampling frame of community-dwelling Veterans who utilized health care and had an address in their records.

Data Collection/Sources

The mailed survey was used to collect demographics/characteristics of Veterans, and provided PRO data using the PROMIS-29® Profile v1.0 scales. Diagnostic data on select health conditions (anxiety, depression, sleep disorders, pain disorders) was ascertained using VA administrative databases/patient medical records.

Variables

Demographics/Characteristics

Demographics and Veteran characteristics included the following: gender (male, female); age; race/ethnicity (White, Black/African American, Hispanic, Asian, native Hawaiian/other Pacific Islander, American Indian/Alaska native, and other); highest level of education completed (did not complete elementary school, elementary (grades 1 through 8), some high school (grades 9 through 11), high school graduate (grade 12 or GED), some college or technical school, college graduate (4 years or more)); current relationship status (married, member of an unmarried couple, separated/divorced, widowed, never married); typical living arrangement (live alone; live with family, friend, or spouse/other; live with formal (hired/paid) caregiver; other); and average distance/travel time to the most typically used VA health care facility.

PROMIS-29®

PROMIS-29® Profile v1.0 domains1 were patient self-report and included the following domains. (Note: The PROMIS-29® Profile is available from HealthMeasures at http://www.healthmeasures.net/.)

Physical function was measured with four questions assessing ability to engage in common activities of daily living (e.g., “Are you able to do chores such as vacuuming or yard work?” “Are you able to go for a walk of at least 15 minutes?”). Response options are on a 5-point scale, ranging from 5 (without any difficulty) to 1 (unable to do).

Anxiety was measured with four questions assessing individual’s experiences of common symptoms of anxiety in the prior week (e.g., “I felt fearful,” “My worries overwhelmed me”). Response options are on a 5-point scale, ranging from 1 (never) to 5 (always).

Depression was measured with four questions assessing individual’s experiences of common symptoms of depression in the prior week (e.g., “I felt worthless,” “I felt hopeless”). Response options are on a 5-point scale ranging from 1 (never) to 5 (always).

Fatigue was measured with four questions assessing individual’s experiences of exhaustion in the prior week (e.g., “I feel fatigued,” “How run-down did you feel on average?”). Response options are on a 5-point scale ranging from 1 (not at all) to 5 (very much).

Sleep disturbance was measured with four questions, one asking directly about sleep quality in the prior week (“In the past 7 days. . . My sleep quality was”), with response options on a 5-point scale ranging from 5 (very poor) to 1 (very good), and three questions tapping into both sleep quality (e.g., “My sleep was refreshing”), with response options on a 5-point scale ranging from 5 (not at all) to 1 (very much), and trouble sleeping (e.g., “I had difficulty falling asleep”), with response options on a 5-point scale ranging from 1 (not at all) to 5 (very much) in the prior week.

Satisfaction with participation in social roles was measured with four questions assessing individual’s perceived satisfaction with their ability to perform work-oriented and daily life–oriented activities in the prior week (e.g., “I am satisfied with my ability to work (include work at home),” “I am satisfied with my ability to perform my daily routines”). Response options are on a 5-point scale ranging from 1 (not at all) to 5 (very much).

Pain interference was measured with four questions inquiring about how much pain has gotten in the way of various facets of individual’s lives in the prior week (e.g., “How much did pain interfere with your day to day activities,” “How much did pain interfere with your ability to participate in social activities”). Response options are on a 5-point scale ranging from 1 (not at all) to 5 (very much).

Pain intensity was measured by a single question, asking “In the past 7 days. . . How would you rate your pain on average?” Response options are on a 0 (no pain) to 10 (worst imaginable pain) scale.

Target Health Conditions

Target health conditions (depression, anxiety, sleep disorders, pain disorders) were extracted from patient medical records using VA administrative databases. We sought expert clinical consensus to select diagnoses codes for each condition based on common definitions and descriptions reported in evidence-based literature and professional organization documentation, e.g., American Psychiatric Association’s Diagnostic and Statistical Manual of Mental Disorders (DSM).23 Diagnoses and corresponding ICD-9 codes used to identify individuals who had each health condition are shown in Table 1. Individuals were considered to fall into a diagnosis category if they had 1 or more occurrence of any of the corresponding ICD-9 codes for that diagnosis in the 3 years prior to when surveys were mailed (e.g., from FY2010 through the time of the survey mailing).

Table 1 Diagnoses and Corresponding ICD-9 Codes for Select Health Conditions
Full size table

Analyses

Representativeness

To assess if our survey respondents differed from the overall sample to whom the survey was mailed, we conducted bivariate comparisons of demographic characteristics and select health diagnoses of individuals who were included in the analyses (n = 3321) compared with those who were not (n = 1796). Because self-report data were not available for non-respondents, administrative data extracted from medical records were used to obtain data for comparisons. For example, we used administrative data to obtain the VA facility nearest to the zip code of the non-respondents; specifically, we used a distance function that outputs the geodetic distance in miles between two zip code locations and used the centroid of each zip code in our calculations.

We used descriptive statistics to examine patient demographics and to get a general overview of Veteran’s scores for each PROMIS-29® Profile domain. We also evaluated item response for each PROMIS® domain.

PROMIS-29® Scoring

To score the PROMIS® domains, item responses were added together, multiplied by the total number of questions in the scale, and divided by the number of questions the participant answered. These scores were then converted to T scores based on conversion tables provided by the scale’s developers. PROMIS® scales produce standardized T scores based on population reference values among adults in the US (m = 50, SD = 10); all PROMIS® scores represent a higher level of the construct being measured by each scale. Specifically, the PROMIS-29® Profile produces a separate score for each domain (physical function, anxiety, depression, fatigue, sleep disturbance, satisfaction with participation in social roles, pain interference); all items must be answered and a T score is provided for each. All items for a given domain were presented on the survey as distributed by the developers. If an item response within a domain was missing for any given respondent, the domain was not scored. The response to the single pain intensity item was used in raw form—it is not transformed into a T score.6

We assessed Veterans’ responses to PRO scales (PROMIS-29®) using the scoring algorithms described above, and examined overall frequencies, proportions, means, ranges, and standard deviations, as appropriate. To examine associations of Veterans’ responses to the PROMIS-29® Profile scales with diagnoses in their electronic patient medical records, we conducted bivariate comparisons (independent samples t tests) for Veterans diagnosed with select health conditions and their corresponding mean PROMIS® scale T scores (e.g., PROMIS® depression scores were compared for Veterans with and without depression diagnoses). We also conducted one-sample t tests to compare the means of our Veteran respondents with standardized normative means (standardized to the general adult US population) for each of the 7 PROMIS® domains.

RESULTS

Surveys were mailed to 5512 Veteran patients. The denominator was adjusted to 5117, as 341 surveys were undeliverable and 54 individuals on the initial mailing list had died. Surveys were completed by 3321 Veterans (65% response rate).

Representativeness

There were no significant differences between the Veteran respondents included in the analyses vs. non-respondents on gender, relationship status, distance/travel time to VA health care facility, and health condition–documented diagnoses, including sleep disorders, and pain. Individuals in the sample (vs. non-respondents) were older, on average (67.9 vs. 64.7, p < .0001); a greater proportion were white (70.9 vs. 62.4, p < .0001) and had more education (college graduate) (29.5 vs. 24.0, p < .0001), and fewer had a documented diagnosis of anxiety (25.4 vs. 29.6, p = .0012) and depression (28.6 vs. 34.2, p < .0001).

Sample Demographics/Characteristics

Veteran respondents were mostly male (94.7%) and white (79.3%) and were 68 years of age, on average. All demographics and characteristics are shown in Table 2.

Table 2 Demographics, Veteran Characteristics, Select Health Conditions, and PROMIS-29® Profile Scores (n = 3321)
Full size table

Associations of Select PROs and Administrative Data (ICD-9 in Medical Record) (Table 3)

Using ICD-9 codes, of our Veteran sample, 28.6% had a depression diagnosis, 25.4% had diagnosed anxiety, 8.2% had a pain disorder, and 24.8% had a sleep disorder diagnosis noted in their medical record. Bivariate comparisons showed that Veterans with depression reported significantly higher PROMIS® depression scores, on average, compared with Veterans without depression (60.3 vs. 49.6, p < .0001). Likewise, Veterans with anxiety reported significantly higher average PROMIS® anxiety scores than Veterans without anxiety (62.7 vs. 50.9, p < .0001). Further, Veterans with a pain disorder reported significantly higher pain interference (65.3 vs. 57.7, p < .0001) and pain intensity (6.4 vs. 4.4, p < .0001) scores (vs. Veterans without a pain disorder). Veterans with a sleep disorder (vs. Veterans without a sleep disorder) reported significantly higher sleep disturbance (55.8 vs. 51.2, p < .0001) and fatigue (57.5 vs. 51.8, p < .0001) PROMIS® domain scores.

Table 3 Select PROMIS-29® Domains vs. ICD-9 Diagnoses Recorded in VA Patient Administrative Data (n = 3321)
Full size table

PROMIS-29® Scores for Veterans (Relative to Population Reference Values)

Overall PROMIS-29® Profile domain scores for Veterans were as follows: physical function, 41.08 (range = 22.90–56.90, SD = 10.05); anxiety, 53.86 (range = 40.30–81.60, SD = 11.56); depression, 52.65 (range = 41.00–79.40, SD = 11.07); fatigue, 53.22 (range = 33.70–75.80, SD = 11.16); sleep disturbance, 52.31 (range = 32.00–73.30, SD = 9.45); satisfaction with participation in social roles, 44.95 (range = 29.00–64.10, SD = 10.25); pain interference, 58.31 (range = 41.60–75.60, SD = 10.25) (Table 4). Pain intensity was scored separately as it is on a different scale (4.57 (SD = 2.83)) and is shown as a footnote in Table 4. Relative to general population norms, Veterans reported worse scores across all PROMIS-29® domains. More specifically, anxiety, depression, fatigue, sleep disturbance, and pain interference scores were all higher (worse) than general population normative values. In fact, the largest difference in Veteran and normative scores was for pain interference. Veteran scores for physical function and satisfaction with social role were lower (worse) than US general population norms. Item response for each of the seven PROMIS-29® domains was 93% or better (Table 4).

Table 4 Mean PROMIS-29® Profile Domain Scores (Pain Intensity Scored Separately. Pain Intensity (n = 3226), Mean = 4.57, Standard Deviation = 2.83): Veterans vs. Population Norms (n = 3321)
Full size table

DISCUSSION

As health care providers embrace the concept of considering patient reports of their conditions, in tandem with outcomes that are clinically documented, an increased emphasis on PROs is needed. The present study provides PROMIS® domain outcomes for a Veteran population, examines the associations between PROMIS® scores, and documented medical record diagnosis, and then the PROMIS® item measurements are referenced to the general population, offering the VA a way to compare the health status of Veterans with the adult population at large. Our findings provide data for Veterans using a common, accessible metric(s) of health (physical, mental, and social), functioning, and quality of life measures that are reported directly by patients. These metrics are applicable across many health populations; thus, having these measures available for Veterans provides comparability and findings that are interpretable across research studies and clinical settings. This may facilitate the representation of Veteran cohorts in this era of patient-centered outcome research and comparativeness effectiveness research and is an early step toward clinical practice application of Veteran PRO data.

Concordance of PROMIS-29® Domains and Medical Record Diagnosis in a US Veteran Cohort

In our work, Veterans with a clinical diagnosis of depression and anxiety reported significantly higher PROMIS® depression and anxiety scores, respectively. Those with a clinical diagnosis indicating a pain disorder reported higher PROMIS®-measured pain interference and pain intensity. Likewise, Veterans with a clinically documented sleep disorder reported higher sleep disturbance and fatigue PROMIS® domain scores.

In general, a 5-point difference in scores on PROMIS® measures is considered clinically significant. In addition, literature suggests that a difference of half of a standard deviation on patient-reported outcome measures represents a clinically significant difference.24 The difference in the reported scores between Veterans in the clinical diagnosis and those in the no clinical diagnosis groups on all PROMIS® measures meets either one or both of these criteria. Because of this, we consider these differences to be clinically meaningful.

Our PROMIS-29® Veteran measures may be compared with other cohorts as well. For example, Cella et al.25 found higher PROMIS® fatigue in six clinical samples (chronic obstructive pulmonary disorder (COPD), congestive heart failure (CHF), major depressive disorder (MDD), cancer, back pain, and rheumatoid arthritis (RA)) than that exhibited by the general US population. In our study, the PROMIS® fatigue score for the general Veteran sample (53.22) was lower than that for all six clinical samples, but the score of Veterans with clinically diagnosed fatigue (57.54) was higher than Cella’s study samples with back pain, cancer, RA, and stable COPD.25 This suggests that our Veteran sample is a general sample and includes people with a variety of chronic conditions and clinical concerns.

Similarly, Schalet et al.26 found that a US population sample of individuals diagnosed with MDD had a PROMIS depression T score of 61.9. This is close to the T score of 60.3 in our Veteran sample having a clinically documented depression diagnosis. Schalet and colleagues26 also found anxiety T scores of 61.7 in persons with MDD. In comparison, Veterans with anxiety-related diagnoses had anxiety T scores of 62.65. There is likely significant overlap (rendering similar scores) in anxiety and depression symptoms and diagnoses, as is supported by high correlation between PROMIS-29® anxiety and depression scores (r = 0.86) in our Veteran sample (data not shown).

The mean differences in PROMIS® scale scores measuring constructs congruent with select clinical diagnoses suggest that PROs may be an appropriate indicator of condition symptomology among Veterans. The directional pattern of T scores with clinical diagnoses in the Veteran sample and in the literature suggests that PROMIS® scores provide clinically and practically meaningful metrics. Collectively, these results indicate that the PROMIS-29® Profile is appropriate for use in a Veteran population receiving VA health care, suggesting clinical validity of PROMIS® domains across multiple metrics. However, future research to examine domain scores across demographic variables, such as age group and gender, would provide additional insight and is warranted.

PROMIS-29® Scores for Veterans Relative to Population Reference Values

PROMIS-29® domain scores were significantly lower (worse) than population norms in the areas of physical function and satisfaction with social role, while Veterans reported significantly higher (worse) scores for anxiety, depression, fatigue, sleep disturbance, and most notably, pain interference. Across the board, our data indicate that Veterans utilizing VA health care services are faring poorly compared with the US general population on all PROMIS-29 domains.

Limitations

The data presented in this paper are self-reported and collected via a survey and are subject to recall and response bias. Our sample is limited in that it included only 5.31% women, which may limit generalizability to all Veterans. However, we removed women from the sample and conducted sensitivity analyses (data not shown), on all study data, including comparisons of (1) PROMIS-29® domains vs. ICD-9 documented diagnoses and (2) mean PROMIS-29® Profile domain scores of Veterans vs. population norms and found comparable results, with no differences in magnitude or significance levels. Additionally, the sample was comprised of patients receiving VA health, which may limit generalizability of the results (e.g., associations between clinical diagnoses and PROs) beyond large integrated health care systems and to Veterans who receive non-VA health care. We noted some differences between the sample and non-respondents as well that could limit the generalizability of the results overall. We included the PTSD ICD-9 code with the anxiety conditions; this is an area of ongoing debate among DSM-5 users and experts 27,28 and may be a limitation in that inclusion yields a higher number of anxiety category diagnostic “cases.”

This study contributes to the literature in a few key ways. First, we presented PROMIS-29® data on a sizable general Veteran cohort; to our knowledge, this is the first published account of these data in the US Veteran population. We also found that the domains in the PROMIS-29® are selectively sensitive to expected differences between clinically defined groups, such as those with pain disorders and mental health conditions ranging from depression and anxiety to sleep disorders. Our findings suggest that use of PROMIS-29® provided an accurate PRO assessment of symptoms/conditions among US Veterans, as compared with clinical documentation of diagnoses in electronic medical records. Finally, our findings show how this Veteran cohort compares with population reference values across the PROMIS-29® domains.

Conclusions

PROMIS® items provide measurements normed to the general population, offering the VA a way to easily compare the health status of Veterans with the adult population at large. PROs may be a useful tool for VA providers to assess patient’s physical and mental health, as is indicated by the positive associations between PRO scores and clinical documentation, suggesting clinical validity for targeted measures. Veterans, in general, scored worse across all PROMIS® domains compared with the US population normative values, suggesting additional work is needed to develop and implement potential interventions for VA patients in these areas.