INTRODUCTION

Providing high-quality health care is central to our nation’s commitment to veterans. The Veterans Affairs health care system (VA), the nation’s largest integrated health care system, provides comprehensive health services to US military veterans who are enrolled. However, the quality of VA care is a longstanding area of concern.1 , 2 As a result, the Veterans Access, Choice and Accountability Act (VACAA), passed in 2014, mandated an independent assessment of health care capabilities and resources of the Veterans Health Administration.3 As part of this assessment, the Interim Under Secretary for Health for VA called for a comprehensive evaluation of “VA’s ability to deliver high-quality health care to Veterans.”4

Previous studies have systematically reviewed aspects of care at the VA.5 – 7 Two peer-reviewed articles based on one of these7 summarized the available evidence on medical8 and surgical9 care quality through 2009. This review indicated that in most studies, the care provided in the VA compares favorably to non-VA systems. However, quality may have changed in the last 6 years since this review, and concerns about quality of care have continued to mount.10 To assess the quality of care provided by the VA, we performed a systematic review of published comparisons of the quality of care in VA facilities to other settings.

METHODS

Search Strategy

We defined quality of care using the Institute of Medicine’s (IOM) definition, which is “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.”11 Given the existence of a recent systematic review on the quality of health care delivered in VA versus comparable non-VA settings, we chose to explicitly build upon and expand this work.7 We use consistent methods, including the same search terms (Online Appendix) and database (MEDLINE). The dates searched were limited to the past 10 years (January 1, 2005, to January 1, 2015) to ensure a reasonable scope and up-to-date studies.

Article Screening and Data Abstraction

Titles and abstracts identified by our literature search were screened by two reviewers with experience screening and abstracting data for health-related systematic reviews (CO, CH) under the supervision of a senior member of the study team with extensive clinical and systematic review experience (CG) for inclusion in full-text screening. If the article presented comparisons of quality of health care in the VA and non-VA settings in the US, the full text of each article was reviewed independently by both reviewers (CO, CH). We extracted the data sources, geographical areas, clinical conditions, quality dimensions, and comparability of measures in each sample. Studies were excluded if they included no original data; were case reports; or used non-contemporaneous comparisons, unequal or non-representative samples, or dissimilar or indirect quality measures. Data from each article, including the size of the VA and non-VA samples, years of data collection, control variables, and primary and secondary outcomes, were abstracted by one reviewer and double-checked by another reviewer (CO, CH). Disagreements were resolved by a senior team member (CG).

Quality Assessment

Article quality was assessed using the criteria developed in the original review for assessing comparative health care quality studies.7 Briefly, six elements were used, including whether: (1) time periods during which the VA and non-VA patients were studied were sufficiently contemporaneous; (2) VA and non-VA samples were comparable in size/scope; (3) quality measurements were assessed using specified and identical indicators with a similar assessment format; (4) quality measures were meaningfully associated with outcomes; (5) measures were clinically relevant and important; and (6) the statistical methodology was sound. Each factor was graded (A for excellent, B for acceptable, C for unacceptable). The overall subjective assessment of the article was created, but this assessment was not an “average” of individual component grades. Thus, an article with a critical flaw could be graded “C” and rejected, even if it scored well on other criteria. Articles were graded by one reviewer and double-checked by the other reviewer (CO, CH). Disagreements were resolved in consultation with the senior team member (CG).

Study Classification

Although we employed a nearly identical search and abstraction strategy, we organized article categories differently than the original review,7 which split articles into medical8 (non-surgical) and surgical9 care and by Donabedian’s quality triad of structure, process, and outcome.12 To guide decision making about how to improve quality at the VA, we organized our review using the IOM’s six characteristics of high-quality care: safe, timely, equitable, effective, efficient, and patient-centered,11 and then grouped similar kinds of outcomes together. Timeliness as related to delays or wait times for receiving necessary medical care (e.g., time from admission to emergency surgery) was included in this review. Timeliness as it relates solely to access (e.g., wait time for obtaining an appointment)—while a critical issue currently facing the VA13—was not addressed because no studies directly comparing VA and non-VA settings were identified.14 We included articles on efficiency if they assessed appropriateness (necessary care versus overuse). We excluded cost-efficiency articles, as cost comparisons between VA and other settings have a number of limitations.15

We classified each study (both new and from the prior review) according to the direction of the statistically significant differences in performance measures for the VA care relative to a non-VA comparison group. If VA quality of care was shown to be better than non-VA care or if multiple results were reported and VA quality of care was better in some instances and the same in others, the study was classified as “VA better.” If multiple quality measures were reported and VA care was better than non-VA on some and worse on others, the study was classified as “mixed.” If the quality of care in VA and non-VA did not differ, the study was classified as “same.” If VA quality of care was shown to be worse than non-VA, the study was classified as “VA worse,” as were studies with multiple results reported where the quality of care was worse in some instances and the same in others.

Ensuring Comparability of Populations

Wherever possible, we report results adjusted for risk, comorbidities, demographics, or other variables. Unadjusted results are noted. We included and emphasized studies that compare VA patients to veterans receiving care in non-VA settings. We also included studies that compared veterans receiving VA care (referred to as “VA patients”) with individuals who are not identified as veterans receiving non-VA care (referred to as “non-veterans”).

RESULTS

Searches yielded 461 articles, 306 of which were rejected at abstract screening; 155 proceeded to full-text screening. Data were extracted from 69 articles; 31 of these were also included in the previous review,7 yielding 38 new articles (Fig. 1). Included articles were categorized by quality dimension: safety (34), timeliness (1), equity (4), effectiveness (24), efficiency (9), and patient-centeredness (5). Studies were sufficiently heterogeneous to preclude pooling or other meta-analysis, so results are presented narratively. The 34 articles on safety and 24 on effectiveness are summarized in Table 1.

Fig. 1
figure 1

Flow Diagram of Included and Excluded Studies. Of the 461 articles identified using our search strategies, 306 were excluded at title/abstract screening and 155 were screened for inclusion using the full text of the article. Of these, 86 were excluded, leaving 38 new articles for data abstraction. These articles were combined with the 31 articles from the prior review that met inclusion criteria, resulting in the 69 articles included in this systematic review.

Full size image
Table 1 Evidence table for safety and effectiveness
Full size table

Safety

Safety measures focused on avoiding illness/injury to patients resulting from medical care, such as complications following surgical procedures. Studies of morbidity and mortality were also included in this category, as were studies about adherence to safety guidelines or best practices. In 22 of 34 articles on safety, VA generally performed as well (11 studies) or better (11 studies) than non-VA settings. VA fared worse in nine studies and had mixed performance in three studies.

Mortality

VA facilities generally performed comparably or favorably relative to other settings in terms of mortality. Mortality rates associated with specific conditions16 – 19 or following surgery20 – 26 were often similar for VA patients compared with non-veterans, and occasionally lower.17 However, some studies found higher surgical mortality among VA patients compared with non-veterans.19 , 22 , 26 – 29 Lower mortality rates were observed after cataract surgeries for VA patients compared with veterans who are Medicare fee-for-service beneficiaries.30 Mortality among veterans residing in VA nursing homes and community nursing homes was similar.31 Veterans undergoing dialysis treatment in VA and non-VA settings experienced similar mortality.32 Adjusted mortality was lower among male VA patients compared with male Medicare Advantage beneficiaries over 65 years old;33 – 36 results were similar for females.34 Mortality within 1 year of admission after hip fracture was 21 % lower among veterans admitted to non-VA hospitals compared with VA patients.37 Mortality rates have declined more quickly in VA over time than in non-VA settings.38

Morbidity

VA facilities had mixed results with respect to most studies of morbidity. Postoperative morbidity was lower for VA patients compared with non-veterans in some surgeries,22 , 25 , 39 while for other surgeries it was similar.40 – 44 In a study comparing quality of care in nursing homes, veterans in VA nursing homes were less likely to develop a pressure ulcer than veterans in community nursing homes.31 However, other studies found that morbidity among VA patients is worse after pancreatectomy28 and for male patients only after gastric bypass, compared to patients in private sector hospitals (unadjusted).41

Complications

VA facilities had mixed results with respect to complications after surgery. Surgical complication rates were similar among VA patients and non-veterans.21 , 23 , 29 Higher complication rates were observed for VA patients undergoing cataract surgeries than veterans who were Medicare fee-for-service beneficiaries.45 Among all kidney transplant recipients, VA patients had higher graft failure risk than non-veterans.27

Other Safety Measures

VA hospitals were more likely to follow best practices for central venous catheter bloodstream infection prevention compared with non-VA hospitals.46 However, performance on the Agency for Healthcare Research and Quality’s (AHRQ) patient safety indicators was found to be a mix of higher, lower, and similar rates at VA hospitals compared with non-VA hospitals.47 – 49

Effectiveness

Seventeen studies showed better performance in VA facilities, while three had similar performance, one was mixed, and three were worse than non-VA settings.

Outpatient Care

Outpatient care was generally strong in VA facilities. VA patients received higher quality care than non-veterans for one measure of preventive care mammography (85–90 % vs. 64–77 %) and two measures of outpatient diabetic management (annual HbA1c test: 94–96 % vs. 70–81 %; annual eye examination: 67–85 % vs. 68–74 %) (no statistical tests).50 VA patients also received more effective care than non-veterans based on 10 of 11 quality measures in the first study year and all 12 quality measures in the second study year assessing diabetes, cardiovascular, and cancer screening care, with rate differences ranging from 4.3 percentage points (95 % confidence interval [CI] 3.2–5.4) for cholesterol testing in coronary heart disease to 30.8 (95 % CI 28.1–33.5) for colorectal cancer screening.51 Receipt of diabetes education and annual HbA1c tests was higher among VA patients compared with veterans in non-VA care.52 VA patients were more likely than veterans receiving care outside VA to receive recommended diabetes care, including being twice as likely to have a foot examination and 60–70 % more likely to have an eye examination, two or more A1c tests, and two or more providers visits,53 a routine checkup within 2 years (91.6 % among VA patients compared to 80.6 % overall; P < 0.001),54 and influenza and pneumonia vaccinations (rates increased 10–240 %),53 , 55 , 56 but similar rates of cholesterol screening.55 Blood pressure control was higher among male African-American VA patients than male African-American non-VA patients (49.4 % vs. 44.0 %, P < 0.01) though similar among Caucasians.57 VA patients were more likely than non-VA patients to receive recommended ambulatory preventive and disease management services,58 including influenza and pneumococcal vaccination.58 , 59 These studies used a national sample comparison group of non-VA patients, which may have included both veterans and non-veterans. Obese VA patients were more than twice as likely to have received advice to lose weight as veterans receiving non-VA care and non-veterans and equally likely to have received advice to maintain weight.60 However, in a study of veterans receiving primary care at VA-staffed versus contract community clinics, veterans with diabetes at VA-staffed clinics were less likely to receive a retinal examination (odds ratio [OR] 0.72, 95 % CI 0.55–0.93) and veterans with chronic obstructive pulmonary disease at VA-staffed clinics were less likely to receive a flu shot (OR 0.73, 95 % CI 0.55–0.99).61

Non-Ambulatory Care

In non-ambulatory settings, VA care was generally of similar quality to care provided in non-VA facilities. Compared with non-VA patients from the Medicare cancer patient database, VA patients had earlier diagnosis of colon and rectal cancers, higher rates for three quality measures, similar rates for nine, and lower rates for one.62 Comparison of an academic practice and a VA hospital found that appropriate use of stress/rest myocardial perfusion imaging studies did not differ between settings.63 Rates of hemodialysis via arteriovenous fistulas (which are preferred by guidelines over other methods) were not different between VA patients and Medicare patients after accounting for pre-end-stage renal disease care.64 VA patients received higher quality care than non-veterans for nine out of ten measures of inpatient care and performed similarly on one measure.50

Medication Management

Medication management was examined in a number of settings, including outpatient, emergency department, and inpatient. Elderly VA patients were less likely to receive inappropriate medication than were patients in Medicare HMOs,65 and VA patients with acute myocardial infarction were more likely to receive appropriate medications than were non-VA patients.66 Observed compliance by providers with erythropoietin administration guidelines was higher at VA than in the private sector.67 Antibiotic prescribing practices were generally similar between seven VA and seven non-VA emergency departments; however, in the three cities in which prescription rates were not comparable between VA and non-VA sites, VA sites had much higher rates of antibiotic prescriptions.68

Availability of Services

The structure of women’s health care differed at eight VA women’s health centers and 13 Department of Health and Human Services Centers of Excellence. Preventive cancer screening and general reproductive services were available at all centers, while VA centers were less likely to offer extensive reproductive services but more likely to offer mental health care.69 Eight of 15 clinical pharmacy services were more commonly provided in VA hospitals than non-VA hospitals [in-service education was 25 % higher (P = 0.003), clinical research 154 % higher (P < 0.0001), drug protocol management 28 % higher (P < 0.0001), drug therapy counseling 80 % higher (P < 0.0001), participation on rounds 38 % higher (P = 0.001), and admission drug histories 310 % higher (P < 0.0001)].70 VA patients and Medicare/Medicaid-insured patients were less likely to receive kidney transplants than were patients with private insurance.71

End-of-Life Care

VA tended to avoid inappropriate care and utilize palliative care at end of life. Increasing use of chemotherapy at the end of life is associated with higher rates of in-hospital deaths and later admission to hospice, which are linked to lower quality end-of-life care.72 – 75 When comparing male VA patients and Medicare patients with lung and colorectal cancer, VA patients were less likely to receive chemotherapy within 14 days of death or to be admitted to an ICU within 30 days of death, and they were similarly likely to have more than one emergency room visit within 30 days of death.76 Among veterans who died in VA facilities, palliative care consults (67 % vs. 21 %, P < 0.001) and death in a dedicated palliative care, hospice unit, or intensive care unit were more common (47 % vs. 16 %, P < 0.001), and death in a nursing home was less common (10 % vs. 26 %, P < 0.001) than among veterans who died in non-VA facilities (all unadjusted results).77

Timeliness, Equity, Efficiency, and Patient-Centeredness

Since fewer than ten studies were found that related to each of the dimensions of timeliness, equity, efficiency, and patient-centeredness, we do not discuss the results of these studies in detail (results and evidence tables in Online Appendix). The single study that addressed timeliness of care showed worse performance among veterans in VA facilities relative to non-VA facilities for time between hospital admission for hip fracture and surgical repair.37 Four studies examined equity, with one study showing better performance on several chronic disease and preventive measures,51 one study showing worse performance in VA facilities for emergency room visits among cancer patients in the last month of life,76 and two studies showing similar equity in 30-day post-admission mortality78 and graft failure.27 Nine articles compared efficiency using utilization, with six studies showing worse performance in VA facilities on outcomes such as average length of stay and service utilization for patients on dialysis,32 , 61 , 79 – 82 two showing better performance for visits/admissions83 and generic drug utilization,84 and one with mixed results for utilization-related inpatient quality indicators. Five studies examined patient-centeredness; three studies demonstrated better patient and family satisfaction in VA facilities77 , 85 , 86 and two demonstrated similar performance in perceptions of racial discrimination and satisfaction in transitional programs for homeless veterans.87 , 88

DISCUSSION

In this systematic review, we identified 69 studies comparing quality of care provided in VA facilities to non-VA care. Studies of safety and effectiveness indicated generally favorable performance by VA facilities compared to non-VA facilities, with 22 of 34 studies of safety and 20 of 24 studies of effectiveness showing the same or better quality of care in VA facilities and the remainder showing mixed (four studies) or worse (12 studies) performance. In terms of safety, VA facilities performed similarly or better in most, but not all, studies comparing morbidity and mortality. Results from studies about complications and patient safety indicators were more mixed. In terms of effectiveness, VA facilities had similar or superior quality to non-VA facilities with respect to preventive, recommended, and end-of-life care, as well as managing medications. Non-ambulatory care studies indicated similar care quality between VA and non-VA settings. However, studies on service availability had mixed results. A comparison of studies included in the previous review, which included studies from 1990 to 2009, and those identified in this review (2005 to 2014) showed that while numerous studies have been published in the last 5 years, no stark trends can be observed in terms of whether older or newer studies demonstrate systematically better or worse performance in VA settings (Table 2).

Table 2 Comparison of studies included in the prior and current review
Full size table

In keeping with the findings of the prior review, we find that VA is generally more adherent to recommended care processes than other systems of care. However, better processes did not necessarily achieve better outcomes, as we observed few differences in adjusted mortality. Although easy to obtain, mortality may not be the optimal outcome for comparison for several reasons, some of which were noted in the prior review.7 – 9 While nearly every study we included attempted to risk-adjust the veteran and non-veteran populations, some excess baseline mortality risk may still be present, since veterans have comparatively worse health status than the general population.89 – 91 Mortality rates are also known not to be sensitive in detecting differences in the quality of health care provided.92 – 94 Other outcomes, such as complications specific to a disease or procedure of interest, might be more revealing about meaningful quality differences in VA settings than those included in this review.

Our review builds on a previous review conducted in this area,7 – 9 but both updates and expands upon it. Updating the results of the prior review up to 2015 is critically important given the ongoing interest in the quality of care provided at the VA.95 In addition to categorizing the studies published since the prior review, we also categorized the studies contained in the prior review to provide a broad picture of quality at the VA over the last decade along the dimensions of health care quality proposed by the IOM.11. This categorization allows us to highlight the gaps in research by dimension, which may help inform the investment of resources for research and improvement by VA stakeholders.

This study is subject to a number of limitations. The search strategy employed was narrowly defined to align with the methods of the previous review, but this may have excluded relevant studies. Most studies were not conducted with perfectly matched comparison groups (i.e., veterans receiving care in VA and non-veterans in non-VA settings). While we used risk-adjusted or otherwise comparable results wherever possible, this may not have accounted for unobservable differences between veterans and non-veterans. Although we included confidence intervals or levels of significance in the evidence tables (Table 1 and Online Appendix), it was not always possible to determine whether lack of a significant difference was due to an underpowered study versus evidence of a lack of a difference. However, while many of the studies used large, nationally representative samples, some of the smaller studies did observe significant differences. We also did not assess whether the magnitudes of differences were clinically significant, as the thresholds for clinical differences for many of the outcomes investigated may be subjective. In deciding which studies to include, we assessed the study design of each (Table 1) and excluded low-quality studies from this review. However, we did not systematically assess each study’s risk of bias beyond that. It is notable that all but 4 of the 69 articles included had at least one VA-affiliated author or were funded directly by VA.

Nonetheless, the available data indicate overall comparable health care quality in VA facilities compared to non-VA facilities with regard to safety and effectiveness. Rates of complications and availability of services had the least favorable results, but these results were mixed rather than consistently poor. The overall number of studies comparing VA and non-VA care was small, and study quality varied. More studies that examine and compare the quality of VA care with respect to timeliness, equity, efficiency, and patient-centeredness are needed to better assess VA facilities’ performance on these quality dimensions.