Journal of General Internal Medicine

, Volume 32, Issue 1, pp 105–121

Comparing VA and Non-VA Quality of Care: A Systematic Review

  • Claire O’Hanlon
  • Christina Huang
  • Elizabeth Sloss
  • Rebecca Anhang Price
  • Peter Hussey
  • Carrie Farmer
  • Courtney Gidengil
Review Paper

Abstract

Background

The Veterans Affairs (VA) health care system aims to provide high-quality medical care to veterans in the USA, but the quality of VA care has recently drawn the concern of Congress. The objective of this study was to systematically review published evidence examining the quality of care provided at VA health care facilities compared to quality of care in other facilities and systems.

Methods

Building on the search strategy and results of a prior systematic review, we searched MEDLINE (from January 1, 2005, to January 1, 2015) to identify relevant articles on the quality of care at VA facilities compared to non-VA facilities. Articles from the prior systematic review published from 2005 and onward were also included and re-abstracted. Studies were classified, analyzed, and summarized by the Institute of Medicine’s quality dimensions.

Results

Sixty-nine articles were identified (including 31 articles from the prior systematic review and 38 new articles) that address one or more Institute of Medicine quality dimensions: safety (34 articles), effectiveness (24 articles), efficiency (9 articles), patient-centeredness (5 articles), equity (4 articles), and timeliness (1 article). Studies of safety and effectiveness indicated generally better or equal performance, with some exceptions. Too few articles related to timeliness, equity, efficiency, and patient-centeredness were found from which to reliably draw conclusions about VA care related to these dimensions.

Discussion

The VA often (but not always) performs better than or similarly to other systems of care with regard to the safety and effectiveness of care. Additional studies of quality of care in the VA are needed on all aspects of quality, but particularly with regard to timeliness, equity, efficiency, and patient-centeredness.

KEY WORDS

veterans Veterans Affairs Veterans Health Administration quality systematic review 

INTRODUCTION

Providing high-quality health care is central to our nation’s commitment to veterans. The Veterans Affairs health care system (VA), the nation’s largest integrated health care system, provides comprehensive health services to US military veterans who are enrolled. However, the quality of VA care is a longstanding area of concern.1,2 As a result, the Veterans Access, Choice and Accountability Act (VACAA), passed in 2014, mandated an independent assessment of health care capabilities and resources of the Veterans Health Administration.3 As part of this assessment, the Interim Under Secretary for Health for VA called for a comprehensive evaluation of “VA’s ability to deliver high-quality health care to Veterans.”4

Previous studies have systematically reviewed aspects of care at the VA.57 Two peer-reviewed articles based on one of these7 summarized the available evidence on medical8 and surgical9 care quality through 2009. This review indicated that in most studies, the care provided in the VA compares favorably to non-VA systems. However, quality may have changed in the last 6 years since this review, and concerns about quality of care have continued to mount.10 To assess the quality of care provided by the VA, we performed a systematic review of published comparisons of the quality of care in VA facilities to other settings.

METHODS

Search Strategy

We defined quality of care using the Institute of Medicine’s (IOM) definition, which is “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.”11 Given the existence of a recent systematic review on the quality of health care delivered in VA versus comparable non-VA settings, we chose to explicitly build upon and expand this work.7 We use consistent methods, including the same search terms (Online Appendix) and database (MEDLINE). The dates searched were limited to the past 10 years (January 1, 2005, to January 1, 2015) to ensure a reasonable scope and up-to-date studies.

Article Screening and Data Abstraction

Titles and abstracts identified by our literature search were screened by two reviewers with experience screening and abstracting data for health-related systematic reviews (CO, CH) under the supervision of a senior member of the study team with extensive clinical and systematic review experience (CG) for inclusion in full-text screening. If the article presented comparisons of quality of health care in the VA and non-VA settings in the US, the full text of each article was reviewed independently by both reviewers (CO, CH). We extracted the data sources, geographical areas, clinical conditions, quality dimensions, and comparability of measures in each sample. Studies were excluded if they included no original data; were case reports; or used non-contemporaneous comparisons, unequal or non-representative samples, or dissimilar or indirect quality measures. Data from each article, including the size of the VA and non-VA samples, years of data collection, control variables, and primary and secondary outcomes, were abstracted by one reviewer and double-checked by another reviewer (CO, CH). Disagreements were resolved by a senior team member (CG).

Quality Assessment

Article quality was assessed using the criteria developed in the original review for assessing comparative health care quality studies.7 Briefly, six elements were used, including whether: (1) time periods during which the VA and non-VA patients were studied were sufficiently contemporaneous; (2) VA and non-VA samples were comparable in size/scope; (3) quality measurements were assessed using specified and identical indicators with a similar assessment format; (4) quality measures were meaningfully associated with outcomes; (5) measures were clinically relevant and important; and (6) the statistical methodology was sound. Each factor was graded (A for excellent, B for acceptable, C for unacceptable). The overall subjective assessment of the article was created, but this assessment was not an “average” of individual component grades. Thus, an article with a critical flaw could be graded “C” and rejected, even if it scored well on other criteria. Articles were graded by one reviewer and double-checked by the other reviewer (CO, CH). Disagreements were resolved in consultation with the senior team member (CG).

Study Classification

Although we employed a nearly identical search and abstraction strategy, we organized article categories differently than the original review,7 which split articles into medical8 (non-surgical) and surgical9 care and by Donabedian’s quality triad of structure, process, and outcome.12 To guide decision making about how to improve quality at the VA, we organized our review using the IOM’s six characteristics of high-quality care: safe, timely, equitable, effective, efficient, and patient-centered,11 and then grouped similar kinds of outcomes together. Timeliness as related to delays or wait times for receiving necessary medical care (e.g., time from admission to emergency surgery) was included in this review. Timeliness as it relates solely to access (e.g., wait time for obtaining an appointment)—while a critical issue currently facing the VA13—was not addressed because no studies directly comparing VA and non-VA settings were identified.14 We included articles on efficiency if they assessed appropriateness (necessary care versus overuse). We excluded cost-efficiency articles, as cost comparisons between VA and other settings have a number of limitations.15

We classified each study (both new and from the prior review) according to the direction of the statistically significant differences in performance measures for the VA care relative to a non-VA comparison group. If VA quality of care was shown to be better than non-VA care or if multiple results were reported and VA quality of care was better in some instances and the same in others, the study was classified as “VA better.” If multiple quality measures were reported and VA care was better than non-VA on some and worse on others, the study was classified as “mixed.” If the quality of care in VA and non-VA did not differ, the study was classified as “same.” If VA quality of care was shown to be worse than non-VA, the study was classified as “VA worse,” as were studies with multiple results reported where the quality of care was worse in some instances and the same in others.

Ensuring Comparability of Populations

Wherever possible, we report results adjusted for risk, comorbidities, demographics, or other variables. Unadjusted results are noted. We included and emphasized studies that compare VA patients to veterans receiving care in non-VA settings. We also included studies that compared veterans receiving VA care (referred to as “VA patients”) with individuals who are not identified as veterans receiving non-VA care (referred to as “non-veterans”).

RESULTS

Searches yielded 461 articles, 306 of which were rejected at abstract screening; 155 proceeded to full-text screening. Data were extracted from 69 articles; 31 of these were also included in the previous review,7 yielding 38 new articles (Fig. 1). Included articles were categorized by quality dimension: safety (34), timeliness (1), equity (4), effectiveness (24), efficiency (9), and patient-centeredness (5). Studies were sufficiently heterogeneous to preclude pooling or other meta-analysis, so results are presented narratively. The 34 articles on safety and 24 on effectiveness are summarized in Table 1.
Fig. 1

Flow Diagram of Included and Excluded Studies. Of the 461 articles identified using our search strategies, 306 were excluded at title/abstract screening and 155 were screened for inclusion using the full text of the article. Of these, 86 were excluded, leaving 38 new articles for data abstraction. These articles were combined with the 31 articles from the prior review that met inclusion criteria, resulting in the 69 articles included in this systematic review.

Table 1

Evidence table for safety and effectiveness

Author, year

Quality dimension(s) [assessment*]

Data level

VA sample [sample size]

Years collected

Data level

Non-VA sample [sample size]

Years collected

Conditions

Outcomes

Primary findings†

Final grade‡

New articles abstracted for this review

Bean-Mayberry B, et al., 2007

Effectiveness [mixed]

Mult ctrs

VA comprehensive Womens’ Health Centers [N = 8]

2003

Nat’l

Department of Health and Human (DHHS) Services National Centers of Excellence [N = 13]

2003

None

Availability of services

Preventive cancer screening and general reproductive services were uniformly available at all centers, although DHHS centers offered extensive reproductive services on-site more frequently, and VA centers more often had on-site mental health care

B

Boitano LT, et al., 2012

Safety [same]

Single ctr

Veterans at a VA hospital [N = 322]

2006–2009

Single ctr

Patients at Northwestern Memorial Hospital [N = 269]

2006–2009

Vascular surgery

Postoperative outcomes (morbidity and mortality), adjusted

Multivariate analysis showed that hospital setting was not an independent predictor of complications, major adverse events, or death, suggesting no difference in outcomes between the VA and private sector

A/B

Bond CA, et al., 2008

Effectiveness [better]

Nat’l

VA hospitals [N = 84]

2006

Nat’l

Non-VA hospitals [N = 1,041]

2006

None

Clinical pharmacy services offered

8/15 clinical pharmacy services were more common in VA hospitals. In-service education was higher by 25 %, clinical research 154 %, drug protocol management 28 %, drug therapy counseling 80 %, participation on rounds 38 %, and admission drug histories 310 % (P < 0.003)

A

Borzecki AM, et al., 2010

Safety [better]

Nat’l

IQI-related discharge at VA [N = 403,828]

2003–2007

Nat’l

HCUP-NIS IQI-related discharges [sample size not reported]

2003–2007

Multiple

Inpatient quality indicators

Comparing VA and HCUP-NIS risk-adjusted rates from 2003 through 2007, slope estimates for AMI, stroke, hip fracture, pneumonia, and hip replacement mortality rates declined more rapidly in the VA

A

Chakkera HA, et al., 2005

Safety [worse]

Nat’l

VA patients who received pretransplant care in VA or paid by VA [N = 1,646]

1991–2001

Nat’l

Non-VA patients who did not receive pretransplant care in VA or paid for by VA [N = 77,715]

1999–2001

Kidney transplant

Graft and patient survival after kidney transplant, adjusted

Among all recipients, VA users had a 20 % higher risk for graft failure (RR 1.21; 95 % CI 1.12–1.30) and 14 % higher risk of mortality (RR 1.14; 95 % CI 1.07–1.22) compared with non-VA users

A

Choi JC, et al., 2009

Safety [same]

Single ctr

VA patients [N = 682]

2002–2006

Nat’l

Non-veteran non-VA patients [N = 34,572]

2004

Cardio

In-hospital mortality rate after CABG, unadjusted

No significant difference in in-hospital mortality rate for male patients (1.6 % versus 2.4 %; P = 0.20)

B

Fihn SD, et al., 2009

Safety [same]

Nat’l

VA patients [N = 27,494]

2000–2005

Nat’l

Medicare patients in private sector hospitals [N = 789,400]

2000–2005

IHD

Adjusted 30-day mortality following AMI

Adjusted relative odds of death were not significantly different for VA or private sector hospitals (OR 1.02; 95 % CI 0.96–1.08)

A

French DD, et al., 2012a

Safety [worse]

Nat’l

Patients in VA database [N = 20,191]

2007

Nat’l

Patients in Medicare database [N = 137,726]

2007

Cataract surgery

90-Day rates of cataract procedure complications

Adjusted results resulted in significant odds increases of corrective procedures for routine cataract surgeries performed in VA (OR 1.70; 95 % CI 1.58–1.82) and for complex cataract surgery (OR 2.68; 95 % CI 2.24–3.20)

A

French DD, et al., 2012b

Safety [better]

Nat’l

Patients in VA database [N = 19,721]

2007

Nat’l

Patients in Medicare database [N = 129,302]

2007

Cataract surgery

All-cause mortality following cataract surgery, adjusted

Mortality risk did not differ significantly between the two cohorts at time points within 6 months following cataract surgery. Medicare patients experienced 13–17 % excess mortality in both routine and complex cases at 270 and 365 days

A

Gonzales R, et al., 2006

Effectiveness [worse]

Mult ctrs

Patients at VA ED [N = 1,125]

2003–2004

Mult ctrs

Patients at matched non-VA EDs [N = 1,138]

2003–2004

Pulmonary

Antibiotic prescribing patterns in acute respiratory infections

Clinical setting (VA vs. non-VA) was not independently associated with antibiotic prescribing. In four cities, VA and non-VA EDs were very similar, but far apart in four others. In discordant cases, the VA ED sites showed higher adjusted rates of antibiotic treatment

B

Hynes DM, et al., 2006

Effectiveness [better]

Mult ctrs

VA hemodialysis patients [N = 177]

2001–2003

Mult ctrs

Private sector hemodialysis patients [N = 131]

2001–2003

ESRD

Compliance with erythropoietin guidelines, adjusted

Erythropoietin was administered predominantly subcutaneously for 52 % of patients in VA versus 15 % in private-sector facilities (P < 0.001)

B

Keating NL, et al., 2010

Effectiveness [better]

Nat’l

Male VA patients [N = 2,913]

2001–2005

Nat’l

Propensity-matched male Medicare patients [N = 2,913]

2001–2005

Lung cancer, colorectal cancer

End-of-life care indicators, propensity adjusted

Men treated at VA were less likely than those in private sector to receive chemo within 14 days of death (4.6 % vs. 7.5 %; P < 0.001) or to be admitted to an ICU within 30 days of death (12.5 vs. 19.7; P < 0.001), and similarly likely to have >1 ER visit within 30 days of death (13.1 vs. 14.7; P = 0.09)

A

Keating NL, et al., 2011

Effectiveness [better]

Nat’l

VA patients [N = 50,573]

2001–2005

Nat’l

FFS Medicare patients [N = 143,504]

2001–2005

Colorectal, lung, or prostate cancer; lymphoma; or multiple myeloma

High-quality cancer care process measures, propensity adjusted

For colon cancer, VA had higher rates of diagnosis at earlier stage (P < 0.001) and resection (P = 0.01), but similar rates of adjuvant chemo. For rectal cancer, VA had higher rates of diagnosis at earlier stage (P = 0.007), but similar rates of resection or adjuvant chemo/radiation. Lung cancer outcomes were non-significant. Outcomes were mixed for prostate and hematologic cancer

A/B

Landrum MB, et al., 2012

Safety [better]

Nat’l

Males over 65 diagnosed/treated for certain cancers at VA [N = 26,718]

2001–2005

Nat’l

Males over 65 with certain cancers using Medicare FFS [N = 118,195]

2001–2005

Colorectal, lung, or prostate cancer; lymphoma; or multiple myeloma

Time to all-cause death and cancer-attributable death), propensity adjusted

VA patients had higher survival rates of colon cancer (HR 0.87; 95 % CI 0.82–0.93) and non–small-cell lung cancer (HR 0.91; 95 % CI 0.88–0.95) and similar survival rates of rectal cancer, small-cell lung cancer, diffuse large–B-cell lymphoma, and multiple myeloma compared to similar FFS Medicare patients

A

Liu CF, et al., 2008

Effectiveness [worse]

Nat’l

Veterans who are primary care users at 76 VA-staffed community clinics [N = 17,060]

2000–2001

Nat’l

Veterans who are primary care users at 32 non-VA contract community clinics receiving capitation [N = 6,842]

2000–2001

DM, pulmonary

Outpatient service utilization and receipt of primary care services, adjusted

Odds ratios for the proportion of contract and VA-staffed clinic diabetic patients receiving a retinal exam were (OR 0.72; 95 % CI 0.55-0.93) and COPD patients receiving a flu shot (OR 0.73; 95 % CI 0.55–0.99)

A

Lu H, et al., 2010

Effectiveness [better]

Mult ctrs

Veterans who died in a VA facility [N = 520]

NR

Mult ctrs

Veterans who died in a non-VA facility [N = 89]

NR

None

Perceptions of the care and services during last month of life

Patients who died in the VA were more likely to have had a palliative care consult (67 % vs. 21 %; P < 0.001) and to have died in a palliative care or hospice unit (47 % vs. 16 %; P < 0.001), but they were more likely to die in ICU (26 % vs. 13 %; P = 0.01) and less likely to die in a nursing home (0 % vs. 26 %; P < 0.001)

B

Lynch CP, et al., 2010

Effectiveness [better]

Nat’l

Veterans who had used VA facilities in the last year [N = 1,342]

2003

Nat’l

Veterans who had not used VA facilities in the last year [N = 3,159]

2003

DM, Preventive care

Quality of diabetes care, adjusted

VA users were more than twice as likely to have received foot exams by a provider (OR 2.59; 95 % CI 1.76–3.83), pneumonia shots (OR 2.30; 95 % CI 1.68–3.14), and flu shots (OR 2.05; 95 % CI 1.44–2.92). VA users had 60–70 % greater likelihood of A1c tests, and 2+ provider visits in the last 12 months

A

Nelson KH, et al., 2011

Effectiveness [same]

Mult ctrs

VA patients [N = 150]

2009

Mult ctrs

Patients at an academic medical center [N = 150]

2009

IHD

Appropriate use of stress/rest myocardial perfusion imaging (MPI), unadjusted

The majority of the studies were ordered for appropriate indication (67.3 % in VA vs. 74 % in academic practice) (P = 0.272)

B

Parikh DS, et al., 2011

Effectiveness [same]

Mult VISNs

VA patients [N = 378]

2005–2006

Mult VISNs

Medicare patients [N = 25,534]

2005–2006

ESRD

Method of vascular access for first outpatient dialysis

Adjusting for patient demographics and comorbidities only, VA patients had greater likelihood of arteriovenous fistulas (AVF) use (preferred by guidelines) (OR = 1.70; 95 % CI 1.31–2.20), but accounting for pre end-stage renal disease care removed the significant difference (OR 1.28; 95 % CI 0.98–1.66)

A/B

Richardson KK, et al., 2013

Safety[worse]

Nat’l

VA users [N = 9,308]

2002–2008

Nat’l

Veterans at non-VA hospitals [N = 1,881]

2002–2008

Orthopedic surgery

Time between admission and repair, 1-year mortality

Likelihood of death within 1 year of admission was 21 % less for veterans admitted to non-VA hospitals (RR = 0.79; 95 % CI 0.71–0.88; P < 0.001)

A

Rivard PE, et al., 2010

Safety [worse]

Nat’l

VA hospitals [N = 116]

2003–2004

Nat’l

Non-VA hospitals (HCUP-NIS) [N = 992]

2003–2003

None

Patient Safety Indicators (PSIs), adjusted

VA had higher composite PSI [VA: 1.118 95 % CI (1.071–1.164), HCUP-NIS: 0.987 95 % CI (0.977–0.997)]. VA also had higher foreign body left in during procedure, iatrogenic pneumothorax, postoperative hemorrhage, postoperative wound dehiscence. HCUP-NIS had more postoperative sepsis than VA, with a small overlap of confidence intervals. No significant differences for other PSIs evaluated

A

Rosen AK, et al., 2005

Safety [mixed]

Nat’l

VA users [N = 281,423]

2000–2001

Nat’l

Non-VA (HCUP-NIS and Medicare) [sample sizes not reported]

2000–2000

None

PSIs, risk-adjusted

VA-risk-adjusted rates are lower than HCUP-NIS and Medicare event rates for decubitus ulcer, infection due to medical care, postoperative respiratory failure, and postoperative sepsis. VA PSI event rates were higher than HCUP-NIS and Medicare event rates for postoperative physiologic and metabolic derangements and technical difficulty with procedure. VA PSI event rates were higher than HCUP-NIS event rates, but lower than Medicare event rates, for the remaining indicators

A

Selim AJ, et al., 2010

Safety [better]

Nat’l

VA users [N = 35,876]

1999–2003

Nat’l

Medicare Advantage cohort [N = 71,424]

1999–2003

None

2-Year mortality, probability same or better physical or mental health at 2 years; adjusted

2-Year mortality rates were 11.8 % and 9.9 % for the Medicare and VA, respectively; probability of being alive with the same or better physical health at 2 years higher VA compared to Medicare; same or better mental health at 2 years was also significantly higher at VA than in Medicare. Propensity score matched analyses had comparable results

A

Tarlov E, et al., 2012

Safety [same]

Nat’l

VA users [N = 1,465]

1999–2001

Nat’l

Medicare FFS users [N = 1,042]

1999–2001

Colon cancer

Overall and event-free 36-month survival, adjusted

Overall survival hazard ratios were similar at stages 1–3. Event-free survival hazard ratios were also not significantly different

A

Trivedi AN, et al., 2011

Effectiveness [better]

Nat’l

VA patients [N = 293,554]

2000–2007

Nat’l

Medicare Advantage (MA) enrollees [N = 5,768,573]

2000–2007

DM, IHD, HTN, Preventive care

Healthcare Effectiveness Data and Information Set (HEDIS), External Peer Review Program (EPRP) indicators

The VA had higher aggregate performance than MA for 10 of 11 measures in the initial year of assessment, and all 12 measures in the final year. Adjusted differences range from 4.3 percentage points (95 % CI 3.2–5.4) for cholesterol testing in CHD to 30.8 percentage points (95 % CI 28.1–33.5) for colorectal cancer screening

A/B

Vaughan-Sarrazin MS, et al., 2007

Safety [worse]

Nat’l

Male VA users [N = 139,331]

1996–2002

Nat’l

Male Medicare patients [N = 1,212,729]

1996–2002

Cardio

30-, 90-, 365-Day mortality, adjusted

Adjusted mortality after CABG was higher (P < 0.001) in VA users compared with nonusers at 30, 90, and 365 days (ORs 1.07, 1.07, 1.09). For PCI, adjusted mortality at 30 and 90 days was similar (NS), but higher for VA users at 365 days (OR 1.09)

A

Wang A, et al., 2005

Effectiveness [better]

Nat’l

VA users [N = 3,391]

2000

Nat’l

Non-VA users [N = 178,735]

2000

Overweight/obesity

Association between obesity and weight advice, adjusted

Obese VA users were twice as likely to have received professional advice to lose weight (OR 2.06; 95 % CI 1.64 to 2.59) and as likely to have received professional advice to maintain weight (OR 1.72; 95 % CI 0.75 to 3.97)

B

Wang V, et al., 2013

Safety [same]

Mult VISNs

VA dialysis users [N = 381]

2007–2008

Mult VISNs

Veterans who received outpatient dialysis exclusively in VA-outsourced settings [N = 659]

2007–2008

ESRD

Adjusted all-cause hospitalization and mortality at 1 year, adjusted

There was no difference in mortality outcomes among veterans who received outpatient dialysis exclusively in VA-outsourced compared to VA dialysis users (OR 0.80; 95 % CI 0.48–1.3)

A

Weeks WB, et al., 2009

Effectiveness [better]

Nat’l

VA patients [sample size not reported]

2005–2006

Nat’l

Medicare FFS patients [sample size not reported]

2005–2006

Multiple

Outpatient and inpatient quality measures

The VA outperformed Medicare fee-for-service performance in one measure of mammography and two measures of outpatient diabetic management. In 2005 and 2006, the VA performed better than hospitals contributing to Hospital Compare

B

West A, et al., 2006

Effectiveness [better]

Nat’l

VA users [N = 1,928]

2000

Nat’l

Veteran VA non-users and non-veterans [N = 12,461]

2000

None

Routine checkup within last 2 years

VA patients reported higher rates of seeing a doctor for a routine checkup within the past 2 years (91.6 %; P < 0.001)

B

Articles from previous review (with additional information abstracted as necessary)

Bansal D, 2005

Effectiveness [better]

Single ctr

VA patients [N = 117]

2002

Nat’l

Registry not further described [sample size not reported]

2002

IHD

Use of certain medications among patients with MI

Use of aspirin, beta-blockers, ACE inhibitors, heparin, GP2b/3a inhibitors was higher in the Little Rock VA compared to the rest of Arkansas and the entire US

B

Barnett MJ, et al.,2006

Effectiveness [better]

Nat’l

VA patients [N = 123,633]

2002–2003

Nat’l

Medicare HMO patients [N = 157,517]

2000–2001

Other safety

Use of potentially inappropriate medications

Compared with private sector patients, VA patients were less likely to receive any inappropriate medication (21 % vs. 29 %; P < 0.001), and in each classification: always avoid (2 % vs. 5 %; P < 0.001), rarely appropriate (8 % vs. 13 %; p < 0.001), and some indications (15 % vs. 17 %; P < 0.001)

B

Berlowitz DR, et al., 2005

Safety [mixed]

One VISN

Veterans in VA nursing homes [N = 3,802]

1997–1999

Lrg geo area

Veterans in contract nursing homes [N = 961]

1997–1999

Other medical/ nonsurgical condition

Risk-adjusted rates of pressure ulcer, functional and behavioral decline, mortality

Veterans in VA nursing homes were significantly (P < 0.05) less likely to develop a pressure ulcer (OR 0.63) but more likely to experience functional decline (OR 1.6) than veterans in community nursing homes. Veterans in VA nursing homes were similar likely to die experience behavioral decline

A

Bilimoria KY, et al., 2007

Safety [same]

Nat’l

VA patients [N = 513]

1985–2004

Nat’l

Academic hospital patients [N = 12756]

1985–2004

General surgical, surgical oncology

60-Day and 3-year mortality

Unadjusted and adjusted mortality rates at 60 days and 3 years were comparable between VA, academic and community hospital settings for resection of stage I and II pancreatic cancer

B

Community hospital patients [N = 18,299]

Campling BG, et al., 2005

Safety [worse]

One VISN

Male VA patients [N = 862]

1995–1999

Lrg geo

Male non-VA patients [N = 27,936]

1995–1999

Cancer

Survival following diagnosis of lung cancer

The median survival was 6.3 months for VA patients compared with 7.9 months for patients in the rest of the state, and the 5-year overall survival rate was 12 % for VA patients compared with 15 % for patients in the rest of the state. The hazard ratio for VA patients compared with non-VA patients is 1.22 (P < 0.001)

B

Chi RC, et al.,2006

Effectiveness [better]

Nat’l

VA users [N = 3,265]

2003

Nat’l

Veteran non-VA users [N = 10,677]

2003

Preventive care

Influenza and pneumococcal vaccination

Among veterans, influenza and vaccination rates higher for VA users compared to non-users. For veterans, VA care was independently associated with influenza vaccination (adjusted OR 1.8; 95 % CI 1.5–2.2) and pneumococcal vaccination (adjusted OR 2.4; 95 % CI 2.0–2.9)

A

Non-veterans [N = 40,331]

Fink AS, et al., 2007

Safety [better]

Nat’l

Female VA patients [N = 5,157]

2001–2004

Mult. ctrs

Female private sector patients [N = 27,467]

2001–2004

General surgical

30-Day postoperative morbidity and mortality

Risk-adjusted mortality rates are comparable between PS and VA patients, although setting of care did not enter the mortality regression model. Risk-adjusted morbidity was higher in the PS compared with the VA OR 0.8 (95 % CI 0.71–0.90)

B

Gill JS, et al., 2007

Effectiveness [worse]

Nat’l

VA patients [N = 7,395]

1995–2004

Nat’l

Privately insured patients [N = 144,651]

1995–2004

Other surgical

Time to treatment

Both VA-insured and Medicare/Medicaid-insured patients were 35 % less likely to receive transplants than privately insured patients (HR 0.65; 95 % CI 0.60–0.70; P < 0.001). VA patients were less likely to be placed on the wait-list (HR 0.71; 95 % CI 0.67–0.76), but even those listed received transplants less frequently than the privately insured (HR 0.89; 95 % CI 0.82–0.96)

A

Medicare/Medicaid patients [N = 357,345]

Glasgow RE, et al.,2007

Safety [worse]

Nat’l

VA patients [N = 377]

2001–2004

Mult. ctrs

Private sector patients [N = 692]

2001–2004

Other surgical

Postoperative outcomes

Adjusting for case mix differences, postoperative morbidity and mortality rates for pancreatectomy were higher in the VA compared with the PS (OR 1.58; 95 % CI 1.08–2.31 and 2.53 95 % CI 1.02–6.29 respectively)

A/B

Hall BL, et al., 2007

Safety [same]

Nat’l

VA patients [N = 2,814]

2001–2004

Mult. ctrs

Private sector patients [N = 4,268]

2001–2004

General surgical, head and neck

30-Day morbidity and mortality;

Overall 30-day morbidity and mortality do not differ significantly in the VA vs. PS in risk-adjusted model. Mortality event rate is too low to accurately evaluate, odds ratio for morbidity associated with VA care is 1.25 (95 % CI 0.87–1.78)

B

Adverse event rates, LOS

Henderson WG, et al., 2007

Safety [worse]

Nat’l

Male VA patients [N = 9,409,818]

2001–2004

Mult. ctrs

Male private sector patients [N = 18,399]

2001–2004

General surgical

30-Day postoperative morbidity and mortality

After risk adjustment for patient comorbidities and severity of illness, the odds of mortality at 30 days were higher in the VA compared with the PS (OR 1.23; 95 % CI 1.08–1.41). There was no significant difference in morbidity at 30 days among the sites

A/B

Hutter MM, et al., 2007

Safety [better]

Nat’l

Male VA patients [N = 30,058]

2001–2004

Mult. ctrs

Male private sector patients [N = 5,174]

2001–2004

Vascular

30-Day postoperative morbidity and mortality

Risk-adjusted mortality was comparable among the two groups. Accounting for comorbidities and severity of illness, postoperative morbidity rates were lower in the VA population, OR 0.84 (95 % CI 0.78–0.92)

A/B

Jha AK, et al., 2007

Effectiveness [better]

Nat’l

VHA patients [N = 33,504-74,250]

1995–2003

Nat’l

Representative community sample [sample size not reported]

1995–2003

Preventive care

Vaccination rates

Trends in influenza and pneumonia vaccination rates were significantly different in the VA compared to in the BRFSS (P < 0.001). Pneumonia hospitalization decreased by 50 % among elderly VA enrollees but increased among Medicare enrollees by 15 % (P < 0.001)

B

Johnson RG et al., 2007

Safety [better]

Nat’l

VA patients [N = 458]

2001–2004

Mult. ctrs

Private sector patients [N = 3,535]

2001–2004

Vascular

30-Day postoperative morbidity and mortality

After risk adjustment, no significant difference in 30-day mortality rates among VA and PS female vascular patients. After adjusting for severity of illness, 30-day complication/morbidity rates were significantly lower in VA compared with PS (OR 0.60; 95 % CI 0.44–0.81)

B

Keyhani S, et al., 2007

Effectiveness [better]

Nat’l

Veterans receiving VHA/VHA and FFS Medicare/ VHA and Medicare HMOs [N = 171/1,009/145]

2000–2003

Nat’l

Veterans receiving FFS Medicare/Medicare HMO [N = 3,552/576]

2000–2003

Preventive care

Influenza a and pneumonia vaccination, cholesterol screening

Veterans receiving care through VA reported 10 % greater use of influenza vaccination (P < 0.05), 14 % greater use of pneumococcal vaccination (P < 0.01), and a non-significant 6 % greater use of serum cholesterol screening (P = 0.1) than veterans receiving care through Medicare HMOs

B

Krein SL, et al., 2007

Safety [better]

Nat’l

VA hospitals [N = 119]

2005

Nat’l

Non-VA hospitals [N = 421]

2005

Other medical/ nonsurgical condition

Regular use of specific prevention modalities and a composite measure

Adjusted findings revealed that VA hospitals were significantly more likely to use chlorhexidine gluconate (OR 4.8; 95 % CI 1.6–15.0) and/or to use a composite approach (OR 2.1; 95 % CI 1.0–4.2) as compared with non-VA hospitals

B

Lancaster RT, et al., 2007

Safety [same]

Nat’l

Procedures at VA hospitals [N = 237]

2001–2004

Mult. ctrs

Procedures at university hospitals [N = 783]

2001–2004

General surgical

Postoperative 30-day morbidity and mortality LOS, need for re-operation, postoperative events

Risk-adjusted outcomes suggest that 30-day postoperative morbidity and mortality rates in the VA compared with the PS for hepatic resections do not vary significantly. After risk adjustment, morbidity rates and mortality were comparable in VA and PS. Comparing morbidity of VA w/ PS OR was 0.94 (95 % CI 0.62–1.42) and mortality OR was 1.62 (95 % CI 0.61–4.32)

A/B

Lautz DB, et al., 2007

Safety [worse]

Nat’l

VA patients [N = 374]

2001–2004

Mult. ctrs

Private sector patients [N = 2,064]

2001–2004

Other surgical

30-Day postoperative outcomes: morbidity and mortality, LOS

No significant difference in postoperative morbidity or mortality among women in the VA versus non-VA settings (16.07 vs. 12.02 %; P = 0.21 and 0.89 vs. 0.42 %; P = 0.47). Unadjusted and adjusted morbidity rates were higher among men treated at the VA versus non-VA (OR 2.77; 95 % CI 1.78–4.31 unadjusted and OR 2.29; 95 % CI 1.28–4.10 adjusted)

A/B

Nelson KM, et al., 2005

Effectiveness [better]

Nat’l

Veterans with some VA care [N = 254]

2000

Nat’l

Adults with other insurance types [N = 10,632]

2000

DM

Diabetes self- management and preventive care practices

Persons who received care through the VA were more likely to report taking a diabetes education class and HbA1c testing than those covered by private insurance

B

Veterans with all VA care [N = 281]

Neumayer L, et al., 2007

Safety [same]

Nat’l

VA patients [N = 644]

2001–2004

Mult. ctrs

Private sector patients [N = 3,179]

2001–2004

General surgical

30-Day postoperative morbidity and mortality, LOS

After adjusting for comorbidities and preoperative factors, there was no significant difference in 30-day morbidity or mortality in female patients at the VA compared with the PS (OR 1.40; 95 % CI 0.89–2.20)

B

Rehman SU, et al., 2005

Effectiveness [better]

One VISN

VA patients [N = 12,366]

2001–2003

Lrg geo

Non-VA patients [N = 7,734]

2001–2003

HTN

Control of blood pressure below 140/90 mmHg

Blood pressure control was comparable among white hypertensive men at VA (55.6 %) and non-VA (54.2 %) settings (P = 0.12). In contrast, BP control was higher among African American hypertensive men at VA (49.4 %) compared with non-VA (44.0 %) settings (P < 0.01), even after risk adjustment

A

Ross JS, et al., 2008

Effectiveness [better]

Nat’l

Adults receiving care at VAMCs [N = 10,007]

2000, 2004

Nat’l

Adults receiving care elsewhere [N = 393,873]

2000, 2004

DM, IHD, HTN, preventive care

Self-reported use of 17 recommended health care services

VA care was associated with greater use of recommended services in both years of study (6/17 services more used in 2000, 12/17 more used in 2004)

B

Selim AJ, et al., 2007

Safety [better]

Nat’l

VHA patients [N = 16,725 at baseline and 12,177 at follow-up]

1998–2000

Nat’l

Medicare Advantage Program patients [N = 62,614 at baseline and 26,225 at follow-up]

1998–2000

None

Risk-adjusted 2-year mortality, change in physical and mental health status

Lower risk-adjusted mortality in the VA compared to MA (2-year mortality 7.6 % in VA vs. 9.2 % in MA); VA patients had a slightly higher probability than Medicare patients of being alive with the same or better mental health (71.8 % vs. 70.1 %; P = 0.002)

B

Selim AJ, et al., 2006

Safety [better]

Nat’l

VHA patients [N = 420514]

1999–2004

Nat’l

Medicare Advantage Program [N = 584294]

1998–2004

Other medical/nonsurgical condition

Risk-adjusted mortality

After adjusting for case mix, the HR for mortality in Medicare was significantly higher than that in VA (HR, 1.40; 95 % CI = 1.38–1.43)

B

Selim, AJ, et al., 2009

Safety [better]

Nat’l

Medicaid-eligible VHA patients [N = 2,361]

1999–2000

Nat’l

Medicaid-eligible Medicare Advantage patients [N = 1,912]

1999–2000

Other medical/ nonsurgical condition

3-Year risk-adjusted mortality rate

The adjusted HR of mortality in the MA dual enrollees was significantly higher than in VA dual enrollees (HR 1.26; 95 % CI 1.04–1.52)

B

Turrentine FE, et al., 2007

Safety [same]

Nat’l

VA patients [N = 178]

2001–2004

Mult. ctrs

Private sector patients [N = 371]

2001–2004

Other surgical

30-Day morbidity and mortality

Mortality event rate was too low for adjustment. Adjusting for comorbidities, the 30-day postoperative morbidity ratio in VA versus the PS was no longer significant (adjusted OR1.33, 95 % CI 0.49–3.61 compared with unadjusted OR 2.75; 95 % CI: 1.55–4.91)

B

Weeks WB, et al., 2008

Safety [mixed]

One VISN

Male VA enrollees receiving care within VA [N = 50,429]

1998-2000

Lrg geo

Male VA enrollees receiving care outside VA [N = 74,017]

1998-2000

Patient Safety Indicators

Risk-adjusted rates of non- obstetric PSIs

Rates similar for 9 of 15 PSIs, ulcer, sepsis, iatrogenic infection, postoperative respiratory failure, postoperative metabolic derangement lower in VA, mortality higher in VA for low-risk DRGs

B

Weiss JS, et al., 2006

Safety [same]

One VISN

VA patients [N = 140]

1997–2002

Lrg geo

Private sector patients [N = 6,949]

1997–2002

Vascular

Perioperative mortality, stroke, and cardiac complications

After risk adjustment, having surgery at the VA was not a significant predictor of death (OR 2.98; 95 % CI 0.51-17.6), stroke (OR 0.95; 95 % CI 0.3–3.4), or cardiac complications (OR 1.07; 95 % CI 0.37–3.10)

B

Table abbreviations: BRFSS Behavioral Risk Factor Surveillance System, CI confidence interval, COPD chronic obstructive pulmonary disease, CABG coronary artery bypass grafting, DM diabetes mellitus, DRG diagnosis-related group, ED emergency department, FFS fee for service, ESRD end stage renal disease, HR hazard ratio, HCUP-NIS Healthcare Cost and Utilization Project Nationwide Inpatient Sample, HTN hypertension, IQI inpatient quality indicator, ICU intensive care unit, IHD ischemic heart disease, MA Medicare Advantage, MI myocardial infarction, NR not reported, OR odds ratio, PSI patient safety indicator, PS private sector, RR relative risk, TIA transient ischemic attack, VA Veterans Affairs, VISN Veterans Integrated Service Network

*We assessed each study in the review according to the statistically significant differences in performance on quality of care measures for VA care relative to a non-VA comparison group. If the VA quality of care was shown to be better than quality for non-VA care, the study was classified as “VA better.” If VA quality of care was better in some instances and the same in other instances compared to non-VA care in the same study, the study was also classified as “VA better.” If multiple quality measures were reported in the study and VA care was better than non-VA on some and worse on others, the study was classified as “mixed.” If the quality of care in the VA and non-VA did not differ, the study was classified as “same.” If the VA quality of care was shown to be worse than non-VA, the study was classified as “VA worse” (as were studies where the quality of care was worse in some instances and the same in other instances)

†The Primary Findings text has been drawn directly from the reviewed articles and in some cases may be similar or identical to the article’s text

‡Each article was given an overall assessment of quality shown in the Final Grade column. This assessment was based on the following criteria: time frames, samples (both VA and non-VA), quality measurements, outcomes, importance of measures, and statistical methods. Each of these factors was assigned a grade (A, B, or C) based on the data abstraction grading guidelines developed in the original systematic review. The overall assessment was predicated on the global assessment of the article considering the individual components, but was not an average. Thus, an article that had, for example, a critical flaw in methodology would be a “C,” even if other issues were satisfactory. Articles that received an overall grade of “C” were rejected from the review

Safety

Safety measures focused on avoiding illness/injury to patients resulting from medical care, such as complications following surgical procedures. Studies of morbidity and mortality were also included in this category, as were studies about adherence to safety guidelines or best practices. In 22 of 34 articles on safety, VA generally performed as well (11 studies) or better (11 studies) than non-VA settings. VA fared worse in nine studies and had mixed performance in three studies.

Mortality

VA facilities generally performed comparably or favorably relative to other settings in terms of mortality. Mortality rates associated with specific conditions1619 or following surgery2026 were often similar for VA patients compared with non-veterans, and occasionally lower.17 However, some studies found higher surgical mortality among VA patients compared with non-veterans.19,22,2629 Lower mortality rates were observed after cataract surgeries for VA patients compared with veterans who are Medicare fee-for-service beneficiaries.30 Mortality among veterans residing in VA nursing homes and community nursing homes was similar.31 Veterans undergoing dialysis treatment in VA and non-VA settings experienced similar mortality.32 Adjusted mortality was lower among male VA patients compared with male Medicare Advantage beneficiaries over 65 years old;3336 results were similar for females.34 Mortality within 1 year of admission after hip fracture was 21 % lower among veterans admitted to non-VA hospitals compared with VA patients.37 Mortality rates have declined more quickly in VA over time than in non-VA settings.38

Morbidity

VA facilities had mixed results with respect to most studies of morbidity. Postoperative morbidity was lower for VA patients compared with non-veterans in some surgeries,22,25,39 while for other surgeries it was similar.4044 In a study comparing quality of care in nursing homes, veterans in VA nursing homes were less likely to develop a pressure ulcer than veterans in community nursing homes.31 However, other studies found that morbidity among VA patients is worse after pancreatectomy28 and for male patients only after gastric bypass, compared to patients in private sector hospitals (unadjusted).41

Complications

VA facilities had mixed results with respect to complications after surgery. Surgical complication rates were similar among VA patients and non-veterans.21,23,29 Higher complication rates were observed for VA patients undergoing cataract surgeries than veterans who were Medicare fee-for-service beneficiaries.45 Among all kidney transplant recipients, VA patients had higher graft failure risk than non-veterans.27

Other Safety Measures

VA hospitals were more likely to follow best practices for central venous catheter bloodstream infection prevention compared with non-VA hospitals.46 However, performance on the Agency for Healthcare Research and Quality’s (AHRQ) patient safety indicators was found to be a mix of higher, lower, and similar rates at VA hospitals compared with non-VA hospitals.4749

Effectiveness

Seventeen studies showed better performance in VA facilities, while three had similar performance, one was mixed, and three were worse than non-VA settings.

Outpatient Care

Outpatient care was generally strong in VA facilities. VA patients received higher quality care than non-veterans for one measure of preventive care mammography (85–90 % vs. 64–77 %) and two measures of outpatient diabetic management (annual HbA1c test: 94–96 % vs. 70–81 %; annual eye examination: 67–85 % vs. 68–74 %) (no statistical tests).50 VA patients also received more effective care than non-veterans based on 10 of 11 quality measures in the first study year and all 12 quality measures in the second study year assessing diabetes, cardiovascular, and cancer screening care, with rate differences ranging from 4.3 percentage points (95 % confidence interval [CI] 3.2–5.4) for cholesterol testing in coronary heart disease to 30.8 (95 % CI 28.1–33.5) for colorectal cancer screening.51 Receipt of diabetes education and annual HbA1c tests was higher among VA patients compared with veterans in non-VA care.52 VA patients were more likely than veterans receiving care outside VA to receive recommended diabetes care, including being twice as likely to have a foot examination and 60–70 % more likely to have an eye examination, two or more A1c tests, and two or more providers visits,53 a routine checkup within 2 years (91.6 % among VA patients compared to 80.6 % overall; P < 0.001),54 and influenza and pneumonia vaccinations (rates increased 10–240 %),53,55,56 but similar rates of cholesterol screening.55 Blood pressure control was higher among male African-American VA patients than male African-American non-VA patients (49.4 % vs. 44.0 %, P < 0.01) though similar among Caucasians.57 VA patients were more likely than non-VA patients to receive recommended ambulatory preventive and disease management services,58 including influenza and pneumococcal vaccination.58,59 These studies used a national sample comparison group of non-VA patients, which may have included both veterans and non-veterans. Obese VA patients were more than twice as likely to have received advice to lose weight as veterans receiving non-VA care and non-veterans and equally likely to have received advice to maintain weight.60 However, in a study of veterans receiving primary care at VA-staffed versus contract community clinics, veterans with diabetes at VA-staffed clinics were less likely to receive a retinal examination (odds ratio [OR] 0.72, 95 % CI 0.55–0.93) and veterans with chronic obstructive pulmonary disease at VA-staffed clinics were less likely to receive a flu shot (OR 0.73, 95 % CI 0.55–0.99).61

Non-Ambulatory Care

In non-ambulatory settings, VA care was generally of similar quality to care provided in non-VA facilities. Compared with non-VA patients from the Medicare cancer patient database, VA patients had earlier diagnosis of colon and rectal cancers, higher rates for three quality measures, similar rates for nine, and lower rates for one.62 Comparison of an academic practice and a VA hospital found that appropriate use of stress/rest myocardial perfusion imaging studies did not differ between settings.63 Rates of hemodialysis via arteriovenous fistulas (which are preferred by guidelines over other methods) were not different between VA patients and Medicare patients after accounting for pre-end-stage renal disease care.64 VA patients received higher quality care than non-veterans for nine out of ten measures of inpatient care and performed similarly on one measure.50

Medication Management

Medication management was examined in a number of settings, including outpatient, emergency department, and inpatient. Elderly VA patients were less likely to receive inappropriate medication than were patients in Medicare HMOs,65 and VA patients with acute myocardial infarction were more likely to receive appropriate medications than were non-VA patients.66 Observed compliance by providers with erythropoietin administration guidelines was higher at VA than in the private sector.67 Antibiotic prescribing practices were generally similar between seven VA and seven non-VA emergency departments; however, in the three cities in which prescription rates were not comparable between VA and non-VA sites, VA sites had much higher rates of antibiotic prescriptions.68

Availability of Services

The structure of women’s health care differed at eight VA women’s health centers and 13 Department of Health and Human Services Centers of Excellence. Preventive cancer screening and general reproductive services were available at all centers, while VA centers were less likely to offer extensive reproductive services but more likely to offer mental health care.69 Eight of 15 clinical pharmacy services were more commonly provided in VA hospitals than non-VA hospitals [in-service education was 25 % higher (P = 0.003), clinical research 154 % higher (P < 0.0001), drug protocol management 28 % higher (P < 0.0001), drug therapy counseling 80 % higher (P < 0.0001), participation on rounds 38 % higher (P = 0.001), and admission drug histories 310 % higher (P < 0.0001)].70 VA patients and Medicare/Medicaid-insured patients were less likely to receive kidney transplants than were patients with private insurance.71

End-of-Life Care

VA tended to avoid inappropriate care and utilize palliative care at end of life. Increasing use of chemotherapy at the end of life is associated with higher rates of in-hospital deaths and later admission to hospice, which are linked to lower quality end-of-life care.7275 When comparing male VA patients and Medicare patients with lung and colorectal cancer, VA patients were less likely to receive chemotherapy within 14 days of death or to be admitted to an ICU within 30 days of death, and they were similarly likely to have more than one emergency room visit within 30 days of death.76 Among veterans who died in VA facilities, palliative care consults (67 % vs. 21 %, P < 0.001) and death in a dedicated palliative care, hospice unit, or intensive care unit were more common (47 % vs. 16 %, P < 0.001), and death in a nursing home was less common (10 % vs. 26 %, P < 0.001) than among veterans who died in non-VA facilities (all unadjusted results).77

Timeliness, Equity, Efficiency, and Patient-Centeredness

Since fewer than ten studies were found that related to each of the dimensions of timeliness, equity, efficiency, and patient-centeredness, we do not discuss the results of these studies in detail (results and evidence tables in Online Appendix). The single study that addressed timeliness of care showed worse performance among veterans in VA facilities relative to non-VA facilities for time between hospital admission for hip fracture and surgical repair.37 Four studies examined equity, with one study showing better performance on several chronic disease and preventive measures,51 one study showing worse performance in VA facilities for emergency room visits among cancer patients in the last month of life,76 and two studies showing similar equity in 30-day post-admission mortality78 and graft failure.27 Nine articles compared efficiency using utilization, with six studies showing worse performance in VA facilities on outcomes such as average length of stay and service utilization for patients on dialysis,32,61,7982 two showing better performance for visits/admissions83 and generic drug utilization,84 and one with mixed results for utilization-related inpatient quality indicators. Five studies examined patient-centeredness; three studies demonstrated better patient and family satisfaction in VA facilities77,85,86 and two demonstrated similar performance in perceptions of racial discrimination and satisfaction in transitional programs for homeless veterans.87,88

DISCUSSION

In this systematic review, we identified 69 studies comparing quality of care provided in VA facilities to non-VA care. Studies of safety and effectiveness indicated generally favorable performance by VA facilities compared to non-VA facilities, with 22 of 34 studies of safety and 20 of 24 studies of effectiveness showing the same or better quality of care in VA facilities and the remainder showing mixed (four studies) or worse (12 studies) performance. In terms of safety, VA facilities performed similarly or better in most, but not all, studies comparing morbidity and mortality. Results from studies about complications and patient safety indicators were more mixed. In terms of effectiveness, VA facilities had similar or superior quality to non-VA facilities with respect to preventive, recommended, and end-of-life care, as well as managing medications. Non-ambulatory care studies indicated similar care quality between VA and non-VA settings. However, studies on service availability had mixed results. A comparison of studies included in the previous review, which included studies from 1990 to 2009, and those identified in this review (2005 to 2014) showed that while numerous studies have been published in the last 5 years, no stark trends can be observed in terms of whether older or newer studies demonstrate systematically better or worse performance in VA settings (Table 2).
Table 2

Comparison of studies included in the prior and current review

 

Outcomes

Articles

Better

Same

Mixed

Worse

Total

Safety

 Studies included in prior review

7

6

2

4

19

 Studies identified in current review

4

5

1

5

15

Effectiveness

 Studies included in prior review

8

0

0

1

9

 Studies identified in current review

10

2

2

2

16

In keeping with the findings of the prior review, we find that VA is generally more adherent to recommended care processes than other systems of care. However, better processes did not necessarily achieve better outcomes, as we observed few differences in adjusted mortality. Although easy to obtain, mortality may not be the optimal outcome for comparison for several reasons, some of which were noted in the prior review.79 While nearly every study we included attempted to risk-adjust the veteran and non-veteran populations, some excess baseline mortality risk may still be present, since veterans have comparatively worse health status than the general population.8991 Mortality rates are also known not to be sensitive in detecting differences in the quality of health care provided.9294 Other outcomes, such as complications specific to a disease or procedure of interest, might be more revealing about meaningful quality differences in VA settings than those included in this review.

Our review builds on a previous review conducted in this area,79 but both updates and expands upon it. Updating the results of the prior review up to 2015 is critically important given the ongoing interest in the quality of care provided at the VA.95 In addition to categorizing the studies published since the prior review, we also categorized the studies contained in the prior review to provide a broad picture of quality at the VA over the last decade along the dimensions of health care quality proposed by the IOM.11. This categorization allows us to highlight the gaps in research by dimension, which may help inform the investment of resources for research and improvement by VA stakeholders.

This study is subject to a number of limitations. The search strategy employed was narrowly defined to align with the methods of the previous review, but this may have excluded relevant studies. Most studies were not conducted with perfectly matched comparison groups (i.e., veterans receiving care in VA and non-veterans in non-VA settings). While we used risk-adjusted or otherwise comparable results wherever possible, this may not have accounted for unobservable differences between veterans and non-veterans. Although we included confidence intervals or levels of significance in the evidence tables (Table 1 and Online Appendix), it was not always possible to determine whether lack of a significant difference was due to an underpowered study versus evidence of a lack of a difference. However, while many of the studies used large, nationally representative samples, some of the smaller studies did observe significant differences. We also did not assess whether the magnitudes of differences were clinically significant, as the thresholds for clinical differences for many of the outcomes investigated may be subjective. In deciding which studies to include, we assessed the study design of each (Table 1) and excluded low-quality studies from this review. However, we did not systematically assess each study’s risk of bias beyond that. It is notable that all but 4 of the 69 articles included had at least one VA-affiliated author or were funded directly by VA.

Nonetheless, the available data indicate overall comparable health care quality in VA facilities compared to non-VA facilities with regard to safety and effectiveness. Rates of complications and availability of services had the least favorable results, but these results were mixed rather than consistently poor. The overall number of studies comparing VA and non-VA care was small, and study quality varied. More studies that examine and compare the quality of VA care with respect to timeliness, equity, efficiency, and patient-centeredness are needed to better assess VA facilities’ performance on these quality dimensions.

Acknowledgments

The authors gratefully acknowledge the authors of the original systematic review on which this systematic review was based: Paul Shekelle, Steven Asch, Peter Glassman, Sierra Matula, Amal Trivedi, and Isomi Miake-Lye. The authors would also like to thank Jody Larkin, Roberta Shanman, Brian Briscombe, Aneesa Motala, Ian Stefanison, and Henry Ebron for their assistance.

Author Contributions

Dr. Gidengil had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design; drafting of the manuscript: CG, CO, CH

Acquisition of data; analysis and interpretation of data: CG, CO, CH

Critical revision of the manuscript for important intellectual content: RAP, ES, PH, CF

Administrative, technical, or material support: PH, CF

Study supervision: CG, RAP, ES, PH, CF

Financial Disclosures

None reported.

Funding/Support

This work was completed under a subcontract from The MITRE Corp. for the US Department of Veterans Affairs as called for by the Veterans Access, Choice, and Accountability Act of 2014 Section 201. The report was prepared under Prime Contract No. HHS-M500-2012-00008I, Prime Task Order No. VA118A14F0373.

Role of the Sponsor

The funding organization had no role in the design or conduct of the study; the collection, analysis, or interpretation of the data; or the preparation of the manuscript. VA had the opportunity to review the manuscript before submission, but submission for publication was not subject to VA approval.

Compliance with Ethical Standards

Conflict of Interest

The authors declare no conflicts of interest.

Supplementary material

11606_2016_3775_MOESM1_ESM.docx (85 kb)
ESM 1(DOCX 85 kb)

Copyright information

© Society of General Internal Medicine 2016

Authors and Affiliations

  • Claire O’Hanlon
    • 1
    • 2
  • Christina Huang
    • 1
    • 2
  • Elizabeth Sloss
    • 3
  • Rebecca Anhang Price
    • 3
  • Peter Hussey
    • 4
  • Carrie Farmer
    • 5
  • Courtney Gidengil
    • 4
  1. 1.Pardee RAND Graduate SchoolSanta MonicaUSA
  2. 2.RAND Corp.Santa MonicaUSA
  3. 3.RAND Corp.ArlingtonUSA
  4. 4.RAND Corp.BostonUSA
  5. 5.RAND Corp.PittsburghUSA

Personalised recommendations