Advertisement

Effectiveness of Models Used to Deliver Multimodal Care for Chronic Musculoskeletal Pain: a Rapid Evidence Review

  • Kim PetersonEmail author
  • Johanna Anderson
  • Donald Bourne
  • Katherine Mackey
  • Mark Helfand
Review Paper

Abstract

Background

Primary care providers (PCPs) face many system- and patient-level challenges in providing multimodal care for patients with complex chronic pain as recommended in some pain management guidelines. Several models have been developed to improve the delivery of multimodal chronic pain care. These models vary in their key components, and work is needed to identify which have the strongest evidence of clinically-important improvements in pain and function. Our objective was to determine which primary care-based multimodal chronic pain care models provide clinically relevant benefits, define key elements of these models, and identify patients who are most likely to benefit.

Methods

To identify studies, we searched MEDLINE® (1996 to October 2016), CINAHL, reference lists, and numerous other sources and consulted with experts. We used predefined criteria for study selection, data abstraction, internal validity assessment, and strength of evidence grading.

Results

We identified nine models, evaluated in mostly randomized controlled trials (RCTs). The RCTs included 3816 individuals primarily from the USA. The most common pain location was the back. Five models primarily coupling a decision-support component—most commonly algorithm-guided treatment and/or stepped care—with proactive ongoing treatment monitoring have the best evidence of providing clinically relevant improvement in pain intensity and pain-related function over 9 to 12 months (NNT range, 4 to 13) and variable improvement in quality of life, depression, anxiety, and sleep. The strength of the evidence was generally low, as each model was only supported by a single RCT with imprecise findings.

Discussion

Multimodal chronic pain care delivery models coupling decision support with proactive treatment monitoring consistently provide clinically relevant improvement in pain and function. Wider implementation of these models should be accompanied by further evaluation of clinical and implementation effectiveness.

KEY WORDS

multimodal multidisciplinary musculoskeletal pain chronic pain rapid review 

INTRODUCTION

In light of the current prescription opioid overdose epidemic, chronic non-cancer pain is a public health concern of growing importance. When defined as pain lasting longer than 3 months, an estimated 100 million US adults suffer from chronic pain, at a cost of $560 to $635 billion per year due to medical treatment and lost productivity.1 Chronic pain is complex, involving dynamic interactions between biological, psychological, and social factors unique to each individual,2 and most patients suffer from comorbid conditions.36 To address this complexity, some pain management guidelines recommend multimodal pain care,1, 79 which is typically defined as the use of more than one type of therapy and can include more than one discipline when available (multidisciplinary). Common modalities include self-management; complementary and integrative health; pharmacological, psychological, physical, or restorative therapy; and procedural treatments. Primary care providers (PCPs) are responsible for the majority of pain management.10 However, PCPs face many system- and patient-level challenges in providing the recommended multimodal interventions.1, 10, 11 In their 2011 “Blueprint for Transforming Prevention, Care, Education, and Research,” the Institute of Medicine’s Committee on Advancing Pain Research, Care and Education found that reimbursement limitations and short primary care visits often provide inadequate time and resources for complex treatment planning and coordination of multimodal care, monitoring, and patient education and activation activities.12 A 2015 Patient-Centered Outcomes Research Institute (PCORI) multistakeholder workgroup suggested that system-level care delivery models are needed to support PCPs and provide better tools for managing chronic pain.11 Ideally, system-level multimodal chronic pain care delivery models would include (1) medication management, (2) access to multidisciplinary treatment modalities tailored to comorbidity needs, (3) decision support for clinicians, (4) self-management support for patients, (5) dedicated care management staffing to support ongoing interdisciplinary treatment coordination and monitoring, and (6) health information technology (HIT) enhancements, such as telecare, to better facilitate timely communication and monitoring. In the past few years, several such care delivery models have emerged. However, these differ substantially in their key components, and work is needed to identify which have the best evidence of clinically important improvements in pain and function.

We reviewed evidence of the effectiveness of models for improving the delivery of multimodal chronic pain care within the primary care setting. Our objective is to determine which multimodal care delivery models, integrated within the primary care setting, provide clinically relevant benefits in pain and function and minimize unintended consequences in adults with chronic musculoskeletal pain; to define key elements of these models; and to identify patients who are most likely to benefit.

METHODS

This rapid review13 was conducted in response to an urgent request by a US Department of Veterans Affairs’ Health Services Research & Development (HSR&D) Pain Management State-of-the-Art (SOTA) committee for timely and accessible evidence for use in informing their November 2016 conference. Although guided by current standard AHRQ systematic review methods,14 to meet a condensed timeframe of 3 months, we streamlined our process by focusing on a subset of the highest priority outcomes and settings and used sequential instead of independent dual review processes to minimize bias and error. We report this review based on PRISMA guidelines.15 The complete description of our methods can be found on the PROSPERO international prospective register of systematic reviews (http://www.crd.york.ac.uk/PROSPERO/; registration number CRD42016050272) and in our full evidence report.16

We searched MEDLINE® (Ovid) and CINAHL from 1996 to October 2016 using terms for chronic pain and health care delivery models (e.g., “primary care planning,” “disease management,” “multi-component,” “chronic pain”). We searched numerous other sources, including Cochrane Database of Systematic Reviews, Agency for Healthcare Research and Quality, Google Scholar, and more to identify existing systematic reviews and gray literature; a complete list of our search strategy can be found in our full report.16 Additional citations were identified from hand-searching reference lists and consultation with content experts. We limited the search to articles involving human subjects available in the English language.

Study selection was based on the eligibility criteria described in Table 1. We abstracted data from all included studies on the setting, model components, follow-up duration, patient demographics, comorbidities, pain characteristics, and results for each included outcome. We rated study quality using the Drug Effectiveness Review Project methods for randomized controlled trials (RCTs)18 and risk of bias using the  Cochrane’s Risk of Bias Tool for cohort studies.12, 19, 20 Study selection, data abstraction, and quality assessment were first completed by one reviewer and checked by at least one additional reviewer. Disagreements were resolved by consensus.
Table 1

Eligibility Criteria

Population: Adults with chronic musculoskeletal pain (persistent for 3 months or longer)

• Potential effect modifiers of interest include (1) the specific location and/or type of pain, (2) patient demographics (e.g., age, race, ethnicity, and gender), (3) patient comorbidities (including past or current alcohol or substance use disorders, mental health disorders, medical comorbidities, and those at high risk for substance use disorders).

Intervention: Any model with system-based mechanisms aiming to increase the uptake and organization of multimodal care (e.g., collaborative care, care management, integrated care, telecare, peer-delivered care, informal caregiving, stepped care models, and algorithms).

Comparator: Any.

O utcomes:

• Effectiveness: Percentages of patients obtaining reductions in pain intensity and pain-related function from baseline of at least 30 or 50%,17 quality of life, depression, anxiety, sleep, and opioid doses.

• Unintended consequences: Adverse effects on patient satisfaction, provider satisfaction, time burden, sustainability.

Timing: Any study follow-up durations.

Setting: Integrated within primary care; not to include interventions occurring entirely within intensive pain rehabilitation, specialty, or tertiary care.

Study design: Systematic reviews, randomized controlled trials, or concurrently-controlled cohort studies.

We graded the strength of the evidence based on the AHRQ Methods Guide for Comparative Effectiveness Reviews.21 This approach incorporates five key domains: study limitations (includes study design and aggregate quality), consistency, directness, precision of the evidence, and reporting biases. Ratings range from high to insufficient, reflecting our confidence that the evidence reflects the true effect. Strength of evidence ratings was first completed by one reviewer and then checked by another, and we resolved disagreements using consensus.

Models of multimodal chronic pain care differ substantially in the types of system interventions they use to promote guideline-concordant multimodal chronic pain management in the primary care setting. Sources of heterogeneity include number, breadth, intensity, frequency, and duration of components. This type of heterogeneity is often characteristic of complex multicomponent interventions and can be a challenge to constructing a framework for organizing the evidence synthesis. This is because interventions can be conceptually lumped or split by various types of characteristics and there is no agreed-upon single best approach for doing so.22 Given the model heterogeneity, we did not perform any meta-analysis. Instead, we qualitatively described the model characteristics, organized by the four most common system interventions they used to promote guideline-concordant multimodal chronic pain management (Table 3): (1) decision support, (2) additional care coordination resources, (3) enhanced patient education and activation, and (4) increased access to a broader range of treatments. We summarized effects on outcomes by grouping them into three categories based on the models’ main focus: (1) decision support coupled with proactive symptom monitoring, (2) risk-matched treatment pathways, and (3) increased access.

RESULTS

Study design and quality

Searches resulted in 901 potentially relevant articles (Fig. 1). Of these, we included eight RCTs (in ten publications)2, 2325, 2729, 3133 and one retrospective cohort study.26 Overall, most studies were fair or good quality (Table 2, Fig. 2). Common limitations among fair-quality studies included greater than 20% attrition and baseline differences in potential prognostic factors. Poor-quality studies also excluded 34 to 47% of patients from analyses. Fidelity to case management protocols was adequate but was rarely mentioned for provider training and enactment of skills. Despite robust methodology, the strength of the evidence is generally low as each intervention is only supported by a single, imprecise study. All but one study27 were randomized at the patient level, and most interventions were compared to usual care, often minimally described as regular access to primary and specialty care.
Fig. 1

Literature flowchart.

Table 2

Characteristics of Included Studies

Author year

N

Setting

Major intervention components

Follow-up (months)

Gender (% male)

Mean age (years)

Baseline pain intensity*

Mental health comorbidities

Study quality

Ahles 200123

396

4 PC practices (USA)

Computer-based tailored “prescription” algorithm + nurse educator

6

39

49

NR

27% emotional distress

Poor

Ahles 200624

1066

14 PC practices (USA)

Computer-based tailored “prescription” algorithm + nurse educator

12

48

48

NR

1% SUD

Fair

Angeles 201325

63

Single center (Canada)

Group multidisciplinary education co-facilitated by an occupational therapist and a social worker

6

38

55

NR

19.3% possible or probable SUD

Poor

Bair 20152 (ESCAPE)

241

5 GM clinics (Ind. VAMC)

Stepped care with analgesics, self-management, and CBT delivered by 2 NCM

9

88

37

6.6

Mean PTSD Scorea = 26.4

Mean depression Scoreb = 11.2

Good

Burnham 201026 (CAPRI†)

82

Single rural center (Canada)

Weekly multidisciplinary group sessions added to analgesic optimization

18

31

47

7.7

NR

Poor

Dobscha 200927 (SEACAP)

401

5 PC clinics (Portland VAMC)

Collaborative care delivered by psychologist care manager

12

92

62

5.2

18% MDD, 16% PTSD

Good

Hay 200628

216

15 practices (England)

Pharmacist-led pharmacological treatment optimization

12

36

62

6.1

NR

Fair

Hill 201129 (STarT Back)

851

10 practices (England)

Physiotherapist-led stratified care using STarT Back Screening Tool

12

41

50

5.3

NR

Fair

Kroekne 200930 (SCAMP)

250

5 GM clinics (Ind. VAMC)

Stepped care with antidepressants and self-management delivered by a NCM

12

47

56

6.2

75% MDD

Mean anxiety scorec = 8.9

Good

Kroenke 201431 (SCOPE)

250

5 PC clinics (Ind. VAMC)

Automated symptom monitoring and optimized analgesic management by NCM and PC pain specialist team

12

83

55

5.1

24% MDD, 17% PTSD

Good

Table does not include Thielke 2015, secondary publications of already included studies

Abbreviations: PC primary care, NR not reported, SUD substance use disorder, ESCAPE Evaluation of Stepped Care for Chronic Pain, GM general medicine, CBT cognitive behavioral therapy, NCM nurse case manager, PTSD post-traumatic stress disorder, CAPRI Central Alberta Pain and Rehabilitation Institute, Ind. = Indianapolis, OBS observational, SEACAP Study of the Effectiveness of a Collaborative Approach to Pain, MDD major depressive disorder, SCAMP Stepped Care for Affective Disorders and Musculoskeletal Pain, SCOPE Stepped Care to Optimize Pain Care Effectiveness, GADS Generalized Anxiety Disorder scale

*Mean score on a 10-point scale

†All studies RCT except Burnham 2010 = retrospective cohort

aDetermined using the Posttraumatic Stress Disorder Check List–17. Scores range from 0 to 68

bDetermined using Patient Health Questionnaire–9. Scores range from 0 to 27

cDetermined using the Generalized Anxiety Disorder scale. Scores range from 0 to 21

Fig. 2

Risk of bias assessment of included RCTs.

Setting and Subjects

Most studies involved multiple primary care practices in the USA2, 23, 24, 27, 30, 31 or England (Table 2).28, 29 Four interventions were evaluated within either the Indianapolis Veterans Affairs Medical Center (VAMC)2, 30, 31 or the Portland VAMC.27 Two studies took place in single centers in Canada.25, 26 Sample sizes were ≤ 250 patients in the majority of the studies (range, 63 to 1066). Follow-up duration was 12 months in the majority of studies (range, 6 to 18 months). The proportion of male patients ranged from 31 to 92%, with higher proportions in those studies within the VA. The mean patient age ranged from 37 to 62 years old. Most studies reported baseline pain intensity (n = 7), which ranged from 5.1 to 7.7 on a 10-point scale. Commonly reported mental health comorbidities were major depressive disorder, post-traumatic stress disorder, and substance use disorder, but baseline prevalence of these conditions was low in most studies (range, 1 to 24%). The exception was that one study specifically targeted patients with comorbid musculoskeletal pain and depression.30

Overview of Multimodal Chronic Pain Care Model Components

We identified nine diverse models of multimodal chronic pain care (Tables 3 and 4). All but one model25, 29 involved multiple components for improving pain care delivery. The majority of interventions included a decision-support component—most commonly algorithm-guided treatment and/or stepped care—coupled with proactive ongoing treatment monitoring.2, 23, 24, 27, 28, 30, 31 For the 55% of models that involved algorithm-guided care,2, 26, 28, 30, 31 all but one focused on analgesic optimization. A typical stepped care analgesic optimization algorithm starts with acetaminophen and nonsteroidal anti-inflammatory drugs and progresses to (1) tricyclic antidepressants or cyclobenzaprine, (2) tramadol, (3) gabapentoids, (4) topical analgesics, and (5) opioid. Treatment is advanced at each step when the patient meets the prespecified criteria of less than 30% improvement in pain score, global improvement not at least moderate or better, or patient desire for a change in treatment.31 In two studies, the decision support was in the form of a stratified approach by way of prognostic screening with matched treatment pathways.26, 29 One stratified model29 focused on adults with back pain from ten general practices within the Keele General Practice Research Partnership in England and used the validated Keele STarT Back Screening Tool, which is a nine-item inventory that queries patients about referred leg pain, comorbid pain, disability (two items), bothersomeness, catastrophizing, fear, anxiety, and depression to categorize patients into low-, medium-, and high-risk groups.20 Alternatively, the stratified approach used in the Central Alberta Pain and Rehabilitation Institute (CAPRI) triaged patients using an unspecified 1.5- to 2-h assessment process to differentiate one of four care pathways based on the extent of their medication management, psychosocial, and/or comorbid medical illness issues (i.e., minimal, high, complex).26 The most intense example of active provider education was from the Study of the Effectiveness of a Collaborative Approach to Pain (SEACAP) study, in which providers participated in two 90-min education sessions addressing chronic pain care skills and knowledge, modeled after the MacArthur Depression Education Program.27, 34
Table 3

Chronic Pain Care Model Processes

Author and Year 

Decision support

Increasing access to and coordination of multimodal care

Additional care coordination resources

Active patient education, activation

Ahles 2001/200623, 24

Algorithm-guided treatment recommendations; nurse educator support for patients with psychosocial problems

 

Weekly telephone contact with nurse educator

 

Angeles 201325

 

Centralization: multidisciplinary program developed by available providers, tailored to setting, delivered by group visits

 

Group sessions

Bair 20152 (ESCAPE)

Algorithm-guided stepped carea with analgesics and CBT, delivered by NCM

 

Biweekly by NCM

 

Burnham 201026 (CAPRI)

4 care pathways based on complexity: (1) self-management, (2) spinal block, (3) medication management, (4) multidisciplinary care

Establishment of a multidisciplinary program in a rural setting

Weekly for complex patients

Weekly 5-h group multidisciplinary sessions for complex patients

Dobscha 200927

(SEACAP)

Clinician education; stepped carea; expert decision support

 

Every 2 months by psychologist and internist team

Optional 4-session group workshop

Hay 200628

Enhanced pharmacy review: pharmacist-led and algorithm-guided

 

Biweekly by pharmacist and nurse

3–6 20-min sessions with pharmacist

Hill 201129 (STarT Back)

Risk stratification using validated tool; risk-matched treatment pathways

   

Kroenke 200930 (SCAMP)

Algorithm-guided stepped carea with antidepressants and self-management

 

Biweekly to monthly by depression pain clinical specialist

6 30-min sessions with NCM

Kroenke 201431 (SCOPE)

Algorithm-guided stepped carea with analgesics

 

Automated monitoring via IVR or internet that would prompt nurse contacts

 

aStepped care refers to the sequential progression of treatment intensiveness according to an algorithm. For example, step 1 may involve self-management strategies alone and subsequent steps involving adding medications from first-line analgesic medications up to adding long-acting opioids

Abbreviations: NCM nurse case manager, CBT cognitive behavioral therapy, IVR interactive voice response, ESCAPE Evaluation of Stepped Care for Chronic Pain, SEACAP Study of the Effectiveness of a Collaborative Approach to Pain, SCAMP Stepped Care for Affective Disorders and Musculoskeletal Pain, SCOPE Stepped Care to Optimize Pain Care Effectiveness, CAPRI Central Alberta Pain and Rehabilitation Institute

Table 4

Summary of Common Model Components

Author year

Facilitate interaction between providers

Primary care provider education, activation

Pharmaco-therapy algorithm

Proactive symptom monitoring frequency

Case management team

Mental health treatment

Patient self-management supporta

Ahles 2001/200623, 24

NS

Passive

NS

UC: NS

Int: weekly, descending

UC: PCP

Int.: PCP, nurse

UC: NS

Int: required

UC: passive

Int: both

Angeles 201325

Weekly CM

Active

NS

Weekly for 8 weeks

Occupational therapist, social worker

NS

Active

Bair 20152 (ESCAPE)

Weekly CM

NS

Analgesic

Biweekly

Nurses

Required CBT

Passive

Burnham 201026 (CAPRI)

Int. 1*: NS

Int. 2*: weekly CM

Active

Arm 1: analgesic

Arm 2: NS

Int. 1*: NS

Int. 2*: weekly for 12 weeks

Int. 1*: PCP

Int. 2*: PCP, physiatrist, psychologist, physical therapist, kinesiologist, nurse, dietician

Int. 1*: NS

Int. 2*: required, 1+ h psychotherapy

Int. 1*: passive

Int. 2*: both

Dobscha 200927

(SEACAP)

NS

Active

NS

Every 2 months

Psychologist, internist

Optional

Both

Hay 200628

NS

NS

Analgesic

Biweekly

Pharmacist, nurse

NS

Both

Hill 201129 (STarT Back)

NS

Active

NS

NS

Physiotherapist, nurse

Required, high-risk patients received “psychologically informed physiotherapy”

Passive

Kroenke 200930 (SCAMP)

Weekly CM

NS

Antidepressant

Biweekly to monthly

Depression-pain clinical specialist

Optional

Active

Kroenke 201431 (SCOPE)

Weekly CM

NS

Analgesic

Automated, descending

Nurse, physician pain specialist

Optional

Passive

*Intervention 1 = supervised medication management, intervention 2 = full multidisciplinary program

a“Active” patient education refers to patient participation in the learning process, such as choosing and practicing self-care strategies. “Passive” patient education refers to the provision of educational materials to the patients for independent study, such as in the form of a written guide

Abbreviations: NS none specified, UC usual care, Int. = intervention, CM case management, CBT cognitive behavioral therapy, HIT health information technology, PCP primary care provider, ESCAPE Evaluation of Stepped Care for Chronic Pain, SEACAP Study of the Effectiveness of a Collaborative Approach to Pain, SCAMP Stepped Care for Affective Disorders and Musculoskeletal Pain, SCOPE Stepped Care to Optimize Pain Care Effectiveness, CAPRI Central Alberta Pain and Rehabilitation Institute

In the majority of studies, designated case managers from various disciplines delivered the treatment monitoring component of the intervention primarily via phone contacts at various frequencies, from weekly to every 2 months. One notable exception was in the Stepped Care to Optimize Pain Care Effectiveness (SCOPE) study in which patients in the intervention group underwent automated symptom monitoring, either by interactive voice-recorded telephone calls or by the Internet, which prompted live case manager follow-up on an as-needed basis.31 Monitoring typically comprised assessment of changes in pain severity and interference, global improvement, and patients’ adherence to and desire for a change in treatment. In half of the models, patients participated in “active” education, in which the patient participated in the learning process, such as choosing and practicing self-care strategies. Most active patient education was in the form of group education sessions. An example patient education curriculum included learning about pain triggers and flare-ups, emotional coping mechanisms, strategies for physical activity, muscle relaxation, deep breathing, distraction, and sleep hygiene, and how to work with employers and clinicians.30 The other half of the models included “passive” patient education, in which educational materials were provided to the patients for independent study, such as a written guide. Two models featured increasing capacity for25 and access to26 multimodal care. The McMaster Family Health Team (MFHT) in Hamilton, Ontario, sought to use existing resources to increase capacity and access to multimodal care by centralizing services via weekly 2-h group sessions that incorporated physician, pharmacist, dietician, and physiotherapist resource persons.25 The CAPRI represents an example of a Canadian health region administration providing funding for developing a new multidisciplinary program designed specifically to increase access to multimodal chronic pain care in a previously underserved rural setting in Lacombe, Alberta. It featured decision support via risk stratification with matched treatment pathways and weekly symptom monitoring and weekly 5-h group multidisciplinary education and activation sessions as needed.26

Patient Outcomes

Decision Support Coupled with Proactive Treatment Monitoring

Among the six models that coupled decision support with proactive treatment monitoring, the proportion of patients with clinically significant improvement in pain intensity or pain-related function based on a 30% or greater reduction in scores on the Roland-Morris Disability Questionnaire (RMDQ), Brief Pain Inventory (BPI), or Outcome Measures in Rheumatology-Osteoarthritis Research Society International (OMERACT-OARSI) was significantly increased in Evaluation of Stepped Care for Chronic Pain (ESCAPE),2 SEACAP,27 Stepped Care for Affective Disorders and Musculoskeletal Pain (SCAMP),30 and SCOPE,31 (NNT range, 4.1 to 12.7 in 12 months), was unchanged in a model that emphasized enhanced pharmacy review and physiotherapy,28 and was unmeasured in model that emphasized rapid assessment and management via computer-based assessment (Table 5, Figure 3).24 In the model that emphasized rapid computer-based assessment, pain intensity and function were measured based on the Medical Outcomes Study SF-36.24 Improvement from baseline on the bodily pain score (lower score = higher levels of pain) was greater in the intervention group compared to the control at 6 months (7.6 vs 2.2; P = 0.011), but not at 1 year (7.8 vs 3.6; P = 0.06). Intervention group patients also improved significantly more in the functional interference estimate (range 0 = “pain usually or severely interferes” to 6 = “pain rarely interferes”) at 6 months (0.96 vs − 0.98; P = 0.027) and 1 year (1.5 vs 0.65; P = 0.02). Three of the models27, 30, 31 also showed improvements on at least one of the additional important outcomes of quality of life,30, 31 depression,27, 30, 31 anxiety,30 and sleep.31
Table 5

Summary of Findings (Intervention vs Control)

Author year

Clinically significant* improvement in pain intensity or pain-related function

QOL

Depression

Anxiety

Sleep

Opioid use

Unintended consequences/treatment satisfaction

Decision support coupled with case management

Ahles 200123

NR

SF-36 mean:

Pain component: 59.7 vs 46.9, P < 0.005

Role physical: 54.8 vs 37.5, P < .03

Role emotional: 81.9 vs 62.0, P < 0.001

Role social: 79.5 vs 64.5, P < 0.001

NR

NR

NR

NR

NR

Ahles 200624

NR

SF-36 mean change:

Role emotional: 13.9 vs 3.8, P = 0.046

Vitality: 7.4 vs 3.7, P = 0.048

NR

NR

NR

NR

NR

Bair 20152 (ESCAPE)

RMDQ: RR = 1.52 (95% CI 1.22 to 1.99)

NNT = 8 (95% CI, 4 to 294)

NR

NR

NR

NR

NR

NR

Dobscha 200927

(SEACAP)

RMDQ: 21.9 vs 14.0%, P = 0.04

NNT = 13 (95% CI 7 to 271)

Mean change EQ-5D: − 0.02 vs − 0.04, P = 0.17

Mean change PHQ-9: − 3.7 vs − 1.2, P = 0.003

NR

NR

Any opioid prescribed: 65 vs 61%, P = 0.56

Mean change global treatment satisfaction: − 0.27 vs − 0.36, P = 0.44

Hay 200628

OMERACT-OARSI (high improvement): 27 vs 28%; P = 0.8

NR

HADS depression:†

0.01 (95% CI − 0.7 to 0.7)

HADS anxiety:†

− 0.23 (95% CI − 1.1 to 0.6)

NR

NR

Satisfaction with treatment:†

− 19% (95% CI −32 to −4)

Kroenke 200930 (SCAMP)

BPI: 41.5 vs 17.3%; RR = 2.4 (95% CI 1.6 to 3.2)

NNT = 4.1 (95% CI 3.0 to 6.5)

SF-36:**

General health: 11.1 (95% CI 4.2 to 18.0)

Social functioning: 6.1 (95% CI −1.3 to 13.5)

Vitality: 8.8 (95% CI 3.6 to 14.0)

≥ 50% decrease in HSCL-20 from baseline:

RR = 2.3 (95% CI 1.5 to 3.2)

GAD-7:**

− 2.2 (95% CI − 3.5 to − 0.9)

NR

Any opioid prescribed: 54 vs 53%, P = 0.35

NR

Kroenke 201431 (SCOPE)

BPI: 51.7 vs 27.1%; RR = 1.9 (95% CI 1.4 to 2.7)

NNT = 4.1 (95% CI 3.0 to 6.4)

SF-12:**

Physical: 2.5 (95% CI 0.0 to 5.0)

Mental: 0.2 (95% CI − 2.9 to 3.3)

SF-36:**

Social functioning: 5.3 (95% CI − 1.6 to 12.2)

Vitality: 2.2 (95% CI − 3.9 to 8.2)

PHQ-9:**

− 1.8 (95% CI − 3.4 to − 0.2)

GAD-7:**

− 0.7 (95% CI − 1.9 to 0.5

PROMIS sleep:**

− 1.0 (95% CI − 2.0 to 0.0)

Mean # of months taking opioids: 2.0 vs 1.6, P = 0.27

NR

Risk/complexity-matched treatment pathways

Burnham 201026 (CAPRI)

NR

NR

NR

NR

NR

NR

NR

Hill 201129

RMDQ: 65 vs 57%; OR = 1.48 (95% CI 1.02 to 2.15)

NNT = 10.8 (95% CI 5.8 to 206)

SF-12:**

Physical: − 2.93 (95% CI − 4.31 to − 1.56)

Mental: − 0.69 (95% CI − 2.39 to 1.01)

HADS depression:**

0.62 (95% CI 0.07 to 1.17)

HADS anxiety**:

0.45 (95% CI − 0.10 to 1.01)

NR

NR

Satisfaction with care (not satisfied): 27 vs 36%

Increasing access via group multidisciplinary intervention sessions

Angeles 201325

NR

SF-36 mean change:

Physical: − 15.3 vs 3.4, P = 0.01

Emotional: 2.6 vs 3.7, P = .92

Social: 3.2 vs 2.7, P = 0.95

Mental: 3.6 vs 3.6, P > 0.99

NR

NR

NR

NR

NR

Abbreviations: QOL quality of life, RMDQ Roland-Morris Disability Questionnaire, SF-36 Medical Outcomes Study Short form-36, SF-12 Short form-12, EQ-5D EuroQol health-related quality of life, PHQ-9 Patient Health Questionnaire-9, OMERACT-OARSI Outcome measures in rheumatology-Osteoarthritis Research Society International, HADS hospital anxiety and depression, BPI brief pain inventory, HSCL-20 Hopkins symptom checklist, GAD-7 generalized anxiety disorder, PROMIS patient-reported outcomes measurement information system, ESCAPE Evaluation of Stepped Care for Chronic Pain, SEACAP Study of the Effectiveness of a Collaborative Approach to Pain, SCAMP Stepped Care for Affective Disorders and Musculoskeletal Pain, SCOPE Stepped Care to Optimize Pain Care Effectiveness, CAPRI Central Alberta Pain and Rehabilitation Institute

*≥ 30% decrease from baseline

**Between-group mean difference (intervention-control)

†Between-group mean difference (control-intervention)

Fig. 3

Forest plot: improvement in pain and pain-related function. Measured by Roland-Morris Disability Questionnaire. †Measured by Brief Pain Inventory, ‡Rate of improvement for model vs usual care. Abbreviations: ESCAPE = Evaluation of Stepped Care for Chronic Pain; NNT = number needed to treat; NR = not reported; RCT = randomized controlled trial; RB = relative benefit; SEACAP = Study of the Effectiveness of a Collaborative Approach to Pain; STarT Back = stratified primary care management for low back pain; SCAMP = Stepped Care for Affective Disorders and Musculoskeletal Pain; SCOPE = Stepped Care to Optimize Pain Care Effectiveness.

Risk/Complexity-matched Treatment Pathways

Risk stratification coupled with risk-matched treatment pathways using the validated STarT Back screening tool for back pain resulted in greater clinically significant improvement in pain intensity or pain-related function (≥ 30% decrease in RMDQ scores) than non-stratified current best practice at 12 months.29 The STarT Back intervention also improved depression scores and quality of life at 12 months. However, the StarT Back intervention did not impact anxiety or satisfaction with care. Findings for STarT Back are limited by greater than 20% attrition and are most applicable to females with a mean age of 50 years and unknown mental health comorbidities. Another approach for risk stratification, CAPRI, has not been evaluated in a comparative study.26

Increasing access via group multidisciplinary intervention sessions

The McMaster Family Health Team (MFHT) in Hamilton, Ontario, implemented weekly group sessions to increase access to and coordination of specialty services.25 After 6 months of follow-up, there was a statistically significant improvement in the SF-36 physical domain. But this model’s single small study (N = 63) with only 50% adherence provides insufficient evidence on which to draw conclusions.

DISCUSSION

To our knowledge, this is the first review to evaluate the effectiveness of models to improve multimodal chronic pain care delivery in the primary care setting. Eight RCTs2, 2325, 2729, 3133 and one retrospective cohort26 of mostly fair or good quality identified five models that primarily coupled decision support with proactive treatment monitoring as having the strongest evidence of providing clinically significant improvement in pain intensity and pain-related function over 9 to 12 months (NNT range, 4 to 13): ESCAPE,2 SEACAP,27 STarT Back,29 SCAMP,30 and SCOPE.31 Four of the models27, 2931 also showed improvements on at least one of the additional important outcomes of quality of life,2931 depression,27, 2931 anxiety,30, 31 and sleep.31 Despite strong methodology, however, the strength of the evidence is generally low, as each intervention is only supported by a single study with imprecise findings.

Our ability to rank models from best to worst is somewhat limited by heterogeneity in outcome assessment methods, patient populations, and setting. The SCAMP30 and SCOPE31 models stand out as having the potential to be better than others, because they resulted in the greatest clinically significant improvements in pain intensity or pain-related function (NNT’s of 4 compared with NNT’s of 8 to 13). SCAMP and SCOPE share many components with other models, such as weekly case management meetings, pharmacotherapy algorithms, care coordination teams, and access to mental health treatment. However, SCAMP and SCOPE are the only models to include a clinical pain and/or depression specialist on their case management team, which may have contributed to their greater patient improvements. The least comprehensive model, in which pharmacists and physiotherapists focused on medication optimization, exercise, and self-management, produced no clinically important improvement at 12 months.28 Compared to other models, this model was the least comprehensive in addressing the complex range of biological, psychological, and social factors faced by individuals with chronic pain. For example, this model did not appear to include any access to mental health treatment. A limitation of the evidence base is its generalizability. These models were primarily implemented within a large integrated VA health care system with a developed patient-centered medical home model for primary care that shares a common electronic health record with specialists and hospitals and can include integrated mental health services. Implementing the models identified in this review may be more challenging in smaller, less integrated health care settings. Also, they are most applicable to middle-aged male populations seen at VA primary care clinics for low back pain and may not be widely reflective of all patients with chronic pain or those seen in a broader range of specialty settings, including multidisciplinary pain clinics and rehabilitation centers. We were unable to determine the patients who are most likely to benefit from these models due to under-reporting of key patient characteristics such as pain duration, opioid use at baseline, and prevalence of common medical and mental health comorbidities. Additional limitations of the evidence base include (1) each model was supported by a single study; (2) incomplete evaluation of model fidelity; (3) only half to a small minority of studies measured IMMPACT-recommended outcomes17—50% for depression, 40% for anxiety, 10% for sleep, 30% for opioid use, and 20% for unintended consequences; and (4) that sustainability is unknown because follow-up was limited to 12 to 18 months.

The primary limitations of our findings related to our review methods include (1) our literature search, (2) our focused scope, and (3) our use of sequential instead of independent dual assessment. For our literature search, limiting to English-language studies from MEDLINE and CINAHL, coupled with the inconsistent terminology used in the literature on complex multicomponent health care interventions,22 may have increased our risk of missing relevant studies. We addressed this challenge by including a wide variety of terminology in our search strategy (e.g., “primary care planning,” “disease management,” “multi-component,” “chronic pain”), as well as searching a wide range of other sources. Second, to meet our condensed timeframe, we focused our scope to the setting of primary care, because it is responsible for the majority of pain management, and, for benefits, to IMMPACT-recommended outcomes. Although our scope captures the highest priority needs, it limits applicability to other settings in which pain care occurs, such as multidisciplinary pain clinics and rehabilitation centers and to other outcomes of pain care, such as patient satisfaction. Third, although sequential dual review is a widely-used method, its comparison to independent dual review has not yet been empirically studied and may have increased the risk of error and bias.

Chronic pain management is anticipated to continue to be an important clinical area in the future, and rapid evidence development is expected. For example, several additional multimodal chronic pain care models have already shown promise for improving patient outcomes in single-arm studies,3540 and we also identified several ongoing studies4151 which may fill gaps in existing research or provide further support for various models of pain care. For example, the STarT Back Screening Tool is now also being evaluated in an ongoing study in six large primary care clinics in the integrated Group Health system in Washington state.42 In response to our findings, and as more emerge, large national health care systems with established electronic health information systems for use in quality improvement and research may consider wider implementation of one or more of the models with a clear plan for further evidence development that addresses shortcomings of previous research. Also, as a main focus of these models is to reduce the numerous known challenges to primary care providers in managing the complexities of patients with chronic pain, it is also important to understand how these models are affecting providers’ experiences. As provider perspectives were largely unexplored in previous studies, we suggest future research consider assessing the three domains identified as important in interviews of providers at the Indianapolis VAMC (1) patient-centered communication skills, (2) extent of shared decision-making, and (3) provider burnout.52

CONCLUSIONS

Five models primarily coupling a decision-support component—most commonly algorithm-guided treatment and/or stepped care—with proactive ongoing treatment monitoring have the best evidence from good-quality RCTs of providing clinically relevant improvement in pain intensity and pain-related function over 9 to 12 months, as well as variable improvement in other important core outcomes. National health care systems may be encouraged to consider wider implementation of any of those models with a clear plan for further evidence development to addresses shortcomings of previous research.

Notes

Acknowledgements

The VA Evidence-based Synthesis Program (ESP) is funded by Quality Enhancement Research Initiative (QUERI). The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of QUERI. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the US Government.

Compliance with Ethical Standards

Conflicts of Interest

The authors declare that they do not have a conflict of interest.

References

  1. 1.
    Institute of Medicine (US) Committee on Advancing Pain Research, Care, and Education. Relieving pain in America: a blueprint for transforming prevention, care, education, and research. Washington DC: National Academy of Sciences.; 2011.Google Scholar
  2. 2.
    Bair MJ, Ang D, Wu J, et al. Evaluation of Stepped Care for Chronic Pain (ESCAPE) in Veterans of the Iraq and Afghanistan Conflicts: a randomized clinical trial. JAMA Intern Med. 2015;175(5):682–9.CrossRefPubMedGoogle Scholar
  3. 3.
    Narouze S, Souzdalnitski D. Obesity and chronic pain: systematic review of prevalence and implications for pain practice. Reg Anesth Pain Med. 2015;40(2):91–111.CrossRefPubMedGoogle Scholar
  4. 4.
    Stubbs B, Koyanagi A, Thompson T, et al. The epidemiology of back pain and its relationship with depression, psychosis, anxiety, sleep disturbances, and stress sensitivity: data from 43 low- and middle-income countries. Gen Hosp Psychiatry. 2016 Nov - Dec;43:63–70.CrossRefPubMedGoogle Scholar
  5. 5.
    Otis JD, Keane TM, Kerns RD. An examination of the relationship between chronic pain and post-traumatic stress disorder. J Rehabil Res Dev. 2003;40(5):397–405.CrossRefPubMedGoogle Scholar
  6. 6.
    Hooten WM. Chronic pain and mental health disorders: shared neural mechanisms, epidemiology, and treatment. Mayo Clin Proc. 2016;91(7):955–70.CrossRefPubMedGoogle Scholar
  7. 7.
    Rosenquist RW, Benzon HT, Connis RT, et al. Practice guidelines for chronic pain management: an updated report by the American Society of Anesthesiologists Task Force on Chronic Pain Management and the American Society of Regional Anesthesia and Pain Medicine. Anesthesiology. 2010;112(4):810–33.CrossRefGoogle Scholar
  8. 8.
    Kerns RD, Philip EJ, Lee AW, Rosenberger PH. Implementation of the veterans health administration national pain management strategy. Transl Behav Med. 2011;1(4):635–43.CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    Chou R, Qaseem A, Snow V, et al. Diagnosis and treatment of low back pain: a joint clinical practice guideline from the American College of Physicians and the American Pain Society. Ann Intern Med. 2007;147(7):478–91.CrossRefPubMedGoogle Scholar
  10. 10.
    Matthias MS, Parpart AL, Nyland KA, et al. The patient-provider relationship in chronic pain care: providers’ perspectives. Pain Med. 2010;11(11):1688–97.CrossRefPubMedGoogle Scholar
  11. 11.
    Patient-Centered Outcomes Research Institute. Systems interventions to improve the management of chronic musculoskeletal pain: topic brief. 2015.Google Scholar
  12. 12.
    Higgins JP, Altman DG, Gotzsche PC, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Hartling L, Guise J-M, Kato E, et al. EPC methods: an exploration of methods and context for the production of rapid reviews. Rockville (MD): Agency for Healthcare Research and Quality, 2015.Google Scholar
  14. 14.
    Agency for Healthcare Research and Quality. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Rockville, MD: 2014.Google Scholar
  15. 15.
    Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS med. 2009;6(7):e1000097.CrossRefPubMedPubMedCentralGoogle Scholar
  16. 16.
    Peterson K, Anderson J, Bourne D, Mackey K, Helfand M. Evidence brief: effectiveness of models used to deliver multimodal care for chronic musculoskeletal pain. VA ESP Project #09–1992017 May 3, 2017. Available from: https://www.hsrd.research.va.gov/publications/esp/chronicpain.cfm.
  17. 17.
    Dworkin RH, Turk DC, Farrar JT, et al. Core outcome measures for chronic pain clinical trials: IMMPACT recommendations. Pain. 2005;113(1–2):9–19.CrossRefPubMedGoogle Scholar
  18. 18.
    McDonagh M, Jonas D, Gartlehner G, et al. Methods for the drug effectiveness review project. BMC Med Res Methodol. 2012;12:140.CrossRefPubMedPubMedCentralGoogle Scholar
  19. 19.
    Sterne J, Higgins J, Reeves B. A Cochrane risk of bias assessment tool: for non-randomized studies of interventions (ACROBAT-NRSI). 2014.Google Scholar
  20. 20.
    Hill JC, Dunn KM, Lewis M, et al. A primary care back pain screening tool: identifying patient subgroups for initial treatment. Arthritis Care & Research. 2008;59(5):632–41.CrossRefGoogle Scholar
  21. 21.
    Berkman ND, Lohr KN, Ansari M, et al. Grading the strength of a body of evidence when assessing health care interventions for the effective health care program of the Agency for Healthcare Research and Quality: an update methods guide for effectiveness and comparative effectiveness reviews. Rockville MD: Agency for Healthcare Research and Quality, 2013.Google Scholar
  22. 22.
    Guise J-M, Chang C, Viswanathan M, et al. Agency for Healthcare Research and Quality Evidence-based Practice Center methods for systematically reviewing complex multicomponent health care interventions. Journal of clinical epidemiology. 2014;67(11):1181–91.CrossRefPubMedGoogle Scholar
  23. 23.
    Ahles TA, Seville J, Wasson J, Johnson D, Callahan E, Stukel TA. Panel-based pain management in primary care. A pilot study. J Pain Symptom Manage. 2001;22(1):584–90.CrossRefPubMedGoogle Scholar
  24. 24.
    Ahles TA, Wasson JH, Seville JL, et al. A controlled trial of methods for managing pain in primary care patients with or without co-occurring psychosocial problems. Ann Fam Med. 2006;4(4):341–50.CrossRefPubMedPubMedCentralGoogle Scholar
  25. 25.
    Angeles RN, Guenter D, McCarthy L, et al. Group interprofessional chronic pain management in the primary care setting: a pilot study of feasibility and effectiveness in a family health team in Ontario. Pain Research & Management. 2013;18(5):237–42.CrossRefGoogle Scholar
  26. 26.
    Burnham R, Day J, Dudley W. Multidisciplinary chronic pain management in a rural Canadian setting. Canadian Journal of Rural Medicine. 2010;15(1):7–13.PubMedGoogle Scholar
  27. 27.
    Dobscha SK, Corson K, Perrin NA, et al. Collaborative care for chronic pain in primary care: a cluster randomized trial. JAMA. 2009;301(12):1242–52.CrossRefPubMedGoogle Scholar
  28. 28.
    Hay EM, Foster NE, Thomas E, et al. Effectiveness of community physiotherapy and enhanced pharmacy review for knee pain in people aged over 55 presenting to primary care: pragmatic randomised trial. BMJ. 2006;333(7576):995.CrossRefPubMedPubMedCentralGoogle Scholar
  29. 29.
    Hill JC, Whitehurst DGT, Lewis M, et al. Comparison of stratified primary care management for low back pain with current best practice (STarT Back): a randomised controlled trial. Lancet. 2011;378(9802):1560–71.CrossRefPubMedPubMedCentralGoogle Scholar
  30. 30.
    Kroenke K, Bair MJ, Damush TM, et al. Optimized antidepressant therapy and pain self-management in primary care patients with depression and musculoskeletal pain: a randomized controlled trial. JAMA. 2009;301(20):2099–110.CrossRefPubMedPubMedCentralGoogle Scholar
  31. 31.
    Kroenke K, Krebs EE, Wu J, Yu Z, Chumbler NR, Bair MJ. Telecare collaborative management of chronic pain in primary care: a randomized clinical trial. JAMA. 2014;312(3):240–8.CrossRefPubMedGoogle Scholar
  32. 32.
    Kroenke K, Krebs EE, Bair MJ. Pharmacotherapy of chronic pain: a synthesis of recommendations from systematic reviews. General Hospital Psychiatry. 2009;31(3):206–19.CrossRefPubMedGoogle Scholar
  33. 33.
    Thielke S, Corson K, Dobscha SK. Collaborative care for pain results in both symptom improvement and sustained reduction of pain and depression. General Hospital Psychiatry. 2015;37(2):139–43.CrossRefPubMedGoogle Scholar
  34. 34.
    Cole S, Raju M, Barrett J, Gerrity M, Dietrich A. The MacArthur foundation depression education program for primary care physicians: background, participant's workbook, and facilitator’s guide. General hospital psychiatry. 2000;22(5):299–358.CrossRefPubMedGoogle Scholar
  35. 35.
    Unützer J, Hantke M, Powers D, et al. Care management for depression and osteoarthritis pain in older primary care patients: a pilot study. International Journal of Geriatric Psychiatry. 2008;23(11):1166–71.CrossRefPubMedGoogle Scholar
  36. 36.
    Dorflinger L, Moore B, Goulet J, et al. A partnered approach to opioid management, guideline concordant care and the stepped care model of pain management. JGIM: Journal of General Internal Medicine. 2014;29:870–6.CrossRefPubMedPubMedCentralGoogle Scholar
  37. 37.
    Briggs M, Closs SJ, Marczewski K, Barratt J. A feasibility study of a combined nurse/pharmacist-led chronic pain clinic in primary care. Quality in Primary Care. 2008;16(2):91–4.PubMedGoogle Scholar
  38. 38.
    Chelminski PR, Ives TJ, Felix KM, et al. A primary care, multi-disciplinary disease management program for opioid-treated patients with chronic non-cancer pain and a high burden of psychiatric comorbidity. BMC Health Services Research. 2005;5(1):1.CrossRefGoogle Scholar
  39. 39.
    Gardiner P, Dresner D, Barnett KG, Sadikova E, Saper R. Medical group visits: a feasibility study to manage patients with chronic pain in an underserved urban clinic. Global Advances in Health and Medicine. 2014;3(4):20–6.CrossRefPubMedPubMedCentralGoogle Scholar
  40. 40.
    Wiedemer NL, Harden PS, Arndt IO, Gallagher RM. The opioid renewal clinic: a primary care, managed approach to opioid therapy in chronic pain patients at risk for substance abuse. Pain Medicine. 2007;8(7):573–84.CrossRefPubMedGoogle Scholar
  41. 41.
    Aragonès E, López-Cortacans G, Caballero A, et al. Evaluation of a multicomponent programme for the management of musculoskeletal pain and depression in primary care: a cluster-randomised clinical trial (the DROP study). BMC psychiatry. 2016;16(1):1.CrossRefGoogle Scholar
  42. 42.
    Cherkin D, Balderson B, Brewer G, et al. Evaluation of a risk-stratification strategy to improve primary care for low back pain: the MATCH cluster randomized trial protocol. BMC Musculoskeletal Disorders. 2016;17(1):361.CrossRefPubMedPubMedCentralGoogle Scholar
  43. 43.
    Chouinard M-C, Hudon C, Dubois M-F, et al. Case management and self-management support for frequent users with chronic disease in primary care: a pragmatic randomized controlled trial. BMC Health Services Research. 2013;13:49.CrossRefPubMedPubMedCentralGoogle Scholar
  44. 44.
    de Heer E, de Wilde-Timmerman L, Dekker J, et al. Efficacy of collaborative care versus antidepressant treatment in chronic pain and major depression: a multi center proof of concept study. Journal of Psychosomatic Research. 2016;85:60–1.CrossRefGoogle Scholar
  45. 45.
    Eaton LH, Gordon DB, Wyant S, et al. Development and implementation of a telehealth-enhanced intervention for pain and symptom management. Contemporary Clinical Trials. 2014;38(2):213–20.CrossRefPubMedPubMedCentralGoogle Scholar
  46. 46.
    Goertz CM, Salsbury SA, Vining RD, et al. Collaborative Care for Older Adults with low back pain by family medicine physicians and doctors of chiropractic (COCOA): study protocol for a randomized controlled trial. Trials [Electronic Resource]. 2013;14:18.Google Scholar
  47. 47.
    Karp JF, Rollman BL, Reynolds CF, 3rd, et al. Addressing both depression and pain in late life: the methodology of the ADAPT study. Pain Medicine. 2012;13(3):405–18.Google Scholar
  48. 48.
    MacDougall P . Nova Scotia Chronic Pain Collaborative Care Network: a pilot study. 2011.Google Scholar
  49. 49.
    Krebs E. Comparative effectiveness of patient-centered strategies to improve pain management and opioid safety for veterans. 2016.Google Scholar
  50. 50.
    Debar LL. Collaborative Care for Chronic Pain in primary care. 2016.Google Scholar
  51. 51.
    Bair M. Care Management for the Effective Use of Opioids (CAMEO). 2015.Google Scholar
  52. 52.
    Matthias MS, Bair MJ, Nyland KA, et al. Self-management support and communication from nurse care managers compared with primary care physicians: a focus group study of patients with chronic musculoskeletal pain. Pain Management Nursing. 2010;11(1):26–34.CrossRefPubMedGoogle Scholar

Copyright information

© Society of General Internal Medicine (outside the USA) 2018

Authors and Affiliations

  • Kim Peterson
    • 1
    Email author
  • Johanna Anderson
    • 1
  • Donald Bourne
    • 1
  • Katherine Mackey
    • 1
  • Mark Helfand
    • 1
  1. 1.Department of Veterans AffairsVA Portland Health Care System, Evidence-based Synthesis Program (ESP) Coordinating CenterPortlandUSA

Personalised recommendations