In a 2011 call to action, the National Academy of Medicine advanced the goal that by the end of the decade, 90% of clinical decisions should be supported by the best available evidence, motivated in part by a recognition that “care that is important is often not delivered [and] care that is delivered is often not important”.1
Instrumental towards this end is a health system in which evidence is both used and applied as a part of routine practice.1 But synthesizing and translating evidence into practice remains a challenge.2 What infrastructure do health systems need to facilitate consistent, evidence-based practice within their organizations?
Health technology assessment (HTA) is one potential mechanism for health systems to systematically synthesize and integrate evidence into clinical practice.3 HTA is a process that uses health services research methods—including rapid or systematic reviews and budget-impact or cost-effectiveness analyses—to determine the comparative benefits, harms, and costs of alternative ways to manage disease or deliver care.3 Most clinicians are familiar with systematic reviews and cost-effectiveness analyses, though the other mentioned methodologies may require brief explanation; rapid reviews are streamlined systematic reviews produced on quicker timelines4, and budget impact analyses are short-term economic evaluations from a payer perspective.5
Health system–based HTA centers are virtual or on-site programs in hospitals or health systems that are staffed by research analysts and clinicians who conduct HTA6 (Fig. 1). These entities: (1) find evidence summaries in the research literature or solicit them from third parties (e.g., the Agency for Healthcare Research and Quality);7 (2) generate evidence summaries if they do not yet exist;7 (3) interpret “external evidence” (the research literature) in the context of “internal evidence” (data derived from a system’s own clinical, utilization and expenditure estimates);8 (4) conduct economic analyses as needed;7 (5) combine results from the HTA process with stakeholder input to forge standardized guidelines, clinical pathways, and clinical decision support tools that directly bridge evidence and stakeholder values and preferences with clinical practice2 and last; (6) analyze results of implementation and strategize needed enhancements.7
Although many US health systems seek evidence to guide acquisition of technology and pharmaceuticals or the implementation of new care delivery models,8 most do not have the infrastructure to obtain or implement evidence in practice.7 Health system–based HTA centers fill this void.
As an example, University of Pennsylvania Health System established the Center for Evidence-based Practice in 2006 to support health system–based HTA and the integration of evidence into practice.9 Startup cost and infrastructure were minimal; initial support partially funded two physician-administrators and a masters-level research analyst. Staffing grew as demand for services grew, to four full-time analysts, one administrative assistant, and two physician faculty co-directors who each dedicated 0.50 full-time equivalents to the effort. At Penn, evidence reviews are generated on topics requested by clinical and administrative leaders. Local patterns of care are analyzed and econometric analyses conducted as needed. The end-products are rapid reviews and cost analyses that put clinical evidence into local context, and ensure that best practices inform care and expenditure decisions. Since 2006, in total, 450 new rapid evidence reviews or economic evaluations have been produced and the Center has disseminated or implemented over 250 clinical pathways and 25 computerized clinical decision support interventions, resulting in improved patient outcomes and the delivery of high-value care.7 For example, as a result of Penn’s healthcare system–based HTA process, the appropriate use of venous thromboembolism prophylaxis has increased 30%.10 The decision to use chlorhexidine as opposed to iodine for preoperative skin antisepsis was estimated to not only decrease surgical site infections but also to incur cost savings of $500,000 per year11—less than the expenditure to initially run the Center for Evidence-based Practice.
Another example of health system–based HTA is the Kaiser Permanente Interregional New Technologies Committee, which has been in existence since the early 1980s.12 It currently consists of physician leaders as well as more than 14 full-time equivalents of staff with expertise in epidemiology and biostatistics.12 Their scope of work is similar to that described for the University of Pennsylvania. One project involved health technology assessment of a foreign object detection program in the operating room, leading to a multistep initiative that deployed bar-coded and radiofrequency-tagged surgical sponges, with a reduction in retained foreign object rates by 50%.12
Though health system–based HTA is still in nascent stages of development within the US,7 the University of Pennsylvania and Kaiser experiences demonstrate that institutional HTA centers can be successful and persist within the US healthcare landscape, particularly when supported by institutional administration,13 and effectively yield the delivery of high-value care—a major priority for payers, clinicians, and patients.
BENEFITS AND CHALLENGES TO ESTABLISHING A HEALTH SYSTEM–BASED HTA INFRASTRUCTURE
The benefits of HTA for a health system are myriad. Health system–based HTA can: (1) marry institutional academic strengths, if available, with operations;6 (2) contribute to a transparent and systematic process for value-based resource allocation, and support a continuous process of quality improvement;9 (3) decrease the time lag between evidence generation and implementation,14 and; (4) help promote the delivery of higher quality lower cost care.2 Further, health system–based HTA is a fundamental component of a learning healthcare system, which aims to use the best evidence to provide value-based and appropriate population-level care.15 It also is part of the “knowledge to action” framework that decreases the time lag between evidence generation and application.14 Synthesizing and adapting the evidence for the local context also increase the likelihood of sustainable implementation.14, 15
The benefits of HTA infrastructure are perhaps most obvious for integrated health systems that insure their own patients, employ their own physicians, have multiple hospital sites, and have a vested interest in delivering value-based care.12 Kaiser Permanente (as mentioned) and the Veterans’ Health Administration have established HTA programs that allow evidence to inform system-level decision-making, and the network allows rapid dissemination of best practices.12
It might appear that institutions that do not assume financial risk for patients’ health outcomes or that are staffed primarily by private practitioners would not have the same motivation for HTA, as they benefit less from value-based contracts and may confront challenges obtaining physician buy-in regarding potential restrictions in the available formularies, supplies, or technologies.3, 9 But health systems in many different payment structures are likely to benefit from HTA, as the goal of HTA analyses is to systematically analyze the benefits and harms of particular expenditures, an appealing concept to a diverse set of stakeholders.16, 17 This is particularly true in an era of increased governmental scrutiny and public reporting. The benefit of HTA to healthcare systems goes beyond cost containment.16, 17 Cost is just one consideration in value assessment.16, 17 Final health system–based HTA recommendations should incorporate diverse stakeholder inputs to address multiple domains, including but not limited to patient-reported outcomes, ethical/legal/social implications, and the availability of alternative treatments.17 The sum of these inputs informs a care decision that aims for sustainable access to high-value care for all patients.
It is unlikely that every health system will see the merit of supporting an internal HTA infrastructure. Not all decisions are amenable or desirable for an HTA process; as such, a mandate that all expenditure decisions undergo an HTA evaluation may not be appropriate.3, 7, 9 That said, every hospital does make expenditures by some mechanism and many would likely be improved by an HTA process. In most cases currently, health systems rely only on the excellence of individual practitioners to identify and integrate evidence where appropriate, whether in local policies, guidelines, or decision support.9 Some systems may choose to partially outsource HTA staffing to independent external organizations. We think a health system–based HTA infrastructure is preferable, given the need for analysis of “internal evidence,” the ability to canvas stakeholders values and preferences and implement clinical pathways and clinical decision support tools, and the desirability of an entity to monitor newly implemented interventions for success. Further, as mentioned above, costs of an internal HTA infrastructure can be mitigated by improved patient outcomes and savings from value-driven expenditures, as well as justified by the necessity of employing skilled staff for credible evidence synthesis and translation efforts.8, 9
LOCAL HTA—NATIONAL IMPLICATIONS
There are barriers to the conduct and influence of nationally coordinated HTA in the USA.3, 18 As compared with other countries, there is relatively little public expenditure on HTA.3 Special interest groups have a strong voice in the USA that often promotes the review or use of expensive, new technologies—even in the absence of a robust evidence base for their adoption over older, lower cost technologies. The nation’s largest public payer, Medicare, is prohibited from considering costs in coverage decisions. Further, were centralized cost-considerations permitted, the contextual nature of US healthcare, given that we are not a single payer system, would make it difficult to produce economic assessments of widespread applicability.3, 18
Though no substitute for a nationally coordinated effort, health system–based HTA could help create and promote a national evidence–based practice ecosystem.6 Rapid reviews, the most frequent HTA output, could be collated by a central agency for dissemination. The rebranded National Guideline Clearinghouse (now ECRI Guidelines Trust™)19 which maintains high-quality clinical practice guidelines, could be a potential repository. A centralized collated repository could further ensure methodologic rigor and other characteristics that end-users identify as important for rapid reviews: the credibility of the review producer, relevance of key questions, and close working relationship between the end-user and producer.8 Ideologically aligned health networks (e.g., High-Value Healthcare Collaborative, Heath Care Systems Research Network) could help collate rapid reviews or lessons-learned from implementation, particularly given that inter-system sharing of best practices is often hindered by competition for patients and bargaining power vis-à-vis insurers. In Europe, a network of health system–based HTA centers20 shares best-practices and evidence reports; a similar arrangement in the US may be possible and align with third parties (e.g., the Agency for Healthcare Research and Quality) to help guide health technology assessment topic selection. Members of a hospital-based HTA network could promote more consistent reporting and sharing of best practices across systems, on specific topics of shared interest, with an added benefit of potential minimization in variations of care delivered across regions and within the same market.21
It is also possible to envision US health system–based HTA having a role in promoting national value–based care. In 2012, Memorial Sloan Kettering Cancer Center decided against the use of ziv-aflibercept for treatment of metastatic colorectal cancer given that existing medications provided similar clinical benefit for approximately half the cost. Later that year, Sanofi offered a 50% national discount on the medication.22 The decentralized nature of US healthcare can make health system- and provider-level decisions influential in whether clinically effective and cost-effective care is provided.3, 18
At the end of the decade, the National Academy of Medicine’s hope that to “the greatest extent possible, the decisions that shape the health and health care of Americans…will be grounded on a reliable evidence base” remains elusive.1 Health system–based HTA could be an important part of the solution. Evidence-based practice will require a continued national emphasis on comparative effectiveness research and also on health system–based evidence synthesis and application.
Institute of Medicine. 2011. Learning What Works: Infrastructure Required for Comparative Effectiveness Research: Workshop Summary. Washington, DC: The National Academies Press. https://doi.org/10.17226/12214.
Flores EJ, Mull NK, Lavenberg JG, Mitchell MD, Leas BF, Williams A, Brennan PJ, Umscheid CA. Using a 10-step framework to support the implementation of an evidence-based clinical pathways programme. BMJ Qual Saf. 2019;28(6):476–485.
Sullivan SD, Watkins J, Sweet B et al. Health technology assessment in health-care decisions in the United States. Value Health. 2009;12 Suppl 2:S39–44. doi: https://doi.org/10.1111/j.1524-4733.2009.00557.x.
Hartling L, Guise JM, Hempel S, Featherstone R, Mitchell MD, Motu’apuaka ML, Robinson KA, Schoelles K, Totten A, Whitlock E, Wilt TJ, Anderson J, Berliner E, Gozu A, Kato E, Paynter R, Umscheid CA. Fit for purpose: perspectives on rapid reviews from end-user interviews. Syst Rev. 2017;6(1):32.
https://www.herc.research.va.gov/include/page.asp?id=budget-impact-analysis. Accessed 9/15/19.
Sampietro-Colom L, Martin J. Chapter 1 Hospital-Based Health Technology Assessment: The Next Frontier. Hospital-Based Health Technology Assessment. The Next Frontier for Health Technology Assessment. In: Sampietro-Colom L, Martin J, eds. Springer; 2016: 3–11.
Jayakumar KL et al. Evidence synthesis activities of a hospital evidence-based practice center and impact on hospital decision making. J Hosp Med. 2016;11(3):185–92. doi: https://doi.org/10.1002/jhm.2498.
Schoelles K, Umscheid CA, Lin JS, et al. A Framework for Conceptualizing Evidence Needs of Health Systems [Internet]. Rockville (MD): Agency for Healthcare Research and Quality (US); 2017 Dec. Available from: https://www.ncbi.nlm.nih.gov/books/NBK493738/.
Umscheid CA, Williams K, Brennan PJ. Hospital-based comparative effectiveness centers: translating research into practice to improve the quality, safety and value of patient care. J Gen Intern Med. 2010;25(12):1352–5.
Umscheid CA, Hanish A, Chittams J, et al. Effectiveness of a novel and scalable clinical decision support intervention to improve venous thromboembolism prophylaxis: a quasi-experimental study. BMC Med Inform Decis Mak. 2012;12:92. doi: https://doi.org/10.1186/1472-6947-12-92.
Lee I, Agarwal RK, Lee BY et al. Systematic review and cost analysis comparing use of chlorhexidine with use of iodine for preoperative skin antisepsis to prevent surgical site infection. Infect Control Hosp Epidemiol. 2010;31(12):1219–29. doi: https://doi.org/10.1086/657134.
Cisneros R, Arthurs S, Hiatt J. Chapter 19 Medical Technology Assessment at Kaiser Permanente: History and Description of Approach (USA). Hospital-Based Health Technology Assessment. The Next Frontier for Health Technology Assessment. In: Sampietro-Colom L, Martin J. Springer; 2016: 227–238.
Guth RM, Herring AA, Merz LR. Implementation of an evidence-based care program within a multihospital health care system. Am J Med Qual 2019;34(2):144–151.
Bindman, A. A shared responsibility for developing a learning health system. J Nurs Care Qual. 2017;32(2):95–98.
IOM (Institute of Medicine). 2010. Redesigning the Clinical Effective- ness Research Paradigm: Innovation and Practice-Based Approaches: Workshop Summary. Washington, DC: The National Academies Press.
Borsky AE, Flores EJ, Berliner E, Chang C, Umscheid CA, Chang SM. AHRQ evidence-based practice center program--applying the knowledge to practice to data cycle to strengthen the value of patient care. J Hosp Med 2019;5;311–314. Published online first February 20, 2019. doi:https://doi.org/10.12788/jhm.3157.
Institute for Clinical and Economic Review. Overview of the ICER value assessment framework and update for 2017–2019. Institute for Clinical and Economic Review. Available at https://icer-review.org/wp-content/uploads/2017/06/ICER-value-assessment-framework-update-FINAL-062217.pdf. Accessed September 25, 2017.
Garber AM. How the patient-centered outcomes research institute can best influence real-world health care decision making. Health Affairs. 2011;30(12):2243–2251. doi: https://doi.org/10.1377/hlthaff.2010.0255.
https://guidelines.ecri.org. Accessed Feb 1, 2019.
Pasternack I, Lach K. Chapter 27 Networks in Hospital-Based HTA. Hospital-Based Health Technology Assessment: The Next Frontier for Health Technology Assessment. In: Sampietro-Colom L, Martin J. Springer International Publishing Switzerland; 2016: 315–327.
Chandra A, Khullar D, Lee TH. Addressing the challenge of gray-zone medicine. N Engl J Med 2015; 372:203–205. doi: https://doi.org/10.1056/NEJMp1409696.
Bach P et al. In Cancer Care, Cost Matters. The New York Times. Oct. 14, 2012. Print.
Conflict of Interest
Dr Sharaf is supported by the National Cancer Institute (K07CA216326 and R01CA211723) and Patient Centered Outcomes Research Institute IHS-2017C3-9211. He is also a paid consultant for the non-profit Institute for Clinial and Economic Review. Dr. Khullar has nothing to disclose. Dr. Umscheid co-founded a hospital evidence-based practice center at the University of Pennsylvania Health System, was Senior Associate Director of the ECRI Institute – Penn Medicine Agency for Healthcare Research and Quality Evidence-Based Practice Center, and is a past member of the Medicare Evidence Development and Coverage Advisory Committee, which uses evidence reports developed by the Evidence-Based Practice Centers of the Agency for Healthcare Research and Quality to inform decision-making.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sharaf, R.N., Khullar, D. & Umscheid, C.A. Health Technology Assessment Centers—an Infrastructure for Health Systems to Translate Evidence into Practice. J GEN INTERN MED 35, 1296–1299 (2020). https://doi.org/10.1007/s11606-019-05534-1