Skip to main content

Use of Imaging Appropriateness Criteria for Decision Support During Radiology Order Entry: The MGH Experience

  • Chapter
  • First Online:
Radiological Safety and Quality

Abstract

This chapter will focus on the institutional perspective of The Massachusetts General Hospital (MGH) in Boston during the first decade of the twenty-first century. We will describe the MGH experience with design, implementation, adoption and current use of a computerized radiology order entry system that displays explicit normative “appropriateness scores” for outpatient CT, MR, and nuclear cardiology procedures and is embedded in routine clinical workflow. The radiology order entry (ROE) and decision support (DS) system has come to be known as ROE-DS within MGH and among a broader community of US radiologists, various industry stakeholders, and policy makers working in the field. One key point that will be emphasized is that a system like ROE-DS in wide use is a necessary, though by no means sufficient, first step in executing comprehensive institutional programs of imaging utilization management, quality improvement, and radiation dose mitigation.

The chapter will take the form of a historical narrative with emphasis on the following aspects over time: external stakeholders (payers, patients), internal stakeholders (physician’s organization cost and quality management), service providers (radiology informatics, radiology leadership), and users (referring clinicians). Historical periods covered include time leading up to decision to create ROE-DS (up to 2003), planning and architecture (2003–2004), implementation and deployment (2004–2005), refinement and evaluation (2005-present), and future directions. Prior to telling the story of ROE-DS at MGH, it is useful to briefly describe a broader historical context about the concepts of utilization variation and appropriateness, the RAND/UCLA Appropriateness Methodology, and the ACR Appropriateness Criteria for diagnostic imaging.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ackerman SJ, Steinberg EP, Bryan RN et al (1997) Patient characteristics associated with diagnostic imaging evaluation of persistent low back problems. Spine 22(14):1634–1640

    Article  CAS  PubMed  Google Scholar 

  2. American Medical Association (1990) Attributes to guide the development of practice parameters. The Association, Chicago

    Google Scholar 

  3. Ayanian JZ, Landrum MB, Normand SL et al (1998) Rating the appropriateness of coronary angiography – do practicing physicians agree with an expert panel and with each other? N Engl J Med 338(26):1896–1904

    Article  CAS  PubMed  Google Scholar 

  4. Bernstein SJ, Hofer TP, Meijler AP et al (1997) Setting standards for effectiveness: a comparison of expert panels and decision analysis. Int J Qual Health Care 9(4):255–263

    Article  CAS  PubMed  Google Scholar 

  5. Boden SD, Swanson AL (1998) An assessment of the early management of spine problems and appropriateness of diagnostic imaging utilization. Phys Med Rehabil Clin N Am 9(2):411–417, viii

    CAS  PubMed  Google Scholar 

  6. Brook RH (1994) The RAND/UCLA appropriateness method. In: McCormick KA, Moore SR, Siegel RA (eds) Methodology perspectives. Public Health Service, U.S. Department of Health and Human Services, Rockville

    Google Scholar 

  7. Brown BB (1968) The delphi process: a methodology used for the elicitation of opinion of experts. The RAND Corporation, Santa Monica

    Google Scholar 

  8. Carey TS, Garrett J (1996) Patterns of ordering diagnostic tests for patients with acute low back pain. The North Carolina Back Pain Project. Ann Intern Med 125(10):807–814

    Article  CAS  PubMed  Google Scholar 

  9. Cascade PN (1994) Setting appropriateness guidelines for radiology. Radiology 192(1):50A–54A

    CAS  PubMed  Google Scholar 

  10. Cascade PN (2000) The American College of Radiology. ACR appropriateness criteria project. Radiology 214(Suppl):3–46

    PubMed  Google Scholar 

  11. Chassin MR, Kosecoff J, Park RE et al (1987) Does inappropriate use explain geographic variations in the use of health care services? A study of three procedures. JAMA 258(18):2533–2537

    Article  CAS  PubMed  Google Scholar 

  12. Dalkey NC (1969) The Delphi method: an experimental study of group opinion. The RAND Corporation, Santa Monica

    Google Scholar 

  13. Dorfman GS (1999) Utilization of diagnostic tests: assessing appropriateness. Acad Radiol 6(Suppl 1):S40–S51

    Article  PubMed  Google Scholar 

  14. Fitch KBS, Bernstein SJ, Aguilar MD et al (2001) The RAND/UCLA appropriateness method user’s manual. The RAND Corporation, Santa Monica

    Google Scholar 

  15. Freeborn DK, Shye D, Mullooly JP et al (1997) Primary care physicians’ use of lumbar spine imaging tests: effects of guidelines and practice pattern feedback. J Gen Intern Med 12(10):619–625

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  16. Hicks NR (1994) Some observations on attempts to measure appropriateness of care. BMJ 309(6956):730–733

    Article  CAS  PubMed  Google Scholar 

  17. Hillman BJ (1991) Practice policies. Rationale, methods of development, and implications for radiologic practice. Invest Radiol 26(7):689–693

    Article  CAS  PubMed  Google Scholar 

  18. Institute of Medicine (1990) Medicare: a strategy for quality assurance. National Academy Press, Washington, DC

    Google Scholar 

  19. Kendrick D, Fielding K, Bentley E et al (2001) The role of radiography in primary care patients with low back pain of at least 6 weeks duration: a randomised (unblinded) controlled trial. Health Technol Assess 5(30):1–69

    CAS  PubMed  Google Scholar 

  20. Kravitz RL, Laouri M, Kahan JP et al (1995) Validity of criteria used for detecting underuse of coronary revascularization. JAMA 274(8):632–638

    Article  CAS  PubMed  Google Scholar 

  21. Kravitz RL, Park RE, Kahan JP (1997) Measuring the clinical consistency of panelists’ appropriateness ratings: the case of coronary artery bypass surgery. Health Policy 42(2):135–143

    Article  CAS  PubMed  Google Scholar 

  22. Kuntz KM, Tsevat J, Weinstein MC et al (1999) Expert panel vs decision-analysis recommendations for postdischarge coronary angiography after myocardial infarction. JAMA 282(23):2246–2251

    Article  CAS  PubMed  Google Scholar 

  23. Leape LL, Park RE, Solomon DH et al (1990) Does inappropriate use explain small-area variations in the use of health care services? JAMA 263(5):669–672

    Article  CAS  PubMed  Google Scholar 

  24. Lehnert BE, Bree RL (2010) Analysis of appropriateness of outpatient CT and MRI referred from primary care clinics at an academic medical center: how critical is the need for improved decision support? J Am Coll Radiol 7(3):192–197

    Article  PubMed  Google Scholar 

  25. McClellan M, Brook RH (1992) Appropriateness of care. A comparison of global and outcome methods to set standards. Med Care 30(7):565–586

    Article  CAS  PubMed  Google Scholar 

  26. McDonnell J, Meijler A, Kahan JP et al (1996) Panellist consistency in the assessment of medical appropriateness. Health Policy 37(3):139–152

    Article  CAS  PubMed  Google Scholar 

  27. Oddone EZ, Samsa G, Matchar DB (1994) Global judgments versus decision-model-facilitated judgments: are experts internally consistent? Med Decis Making 14(1):19–26

    Article  CAS  PubMed  Google Scholar 

  28. Paul-Shaheen P, Clark JD, Williams D (1987) Small area analysis: a review and analysis of the North American literature. J Health Polit Policy Law 12(4):741–809

    Article  CAS  PubMed  Google Scholar 

  29. Quintana JM, Escobar A, Arostegui I et al (2006) Health-related quality of life and appropriateness of knee or hip joint replacement. Arch Intern Med 166(2):220–226

    Article  PubMed  Google Scholar 

  30. Rao JK, Kroenke K, Mihaliak KA et al (2002) Can guidelines impact the ordering of magnetic resonance imaging studies by primary care providers for low back pain? Am J Manag Care 8(1):27–35

    PubMed  Google Scholar 

  31. Rosenthal DI, Weilburg JB, Schultz T et al (2006) Radiology order entry with decision support: initial clinical experience. J Am Coll Radiol 3(10):799–806

    Article  PubMed  Google Scholar 

  32. Shekelle P (2004) The appropriateness method. Med Decis Making 24(2):228–231

    Article  PubMed  Google Scholar 

  33. Shekelle PG, Chassin MR, Park RE (1998) Assessing the predictive validity of the RAND/UCLA appropriateness method criteria for performing carotid endarterectomy. Int J Technol Assess Health Care 14(4):707–727

    Article  CAS  PubMed  Google Scholar 

  34. Shekelle PG, Kahan JP, Bernstein SJ et al (1998) The reproducibility of a method to identify the overuse and underuse of medical procedures. N Engl J Med 338(26):1888–1895

    Article  CAS  PubMed  Google Scholar 

  35. Shye D, Freeborn DK, Romeo J et al (1998) Understanding physicians’ imaging test use in low back pain care: the role of focus groups. Int J Qual Health Care 10(2):83–91

    Article  CAS  PubMed  Google Scholar 

  36. Sistrom C, Drane W (2001) Networked ICD-9 coding system for a radiology department. AJR Am J Roentgenol 176(2):335–339

    Article  CAS  PubMed  Google Scholar 

  37. Sistrom CL (2009) The appropriateness of imaging: a comprehensive conceptual framework. Radiology 251(3):637–649

    Article  PubMed  Google Scholar 

  38. Sistrom CL, Dang PA, Weilburg JB et al (2009) Effect of computerized order entry with integrated decision support on the growth of outpatient procedure volumes: seven-year time series analysis. Radiology 251(1):147–155

    Article  PubMed  Google Scholar 

  39. Tobacman JK, Scott IU, Cyphert ST et al (2001) Comparison of appropriateness ratings for cataract surgery between convened and mail-only multidisciplinary panels. Med Decis Making 21(6):490–497

    CAS  PubMed  Google Scholar 

  40. Tobacman JK, Zimmerman B, Lee P et al (2003) Visual acuity following cataract surgeries in relation to preoperative appropriateness ratings. Med Decis Making 23(2):122–130

    Article  PubMed  Google Scholar 

  41. Vader JP, Burnand B, Froehlich F et al (1997) Appropriateness of upper gastrointestinal endoscopy: comparison of American and Swiss criteria. Int J Qual Health Care 9(2):87–92

    Article  CAS  PubMed  Google Scholar 

  42. Vader JP, Porchet F, Larequi-Lauber T et al (2000) Appropriateness of surgery for sciatica: reliability of guidelines from expert panels. Spine 25(14):1831–1836

    Article  CAS  PubMed  Google Scholar 

  43. Vartanians VM, Sistrom CL, Weilburg JB et al (2010) Increasing the appropriateness of outpatient imaging: effects of a barrier to ordering low-yield examinations. Radiology 255(3):842–849

    Article  PubMed  Google Scholar 

  44. Volinn E, Mayer J, Diehr P et al (1992) Small area analysis of surgery for low-back pain. Spine 17(5):575–581

    Article  CAS  PubMed  Google Scholar 

  45. Wennberg J, Gittelsohn A (1973) Small area variations in health care delivery. Science 182(4117):1102–1108

    Article  CAS  PubMed  Google Scholar 

  46. Zalis M, Harris M (2010) Advanced search of the electronic medical record: augmenting safety and efficiency in radiology. J Am Coll Radiol 7(8):625–633

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Chris L. Sistrom .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Sistrom, C.L., Weilburg, J.B., Rosenthal, D.I., Dreyer, K.J., Thrall, J.H. (2014). Use of Imaging Appropriateness Criteria for Decision Support During Radiology Order Entry: The MGH Experience. In: Lau, L., Ng, KH. (eds) Radiological Safety and Quality. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-7256-4_7

Download citation

  • DOI: https://doi.org/10.1007/978-94-007-7256-4_7

  • Published:

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-94-007-7255-7

  • Online ISBN: 978-94-007-7256-4

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics