Journal of Intelligent Information Systems

, Volume 36, Issue 1, pp 49–72 | Cite as

ST-Audit: guideline-based automatic auditing of electronic patient records

  • Cleo Zanella Billa
  • Jacques Wainer
  • Claudia Barsottini


This work presents the ST-Audit system that audits a patient record for conformance to a particular clinical guideline. The system uses ST-Guide which models a guideline as a set of states and transitions. The audit system tries to find a path in the state/transition diagram that corresponds to the actions taken by the physician, taking into consideration issues such as the unavailability of all the data needed to evaluate the transitions. The system was used to audit an outpatient clinic regarding their procedure for hypertension treatment (using the VI JNC guideline for hypertension), and the results of the number of non-compliant actions were presented and discussed. A follow up auditing showed a small but statistically significant reduction on the number of non-compliant actions for patients treated after the first audit.


Electronic patient record Automatic auditing Clinical guidelines 


  1. Advani, A., & Musen, Y. (2002). Medical quality assessment by scoring adherence to guideline intentions. Journal of the American Medical Informatics Association, 9, 92–97.CrossRefGoogle Scholar
  2. Advani, A., Lo, K., & Shahar, Y. (1998). Intention-based critiquing of guideline-oriented medical care. In Proceedings of the 1998 AMIA annual symposium (pp. 483–487).Google Scholar
  3. Boxwala, A. A., Peleg, M., Tu, S., Ogunyemi, O., Zeng, Q. T., Wang, D., et al. (2004). GLIF3: A representation format for sharable computer-interpretable clinical practice guidelines. Journal of Biomedical Informatics, 37(3), 147–161.CrossRefGoogle Scholar
  4. Chesani, F., Lamma, E., Mello, P., Montali, M., Storari, S., Baldazzi, P., et al. (2008). Compliance checking of cancer-screening CareFlows: An approach based on computational logic. In A. Ten Teije, et al. (Eds.), Computer-based medical guidelines and protocols: A primer and current trends: A primer and current trends (p. 183). Fairfax: IOS.Google Scholar
  5. Clarke, E., Grumberg, O., & Peled, D. (1999). Model checking. Cambridge: MIT.Google Scholar
  6. Corrigan, J. M., Donaldson, M. S., & Kohn, L. T. (2001). Crossing the quality chasm: A new health system for the 21st century. National Academy Press.Google Scholar
  7. de Clercq, P. A., Blom, J. A., Korsten, H. H. M., & Hasman, A. (2004). Approaches for creating computer-interpretable guidelines that facilitate decision support. Artificial Intelligence in Medicine, 31(1), 1–27.CrossRefGoogle Scholar
  8. Dick, R. S., Steen, E. B., & Demer, D. E. (1997) The computer-based patient record: An essential technology for health care. Washington, DC: National Academy.Google Scholar
  9. Ellis, N. T. (1997). C.A.S.M. and C.A.S.T.: A new information based approach to clinical audit. Computer Methods and Programs in Biomedicine, 54, 29–38.CrossRefGoogle Scholar
  10. Fox, J., Johns, N., & Rahmanzadeh, A. (1998). Disseminating medical knowledge: The PROforma approach. Artificial Intelligence in Medicine, 14(1–2), 157–182.CrossRefGoogle Scholar
  11. Grimshaw, J. M., & Russell, I. T. (1993). Effect of clinical guidelines on medical practice: A systematic review of rigorous evaluations. Lancet, 342(8883), 1317–1322.CrossRefGoogle Scholar
  12. Groot, P., Hommersom, A., Lucas, P., Serban, R., ten Teije, A., & van Harmelen, F. (2007). The role of model checking in critiquing based on clinical guidelines. In 11th conference on artificial intelligence in medicine, AIME 2007, Lecture Notes in Computer Science (Vol. 4594, pp. 411–420). Springer, Berlin.Google Scholar
  13. Hazlehurst, B., Sittig, D. F., Stevens, V. J., Smith, K. S., Hollis, F. H., Vogt, T. M., et al. (2005) Natural language processing in the electronic medical record—assessing clinical adherence to tobacco treatment guidelines. American Journal of Preventive Medicine, 29, 434–439.CrossRefGoogle Scholar
  14. Hripcsak, G., Ludemann, P., Pryor, T. A., Wigertz, O. B., & Clayton, P. D. (1994). Rationale for the Arden syntax. Computers and Biomedical Research, 27(4), 291–324.CrossRefGoogle Scholar
  15. Johnson, P. D., Tu, S. W., Booth, N., Sugden, B., & Purves, I. N. (2000). Using scenarios in chronic disease management guidelines for primary care. In Proc. AMIA Symp. (pp. 389–393).Google Scholar
  16. Kinn, S., Lee, N., & Millman, A. (1995). ABC of medical computing: Using computers in clinical audit. BMJ, 311, 739–742.Google Scholar
  17. Kuilboer, M. M., van der Lei, J., de Jongste, J. C., Overbeek, S. E., Ponsioen, B., & van Bemmel, J. H. (1998). Simulating an integrated critiquing system. Journal of the American Medical Informatics Association, 5, 194–202.Google Scholar
  18. Kuilboer, M. M., van Wijk, M. A. M., Mosseveld, M., & van der Lei, J. (2003). AsthmaCritic: Issues in designing a noninquisitive critiquing system for daily practice. Journal of the American Medical Informatics Association, 10(5), 419–424.CrossRefGoogle Scholar
  19. Kuilboer, M. M., van Wijk, M. A. M., Mosseveld, M., van der Does, E., de Jongste, J. C., Overbeek, S. E., et al. (2006). Computed critiquing integrated into daily clinical practice affects physicians’ behavior—a randomized clinical trial with AsthmaCritic. Methods of Information in Medicine, 45(4), 447–454.Google Scholar
  20. Landon, B. E., Normand, S. T., Blumenthal, D., & Daley, J. (2003). Physician clinical performance assessment—prospects and barriers. JAMA, 290(9), 1183–1189.CrossRefGoogle Scholar
  21. McMillan, K. L. (1993). Symbolic model checking. Norwell: Kluwer.MATHGoogle Scholar
  22. Montani, S. (2007). Case-based reasoning for managing non-compliance with clinical guidelines. Proc. case-based reasoning in the health sciences workshop, International Conference on Case Based Reasoning (ICCBR), August.Google Scholar
  23. Monteiro, A., & Wainer, J. (2001). A conceptual framework to model chronic and long-term diseases. In 8th Conference on artificial intelligence in medicine in Europe—AIME, LNAI (Vol. 2101, pp. 405–408). Berlin: Springer.Google Scholar
  24. National Institute for Clinical Excellence (2002). Principles for best practice in clinical audit. Oxford: Radcliffe Medical. Scholar
  25. National Institutes of Health (1997). The sixth report of the join national committee on prevention, detection, evaluation, and treatment of high blood pressure.
  26. Ohno-Machado, L., Gennari, J. H., Murphy, S. N., Jain, N. L., Tu, S. W., Oliver, D. E., et al. (1998). The guideline interchange format. Journal of the American Medical Informatics Association, 5, 357–372.Google Scholar
  27. Schneider, E. C., Riehl, V., Courte-Winecke, S., Eddy, D. M., & Sennet, C. (1999). Enhancing performance measurement—NCQA’s road map for a health information framework. JAMA, 282, 1184–1190.CrossRefGoogle Scholar
  28. Schnoebelen, P. (2002). The complexity of temporal logic model checking. Advances in Modal Logic, 4, 1–44.Google Scholar
  29. Shahar, Y., Miksch, S., & Johnson, P. (1998). The Asgaard project: A task-specific framework for the application and critiquing of time-oriented clinical guidelines. Artificial Intelligence in Medicine, 14(1–2), 29–51.CrossRefGoogle Scholar
  30. Shakib, S., & Phillips, P. A. (2003). The Australian centre for evidence-based clinical practice generic audit tool: Auditmaker for health professionals. Journal of Evaluation in Clinical Practice, 9(2):259–263.CrossRefGoogle Scholar
  31. Shiffman, R. N., Liaw, Y., Brandt, C. A., & Corb, G. J. (1999). Computer-based guideline implementation systems. Journal of the American Medical Informatics Association, 6, 104–114.Google Scholar
  32. Sips, R., Braun, L., & Roos, N. (2006). Applying intention-based guidelines for critiquing. In Workshop: AI techniques in healthcare: Evidence-based guidelines and protocols.
  33. Tu, S. W., & Musen, M. A. (2001). Representation formalisms and computational methods for modeling guideline-based patient care. In B. Heller, M. Loffler, M. Musen, & M. Stefanelli (Ed.), Computer-based support for clinical guidelines and protocols: Proceedings of EWGLP 2000 (pp. 115–132). IOS Press.Google Scholar
  34. van der Lei, J., & Musen, M. A. (1991). A model for critiquing based on automated medical records. Computers and Biomedical Research, 24(4), 344–378.CrossRefGoogle Scholar
  35. van der Lei, J., Musen, M. A., van der Does, E., & Van Bemmel, J. H. (1991). Review of physician decision making using data from computer-stored medical records. Lancet, 338, 1504–1508.CrossRefGoogle Scholar
  36. Wainer, J., Billa, C., Paixao, M., & Sigulem, D. (2008). ST-Guide: A framework for the implementation of automatic clinical guidelines. In 23th ACM symposium on applied computing (SAC) (Vol. 2, pp. 1325–1332).Google Scholar
  37. Wang, D., Peleg, M., Tu, S. W., Boxwala, A. A., Greenes, R. A., Patel, V. L., et al. (2002). Representation primitives, process models and patient data in computer-interpretable clinical practice guidelines: A literature review of guideline representation models. International Journal of Medical Informatics, 68(1–3), 59–70.CrossRefGoogle Scholar
  38. Woolf, S. H., Grol, R., Hutchinson, A., Eccles, M., & Grimshaw, J. (1999). Clinical guidelines. potential benefits, limitations, and harms of clinical guidelines. BMJ, 318(7182), 527–530.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Cleo Zanella Billa
    • 1
  • Jacques Wainer
    • 1
  • Claudia Barsottini
    • 2
  1. 1.Institute of ComputingUniversity of CampinasCampinasBrazil
  2. 2.Department of Health InformaticsFederal University of Sao PauloSao PauloBrazil

Personalised recommendations