Early Experience Using an Online Reporting System for Interventional Radiology Procedure-Related Complications Integrated with a Digital Dictation System
The absence of user-friendly systems for reporting complications is a major barrier to improving quality assurance (QA) programs in interventional radiology (IR) services. We describe the implementation of a QA application that is completely integrated with the radiology dictation system. We implemented an IR QA process as a module within the electronic medical record and radiologist dictation system applications used at our institution. After a radiologist completes a dictation, he or she must select from a drop-down list of complications before proceeding to the next case. Delayed QA events can be entered using the same applications. All complication entries are sent to a database, which is queried to run reports. During the study period, all the 20,034 interventional procedures were entered in the QA database, 1,144 complications were reported, 110 (9.6%) of which were classified as major. Although majority of the complications (996) were entered at the time of dictation, 148 complications (12.9%) were entered afterwards. All major complications were referred to the IR peer review committee, and 30 of these were discussed in the morbidity and mortality meetings. We studied post-lung-biopsy pneumothorax and chest tube rates and initiated a quality improvement process based on the results.The integration of the IR QA reporting system into the workflow process and the mandatory requirements for completion has the potential to minimize the work effort required to enter complication data, and improve participation in the QA process.
Key wordsInterventional radiology quality assurance complications data collection workflow
The Joint Commission on Accreditation of Healthcare Organizations requires all hospital organizations to assess the performance of practicing health care providers for the purpose of renewing privileges during reappointments. According to the guidelines, practitioner-specific data must be accumulated and compared with aggregate data.1, 2, 3, 4, 5
Evaluating radiology faculty’s performance is especially challenging because outcome data are limited. Major efforts are being made to define performance metrics in the field of diagnostic radiology.3,6, 7, 8, 9, 10 In most radiology departments, the most common parameter used to evaluate radiologists’ performance is a discrepancy in the formal interpretation of an imaging study, with little or no emphasis on interventional radiology procedures.3,6, 7, 8, 9, 10, 11, 12 However, since interventional radiologists perform invasive procedures and do not interpret imaging studies, a review of formal interpretation is not a good measure of assessing interventional radiologists’ competence. According to Society of Interventional Radiology (SIR) guidelines, the safety- or procedure-related complication rate is one of the most important indicators for evaluating the success of any invasive procedure13. However, a major barrier to reporting complications or participating in quality assurance (QA) activities is the fact that most reporting systems are stand-alone systems that are not integrated into the routine workflow and require the physician/radiologist to interrupt his or her routine clinical or academic activities, log on to the system, find the case, and enter the event.
Here, we describe the implementation of a QA program that was integrated with a digital dictation system to facilitate the mandatory reporting of procedure-related complications with minimal impact on the radiologist’s workflow efficiency. The entries populate a secure database that can be used for identifying and managing trends.
Materials and methods
Our institution uses an internally developed electronic medical record (EMR) system called ClinicStation that provides physicians a complete and current view of patients’ data. The Radiologist Dictation Application (RadStation) integrates Philip’s iSite Radiology Application and ClinicStation. RadStation gives the radiologist a dedicated application for dictating interventional radiology reports; it also gives the radiologist immediate access to each patient’s EMR. We collaborated with our institution’s EMR department to develop an Interventional Radiology (IR) QA component to be used in conjunction with the ClinicStation and RadStation applications to provide efficient QA workflow for interventional radiologists. Our institutional review board (IRB) reviewed this protocol and deemed that this was not human subjects research, but rather a QA project, and hence did not need IRB approval.
The QA data entry program was designed to be compliant with Health Insurance Portability and Accountability Act (HIPAA) regulations. Only physicians and other staff members who have successfully completed the HIPAA compliance course are allowed data entry privileges.
When a user logs in to RadStation, the application recognizes the person as an interventional radiologist based on the user’s profile. The administrative task of designating a user as an interventional radiologist is performed via an administrative tool that can be used to grant or revoke access to users. In addition to all IR faculty, IR fellows and radiology residents can access the IR QA system during their IR rotation and enter a complication, if appropriate, to an IR procedure. By departmental policy, access to fellows and residents is revoked when they finish their rotations.
The radiologist can select an undictated interventional procedure from one of the worklists available in RadStation. Alternatively, the radiologist can access the procedure number using the barcode reader function of the handheld Dictaphone. Once a procedure is selected for dictation, RadStation displays the patient’s name and medical record number, procedure name, and the name of the radiologist who performed the procedure. The radiologist can view the patient’s clinical and laboratory records on the primary workstation monitor and the diagnostic images on the two secondary monitors.
If no complications occurred, the radiologist makes a “none” entry in the complication box by pressing the lower of the two left-hand buttons on the Dictaphone or by selecting the “none (all)” tab on the complication panel (Fig. 1). If, however, a complication did occur, the radiologist can enter information about the complication by selecting from a drop-down list in the complication panel (Fig. 2). The system them prompts the radiologist to classify the complication as major or minor (using the standard SIR guidelines). Generally, entering “no complication” or “none” takes less than a second, and entering a specific complication takes less than 5 seconds.
In addition to the above-described mandatory entry of complications during IR procedure dictation using RadStation, IR radiologists can also perform QA on a previous IR procedure using ClinicStation. In ClinicStation, the radiologist can select any previously dictated IR procedure and add a complication to the procedure using the same complication panel buttons described for RadStation. If the complication occurs within 30 days of the procedure, the radiologist selects the “<30 days” tab from the complication panel (Fig. 1). However, if the complication occurs more than 30 days after the procedure, the radiologist selects the “>30 days” tab, and this entry appears as a delayed complication in the database. This allows entry of complications that do not happen at the time of the procedure but happen or are discovered later on. The radiologists can enter multiple complications for each procedure. Because ClinicStation also serves as the institution’s EMR system, IR staff can enter complication events from any institutional computer, even while on rounds in the hospital.
Database, Data Review, Query Process
We implemented a separate application, “IR Query,” to identify and manage trends in events collected in the QA database. This application’s functions include search capabilities, predefined reports, and graphic data reports, which provide visual representations of data. This application can also be used to edit the data if necessary; all changes are tracked by SQL database events. Currently, the IR QA director generates monthly reports for all complications and classifies them as major or minor according to SIR criteria.
From the time of the IR QA application’s implementation on November 1, 2005, until May 8, 2008, 20,034 IR procedures were entered in the QA database (100% compliance). During that interval, 1,144 complications were reported; 110 (9.6%) of these events were classified as major using the SIR guidelines, and the remaining complications (1,034; 90.4%) were classified as minor. Most procedures (n = 1116) had one complication each; however, two complications each were reported for 14 procedures. The vast majority of complications (996; 87.1%) were entered at the time of dictation by the faculty member (780; 78.3%) or the trainee (217; 21.7%) dictating the procedure; the remaining 148 events (12.9%) were entered afterwards by the same faculty member who did the initial dictation or a different faculty member using either ClinicStation or RadStation. More than three-fourths (123; 83.1%) of these later entries represented events that occurred within 30 days of the procedure, while the rest (25; 16.9%) represented events that happened after the 30-day period. The faculty members who performed the procedures reported 51 (34.5%) of these late entries, while the remaining later events (97; 65.5%) were entered by a different faculty member.
Most recent articles on online radiology QA reporting systems have focused on diagnostic radiology and are primarily related to errors in interpreting images, with little emphasis on IR procedures.6,9,13, 14, 15, 16 The system discussed in this paper focuses on entering complications related to IR procedures and can therefore be used for QA programs in interventional radiology departments.
Some of the previously reported methods of tracking IR-related complications require physicians, technologists, secretaries, nurses and system operators to manually enter patient- and procedure-related information.17,18 One of these systems requires the physician to enter all demographic and procedure-related details in on hard-copy data sheets, which are stored in binders and subsequently entered into an electronic database by a technologist, taking 2 minutes of technologist time per procedure.18
HI-IQ is a commercially available free-standing electronic database that allows for online entry and tracking of procedure-related complications; however, the stand-alone QA program requires the operator to specifically log on to the database, thereby interrupting routine workflow9 This interruption, in our opinion, could be perceived as an obstacle by the interventional radiologists, potentially resulting in an underreporting of complications.
The IR QA program described here avoids the expense and complexity of maintaining myriad interfaces to support a stand-alone QA program by being specifically integrated into pre-existing systems containing patients’ EMRs. One limitation of non-integrated information systems is the difficulty of sharing of data between the systems. Our system differs from the previous systems in that it is completely integrated with the online dictation system and thereby results in minimal, if any, workflow interruption for the interventional radiologist, as QA event recording becomes an extension of the existing dictation process.19, 20, 21 More important, our system makes data entry mandatory and prevents the radiologist from ending the dictation and proceeding to the next dictation without completing the QA component. This step allowed all of the interventional procedures performed during the study period to be entered into the system. Since this database is linked to the institutional EMR system (ClinicStation), other patient- and procedure-related data are automatically acquired from existing information in ClinicStation, thus obviating the need for duplicate entries, as is required by some other database systems.19, 20, 21
Our system has the potential to be used in many types of QA and peer review programs within our department, and even for IR departments outside our own. Using the database, we can calculate the rate of major complications (overall and for different kinds of procedures) for the entire group and that for individual faculty members over a given time period. These rates can be compared with the thresholds described in SIR guidelines, and if indicated and appropriate, can lead to Quality Improvement initiatives, potentially resulting in better and safer patient care. The complication rates for individual faculty members can provide criteria to be used for peer review. We can also follow the trends in complications rates over a period of time to assess the usefulness of quality improvement initiatives.
Institution of the IR QA also allows radiologists to easily identify and retrieve cases for discussion in morbidity and mortality sessions. Before instituting the current IR QA program, we had to ask individual faculty members to voluntarily contribute such cases for analysis. With the current system, however, a quick query of the IR QA program can identify all complications during any given time period and thereby allow the peer review committee to select appropriate cases to be discussed.
This system’s design has its limitations. However, we have implemented or are in the process of implementing modifications to address those limitations to maximize the system’s usefulness. In our opinion, the entry of complications discovered after the initial dictation was not reported at 100% compliance, as evidenced by the fact that only 13% of all entries were made after the initial dictation. Although, we do not have evidence to validate this statement, the low proportion of late entries does suggest that many of the complications discovered after the initial dictation are not being captured. We feel that the most important reason for not entering data was that it required a voluntary effort by the radiologist to access a computer, log on to ClinicStation, find the appropriate IR procedure, and enter the complication. Although the IR QA is popular among radiologists, any process that involves interrupting routine clinical or academic activities is likely to achieve less than 100% participation. After the initial months, we implemented the automatic prompt of “please QA previous report. We have not yet evaluated whether this option has increased the rate of reporting of delayed events. Although this prompt reminds the physician to QA the previous procedure, it does not make it a mandatory step, as acknowledging the prompt allows the radiologist to end the current dictation. We plan to modify the current system so acknowledging the prompt will automatically lead the radiologist to the previous procedure. The radiologist will then be forced to make an entry into the dictation (either “none” or a complication, if indicated) before starting the next dictation.
In addition, we have recently modified the system to allow IR physician assistants to enter delayed complications through ClinicStation. As an extension of this plan, physician assistants will be prompted to enter complications events to previous IR procedures if indicated, whenever they dictate a follow-up note regarding IR patients. Our physician assistants are involved in routine post-procedure evaluation of inpatients and outpatients and are aware of delayed complications. We hope that this requirement will increase our capture rate of information about delayed complications.
Since we did not have any formal method of collection of complication events related to interventional procedures prior to the implementation of this program, we are not able to conclusively show that our system results in improved participation in the QA program. Another limitation is the fact that since this program is internally developed at our institution and not commercially available, this could limit broad applicability of the program by other departments. However, the goal of the study is to introduce the concept of integration of the QA step into the routine workflow of the interventional radiologist and use of force function that makes the data entry step a mandatory step. Similar integration concept could easily be applied to other commercially available QA programs (e.g., Hi-IQ) that will integrate the QA entry step to the routine workflow of the interventional radiologists and will force the individual to record the event.
In conclusion, the integration of IR QA reporting system into the workflow process and making completion mandatory has the potential to minimize the work required for data entry and to improve participation in the QA process compared to data entry requirements for stand-alone software programs. Further experience is necessary to identify the limitations of the system, and implement modifications to improve the usefulness of the QA program.
- 13.Guidelines for establishing a quality assurance program in vascular and interventional radiology. J Vasc Interv Radiol 14:S203-207, 2003Google Scholar
- 19.McEnery KW, Suitor CT, Hildebrand S, Downs R. RadStation: client-based digital dictation system and integrated clinical information display with an embedded Web-browser. Proc AMIA Symp :561-564, 2000Google Scholar