Keywords

Background

In 2012, a US Agency for International Development (USAID)-funded project for improving health-care quality in the Central Asian Republics began working with partners in the Kyrgyz Republic to promote the rational use of antibiotics (RUA) among prescribers by using principles of quality improvement. Antibiotic resistance among disease-causing bacteria is a worsening global health threat, making it more difficult and more expensive to treat common infections such as pneumonia, bladder infections, and skin infections. The World Health Organization (WHO) developed a global strategy for the containment of antimicrobial resistance (AMR) in 2001, which included promoting RUA as a key component of national strategies on AMR, knowing that indiscriminate use of antibiotics is strongly correlated with high levels of antibiotic resistance. Promoting RUA among clinicians involves provision of evidence-based guidelines on diagnostic and treatment strategies for common infections; the education of providers on AMR and prescribing guidelines; and institutionalization of regular audits of prescribing practices, providing feedback to clinicians using comparison with peer groups or external standards.

When the RUA initiative began in 2012, no routine surveillance of antibiotic resistance levels for common infections was conducted in the Kyrgyz Republic ; therefore, only limited data were available to characterize the scope of the problem. However, the rate of multidrug-resistant tuberculosis (TB) in Kyrgyz Republic is one of the highest in the world (25% among new cases of TB; 56% among previously treated cases). Likewise, no routine auditing of antibiotic prescribing practices was conducted in the country prior to this initiative, but a survey of antibiotic prescribing patterns conducted by CitiHope International in one Kyrgyz hospital in 2007 showed that antibiotics were inappropriately prescribed in 73% of cases, were not indicated in 48% of cases where prescribed, and indicated but incorrectly selected in 25% of cases.

Although containment of AMR was not prioritized in the Kyrgyz national health strategy in effect from 2012 to 2015, the RUA initiative was strongly supported by the Ministry of Health (MOH), local partners, and international development partners, including the World Health Organization and the Medicines Transparency Alliance. In fact, a national working group on AMR was established shortly after the initiative began. While the national working group was informed of the activities, they were not directly involved in overseeing the RUA initiative.

One of the first activities under the RUA initiative of the project was to identify and characterize gaps in knowledge, attitudes, and practices (KAP) through a survey related to antibiotic use among consumers and prescribers to help establish the need for intervention and to ensure that planned activities were appropriately focused. The KAP survey showed that adult consumers take, on average, over two courses of antibiotics per year and identified many misconceptions about the usefulness of and indications for antibiotics among both consumers and physicians.

Figure 8.1 shows the key stakeholders in the RUA initiative in the Kyrgyz Republic. In addition to the MOH and project staff, two professional associations—the Hospital Association of the Kyrgyz Republic (HAKR) and the Family Group Practice and Nurses Association (FGPNA)—played a central role. Both associations were created in the late 1990s to provide advocacy and support for their constituents. Both organizations began to coordinate quality improvement initiatives through previous health reform projects funded by USAID between 1996 and 2009, which focused broadly on health system strengthening with a focus on improving primary care and reforming health financing. This project included a component on “Other Public Health Threats,” under which the project launched the initiative to promote the rational use of antibiotics.

Fig. 8.1
figure 1

Key stakeholders and their roles

Full size image

Located in the heart of Central Asia, the Kyrgyz Republic is a mountainous country with a population of nearly six million and is one of the poorest of the former Soviet Republics. The socialized health-care system includes an extensive network of primary health-care (PHC) facilities and hospitals, which are managed and financed independently from one another at the district and regional levels. Regional (oblast) Family Medicine Centers (FMCs) represent the highest administrative level of service delivery at the ambulatory level; district (rayon) FMCs are financed and managed independently from regional FMCs but are under the leadership of the regional FMC director, who also serves as the regional health coordinator. Health facilities are managed by physician directors; deputy directors are tasked with ensuring the quality of clinical services, including the coordination of continuing professional development of their health-care staff through the Kyrgyz State Medical Institute for Retraining and Continuing Education (KSMIRCE). The KSMIRCE is a State institute responsible for providing continuing medical education (CME) to all physicians and nurses throughout the country and has a network of regional-level family medicine training centers, each with at least two full-time family medicine physician and nurse trainers. The Family Medicine Department of the KSMIRCE has worked closely with the FGPNA since its inception to respond to training needs voiced by FGPNA members and to coordinate continuing medical education topics with QI initiatives.

Organizing the Improvement Effort

Foundational Work with Implementing Partners

The project’s first task was to meet with local partners, who would serve as the primary implementers, to outline a general strategy and assign specific roles and responsibilities. The FGPNA and the HAKR were natural choices for key implementing partners: they were recipients of grants under the project, had worked on previous QI initiatives, and had institutionalized roles in supporting their constituents at the PHC and hospital levels to provide quality care. Within each organization were two or more individuals who had worked as external quality advisors on previous quality initiatives supported by USAID. During these first meetings, the FGPNA and the HAKR, together with the regional quality advisor (who was also a physician), agreed upon the strategic approach that would be used in the RUA initiative (see Fig. 8.2).

Fig. 8.2
figure 2

Strategic approach used in the RUA initiative

Full size image

The project’s regional quality advisor also met with family medicine trainers from the KSMIRCE to solicit their help in developing the training materials. As the KSMIRCE is responsible for approval of continuing medical education (CME) and annual CME training calendars , working through them was important for institutionalization and future scale-up of training. Figure 8.3 shows the overall timeline of the initiative to rationalize the use of antibiotics in Kyrgyz Republic.

Fig. 8.3
figure 3

Rational use of antibiotics initiative timeline

Full size image

Choosing Improvement Priorities: Selecting Index Conditions

Focus groups of clinicians were then organized by the FGPNA and the HAKR to define a list of “index” conditions for which antibiotics were often unnecessarily or incorrectly prescribed. This was done at the national level rather than the facility level for two reasons: first, all key stakeholders agreed that these conditions were unlikely to vary significantly between facilities; second, using one common set of conditions would allow the project to use the same audit instruments and training curriculum for all participants, compare results between facilities (peer comparisons), and collate data at district, regional, and national levels. The FGPNA gathered 14 physicians representing both urban (nine) and rural (five) PHC facilities, while the HAKR gathered 9 hospital-based physicians from the Bishkek facilities.

  • Brainstorming was used to develop a list of conditions that group participants viewed as significant with regard to unnecessary or incorrect use of antibiotics.

  • General voting was used at the PHC level to narrow down the list to approximately four conditions, based on frequency of disease and frequency of incorrect prescription/selection of antibiotics.

  • At the hospital level, conditions were selected using a prioritization matrix that included the condition frequency based on the experts’ experience, frequency of incorrect use of antibiotics, problems with over- or underdiagnosis, and likelihood of changing providers’ behaviors. No attempt was made to verify the frequency of the selected conditions, as there was general consensus among focus group members. Baseline data collection on the selected conditions allowed the project to confirm the frequency of inappropriate antibiotic prescriptions and use of inappropriate diagnostic criteria.

  • Four conditions were selected to target at the PHC level: common cold, acute bronchitis, acute sinusitis, and acute diarrheal disease; five conditions were selected to target at the hospital level: pneumonia, asthma, chronic obstructive pulmonary disease, cystitis, and pyelonephritis. Pneumonia was selected because of the frequency of the condition and inappropriate selection of antibiotics. Pyelonephritis is often overdiagnosed in Kyrgyz Republic (as in all countries of the former Soviet Union), particularly “chronic pyelonephritis” for which antibiotics are inappropriately prescribed. At the time the study began, women with cystitis were often hospitalized for treatment.

The next step was to determine what, if any, evidence-based clinical guidance existed to use as a standard for the development of audit instruments on the index conditions. Kyrgyz physicians are accustomed to using nationally approved clinical protocols (CPs) to guide their practice decisions following a broad initiative to develop CPs in 2000–2001; however, the number of approved CPs was limited, and principles of evidence-based medicine (EBM) were not known or widely applied to CP development prior to 2004, so there was concern that CPs on the index conditions might not be available or, if available, might not be evidence-based. For this reason, a desk review of existing CPs on index conditions was conducted by the regional quality advisor in conjunction with the EBM unit within the Kyrgyz Ministry of Health. No protocols existed on the four topics targeted at the PHC level. Of the five conditions targeted at the hospital level, evidence-based guidelines had been approved for three (pneumonia, asthma, and chronic obstructive pulmonary disease). No guidance was available for cystitis, and the CP available on pyelonephritis from 2003 contained many non-evidence-based recommendations on risk factors, diagnosis, and treatment. A decision was made at that point that the needed clinical protocols would be developed simultaneously with the training curriculum for the initiative, as evidence-based source materials were needed for both. CP development teams were composed of KSMIRCE family medicine trainers (for PHC protocols) and clinical specialists, with technical support from the regional quality advisor and with methodological support from the MOH EBM consultant.

Developing Audit Instruments

The regional quality advisor worked with the QI coordinators from the professional associations to develop audit instruments (checklists), from which QI indicators would be taken for each index condition. The intended use of the audit instruments was that they would be applied quarterly by facility-level curators (FMC deputy directors) in “audit cycles” and that feedback would be provided to health-care providers after each audit with assistance from the FGPNA QI coordinators.

No information on antibiotic prescription was collected and reported in the existing health information system, nor was it retrievable from the database of the mandatory health insurance fund, the State single-payer entity that regularly conducted quality assurance reviews. This was, in part, because antibiotics were widely available in pharmacies without a prescription, so doctors rarely wrote prescriptions for them, making it difficult to track their use without conducting chart reviews. Although the use of checklists like the ones created was not routine in Kyrgyz Republic and no MOH-approved process existed for their creation, the FGPNA and the HAKR had experience developing such audit instruments from previous national continuous quality improvement efforts in antenatal care and asthma, so the format was familiar to them.

For conditions with existing evidence-based guidance in the form of a national CP, the national CP was used as the source of standards on which audit instruments were developed. For conditions without an existing CP (e.g., cystitis) or if the existing CP was considered non-evidence-based (e.g., pyelonephritis), the regional quality advisor and QI coordinators collected evidence-based recommendations by reviewing clinical guidelines from other countries. For conditions where the decision to prescribe antibiotics is based on a set of clinical rules (e.g., sinusitis), the project included documentation of pertinent clinical findings in the audit instrument. The checklists did not exceed one page per condition, and about half of the included questions were common to all checklists. Examples of checklist items common to each index condition included:

  • Were antibiotics prescribed? If so, how many, types, doses, route (oral, intramuscular, intravenous), frequency, and duration of each.

  • Were medications prescribed for symptomatic care? If so, please list.

  • Were non-pharmacologic measures recommended? If so, please list.

QI coordinators from the FGPNA and the HAKR tested the draft audit instruments in urban facilities prior to general use by reviewing 10–20 clinical records for each condition. This allowed the coordinators to identify data that were not feasible to collect, ambiguities in the instruments, and problems with identifying and finding records based on a specific diagnosis. For example, they found that patients with presumed viral “sinusitis” were sometimes diagnosed with “viral upper respiratory infection,” as clinicians often reserved the diagnosis of sinusitis for patients with signs of a bacterial sinus infection.

Selecting Quality Indicators

Selection of the indicators (see examples in Fig. 8.4) was largely based on the original discussions that took place between QI coordinators from the professional associations and the regional quality advisor to develop the disease-specific audit instruments. Given that the overall aim was to reduce unnecessary antibiotic prescription, the percent of patients treated with antibiotics was included as an indicator for each index condition. Some of the index conditions required treatment with antibiotics (e.g., pneumonia), but a known problem with overdiagnosis prompted the project to include indicators related to establishing the diagnosis based on well-defined diagnostic criteria. For example, in all the study sites, chronic pyelonephritis (kidney infection) was diagnosed much more often than acute pyelonephritis and was likely to be treated with antibiotics despite an absence of clinical signs or symptoms suggesting an active infection (diagnosis was typically based on the presence of back pain and an “abnormal” renal ultrasound). For this reason, the percent of patients hospitalized with a diagnosis of pyelonephritis that had documented signs of infection, such as pyuria (defined as over 10 white blood cells per high-power field on urinalysis), fever, or bacterial growth on a urine culture, was included as an indicator. Similarly, the percent of patients diagnosed with sinusitis who were assessed using recommended criteria suggesting bacterial infections that would justify antibiotic use was included as an indicator. Such indicators related to diagnosis were likewise included for sinusitis, cystitis, and pneumonia.

Fig. 8.4
figure 4

Examples of audit indicators

Full size image

Other indicators were included based on regional disease patterns and priorities. For example, the percentage of patients diagnosed with bronchitis who were asked about smoking status was included since addressing high mortality from noncommunicable diseases was a national priority in Kyrgyz Republic. Similarly, the percentage of charts of patients diagnosed with acute bronchitis with documentation of the duration of cough was included because of the prevalence of tuberculosis (referring all patients with a cough duration of greater than 2 weeks for TB testing was a standard of care). In total, facilities routinely analyzed and tracked 8 indicators on 4 conditions at the PHC level and 15 indicators on 5 conditions at the hospital level. The database-generated reports , however, included additional indicators on each condition as well as data about which antibiotics were prescribed, antibiotic dosing regimens, and medications prescribed for symptom control. Facilities had the option of using this additional data to better understand physician prescribing behaviors.

Developing a Database to Track and Analyze Quality Data

Early in the organizational stage, it was necessary to think ahead and decide how audit results would be recorded, analyzed, and tracked. Based on prior experience implementing QI initiatives, one of the main roadblocks to the improvement cycle is the limited ability of health managers and leading clinicians to analyze collected data in a way that promotes problem solving and formation of improvement plans. Large volumes of data were routinely collected by health facilities, but most often, this data was simply passed up the chain of command and summarized at the national level rather than being used by facilities for decision-making.Footnote 1 During previous QI initiatives, advisors found that they were frequently retraining managers on how to calculate and graph relatively simple indicators. On the other hand, managers and providers were usually able to interpret audit results when external QI advisors presented them with time series charts (line graphs showing changes in one or more indicators over time). During previous projects, quality advisors and/or QI coordinators from professional associations entered all data into Microsoft Excel-based spreadsheets, generated graphs, and presented them to physicians during coaching visits. The Excel-based system was cumbersome, prone to data entry errors, and placed a heavy data entry and processing burden on the professional associations. Further, it did not provide flexibility with regard to the type of data analysis, making it infeasible to collate data at the district or regional level.

Therefore, to overcome these limitations for this initiative, the project developed a database that would have the following features:

  • Data entry could be managed at the facility level with a user-friendly interface that minimized the risk of data-entry error.

  • Data could be automatically collated at district, regional, and national levels and by selected dates.

  • All indicators would be graphed automatically to facilitate analysis.

  • Indicator results could be presented by provider or facility code to maintain confidentiality and avoid the risk of facilities being penalized through traditional lines of reporting.

  • Results could be presented by QI coordinators or coaches with relative ease (either using a projector or printing hard copies of graphs for distribution during QI meetings).

  • The database could be used on computers without the latest versions of operating systems or productivity programs such as Microsoft Word, Excel, or Internet Explorer.

An information technology specialist and systems analyst working on the project developed the database based on a detailed list of specifications submitted by the project’s regional quality advisor. This included a list of indicators and detailed definition of each indicator; the numerator and denominator of each indicator; and a description of how the results should be displayed, depending on whether the desired report contained data from a single audit or from multiple audits over time.

The original version was created for offline use out of concern that some PHC facilities might not have Internet access. However, as collation of data from multiple facilities and districts was essential, the information technology specialist included a feature that would allow QI coordinators at the regional or national level to import updated database files from facilities, allowing them to keep a “master” database.

To facilitate data entry, the user interface looked almost identical to the paper-based audit instruments developed for each index condition. Once data from the audit instruments was entered, reports could be generated by selecting the condition, audit cycle (baseline, first follow-up, second follow-up, etc., or any combination), and the level of data analysis (provider, facility, district, region, country). The database also generated lists of antibiotics and medications prescribed for symptom control, displayed by frequency of their use, from the most recent audit cycle included in the report.

Selection of Study Sites

The FGPNA and the HAKR selected study sites to pilot the audit instruments in consultation with the regional quality advisor. The following issues were considered in the selection of pilot facilities:

  • Selected sites should represent both rural and urban facilities due to potential variations in practice patterns.

  • Selected sites should represent the northern and southern regions of the country due to potential variations in practice patterns.

  • If possible, selected facilities would be from districts in which the FGPNA and the HAKR already conduct monitoring (for cost containment and efficiency).

  • Selected hospitals and PHC facilities should be from the same districts. In addition, the regional “parent” hospital, often located outside of the district, would be included to ensure consistency of practice patterns between regional and district facilities.

Three pilot districts were selected for the PHC-level interventions (one rural district from the north; one rural from the south; and three Family Medicine Centers in Bishkek, the capital). The hospitals in each of these rural districts were selected, along with their respective regional hospitals and four Bishkek facilities, for the hospital-level initiative. The MOH concurred with and approved the facilities proposed for the initiative.

Representatives from the HAKR and the FGPNA conducted baseline monitoring in all study facilities. This monitoring occurred prior to any educational intervention and provided verification that gaps in quality existed for each index condition, helped to better focus the content of the educational intervention, and allowed each facility to see the “starting point” from which they could later measure improvement. At the PHC level, at least five patient records on each index condition were reviewed from each Family Group Practice. (See “Audit and Feedback” section below for details.) There are 10–15 Family Group Practices in each district, and the aim was for no less than 30 records per condition per district. At the hospital level, the data collectors attempted to review 30 records per condition in each facility.

Developing Training Materials

In parallel to the development of the audit instruments and database, family medicine trainers from the KSMIRCE, the MOH chief pulmonologist, and two urologists with academic appointments began to develop training materials and clinical protocols on the index conditions with a focus on appropriate antibiotic use. Those involved in this QI initiative were convinced that an educational intervention would be necessary to effect change in the antibiotic prescribing practices of target physicians based on awareness of existing practice patterns (verified by the results of the baseline audit), results of the KAP survey, and the lack of approved, evidence-based guidance on the selected index conditions. The project aimed to develop a training program that was focused and effective, relying heavily on the practical application of knowledge to solve clinical vignettes. A 2-day training for PHC providers and a separate 2-day training for hospital providers on the selected conditions were developed with a mixture of didactic presentations and case-based, small group activities. The lectures focused on clinical features, diagnosis, and the evidence for or against the prescription of antibiotics. Clinical vignettes were developed to be discussed in small groups where participants would be asked to make a diagnosis and treatment plan and to defend their answers using information from the lectures. The regional quality advisor helped coordinate these efforts and gave ongoing technical support by providing evidence-based source materials, reviewing draft versions of all training materials (schedules, handouts, lectures, clinical vignettes, and written knowledge assessments), and providing feedback to the developers. All partners working on the educational materials had previously been trained in principles of evidence-based medicine under USAID health reform projects and were encouraged and supported by the regional quality advisor to ensure that all clinical recommendations included in training materials were supported by evidence. Once developed, the training materials were approved through the KSMIRCE, which enabled the project to offer CME credit to all training participants.

Carrying Out the Improvement Effort

The basic design of this improvement effort was to train family medicine providers from pilot districts on evidence-based standards of diagnosis and management of the index conditions, followed by regular use of audit and feedback, combined with facility-level discussion of results and formation of improvement plans, to promote adherence to the standards. Feedback was based on the previously described set of indicators that was common to all study sites at the PHC or hospital level, which allowed results to be collated at the district, regional, and national levels. In addition to training, the educational intervention included a set of job aids that served as reminders of key standards of care.

Training

Training was conducted in all target districts over a period of 3 months, with KSMIRCE family medicine trainers, the MOH chief pulmonologist, and the regional quality advisor serving as trainers. At the end of each 2-day training session, QI coordinators from the FGPNA or the HAKR presented results of baseline monitoring. Participants from the hospitals and PHC facilities were asked to critique their own prescribing patterns and, where indicators showed a need for improvement, to discuss how to implement recommendations presented during the training. Each district was able to see results from other districts, which seemed to be a particularly strong motivator for change when providers saw that their diagnostic or prescribing patterns were further out of compliance with standards than those of providers in other districts.

Job Aids

In preparing for this initiative on reducing unnecessary antibiotic prescriptions , the project studied similar initiatives carried out in other countries, including the United States, United Kingdom, and Italy. One practice common to most initiatives was the use of a job aid that allowed providers to give something to patients in place of an antibiotic prescription. The project developed a non-antibiotic prescription pad for common respiratory illnesses that allowed doctors to simply check one of the several common diagnoses, check one or more recommended non-pharmacologic approaches for symptom control, and make a recommendation for non-prescription pain medicine or fever reducers. Space was also provided to indicate when the patient should return should symptoms not improve. This “prescription” incorporated key messages to reduce the amount of time physicians spent counseling patients.

“How to take antibiotics correctly” was another job aid printed on a prescription-sized pad that informed patients how to properly take antibiotics (take as prescribed, finish course, don’t share, don’t save for future use). Both job aids were distributed to all study facilities along with posters containing key messages on antibiotic use for consumers targeting knowledge gaps that were identified through the previously mentioned KAP survey. The posters and leaflets were approved by the MOH, but facilities were not mandated to use them. The project found that the pilot facilities were universally eager to use these materials.

“When to prescribe antibiotics,” a one-sided, letter-sized sheet with antibiotic prescribing indications for pharyngitis, otitis, and sinusitis was a job reminder borne out of the first seminar the trainers delivered to PHC providers on index conditions. When reviewing key points with the participants at the end of the second day, the trainers realized it would be helpful for busy providers to have the commonly used clinical scoring systems and criteria for starting antibiotics summarized on a page that could be kept on the desktop or at arm’s reach. As these criteria were new to the providers, it did not seem realistic that they would remember them, and the trainers wanted to do everything possible to ease the translation of this new knowledge into practice .

Audit and Feedback

QI coordinators from the HAKR and the FGPNA conducted a follow-up audit visit approximately 2 months after training. During visits to each participating facility, the QI coordinator worked side by side with either the deputy director of an FMC (PHC level) or the chief physician (hospitals) to identify and review medical records. These particular positions were chosen to act as “QI curators” because they were, by decree, responsible for ensuring the quality of clinical services in their respective facilities.

As part of the country’s routine health information system, physicians complete a “clinical information form” for all PHC visits and hospital admissions, assigning one or more diagnoses with corresponding International Classification of Diseases, tenth revision (ICD-10) codes, which are subsequently entered into a database by a data entry clerk. The QI coordinator and QI curator used this database to identify medical records for audit by the ICD-10 code of index conditions. Depending on the number of patients diagnosed during the audit period with the condition of interest, either all applicable charts were reviewed (if less than five for a PHC facility or 30 for a hospital) or an appropriate subset. The QI coordinator and QI curator would then complete audit instruments (checklists) for each condition. Data from the review of five medical records could be entered on a one-page checklist. At the hospital level, the QI coordinator helped the QI curator enter data into the database; at the PHC level, the audit instruments were collected and entered into the database by the FGPNA’s QI coordinators after returning to their national office. External coaches informed facility QI curators of their visits in advance so that an internal QI meeting could be scheduled with all relevant clinicians immediately after the audit.

During these feedback meetings, the external QI coordinator and facility QI curator presented indicator results to facility providers using time series charts and tables automatically generated from the database during the same visit (hospital level) or a subsequent visit (PHC level). After displaying the result of a quality indicator, the coach would ask clinicians to recall the standard of care and evaluate their facility’s performance based on the indicator result. Typically, the providers would be asked to comment on each indicator result, noting where improvements had been made and pointing out where there were persistent problems with adherence to clinical standards and offering an explanation as to why (barriers to improvement), followed by a discussion of potential steps to improve adherence to standards. QI coordinators also showed comparison data for neighboring districts and regions .

Data Collection and Analysis

Although installation of the offline database described above was quite simple using a self-extracting file, it had to be done in each facility by the QI coordinators from the professional associations. In addition, there were occasional problems with exporting audit report data to save in document or spreadsheet format, depending on the end user’s software. A more significant problem was maintaining a “master” database at the national level and getting results to the facilities in a timely manner. Theoretically, database files updated at the facility level could be e-mailed to the QI coordinator to import into the master database. However, this rarely happened, either because end users had difficulty locating the database file or were not skilled enough at sending e-mail with file attachments. In reality, the QI coordinators gathered audit data in the field using paper forms and then returned to their central offices where they would update the master database file. They would then provide facilities with the updated database files during their next planned visit. As a result, the QI discussions that occurred during visits by the QI coordinators were always focused on results from the prior audit cycle rather than “real-time” data. This was not universally true at the hospital level, where the QI coordinators occasionally had time to help the facility QI curators enter data and generate reports that were discussed during the same visit.

These problems prompted the project to move to an online database in year two. The transition to using an online database was fairly seamless and resolved all of these issues, while creating a few unexpected challenges. Computer and Internet access was often limited to one office within each facility, but this did not pose a significant barrier to utilization. Most users were trained in person on how to find and log on to the database and were able to use it well afterward. Although not an issue in the Kyrgyz Republic, other countries where the project implemented identical initiatives were hesitant to have their QI data stored on a server outside the country (the project rented server space from a company based in the United States for the online database). In the Kyrgyz Republic, there remain challenges with transferring the database to an MOH-based server, as they have limited capacity to provide 24-hour server support and maintenance.

The online database was designed to assign various roles to users, each with a customizable set of privileges. For example, a national-level QI coordinator from a professional association or the Ministry of Health might have full privileges to enter, edit, and view data from any district, while a facility-level QI curator would be allowed to enter and edit data only for their facility, but could view data from any facility or district. Facility- and provider-level data were always displayed by code to maintain confidentiality.

Support for Improvement Coaching

QI coordinators from the FGPNA (three) and the HAKR (two) were trained in QI and gained QI coaching experience under previous USAID projects and other QI initiatives during the first 2 years of the project. The regional quality advisor provided support and supervision to each of them during the RUA initiative through frequent face-to-face meetings, phone conversations, and electronic communication. Because the regional quality advisor lived in the Kyrgyz Republic, it was possible for the QI coordinators to discuss issues with him on an “as-needed” basis, even during on-site monitoring visits.

In turn, QI coordinators from the FGPNA and the HAKR trained facility-based QI curators on the following topics:

  • QI methodologies

  • How to select charts for audit

  • How to use the audit instruments (checklists)

  • Use of the database (enrollment, data entry, generation of reports)

  • Facilitation of feedback sessions

Training/coaching was tailored to the needs and skills of each QI curator, as some of the curators had participated in QI training and audits organized under previous USAID projects. During each monitoring visit made by the QI coordinators, several hours were spent supporting the QI curators in conducting audits, entering results into the database, and conducting feedback sessions so that they could eventually conduct these QI cycles without outside support.

One deficiency identified during monitoring and support visits conducted by the FGPNA and the HAKR was the absence of a clinical specialist to answer questions posed by the physician participants. While skilled in QI methodologies, the QI coordinators were not necessarily competent to answer clinical questions that might not have been addressed in the training materials. When possible, experienced clinicians involved in the development of the CPs and/or training materials traveled with the QI coordinators to participate in the feedback decisions (e.g., the chief pulmonologist or the quality project’s regional QI advisor).

Financial Support

The FGPNA and the HAKR received USAID-funded grants through the project for their work on the RUA QI initiative . These grants covered travel expenses and some salary support for staff members, including the QI coordinators. The project made individual consulting agreements with trainers and leading specialists to develop training materials, conduct the seminars, and develop the clinical protocols. Per diems were provided, at rates set by the Kyrgyz Republic Government, for training participants who had to travel more than 25 kilometers, and transportation expenses were reimbursed by receipt. Financial support was not provided to the study facilities or their clinicians to participate in the initiative, to conduct chart audits, or to participate in feedback sessions.

Results

QI coordinators collected baseline data from 26 PHC facilities and 7 hospitals in three districts/municipalities between April and June of 2013. They collected follow-up data 3 and 6 months after the initial training, which was conducted in February 2014. At the PHC level, the FGPNA could collect only 3-month follow-up data.

Following training in pilot sites, fewer patients seeking care for common viral illnesses received unnecessary prescriptions for antibiotics. Compared with baseline prescription rates, the percent of patients receiving antibiotics for the common cold dropped from 31% to 0% (193 and 103 records, respectively); for acute bronchitis, from 89% to 1% (195 and 76 records, respectively); for acute diarrheal disease, from 46% to 0% (126 and 8 records, respectively); and for asthma, from 43% to 25% (41 and 24 records, respectively). Figure 8.5 presents an example of a graph generated from the database on the percentage of patients with asthma for whom antibiotics were prescribed.

Fig. 8.5
figure 5

Example of graph from database

Full size image

Providing facility-specific feedback and comparing performance between districts were powerful motivators of change. For example, acute pyelonephritis (kidney infection), a condition that requires antibiotics, was often overdiagnosed and always treated in the hospital. Baseline monitoring showed that the diagnosis could be confirmed by clinical and laboratory criteria in only 68% of total patients hospitalized with acute pyelonephritis; this indicator ranged from 47% to 98% among pilot regions. During training, the trainers emphasized the typical clinical presentation of acute pyelonephritis and the diagnostic criteria. A QI coordinator from the HAKR then showed the results of baseline monitoring. When the lowest-performing region saw that another pilot region outperformed them (98% versus 47%), there were many gasps and a bit of nervous laughter among participants. During the next two audit cycles (approximately 2 and 5 months after training), this same low-performing region achieved a 100% result on this indicator (the countrywide result increased to 90%). At baseline, physicians were ordering urine cultures for only 17% of patients admitted with the diagnosis of pyelonephritis; during the second audit cycle conducted after training, cultures were ordered in 80% of cases. Not only did the quality of diagnosis improve, but the number of cases being admitted to the hospital dropped dramatically so that during the second follow-up and final 3-month audit cycle, only 10 cases of pyelonephritis could be found in all hospitals from the three pilot regions combined.

The percentage of patients admitted to the hospital with pneumonia who had documented fever and at least one additional clinical or radiographic finding consistent with pneumonia increased from a baseline of 46% to 86% by the second audit cycle. This is important as it reflects either a decrease in unnecessary hospitalizations (fewer admitted patients that do not meet diagnostic criteria) or an improvement in physician documentation of essential clinical data.

Sustaining and Scaling Up the Improvement Effort

From the inception of this initiative to promote RUA, the project implementers knew that the limited budget would only allow for developing the approach and testing its effectiveness in pilot districts. The project implementers also knew that successful national scale-up could only take place after the project ended and would have to rely on institutionalized processes that were supported by the State budget. The following activities were considered prerequisites for future scale-up:

  • MOH approval of clinical protocols that contained evidence-based recommendations on antibiotic use for the index conditions and dissemination of the protocols to end users

  • Integration of the training on index conditions into the curriculum and training calendar of the KSMIRCE

  • Institutionalization of the antibiotic prescription audits and feedback mechanisms

Clinical Protocols

Over the course of this initiative , clinical protocols were developed and approved by the MOH, covering all index conditions for which MOH-approved, evidence-based guidance did not previously exist as well as the following conditions or syndromes, which were identified as high priority by the initiative:

  • Structured approach to the diagnosis and treatment of patients with cough

  • Acute viral infections of the upper respiratory tract

  • Acute bronchitis

  • Acute otitis media

  • Acute sinusitis

  • Acute tonsillopharyngitis

  • Acute cystitis in nonpregnant women

  • Acute pyelonephritis

  • Chronic pyelonephritis

  • Prostatitis

The project implementers prioritized the development and approval of clinical protocols for several reasons. First, physicians in the Kyrgyz Republic are hesitant to follow recommendations made in training (even MOH-approved trainings) if they are not consistent with standards of care in existing CPs, even if those CPs are outdated. Second, physicians in the Kyrgyz Republic know that they can be held to standards of care contained in MOH-approved CPs through external quality assurance audits conducted by The Mandatory Health Insurance Fund. Noncompliance with standards discovered through such audits can result in monetary penalties. Finally, USAID had promoted evidence-based medicine in the Kyrgyz Republic for over 10 years; support and promotion of the development and use of evidence-based CPs were foundational to those efforts.

In fact, the successful development of these CPs was possible only because of groundbreaking work accomplished under previous USAID health reform projects. As a result of those efforts, an evidence-based medicine unit led by a well-trained local EBM consultant existed within the MOH, an approved CP development methodology was in place, EBM training programs were developed for CP developers, and a clear process existed to coordinate CP development from the first steps to final MOH approval. Near the beginning of the initiative to promote RUA, the project implementers proposed a list of topics to the EBM unit to prioritize for development, offered to provide close technical support to the guideline developers, and committed to finance the printing and distribution of finalized CPs. The MOH’s EBM consultant subsequently coordinated the necessary steps to develop and approve the CPs.

The six respiratory conditions were grouped into one collection of protocols; the four urologic conditions into another. The project funded the printing of over 3000 copies of each collection, and they were distributed to all PHC physicians and relevant hospital physicians (urologist, pulmonologists, internists) throughout the country by the FGPNA and the HAKR.

Institutionalization and Scale-Up of Training

Although not originally planned, the KSMIRCE family medicine trainers involved in developing and conducting the seminars on RUA requested that the project implementers organize a “training of trainers” so that all family medicine trainers from the KSMIRCE’s affiliate family medicine training centers in the regions could be trained, which would allow them to scale-up training throughout the country. The national family medicine trainers, with support from the regional quality advisor, decided to combine relevant portions of the hospital-level seminar into the PHC seminar, lengthening the final training to 3 days; trainings on pneumonia, chronic obstructive pulmonary disease, asthma, cystitis, pyelonephritis, and prostatitis were added. Thirty trainers were trained using this combined curriculum, which was approved by the KSMIRCE and added to the following year’s national CME calendar.

Institutionalization of Audit and Feedback Mechanisms

Development of the online database was considered to be a key step toward sustainability of the audit and feedback process, as it reduced the time and skill level required to analyze results. This database was made available to all pilot health facilities and was coordinated by the FGPNA and the HAKR. The project’s regional quality advisor held a number of meetings with the MOH, the FGPNA, and the HAKR to determine where the database should “sit” in the Kyrgyz Republic. It was decided that the Republican Health Information Center would be the most logical choice for long-term sustainability, as they already had information technology specialists trained to manage databases and had available space on hosting servers. The project coordinated training of local information technology specialists from the Republic Health Information Center on technical specifications needed to host the database on a local server, how to troubleshoot problems with the database, and how to create templates for index conditions should local partners wish to audit additional conditions in the future. In the process of training, however, the project’s information technology specialist discovered that the Republic Health Information Center probably did not have the hardware or people with the necessary technical skills to host and maintain the database. It was, therefore, recommended that arrangements be made to continue to pay a small annual fee to maintain the database on the server of hosting company based in the United States.

Since the first quality improvement initiatives were implemented through the professional associations (FGPNA and HAKR) in 2005, there were many national-level discussions about the need to institutionalize QI approaches in all health facilities and how best to approach institutionalization. The selection of professional associations to coordinate QI efforts was a step in that direction. As monitoring conducted by the MOH had traditionally been punitive in nature, mechanisms existed at the facility level to ensure audit results were “satisfactory” (i.e., not always reflecting actual practice). It was important, then, to have QI promoted by organizations that were trusted by health facilities and disconnected, as it were, from disciplinary measures so that facilities felt safe to identify and address gaps in quality of care. Having been founded to advocate for and represent their respective members, the FGPNA and the HAKR were well connected with and respected by health facilities. As NGOs, the FGPNA and the HAKR were also able to receive grants from international organizations, such as USAID, that were interested in promoting QI.

The FGPNA and the HAKR were originally trained in QI principles by personnel from the USAID-funded health reform project ZdravPlus II (2005–2009). As they became more familiar with the principles and practice of QI, the professional associations took on increasingly greater responsibility with training, conducting QI audits, and leading feedback sessions.

The intent of the USAID projects had always been to institutionalize QI efforts at both the PHC and hospital levels. The professional associations were advised to support facilities to conduct the key steps in the QI cycle themselves rather than having the professional associations conduct audits, which carried the risk of turning QI activities into “external monitoring and feedback.” In fact, the project never reached the goal of having PHC facilities independently conduct QI cycles. Through many discussions with FGPNA and project quality advisors, the project identified a number of factors that may have acted as barriers to institutionalization of QI at the PHC level:

  1. 1.

    The FGPNA did not have the personnel and the USAID project did not have funding to conduct QI audit cycle visits to each PHC facility more often than once a quarter. The number and geographic spread of PHC facilities in each district contributed to this challenge, as most districts were served by 15–50 PHC facilities, and the FGPNA might be able to visit only two or three facilities in 1 day. If in-person visits to each facility were not required, medical records could be sent to the district-level family medicine center for review, but this created another set of challenges, as FMC personnel rarely made visits to daughter facilities, and personnel from the daughter facilities typically traveled to the district center only monthly. The FGPNA did its best to coordinate audit cycle visits at the time when routine district-level health meetings were conducted so that results of monitoring could be discussed without personnel having to make additional trips to/from their facilities.

  2. 2.

    The time between visits (typically 3 months) seemed too long for participants to “internalize” steps in the audit cycle. In other words, the QI approach did not become habitual, and FGPNA personnel often felt like they were starting from square one during follow-up visits. This primarily applied to the mechanics of auditing medical records and calculating indicator results. There was not such a problem with facilities forgetting the goals of the QI efforts, their results, or the clinical standards on which the indicators were based.

  3. 3.

    Many of the rural PHC facilities had only one or two physicians, making it impractical for them to conduct their own audits. Successful monitoring of charts from these facilities required a new organizational process of getting the appropriate charts to the district FMC and back. On the surface, this may sound simple enough, but success depended on a number of conditions:

    1. (a)

      Medical records of patients with particular (target) conditions are not easily identifiable. In fact, in the Kyrgyz Republic there is no electronic or paper record of patient visits below the district FMC level. A paper “clinical information form” is completed for each patient visit (or visits, if more than one visit is required to manage the same complaint or condition), which is then sent to the district level and entered in a database, but this often occurs weeks after the visit, and only the district FMC has access to the information. So, in order to select medical records for audit, either an additional journal must be kept at each facility to record the names of patients seen with a target condition or a list of patients seen with the target condition must be generated from the database at the district FMC level and daughter facilities notified by phone of which charts to select for review.

    2. (b)

      Charts selected for audit are physically located in the target facility. Quite often, patients take their medical record from the PHC facility, either because it is needed during a visit to a consulting physician or simply because they are afraid their record will be lost by the PHC facility. The FGPNA raised this issue at the level of the MOH, and an order was issued restricting facilities from releasing the original copies of medical records to patients. However, given the absence of electronic health records and photocopying capabilities in facilities, the order was never fully implemented.

    3. (c)

      Charts selected for audit by one of the processes listed above should be reliable, meaning they are not “enriched” by the treating physician prior to being delivered to the district facility for review.

    4. (d)

      Personnel from the target facility actually travel to the district facility for the monthly meeting. Costs of such travel were typically not reimbursed by the district FMC, leaving nurses and physicians from daughter facilities to pay for the travel themselves.

  4. 4.

    Health-care workers were not comfortable with nurses reviewing physicians’ notes in medical records, effectively eliminating this as a monitoring option in smaller PHC facilities. This resistance was likely due to the limited scope of clinical responsibilities traditionally designated to nurses and a historical approach to clinical audits that was rather subjective and dependent on the clinical expertise of the reviewer (rather than using objective checklists with clearly defined quality indicators).

  5. 5.

    The deputy directors of FMCs and hospitals were responsible for quality of care but often overburdened with administrative and clinical responsibilities, leaving them little time or mental space for conducting quality audits and quality meetings between visits of the FGPNA QI coordinators.

At the hospital level, significant effort was made during the initiative to develop the capacity of personnel to conduct the quality audits without external support. During all follow-up visits after training, the HAKR representative worked side by side with the hospital’s deputy director (responsible for quality) to select and review charts and complete the audit instruments. Beginning with the second follow-up visit, deputy directors entered audit data into the database.

An order was issued by the MOH to task all pilot hospitals to continue with at least quarterly monitoring of antibiotic prescription patterns and to enter results into the database after the closing of the project. In fact, since the close of the project, further audits have been completed (without external funding) and entered into the online database.

The HAKR attempted to activate hospital quality committees that existed by MOH decree, hoping that audit tasks could be shared among committee members rather than falling to one person (the deputy director); however, the project has yet to see that these efforts will lead to sustainable institutionalization of this approach. For the most part, the quality committees in hospitals exist on paper only, and most of the members do not have a vision of what their role should/could be. In 2015, the MOH revised the national QI strategy and scope of work of quality committees, incorporating many of the principles of QI that had been introduced by USAID and other international development partners over the past 10 years, which is seen as a positive development for institutionalization of QI.

Reflection

Specific to the RUA QI Initiative

Many factors that contributed to the success of this QI initiative are important to keep in mind if one is hoping to apply a similar approach in other contexts. First, the project’s approach must be understood as a “next step” in a series of QI initiatives that were undertaken over a period of more than 10 years. Although the topic was new, the key players had worked together on a number of previous QI initiatives, and there was political support from the MOH for facility-level QI work as a result of prior successes. The FGPNA had over 7 years of experience coordinating QI projects when this work on RUA began, and the HAKR had just completed a 2-year QI initiative on cardiovascular disease. Also, the same international and local partners had worked together closely on previous QI initiatives, so all participants benefited from mutual trust, respect, and a knowledge of one another’s strengths and limitations. The project’s regional quality advisor, who coordinated the initiative, had lived in the Kyrgyz Republic for 10 years, spoke the local language, and was very familiar with the local health system and medical practices.

I liked the methodology of continuous quality improvement: clear; detailed; understandable for training; transparent in regards to monitoring; and a universal approach that can be used to solve many different types of quality problems. Secondly, quality improvement involves all levels of hospital services, making this experience unique.

–Staff member, HAKR

Some features of this QI initiative evolved from lessons learned through prior initiatives. The most significant was the development of the online database that simplified data entry, automatically generated user-friendly and customizable reports of indicators, and allowed coordinators to collate data at any level. The database was regarded positively among all users, from rural facilities to the national-level coordinators and MOH. Importantly, it continues to be used in the same pilot hospitals as of 1 year after the close of the project.

The main challenge we faced was physicians’ resistance to implement new recommendations. New clinical protocols were published at the end of the project, which became the basis for subsequent training of hospital providers.

–Official, Issyk-Kul Oblast Merged Hospital

The project also learned lessons to guide future QI initiatives. In the past, the FGPNA selected QI topics taking into consideration whether evidence-based guidance was already available in the form of MOH-approved clinical protocols and/or practice guidelines. In most cases, target providers had already been trained on the new guidelines through CME programs led by KSMIRCE trainers. Because the goal of this particular initiative was to promote RUA, the project needed to select conditions where antibiotics were being overprescribed rather than conditions for which MOH-approved CPs already existed. Developing a new CP and getting it approved in the Kyrgyz Republic is a lengthy process, typically taking 12–18 months. Because of the time limitations, the project did not have the liberty to wait for approved CPs to be published before starting to train physicians in the study sites. This resulted in physicians being trained in new approaches without an official protocol with which they could justify or defend their practice. The project handled this by working through the postgraduate institute (KSMIRCE) to develop the training curriculum, which was then approved by the MOH, providing legitimacy to the content. Trainers informed participants that content of the training would be reflected in new clinical protocols that were currently in the development stage.

Introducing and Institutionalizing QI Methodologies

Workplace culture is largely determined by the larger cultural context in which it exists and can be counter to some of the foundational principles of QI. QI principles have their roots in a Western culture that highly values democracy, teamwork, and problem-solving. Individuals contributing their ideas for the betterment of an organization and achievement of its goals are valued and often rewarded. Different values exist within the cultural context of Central Asia: respect for strong, authoritarian leaders who give clear direction on how things are to be done (the “why” is often secondary); respect for workers who strictly adhere to orders; and disdain for workers who are critical of existing processes (this is typically equated with being critical of leaders, which is not acceptable). Often, those who show too much initiative in changing the status quo in the workplace are looked upon with suspicion or contempt. It is not difficult to imagine the challenge of introducing standard QI approaches in such a context. Almost each step in the QI process demands a willingness on the part of participants to be counterculture:

  • Problem identification. This requires being willing to admit that there are problems with resources, organization, and/or content of care. Health-care workers did not always feel safe or comfortable volunteering their thoughts about quality gaps and, in some countries of the Central Asian Republics, seemingly could not see such gaps. QI approaches to selecting problems are, in the Western context, very democratic, often involving brainstorming and voting. Workers in the Central Asian Republics were not accustomed to participating in such a process and had no reason to think their opinions would be valued by leadership. Likewise, health managers were not always ready to accept a decision made by a group.

  • Problem analysis. As with problem identification, there was no cultural context for team-based activities such as brainstorming or an open discussion with all members of a process about quality gaps. With rare exception, the project found that brainstorming sessions were rarely productive if a health facility director participated, as health-care workers were simply too reluctant to share their true opinions openly.

  • Planning for improvement. Within a cultural context that valued adherence to top-down standards and subtly or overtly discouraged individual initiative, the project found that health-care workers had a very difficult time thinking “outside the box.” Such thinking is not easily taught. Rather, it is, at least partly, a learned ability that grows only when rewarded within a cultural context that values creativity and initiative.

  • Implementation of improvement plans. The project encountered little resistance to implementing audits, likely because some form of chart auditing had been institutionalized for many years, although its effectiveness in improving quality might be questioned. Because of the challenge of soliciting improvement plans that involved anything more than “improving adherence to the standard of care” or purchasing a needed resource, there was often very little to implement in regard to changing the organization of services.

Giving visual feedback in the form of time series charts showing change in performance over time was a significant motivator of change, particularly when providers could see their performance in comparison to the performance of other facilities. Improving the approach to audits included defining standards of care and clearly defining a feasible number of reliable and valid quality indicators that could be tracked over time. The project found that improving the approach to audits and facilitating feedback and discussion of audit results had a measurable impact on quality, even if all steps of QI had not been institutionalized.