Background

The growing success in resource-limited settings of disease-specific interventions for HIV, TB, malaria and other conditions has led to renewed focus on primary care. Integration of disease-specific services into primary care settings has emerged as a strategy to harness specific gains for more generalized improvement in health care delivery, with HIV and reproductive health care most commonly studied [1–7]. Despite this work, relatively little attention has been paid directly to strengthening primary care services for adults and adolescents – the core services with which disease-specific programs should integrate. In many developing-world health systems, particularly in sub-Saharan Africa, the majority of general adult and adolescent ambulatory care occurs in the outpatient department (OPD), which typically operates within front-line health centers (HCs) or as auxiliary clinics to district or regional hospitals. In many settings, the OPD is the first point-of-contact with formalized health infrastructure for patients who are not already enrolled in other specialized ambulatory services (e.g. antenatal or TB care), or care for specific chronic diseases (e.g. HIV). As such, the OPD serves a critical function in the health system through triaging for life-threatening conditions, screening for chronic diseases and modifiable risk factors, diagnosis of acute illness, and appropriate management of both acute and chronic conditions. However, there are currently no benchmarks or targets established to define quality of care for adults and adolescents for most conditions in this primary care setting.

Few rigorous studies have been done on how to develop a standardized, integrated approach across conditions for the initial adult patient screening and work-up in resource-limited settings [8, 9]. The few existing studies suggest that standardization and integration could be an important component of improving primary care and health worker performance [10–14], but more research is needed. Aside from disease-specific analyses and one qualitative survey [15] a literature review did not identify any reports describing the quality of general ambulatory primary care for adults and adolescents in any resource-limited setting.

Many tools have been developed for services for selected conditions delivered within OPDs, including malaria, STIs, and TB. In addition, an integrated guideline for respiratory disease was developed and tested in southern Africa [16–18]. The World Health Organization (WHO) and partners have attempted to standardize general adult and adolescent primary care services, called IMAI – the Integrated Management of Adolescent & Adult Illness [19]. Building off the success of the Integrated Management of Childhood Illness (IMCI) for children under-five years [20], IMAI is targeted for use by front-line health care workers, and consists of simplified, syndromic clinical protocols within a single integrated guideline for common presenting illnesses in adults and adolescents. Like IMCI, these syndromic protocols are organized around symptoms (e.g. fever, cough, diarrhea, etc.) and include a simplified approach to patient history and physical exam. These protocols then provide prescriptive guidance on the management (triage, screening, diagnose, treatment, and follow-up and referral) for a variety of conditions commonly seen in adult outpatient care. Unlike IMCI [21–26], IMAI has undergone little study, with limited, mostly unpublished validations from South Africa Simoes E, et al: Preliminary Analysis of IMAI Validation Studies. data.], Ethiopia [27], and Lesotho [Seung KJ, et al: Validation of Integrated Management of Adolescent and Adult Illness (IMAI) guidelines for patients with respiratory symptoms in Lesotho. Unpublished.].

The objective of this study is to describe the routine quality of care for adults and adolescents at OPDs in a rural district in Rwanda in order to document the current quality of care, identify needs for improvement, and propose possible interventions going forward.

Methods

Study setting

The study was conducted in southern Kayonza District, in Rwanda’s Eastern Province, with an estimated total population of approximately 190,000 people [28]. The study was conducted at all eight of the Ministry of Health (MOH) health centers that fall under the district hospital catchment area, and all of which receive support from Partners In Health-Inshuti Mu Buzima (PIH-IMB). PIH-IMB is a non-governmental organization partnering with the MOH since 2005 to strengthen health systems in three rural districts in Rwanda. Rwandan HCs are generally staffed by nurses holding the equivalent of a secondary school education in nursing. These nurses undergo no specific training in adult and adolescent primary care aside from their degree program and practice experience. There are, however, a number of disease-specific didactic trainings provided by the MOH and partners that some nurses may attend. Routine documentation of delivery of care is done in the OPD register, a standard lined registry that contains basic demographic, diagnostic and treatment data.

Study population

The study was conducted between February 1, 2011, and March 31, 2011. Consecutive patients 13 years or older presenting to the OPD were eligible for the study. There is no clearly defined age cut-off between childhood and adolescence in Rwanda [29]. Age 13 was chosen for this study based on definitions used by the United States’ Centers for Disease Control (CDC) [30]. Data at each health center were collected over one week, with all centers visited in eight successive weeks.

Data collection

Data were collected by a clinical mentor, employed by the MOH and assigned to train and supervise OPD nurses as part of a larger initiative to improve quality of care at health centers in the district [31]. Supervision and clinical mentoring activities had not yet begun at the time of baseline data collection. The clinical mentor had the equivalent of a bachelor’s degree in nursing with five years of relevant clinical experience. The clinical mentor underwent extensive didactic and practical training in the adapted IMAI protocols, mentoring and quality improvement. The standard WHO-IMAI protocols [32] were previously adapted and translated by the study staff and district MOH partners through consensus guideline review and pilot testing so as to adhere to Rwandan national treatment guidelines and reflect local epidemiologic priorities.

Data were collected by direct observation of routine clinical consultations using a standard data collection tool developed to document critical components of the clinical encounter. This observation checklist was structured upon the WHO-IMAI Case Management Observation Form, and included the standard triage checklist for emergency conditions contained within the IMAI Quick Check protocol, part of the larger IMAI guideline [19]. IMAI had not been formally introduced in Rwanda at the time of the study and study nurses had not received training in the adapted IMAI protocols.

The clinical mentor was present in the room during the consultation but was instructed to not intervene in patient care, with the exception of a critically ill patient requiring immediate action. Data collected in the observation checklist are summarized in Table 1. Formal diagnostic and management decisions made by the nurse were recorded by the clinical mentor from observed actions and review of the OPD register.

Table 1 Summary of data categories within the study Observation Checklist
Full size table

The clinical mentor recorded information on up to three chief complaints for each patient consultation. For each patient complaint the nurse, following MOH standards, selected from the 57 MOH Système d’Information Sanitaire (SIS) diagnosis codes used in routine practice. The clinical mentor recorded the SIS diagnosis selected by the nurse and selected his own SIS diagnosis based on his clinical expertise and additional training in IMAI.

Data analysis

The primary outcome of interest was correct diagnosis of patient illness, defined as agreement in diagnosis between the nurse and the clinical mentor, with patient complaint as the unit of observation. For example, a patient complaining of both cough and back pain would receive one diagnosis for each complaint, and contribute two observations to the sample. Cases where the nurse found no appropriate diagnosis within the SIS codes for a given patient complaint were excluded from analysis of diagnosis agreement. Chief complaints were defined as the primary complaint either: (1) offered by the patient voluntarily, (2) given by the patient in response to direct questioning by the nurse, or (3) selected by the nurse independently at any point during the consultation.

Correct treatment of each complaint was also evaluated, defined as 100% agreement between the nurse and clinical mentor in all elements of treatment plan, including labs/diagnostic tests ordered, medications prescribed, referrals or transfers made, and follow-up recommendations. The nurse treatment plan reflected their routine practice, while the clinical mentor’s treatment plan was based on the modified IMAI protocols. Observations where the nurse diagnosis was incorrect were not automatically excluded from the treatment analysis, so as to capture those cases where the nurse may have selected an incorrect diagnosis but still made a correct treatment plan.

Data were double-entered into a Microsoft Access database and analyzed using SASv.9.2 (Cary, NC, USA). The sample was described using frequencies for categorical variables and means for continuous variables. Patient assessment by nurses was described using frequencies of vital sign measurement, triage, and screenings. The primary outcomes – frequencies of correct nurse diagnosis and correct nurse treatment – were calculated for each complaint, along with corresponding 95% confidence intervals. A secondary analysis was conducted to identify factors associated with the quality of nurse case management. Relationships between correct diagnosis and treatment and health center and duration of nurse experience in OPD dichotomized at four years (the sample median) were assessed using Pearson’s chi-squared tests.

Ethics

This study was approved by the Rwandan National Ethics Committee and the Institutional Review Boards (IRB) at Partners Healthcare and the London School of Hygiene & Tropical Medicine. Data were collected as part of routine program monitoring of an ongoing mentorship and quality improvement intervention in the study district. No identifying nurse or patient information was collected. As such, the IRB approved a waiver of informed consent under the routine use of program monitoring data. However, patients were explained the purpose of the data collection and could opt-out if desired.

Results

A total of 470 patient consultations were observed (range 53–69 per health center). The median duration of consultation was six minutes (Interquartile Range (IQR): 5–8 minutes) Approximately two-thirds (68.4%) were female and mean patient age was 35.3 years (standard deviation: 15.9 years). All nurses performing the consultations had the nursing education described above. The median practice experience of the nurses was four years (IQR:3–6 years) (Table 2).

Table 2 Sample characteristics and distribution of patient complaints
Full size table

Chief complaints

The majority of patients (75.3%) had one chief complaint, with 21.7% (n = 102) reporting two complaints, 1.9% (n = 9) reporting three complaints, and 1.1% (n = 5) with no complaint recorded. The most common chief complaints were female genitourinary issues (15.0%), cough (14.2%), epigastric abdominal pain (11.1%), and fever (9.7%). Of note, only 13.1% (n = 61) of patients were asked directly for a chief complaint by the nurse (Table 2).

Vital signs

At least one of six vital signs was recorded by any staff at 70.9% of visits, with no visits having all seven vital signs taken (Table 3). At least one vital sign was taken by a registration clerk prior to seeing the nurse in 56.0% of visits, and in 11.5% of patient visits, more than one provider collected vital signs. Weight was the most commonly taken vital sign (62.6%). Temperature was recorded for 27.5% of patients, and 16.2% had blood pressure recorded. No patient had height, respiratory rate or pulse recorded, nor body-mass index (BMI) calculated. One HC had no available registration clerk during any of the patient care sessions observed.

Table 3 Quality Indicators – Consultations with Vital Signs, Triage and Screenings by Outpatient Department nurse (N = 470) a
Full size table

Triage

Only seven (1.5%) patients were fully screened and triaged based on the standard triage protocol within IMAI. Five patients (1%) required higher level of care per the clinical mentor, of whom four were appropriately diagnosed and managed by the nurse (Table 3). All four patient emergencies identified by the nurse were referred to the district hospital; three with high fevers greater than 38°C and one who was convulsing. One patient, identified only by the clinical mentor, was in circulatory shock and with mentor intervention, resuscitation measures were employed.

Routine screening by history

Low rates were observed for routine screening for a wide range of conditions (Table 3). Only 10% (45) of patients were screened for TB (cough greater than 2 weeks). Among male patients, 10.1% were screened for genitourinary/STI symptoms. Twenty-two percent (66 of 300) of eligible female patients were screened for pregnancy or asked about their last menstrual period. Only 7.3% (32) of patients were asked about current sexual activity, and only 1.8% (7) screened for HIV status. Less than 1% of patients were screened for any of the following: anemia, alcohol use, tobacco use, use of mosquito nets at home or nutrition.

Diagnosis and treatment

In the 470 consultations observed, patients reported a total of 585 complaints. Of these, forty-nine categorized as “Other” (n = 43) and “Mental problem” (n = 6) were not captured by the disease categories included in the checklist and were excluded from analysis. Seven additional complaints (1.2%) had no corresponding diagnosis in the SIS codes, and 32 (5.5%) were missing any recorded diagnosis information, leaving 497 complaints with a diagnosis available for analysis. In addition to the 49 excluded complaints described above, 33 (5.6%) observations were missing information on correct treatment, leaving 503 complaints with available treatment data for comparison (Table 4).

Table 4 Correct nurse diagnosis and treatment of patient illnesses*
Full size table

Nurse diagnoses were in agreement with the clinical mentor 50.1% of the time (95% confidence interval 45.7%–54.5%), with the highest diagnosis agreement seen for epigastric pain (90.0%), skin problems or lumps (83.9%), mouth or throat problems (70.3%), and diarrhea (70.0%). Lowest rates of diagnosis agreement were seen for fever (8.2% correct), headache or neurological conditions (20.5%) and cough or difficulty breathing (31.3%). Agreement varied significantly between HCs, ranging from 35.6% to 60.0% (p = 0.03). There was no association between nurse experience in OPD and correct diagnosis.

The nurse treatment plan was in agreement with the clinical mentor less than half the time (44.9%, 95% confidence interval 40.6% - 49.3%). Complaints of fever (78.6%), headache (77.3%), and back or joint pain (71.4%) had the highest proportion of agreement. Lowest agreement was seen for epigastric pain (18.6%), genitourinary symptoms (26.7% for males, 35.0% for females), cough or difficulty breathing (33.8%), and diarrhea (36.7%). Treatment agreement varied from 32.1% to 58.9% across HCs (p = 0.04), but did not vary with duration of nurse experience in the OPD (Table 5).

Table 5 Factors associated with correct nurse diagnosis and treatment of patient illness
Full size table

Discussion

This is, to our knowledge, the first study assessing the general quality of care and clinical decision-making for adults and adolescents at rural OPDs in a resource-limited setting. Every aspect of routine care that was measured, including triage for life-threatening conditions, screening for chronic disease and modifiable lifestyle risk-factors, diagnosis of acute illness, and appropriate management and treatment, were found to be in need of strengthening and quality improvement. The study demonstrated infrequent collection and use of vital signs, infrequent screening for common illnesses (such as HIV, TB, STIs), and low rates of prevention counseling (such as malaria control with ITNs, and alcohol and tobacco use). With a lack of available standards to support quality, compounded by relatively little investment in general primary care for adults and adolescents, it is unsurprising that the results reflect a great need for improvement of front-line delivery.

Nurse diagnosis and treatment decisions were in agreement with the clinical mentor approximately half of the time or less. Agreement in diagnosis and treatment for the same conditions (e.g. fever, epigastric pain) often varied widely, suggesting large variations in individual practice. Poor decision-making could be influenced by a number of long-term structural and health system needs, including gaps in infrastructure, supplies and human resources [33–36]. However, other factors could be addressed in the shorter-term and could lead to an initial rapid improvement in the quality of adult and adolescent primary care.

As noted, aside from selected disease- or condition-specific protocols there are few decision-support tools available to help OPD providers simplify and standardize their decision-making and care. These tools, particularly for children under-5 and for HIV/AIDS, have been shown to improve quality and health worker performance [37–42]. While experiential learning is the primary foundation of practice improvement in Rwandan OPDs, the results show there was no significant increase in the quality of the decision making with increasing years of nursing experience.

The data also highlight needs with respect to nurse training and standardization of care. The low frequencies in our data across the measured areas of clinical practice suggest that the nurses in our sample need additional support in basic clinical skills, including history-taking, completion and interpretation of vital signs, and screening for chronic conditions, in addition to clinical reasoning and decision-making. While it was beyond the scope of this study to isolate whether these gaps originated in pre-service education, pre- or post-service training, and/or other structural factors such as infrastructure, lack of protocols or time constraints, clinical training of nurses in primary care has been shown to at least modestly improve quality of care for acute respiratory tract infections [43] and medication prescription [44]. The data demonstrate that nurses manage some illnesses better than others, and that performance differs significantly between sites, suggesting that standardization of care, as well as performance measurement coupled with mentoring and systems-based quality improvement are needed. In addressing issues of decision support, integrated training, and standardization of care, protocols such as IMAI could represent an important part of a toolkit for developing a common standard of OPD care for adults and adolescents, and improving quality of these services.

While implementation of care based on protocols such as IMAI is necessary, it is not sufficient to support the delivery of high quality of care. A critical component is ensuring durable knowledge and practice improvement through routine and sustainable mentorship of health care workers. Though there are differences in definition and structure of supervision programs [45], mentorship in primary care has been shown to improve and sustain improvements in nurse performance [46–48]. In Rwanda, robust efforts are now underway to implement a system of routine on-site mentorship to health center nurses in all clinical services [31].

There are a number of limitations to our study. First the clinical mentor was not blinded and therefore could have been influenced by observing nurses’ decisions in real-time, thus biasing the results and possibly over-estimating the level of agreement. Having an observer in the room may have also influenced the nurses to perform better or differently than they would in normal circumstances [49]. Given the low level of consistency found, it is unlikely that either potential bias considerably altered the results. Also, clinical decision-making could not be assessed for approximately 15% of patient complaints due to missing data. Approximately half (n = 43) of complaints missing from the analysis were classified as “Other” complaints that did not fall into the conditions selected for study a priori. The remainder of missing diagnosis or treatment information (approximately 7% of the total sample) was distributed across complaints and is unlikely to have had meaningful impact on the results. We have also presented clinical decision-making on diagnosis and therapeutics as a proxy for quality of care, but these are only two key components. We did not measure clinical outcomes, nor define current costs, infrastructure, and human resources which all impact or help define quality and which could be pursued in future research. Finally, IMAI has yet to be formally validated, particularly in low-HIV prevalence settings such as Rwanda, which represents an important next step in establishing a standard of general adult and adolescent primary care delivery.

Conclusions

This study describes the baseline quality of adult and adolescent primary care at Rwandan health center OPDs and demonstrates a clear need for improving the quality and consistency of care. Care based on IMAI or other integrated protocols, accompanied by a robust system of mentorship and supervision that focuses on individual performance as well as systems-level gaps, could be an important approach to simplifying and standardizing the delivery of care for common acute presentations for adults and adolescents in the ambulatory setting. Refining and validating these protocols is needed. Only these coordinated efforts can ensure that high quality and effective primary care is available for all adults and adolescents in resource limited settings.

Authors’ information

Ashwin Vasan and Manzi Anatole are joint-first authors to this manuscript.