Background

The worldwide prevalence of type 2 diabetes (T2DM) is on the rise, especially in low and middle-income countries (LMIC), reaching up to 14% of the adult population in Mexico [1], twice the current global prevalence [2]. T2DM poses considerable challenges to healthcare systems because it is among the leading causes for ambulatory care, hospital admissions and death [3]. In Mexico, quality of T2DM care has been suboptimal [4, 5], health outcomes poor, [6] and costs of T2DM care and its complications are on the rise [7].

Healthcare for patients with diabetes has both individual and health system goals. At the individual level, the goal is to achieve the best possible health outcomes. At the health system level, the goal is to provide accessible, high quality, effective, and efficient care. Information about progression of the disease and processes of care for patients with T2DM over time is required to assess whether goals are being accomplished.

The electronic health record (EHR) is a tool that can provide necessary input for dynamic improvement of systems for chronic care by a) supporting clinical decisions at the point of care; b) providing routine data to evaluate the quality of care (QC); and c) facilitating longitudinal systems research and efficient information sharing for dynamic care improvement [8, 9]. In publicly financed health care systems, EHR may also help improve transparency and accountability.

LMIC are beginning to use EHR [10]. Argentina, Costa Rica and Peru use EHR in circumscribed settings, such as individual clinics or hospitals, or for research purposes [11]. Mexico was the first Latin American country that introduced an EHR on a large-scale at the Mexican Institute of Social Security (IMSS). IMSS is a nation-wide institution providing social security and healthcare benefits to approximately 47% of the 112 million Mexican people. In 2003, IMSS began to introduce its EHR and evidence-based clinical guidelines to improve performance of its family medicine clinics [12]. The IMSS EHR became a key component of the institutional information system, which also includes administrative, member enrollment, pharmacy and other databases.

The IMSS EHR comprises information that could be used to improve the quality, efficiency and continuity of care; however, it has not been fully used for these purposes. Little is known about the quality of the data that are routinely captured and what relevant quality of care indicators (QCIs) can be constructed based on routine data.

T2DM is a complex and dynamic problem that requires a steady flow of information for healthcare providers to guide their clinical and managerial decision-making. The objectives of this project were: a) to develop QCIs for T2DM in the IMSS health system; b) to determine the feasibility of constructing QCIs using IMSS EHR data; and c) to evaluate the QC provided to IMSS patients with T2DM.

Methods

Design

We used data from four large family medicine clinics located in Mexico City with a catchment area of 585,536 people. Each clinic had between 15 and 30 physician offices, a laboratory, and a pharmacy.

IMSS’ EHR consists of several linked databases which include information on appointments; medical history; physical examination; clinical encounter notes; aspects of care and progress for patients with specific conditions (diabetes, antenatal care, hypertension); social work, occupational health, dietary, and preventive care services. Clinical encounter notes capture the reason for the visit, symptoms, signs, diagnoses (ICD-10 coded), orders for laboratory tests, electronic prescriptions, and disability and referral information. Data fields contain free-text, numeric, or pre-coded, menu-driven entries. Electronic prescriptions (by menu-driven generic product name) are linked to the IMSS essential list of medicines and the clinic’s pharmacy inventory. The laboratory database is linked to the menu-driven, coded catalog of IMSS laboratory exams and contains laboratory test orders. Laboratory examination results reside in different, unlinked databases.

Stage 1: Development of QC Indicators

To design and validate T2DM QCIs, we used the modified version of the RAND/UCLA Appropriateness Method, which comprises scientific evidence and expert opinion [13, 14].

First, we performed a systematic search and review of the literature on QCIs for T2DM using Marshall’s proposed method [15]. We searched the following databases: Medline, Ovid, Cochrane Library, National Guideline Clearinghouse, CMA Infobase: Clinical Practice Guidelines, TRIP database, Institute for Clinical System Improvement, ACP Guideline website, American Academy of Family Physicians, NHS Evidence - National Library of Guidelines, and IMSS-Clinical Guidelines [16]. The keywords were “type 2 diabetes mellitus”, “quality of care indicators”, “clinical guidelines,” and “family medicine” or “primary care services,” “diagnosis” and “treatment”.

The review focused on evidence-based care processes shown to increase the likelihood of achieving the best possible clinical outcomes, following the criteria proposed by Saslow et al. [17]. The review included indicators from the Healthcare Effectiveness Data and Information Set (HEDIS) [18], as well as those developed by the RAND Health Research and Institute for Clinical System Improvement (ICSI) teams. Indicators for four basic components of care for T2DM patients were defined a) timely screening for complications and comorbidity; b) non-pharmacological treatment; c) pharmacological treatment; and d) health outcomes.

Next, we assembled a panel of 8 experts. Panel members had experience in treating T2DM patients, in conducting clinical and health systems research, and in developing clinical guidelines. Each panelist received information about the study objectives, methodology, literature review and set of preliminary indicators. Panelists rated indicators on validity and feasibility according to Shekell’s criteria [19] by assigning a value from 1 to 9 (1 = definitely not valid or not feasible and 9 = definitely valid or feasible). An indicator was valid when it assessed a process of care for which there was enough scientific evidence to support the relationship between the specific process and a potential health benefit or the assumption that health care professionals who perform specific care aspects provide better quality of care than those who do not. An indicator was feasible when there was a high probability that the required information to construct the indicator would be available in a typical health record, or, if not, the lack of information would represent poor quality of care. Panelists used these criteria to rate each proposed indicator. An indicator was valid and feasible if its mean score was ≥ 7 on each domain. Panelists participated in two e-mail rounds of ranking and two face-to-face panel meetings.

Stage 2: Extraction of routine EHR data to construct the QCIs

To evaluate whether the indicators defined in Stage 1 could be constructed, we extracted data from a cohort of all continuously enrolled patients in the four study clinics. All patients with a diagnosis of T2DM according to the International Classification of Diseases, Tenth Revision (ICD-10) codes E111-E119, E140-E149, and E14X in one of three EHR diagnosis fields, who were older than 19 years of age and visited a family doctor at least once during 2009 were eligible.

For the cohort of patients with T2DM, we extracted EHR and laboratory data from the participating Family Medicine Clinics (FMCs) for calendar year 2009 and created an analytic dataset with the following variables:

  1. a.

    Variables from the EHR: general patient characteristics (age, sex, schooling, marital status, membership (subscriber/dependent and employment status); medical history (age at onset and duration of T2DM, history of hypertension, other cardiovascular diseases, or dyslipidemia; chronic complications [peripheral vascular disease, nephropathy, retinopathy, and peripheral neuropathy]; physical measurements (weight, height, and blood pressure); nutritional status (at the beginning and at the end of the year) as measured by body mass index (BMI) (underweight [BMI <18.5 kg/m2], normal weight [BMI 18.5 - 24.9 kg/m2], overweight [BMI 25.0 - 29.9 kg/m2], obese [BMI ≥30.0 kg/m2]); clinical care (number of visits to the family doctor, referral to an ophthalmologist and/or dietitian); prescriptions (insulin and oral hypoglycemic drugs [metformin, glibenclamide, acarbose, rosiglitazone or pioglitazone]; other drugs [acetylsalicylic acid, statins and angiotensin converting enzyme inhibitors or angiotensin-receptor blockers]); laboratory tests ordered.

  2. b.

    Variables from the laboratory database: dates and measured values of hemoglobin A1c (HbA1c), fasting blood glucose, and total cholesterol.

We queried the relational EHR and laboratory database tables of each clinic using standard queries in structured query language (SQL) to retrieve data from tables and create a new analytic database. We then assessed the data for completeness, consistency and accuracy. We predefined non-plausible lowest values for the following variables: blood pressure (systolic blood pressure <50 mmHg or >250 mmHg and diastolic blood pressure <40 mm Hg or >200 mmHg [20]); height (<130 cm or >250 cm), weight (<30 kg [21] or >200 kg), HbA1C (<3.0), fasting blood glucose (<37 mg/dl); and total cholesterol (<100 mg/dl). All non-plausible values were excluded from the analysis. We used the SAS statistical package (V9.2) to construct complex variables from the extracted data. We validated the extracted data by comparing a sample of 80 records in the EHR system with our final dataset. The comparisons were made reviewing data from individual EHR with the extracted data.

Stage 3: Evaluation of T2DM quality of care

We described QC indicators by family medicine clinic. We also characterized clinics by available health personnel (number of medical doctors, nutritionists, social workers, and nurses) and number of examination rooms.

The IMSS’ National Commissions of Research and Ethics reviewed and approved the study protocol.

Results

Stage 1: Development of QC Indicators

Table 1 shows the results of the indicator evaluation process. Scores ranged from 5.3 to 8.7. Based on the predefined threshold of 7, the panel discarded 5 and accepted 18 indicators; some indicators scored high in validity, but were not feasible and were then modified. For example, HbA1c test results were not available in all clinics. Therefore, the outcome indicator was modified from “T2DM patients who had HbA1c <7%” to: “T2DM patients who had HbA1c <7%, or fasting blood glucose ≤130 mg/dl in the last three measurements”. Similarly, the LDL-cholesterol indicator was replaced by an indicator based on total cholesterol, while the indicator of screening for microalbuminuria was discarded.

Table 1 Results from evaluation of candidate indicators
Full size table

Stage 2: Extraction of routine EHR data to construct QCIs

The final list (Table 1) shows that 14 indicators were programmable with data from the EHR and laboratory databases. We decided that four indicators could not be reliably constructed because information was only available in free-text fields with wide variability in entries. There was a high percentage of missing data for employment, schooling, marital status, and duration of diabetes. Also, weight, height and blood pressure had between 5% and 24% non-plausible values.

Stage 3: Evaluation of T2DM quality of care

Table 2 presents the main characteristics of the four participating FMCs. There were 204 family doctors working in the clinics; 67 registered nurses, 86 auxiliary nurses, 6 dietitians, and 36 social workers. Three clinics had a separate department to conduct medical education activities. In 2009, the clinics covered 585,536 members and dependents, with 437,417 >20 years old; approximately 174,266 patients visited the clinics at least once during 2009, of whom 25,130 (14% of patients) had a diagnosis of T2DM.

Table 2 Population and clinic characteristics
Full size table

Table 3 presents the characteristics of the T2DM patients. Most (59%) were women. The average age was 62.3 years. Nearly two in five patients had low education (39% illiterate or primary school education only); almost half were married or lived with their partner. Nearly 30% were housewives; 17% were employed and 10% were retired. Most (76%) patients were dependents of insured members.

Table 3 Type 2 diabetes patients’ general characteristics
Full size table

Table 4 lists the clinical conditions and health care characteristics of the included patients. More than 16% of those reporting had diabetes for over 15 years; however, disease duration was mostly missing (44%-77%). Hypertension was the most frequent comorbidity (60%). Approximately 31% of patients had a chronic complication, mostly peripheral vascular disease (14%) followed by diabetic nephropathy (11%). Only 15% had normal weight; 36% were overweight and 33% obese.

Table 4 Clinical conditions and health care characteristics
Full size table

Patients with T2DM were regular users of health care services; on average, they had 8.8 visits in 2009. Between 10% and 20% of patients did not receive any hypoglycemic prescription; most (77%) received 1 or 2 drugs. Metformin (37%) and glibenclamide (46%) were the most frequently prescribed medications.

Table 5 reports results for the quality of care indicators. There were wide variations among clinics. Two clinics had no HBA1c measurements, while the other two had limited availability. Indicators of timely detection of complications varied widely: between 5%-52% of patients received comprehensive foot evaluations; 5%-22% were referred to an ophthalmologist; and 46%-70% were screened for dyslipidemia. Only 4% received nutritional counseling. Across clinics, metformin was prescribed (63%) to a majority of overweight or obese patients; 57% of patients with hypertension were prescribed ACE inhibitors; 43% of patients with risk factors for cardiovascular events received prophylactic acetylsalicylic acid; and 48% of those with hypercholesterolemia received a statin.

Table 5 Indicators of quality of care
Full size table

Approximately 62% of patients had registered results of blood glucose and total cholesterol tests during 2009; among those without this information, 22% did not have a glucose and cholesterol test orders and 16% had the laboratory order registered, but these patients did not attend to the laboratory. After comparing the characteristics of patients with and without laboratory tests, we did not find statistically significant differences regarding sex, age, literacy or occupation, or co-morbidity. Differences were observed in relation to the number of medical visits to a family doctor, as a higher proportion of patients without laboratory exams had only 1 or 2 visits per year (1.2% overall and 21.5% among patients without laboratory results) (Data not included in the table).

Among patients with laboratory exams, few had achieved desired health outcomes: 23% had HbA1c <7% or fasting glucose ≤130 mg/dl in the last 3 measurements; 52% had total cholesterol <200 mg/dl in the last measurement; 11% had blood pressure <130/80 mmHg in the last 3 measurements; 13% of overweight or obese patients had lost ≥5% body weight in the last year; and only 1% of patients reached the combined therapy goals for blood glucose, cholesterol and blood pressure control.

Discussion

Our results suggest that it is feasible to a) develop T2DM QCIs applicable in the context of a middle-income country; b) measure the QCIs using routinely collected data from a widely implemented EHR system; and c) identify aspects of care for T2DM patients that are in need of improvement. Our work highlights the potential of routine EHR data to contribute to chronic care quality improvement, as well as the need for attention to the quality and completeness of electronic data. These findings are important in the global context of rapidly increasing prevalence of chronic conditions, particularly in low and middle-income countries, which require regular monitoring of processes and outcomes of care to ensure wise use of scarce resources. They also highlight the potential but less than optimal uses of electronic health information systems for routine monitoring of care processes and outcomes [22, 23].

A) QCIs in the context of a middle-income country health care system

Most published QCIs have been constructed for healthcare systems in high-income countries. Using the same QCIs across countries would facilitate international comparisons [24]. However, QCIs should be adjusted according to national variations in financing, structure, and provision of healthcare services, and differences in the characteristics of populations across countries. We originally intended to adopt the HEDIS indicators [18] used by more than 90% of America's health plans to measure performance of care and services. However, since HbA1c, LDL-cholesterol tests and nephropathy screening are not established standards at IMSS family medicine clinics, we needed to develop contextually suitable indicators. Our process for developing, measuring, validating, and reporting QC indicators may be informative in other developing countries.

B) measuring QCIs using routinely collected data from a widely implemented EHR system

The EHR did not contain all data elements needed to identify the denominator population of members with diabetes or to construct the identified QCIs. Clinical services at IMSS’ FMCs are almost paperless; all information is electronically stored. Our results indicate that there are shortcomings in the electronic information, possibly related to flaws in its design and the limited use of some EHR fields. While we generated rules and algorithms to identify plausible values and exclude records with missing values from the analysis, it is crucial for IMSS to improve its electronic record system for quality monitoring to become a routine process. This would include maintaining a reliable registry of patients with important chronic illnesses like diabetes.

We used a conservative approach to manage missing data, by not including cases with missing information in the analysis. This decision was supported by results from comparisons of socio-demographic characteristics of patients with and without laboratory results, which did not have statistically significant differences; we also took into account that the size of the sample was large, thus reducing the number of cases would not reduce statistical power. However, the validity and generalizability of our results are limited to the extent that there was bias in exclusion of members without visits during the observation year or without data needed to measure QCIs.

To progress to a robust eHealth system, it will be necessary to analyze in-depth the functionality and quality of the EHR, develop decision-support systems, and strengthen linkages between the EHR and other electronic applications, such as laboratory and pharmacy databases.

C) identifying aspects of diabetes care in need of improvement

Complex healthcare institutions require ongoing evaluation of their performance. In Mexico, evaluating QC based on EHR data is novel and promising, given the high cost, lengthy process, and lack of scale of more typical paper-based evaluations. The lack of complete data notwithstanding, we identified important aspects of care that require improvement. IMSS clinics have low rates of use of diagnostic tests, including HbA1c, microalbuminuria screening and LDL-cholesterol. IMSS’ current model of care relies heavily on family medicine doctors. As our results show, patients have limited access to dietitians, social workers, nurses and other specialists, hindering the integrated multi-specialty care patients with chronic conditions require to ensure the best possible outcomes [25, 26].

Our results are comparable to those from studies within Mexico and elsewhere. In a study conducted in Mexico, 66.3% of T2DM patients did not have good metabolic control, although those with access to healthcare had better health outcomes. The European Core Indicators Diabetes Project included 19 countries [27]; its results showed wide variability in QCIs among them: HbA1c testing ranged from 51% (Ireland) to 99% (France and the Netherlands); lipid measurements from 45% (Ireland) to 99% (the Netherlands); and eye examinations from 12% (Ireland) to 84% (The Netherlands). In India, only 13% of T2DM patients had at least one HbA1c measurement, 16.2% had an eye examination, 3.1% foot examination, 8.3% nutritional counseling, 32.1% serum cholesterol estimation and 17.5% were prescribed aspirin; [28]. Rates of HbA1c > 7% varied from 32% (Ireland) to 83% (Cyprus); total cholesterol > 5 mmol/l ranged from 14% (Ireland) to 68% (Cyprus); blood pressure > 140/90 mmHg, from 17% (France) to 46% (Sweden). [27] The QUALIDIAB network [29] in Argentina, Brazil, Chile, Colombia, Paraguay, and Uruguay reported that 43% of patients had blood glucose level of <7.7 mmol/L and the proportion of patients reaching combined treatment goals for blood glucose, cholesterol and blood pressure ranged from 1% in Mexico to almost 12% in the United States [30]. These variations should give rise to continued assessment and improvement efforts.

While this study included information from only four clinics, we found important differences between them; for example, two clinics did not have HbA1c data, pointing to heterogeneity in care processes even within the same health delivery system. Further research studies should examine ways to reduce these differences. Although the study clinics may not represent the QC that IMSS provides, results from desk-based medical chart reviews (with smaller sample sizes) are similar [4, 5].

Our results will serve as input to subsequent improvements of the IMSS EHR. The health system in which the EHR is used has a heavy clinical workload (20–25 patients per doctor per 6 hours of consultations, or 15–18 minutes per patient). In this context, the EHR can either hinder or facilitate care. The EHR was originally designed to gather routine clinical information to facilitate the daily work of family doctors and yield epidemiological information. Our study supports the notion that data from the EHR can be used to evaluate quality of care. Further, the EHR could potentially serve other purposes, such as a decision support system to guide clinicians through gathering relevant patient information, including risk factors and health status, and provide drug dosing guidelines and support tools. However the IMSS EHR does not yet include decision support systems. These should be developed as part of the advancement of the EHR. The experience of Mexico in the use of the EHR to evaluate quality of care can be of value for other low and middle-income countries that aim to improve their e-health capabilities.

Conclusions

This study demonstrated that it was feasible to evaluate QC for T2DM patients by using EHR data. Evaluating QC using EHR information can identify the performance of individual clinics or individual providers, guide future interventions aimed at improving QC and evaluate whether these interventions achieve their expected aims.

A well designed EHR, which should include rigorous data-entry and data quality monitoring would improve the capabilities to evaluate and monitor processes and outcomes of care, which in turn would provide useful feedback to authorities, healthcare personnel and patients. It also can serve to make comparisons among clinics, health care systems or countries.

From the clinicians’ perspective, EHR design and use should improve its functionality for the daily work, for example as a clinical decision-making tool, and as a reliable source to provide feedback to the providers.