Background

As in many Latin American countries, segmentation, inefficiencies and inequalities hinder the Mexican health system's potential [1]. Employees in the formal sector, either private or government, have access to social security and health insurance. The rest of the population has access to partially funded services offered by Ministry of Health facilities, with no access to other public health care services. Most of the population has access to private health care though primary care providers and medical offices in pharmacies, with more limited access to private hospitals. Overlaps in coverage given labour rotation create multiple duplications and require massive administration to function [1]. Additionally, given the enormous mobility of workers between formal and informal jobs, the bureaucracy involved in updating their status is a barrier to healthcare [2]. The configuration of the health system creates inequalities as well. By 2003, private health expenditures represented 58% of total health expenditures in the country [3].

To address some of these flaws, in 2004, the Mexican government introduced the System of Social Protection in Health to provide health insurance to the population without social security and reduce out-of-pocket expenditures [4]. Among other elements, the reform established a funding mechanism called Seguro Popular (SP), or Popular Health Insurance, designed to increase free access to a predetermined set of interventions and services to reduce catastrophic expenditures for the uninsured.

By 2012, the SP had enrolled 52 million people, or about 75% of the uninsured population [5]. This achievement was followed by increased use of healthcare services and reduced out-of-pocket expenditures [6]. In the second half of the 2000s and most of the 2010s, multiple studies evaluated the impact of the SP on several dimensions of health, healthcare utilization, and financial protection, as we describe in this study. SP became one of the better-documented examples of access to universal health coverage (UHC) in a middle-income country; nonetheless, as we show in this paper, many of these studies lacked robust methods to establish causality and identify SP's attributable effects. Following the health reform introduced by the 2018–2024 federal administration, the SP was abolished in 2019.

Evaluating the impact of the SP is not a simple task. The SP started in a pilot phase in five states (Aguascalientes, Campeche, Colima, Jalisco and Tabasco) in October 2001 but was formally implemented in 2004 when it legaly constituted the operating arm of the System for Social Protection in Health [7]. Furthermore, affiliation to SP was voluntary, ruling out the possibility of comparing affiliates with non-affiliates for the evaluation due to selection bias as people choosing to affiliate may be different compared to non-affiliates in observed and unobserved characteristics that could be associated with outcomes of interest. Studies addressing selection bias and other potential confounding factors used quasi-experimental approaches by exploiting the program's variability in time of initiation and level of penetration across states. Results from studies failing to address this potential bias result in biased estimations. Studies addressing the selection bias and other potential confounding factors used a quasi-experimental design with no experimental groups by exploiting the programme's variability in time and penetration by state.

The SP was successful in enrolling the uninsured population [8]. However, the picture arising from the body of evidence is less clear concerning the impact on the beneficiaries' access to healthcare and health status. Despite being one of the better-documented UHC programmes globally, there are no published systematic reviews on the SP's impact, which is an important task given some of the studies' methodological limitations. Our study's objective was to conduct a systematic review of published studies that assessed the impact of the SP on healthcare utilization, screening, access to treatment, financial protection and health outcomes. Our review focuses on papers that used econometric methods to explicitly address the potential selection biases, excluding those that failed to use methods to address this bias.

Methods

Information sources and search strategy

For this systematic review, we conducted a search strategy in MEDLINE (PubMed) and the Latin American and Caribbean Literature in Health Sciences (LILACS). The strategy combined keywords, Boolean operators and proximity operators initially designed for MEDLINE and adapted to LILACS. The search was conducted in July and August 2020 and updated in February 2022 to identify the peer-reviewed evaluations of the SP. In the second phase of the search, we reviewed the reference lists of the studies selected through the first strategy to identify additional references. We used the following keywords: seguro popular, health insurance, health reform, Mexican health insurance, and the MeSH term “Health Care Reform”. The final search algorithms were as follows: “((Seguro popular) OR (popular health insurance) OR (popular insurance) OR (public insurance) OR (health reform)) AND ((evaluation) OR (effect) OR (assessment) OR (impact)) AND (Mexico ((United States) OR (European)) Filters: from 2002 – 2022” for MEDLINE. The LILACS algorithm was as follows: “(Seguro popular OR popular insurance OR popular health insurance AND evaluation OR impact OR effect AND Mexico (yearcluster:[2002 TO 2020])”.

Selection process (inclusion and exclusion criteria)

We searched for original, quantitative studies published between 2002 and 2022. We excluded studies that were not peer-reviewed, such as the grey literature, comments, theses and protocols, and papers not written in English or Spanish. Given that SP's affiliation was voluntary, there was a potential selection bias in the analyses that simply compared enrolled and unenrolled households. Those enrolled may be systematically different from unenrolled households in characteristics associated with the outcomes of interest. For example, they may be sicker, or they may be more cautious and healthier. It is impossible to guess, and therefore measure, the magnitude and direction of this potential bias. Thus, evaluations that fail to address it may report biased results. We excluded papers that did not use methods to explicitly address this potential bias [4], such as instrumental variables, propensity score matching or regression discontinuity. We also included studies that used interrupted time series analyses with and without a comparison group.

Quality assessment of studies

We applied the Specialist Unit for Review Evidence (SURE) to evaluate the quality of each paper. SURE evaluates 11 key features of a paper: study design; study question; settings, locations, and dates; participant selection; characteristics of participants; appropriate outcome and exposure measures; sample size; adequate description of the methods and results; sponsorship/conflict of interest and limitations. For each item, we added specific aspects to evaluate the quality of the methods used for impact evaluation (Additional file 1). Two researchers evaluated each paper separately, and differences were discussed between authors until consensus was reached. Each item in SURE scores 1, so the maximum score is 11.

Data extraction

Using the Mendeley Desktop version 1.19.4 reference manager, we were able to detect and eliminate those articles that were duplicated. Data extraction was performed independently by two reviewers (R.G.C. and M.A.C.), who resolved inconsistencies through discussion. We used a data extraction form to collect information on study objectives, study design, study population and main findings.

Data analysis

For each study, we report the outcomes analysed as reported in the paper. We also report data sources, sample size, survey design (cross-sectional, longitudinal, or time series) and population studied (gender, age group, urban/rural, and socioeconomic status). We pay special attention to report the evaluation design, i.e., the analytical strategy used to address potential biases, identify causality, and the comparison group.

Most of the selected studies analysed several outcomes. Thus, we first identified six main categories of outcomes and ordered them following a continuum of care framework: healthcare utilization, screening, treatment, testing/monitoring, health outcomes (morbidity or mortality) and financial protection. We also identified subcategories of outcomes, such as specific diseases, age groups or financial protection indicators, within each of the six broad categories.

Because most of the papers analysed multiple outcomes, we graphically displayed the number of outcome subcategories and whether the results showed a positive or no impact. Finally, as studies could find positive impacts for some outcomes and not for others, we classified and displayed the number of papers that found “positive impact”, “no impact” or “mixed impact”. For data analysis, we used Excel® version 2009 for Microsoft®.

Results

The initial search identified 323 citations, from which we excluded 15 duplicates. We then reviewed titles and abstracts of the remaining 313 papers and excluded 240 studies that were not evaluations of the SP. We reviewed the full text of the remaining 73 articles to assess their impact evaluation methodology. After reading the full texts, we excluded 47 articles that failed to use methods to address potential self-selection biases and 26 that were not impact evaluations of the SP. We included 26 studies in the final selection, as shown in Fig. 1.

Fig. 1
figure 1

Analytical sample

Full size image

Table 1 summarizes the 26 papers included in the review by outcomes analysed, data sources, population studied, evaluation design and the comparison group. The most frequent data sources used were the National Health and Nutrition Surveys, cross-sectional surveys collected at different times (12 studies), the National Income and Expenditure Surveys, and cross-sectional surveys collected every 2 years (six studies). From the 26 selected papers, 22 relied on cross-sectional surveys, four on longitudinal data, and one used time series. To address the potential biases associated with the voluntary affiliation to the SP, the majority of the studies used propensity score matching (13 studies), instrumental variables (nine studies) or both methods combined (two studies). Concerning comparison groups, 17 used uninsured households, six used uninsured and social security, and three used social security beneficiaries.

Table 1 Description of the selected studies evaluating the impact of the Seguro Popular
Full size table

Additional file 1 shows the heat map with results from the SURE quality instrument. From the 11 items, the average score for the 26 papers selected is 10.2, ranging from 8 to 11. All studies comply with the following aspects: study question; settings, locations and dates; participant selection, characteristics of participants; appropriate outcome and exposure measures; sample size; adequate description of the methods and results. Twelve papers had lower scores in the description of the study design in the abstract, and nine papers failed to report sponsorship/conflicts of interest.

From the selected studies, nine evaluated healthcare utilization such as infant, perinatal and general healthcare visits. Three focused on screening for hypertension, diabetes and gynaecological or prostate cancer. Five assessed access to hypertension, diabetes or general care, and three papers focused on diabetes diagnosis. One study evaluated newborn and infant mortality and child development. Finally, 16 studies analysed the impact of the SP on financial protection, including out-of-pocket expenditures, impoverishing spending or catastrophic health expenses.

Table 2 shows the number of outcomes for each of the main categories and subcategories. The 26 studies analysed 137 outcomes: 65 on financial protection (47%), 30 on healthcare utilization (22%), 19 on treatment (14%), 14 on screening (10%), five for testing or monitoring (4%) and only four on morbidity or mortality (3%). From all outcomes, 66 were statistically significant, and 71 reported no significant impact.

Table 2 Impact of the Seguro Popular by broad category of outcomes and specific outcomes
Full size table

Figure 2 shows the number of outcomes reported in the studies by six broad categories. The figure shows the number of statistically significant and nonsignificant results for each category. From the 65 outcomes on financial protection, 55% were statistically significant. Concerning healthcare utilization, 40% of the 30 outcomes analysed were significant. For screening, from the 14 outcomes analyzed, 50% were significant. The authors found statistically significant effects in 21% of the 19 treatment outcomes, four of the five monitoring and testing outcomes, and three of four morbidity and mortality outcomes.

Fig. 2
figure 2

Impact of the Seguro Popular by outcome categories

Full size image

Finally, Additional file 2 shows the number of studies by type of result. From the 26 studies, eight found no significant effect in any of the outcomes analysed by those studies, six reported statistically significant effects in all outcomes, and 12 found mixed results (statistically significant effects for some outcomes but nonsignificant results for others). For more details on the specific results by outcome/study see Additional file 3.

Discussion

We conducted a systematic review of published studies that evaluated the impact of the SP in Mexico between 2007 and 2019. We found 26 papers that used rigorous methods for impact evaluation and 47 that evaluated the SP but were excluded because the methods used failed to address the potential selection bias, as affiliation to SP was voluntary. From the 26 papers included, more than half of the studies estimated the impact of the SP on financial protection. One paper assessed the effect of the SP on morbidity and mortality for neonates, and the rest of the papers estimated the impact on healthcare utilization, screening, treatment and testing/monitoring. The 26 studies analysed 137 outcomes: 65 on financial protection (47%), 30 on healthcare utilization (22%), 19 on treatment (14%), 14 on screening (10%), five for testing or monitoring (4%), and only four on morbidity or mortality (3%).

We found a wide variation in the populations studied by age, sex and urbanicity. About 85% of the papers relied on cross-sectional surveys. Most studies used either propensity score matching or instrumental variables to address self-selection bias, as affiliation to SP is voluntary. Sixty-five percent of the studies compared individuals enrolled in the SP with unenrolled groups, and 23% compared with unenrolled and social security affiliates. The rest used the population insured by social security institutions.

The papers analysed a great variety of outcomes, and their results showed substantial heterogeneity in the impact. Overall, 55% of the outcomes analysed showed a positive impact, and 45% no impact. We found evidence of a positive impact on three morbidity/mortality outcomes, 14 healthcare utilization outcomes, and 18 for financial protection. Concerning screening and access to treatment, the studies documented positive results on eight outcomes. Our results show that 50% of the papers reported mixed results, 31% positive impacts, and 19% no effects in all outcomes analysed.

We found more significant impacts on financial protection than on treatment, screening or utilization. This result is consistent with a key feature of the SP as a funding mechanism to increase access to a predetermined set of services. However, much weaker effects on healthcare utilization and health outcomes have been explained previously by the SP operation as a decentralized programme by state governments, with heterogeneity in the capacity to provide care [9]. We cannot rule out the possibility of flawed study designs as well; however, our results on the papers' quality suggest this is not the most likely cause. For the uninsured population that used private services before the SP, evidence on the use of private services suggests that it reduced their financial burden [10]. Moreover, there was no significant expansion of the supply of services following the establishment of the SP [10, 11]. Therefore, it is possible that the SP did not expand the access to services to more people, but rather it became an affordable alternative for people who already had access to healthcare through private services.

To our knowledge, this is the first systematic review of rigorous impact evaluations of the Mexican SP. Other systematic reviews of similar financial protection mechanisms in other countries [12] have reported impacts on financial protection, health services utilization and health outcomes. Some studies found positive results with reduced out-of-pocket expenses, but negative results for catastrophic expenses [13]. In contrast, an Indian study found no protective effect on out-of-pocket spending but observed reduced mortality in insured compared to uninsured households [14]. A study that evaluated the impact of similar programmes in Asia and Africa found increased use of medical services and financial protection by reducing out-of-pocket expenses, but little evidence of a positive effect on the quality of care, and inconclusive effects regarding the empowerment of communities [12]. However, in contrast to our study, many of these reviews included qualitative or mixed-methods studies, and none considered rigorous impact evaluation methods as an inclusion criterion. Nonetheless, the reviews also found mixed results such as reductions in out-of-pocket expenses but adverse effects on catastrophic expenses [13]. Others showed no impact on financial protection but found reductions in mortality [14].

By 2018, affiliation to the SP reached 53.5 million people, 44.7% of the total population [5]. Affiliation, however, was not followed by an equivalent increase in supply [11]. A recent paper on this topic showed that despite this massive increase in coverage, 46.4% of affiliates to the SP reported using private services in 2018. These private options include medical offices in pharmacies that are not regulated; the quality of services has not been documented and is associated with higher out-of-pocket expenses [10].

One limitation of the study is that we could not summarize the findings using meta-analyses because of the considerable heterogeneity in outcomes and how they were measured (see Additional file 3).

Conclusions

In conclusion, the published papers that estimated the effects of the SP show considerable impact heterogeneity. Despite including only rigorous evaluations in our review, researchers were able to find statistically significant effects in 55% of the financial protection outcomes and 40% for healthcare utilization outcomes evaluated. Furthermore, except for one paper, no evidence exists on the impact of the SP on health outcomes or mortality.