Introduction

Large animal studies are needed to test therapeutic efficacy of novel therapies for myocardial infarction (MI). These studies usually serve as crucial checkpoints before advancing to first-in-man trials1,2. Considerable heterogeneity exists in the models currently used to study MI and its aftermath3. The choice for a specific model may influence the manifestation and progression of the disease and subsequently the potential effect of an intervention or technique under evaluation3.

There is a strong demand for optimal selection of models that represent the human disease best, since many promising therapeutics have shown beneficial effects in the preclinical phases, but fail in the clinical setting4. Methodological flaws and inadequate modeling of human MI have been proposed as partial explanations of this ‘translational failure’, leading to false positive study outcomes and the risk of overestimation of effect size in preclinical studies5,6,7,8. However, systematic analysis of methodological decisions on effect size are currently not available.

Standardization of these animal models could be of value for comparison of individual studies to historical data, for which groups in the field of cardioprotection have put forth the first efforts2,9. Above all, the translational value of large animal MI models can be significantly increased by assessing the effect of model design on primary outcome. This enables selection of animal models that most resemble the clinical course of MI.

In the evolving era of big data and abundant publication, the research community is calling on meta-research to systematically evaluate and improve research methods10,11. Systematic reviews and meta-analyses of preclinical data not only provide us with comprehensive overviews and bias assessments, but can also provide us with additional insights that explain heterogeneity within a specific disease and intervention12. In this perspective, combining and examining control groups of preclinical studies for a certain disease model, provides us with a comprehensive data-heavy method of studying the progression of the disease model and quantify the potential influence of certain variables on standard disease outcomes. The aim of the current study was to systematically explore the natural course of artificially induced MI in different large animal models and ultimately determine which biological and methodological factors act as effect modifiers, influencing disease course, primary endpoints and mortality within studies. Through meta-analysis, we report that functional and anatomical endpoints following MI in large animal models vary significantly due to variability in study design (Fig. 1).

Figure 1
figure 1

Graphical schematic representation of differences in outcomes after MI through study design - A model using male pigs in combination with an LAD-occlusion will differ significantly from a female dog model with LCX-occlusion.

Full size image

Methods

Data from control animals from two previous meta-analyses on large animal MI models were collected7,8. In both datasets infarct size as a ratio of the area at risk (IS/AAR), infarct size as a ratio of the left ventricle (IS/LV) and left ventricular ejection fraction (EF) were extracted and added in the current data if not present. Results on peri- and post-procedural mortality were extracted for all studies; peri-procedural meaning within the timeframe of the infarct-induction process (‘death during surgical procedures’) and post-procedural meaning after the disease-inducing procedure. Any procedural complications not due to the induction of the MI itself were not counted as ‘natural’ mortality. Due to evolving methodology over time in MI modeling with regards to the treatment of ventricular fibrillation (VF) during induction of MI, we recorded whether animals were treated for VF (either by medication or defibrillation) or were excluded immediately and performed a predefined sensitivity analysis to exclude a potential effect of this specific early exclusion. A thorough explanation of methodology on mortality data extraction can be found in the Supplementary section.

Pre-defined variables of interest were species, sex, age, weight, use of immunosuppression, co-medication commonly used in clinical care of MI (defined as being treated for the whole study after MI with one or more of the following compounds: aspirin, clopidogrel, ticagrelor, prasugrel, beta-blockers, ACE-inhibitors, angiotensin receptor blockers, and/or statins), follow-up duration post-MI, study quality and multiple characteristics of the infarct induction procedure: open-thorax vs closed percutaneous procedure, permanent vs temporary occlusion, ischemia duration (if transient occlusion) and type of vessel occluded (left coronary artery (LCA) vs left circumflex artery (LCX) vs left anterior descending (LAD) vs right coronary artery (RCA)). The variable method of quantification (for infarct size measurement or ejection fraction) was added in the phase of revisions to correct for any effect of these methods on regular outcomes. Study quality was assessed using the ‘Collaborative Approach to Meta Analysis and Review of Animal Data from Experimental Studies’ (CAMARADES) quality checklist13. As data on age and weight was scarcely available in the included studies, we conducted a post-hoc sensitivity analysis between minipigs and regular pigs within our species variable, as these substantially differ with regards to total body weight and age. All studies that did not report the strain of pigs were pooled in an ‘unreported’ variable.

Any variable not already assessed prior to this project, was added to the database.

All data has been inserted in the CAMARADES database (available on request)14.

Statistical analysis

Random effects meta-analysis with restricted maximum likelihood was performed due to anticipated heterogeneity between the different models of disease. Forest plots were generated to visualize these. Correlation analysis was performed between IS/AAR and EF using linear regression. Correlation between the actual therapeutic effect of included studies and the values of control animals was also assessed using linear regression.

Univariable meta-regression was performed for the association of chosen variables with our outcomes of interest. All variables were subsequently tested in multivariable meta-regression with the outcomes IS/AAR, IS/LV, EF and mortality, to correct for potential effect modification and to distinguish independent effects. Of note, multivariable meta-regression is especially suitable in the setting of animal studies, as all variables of interest are deliberately kept constant in preclinical study setup as opposed to the clinical setting. This minimalizes the risk of a potential ecological bias in our analysis. A post-hoc Wald test was used for categorical univariable meta-regression with more than two categories and in multivariable meta-regression to determine the individual association per individual variable. We used raw means for the outcomes IS/AAR, IS/LV and EF, since percentages are not expected to differ between the different groups under study.

For mortality outcomes, we used ratios (number of dead animals per total animals) and weighed each measurement on the inversed square root of the total number of animals for each comparison in our meta-regression analysis (1/√n). In the case of two measurements in the same procedural setting (for example mentioning of mortality peri-procedural both before and after randomization), the appropriate ratio was determined by multiplying both proportions (1 − ptotal = (1 − p1) * (1 − p2)). The weighing factor for such a value is the square root of the total number of animals in both measurements, divided by two (1/√((n1 + n2)/2)). A p-value of <0.05 was considered significant.

For our prediction modeling strategy, we used multivariable meta-regression to predict the outcomes for commonly used large animal models. We modeled both a pig and a dog model of temporary 60-minute occlusion with follow-up of 1 day, 1 week and 1 month. We did the same for a chronic occlusion pig model, using the same follow-up times. Statistical analyses were performed using R version 3.1.215 with the additional metafor package16 and Stata version 11 (Statacorp, LP, Texas, USA). The R script is available in the Supplementary section.

Results

A total of 246 studies were used, yielding 1500, 1221 and 775 animals for the outcomes IS/AAR, IS/LV and EF, respectively (Table 1). For the mortality analyses, data of 3622 animals and 1555 animals was studied for peri-procedural and post-procedural mortality, respectively (Table 1).

Table 1 Number of included animals per dataset.
Full size table

Meta-analysis

From our datasets, an average IS/AAR of 49.8% (95%CI 46.0–53.6%), IS/LV of 18.1% (95%CI 16.5–19.7%) and EF of 39.3% (95%CI 37.4–41.2%) were observed after MI induction and follow-up (Table 1). These outcomes are also visualized in Forest Plots (Supplementary Figures 1–3). The average peri-procedural mortality and post-procedural mortality were 16.7% (95% CI 14.7–18.7%) and 5.2% (95% CI 3.6–6.9%) respectively (Table 1).

Correlation between assessed outcomes

To study the effect of the initial damage and therapeutic effect of any drug given, we used linear regression to compare the absolute therapeutic effect within a study and the mean outcome that was assessed in the control animals. For IS/AAR (p = 0.0001), IS/LV (p = 0.001) and EF (p = 0.05) there was a significant correlation between the effect of the study therapeutic and the initial damage in the control animal (Supplementary Figure 4A–C). This indicates that greater cardiac damage leads to a larger effect of the investigated therapeutic.

There was no correlation observed between IS/AAR and EF if measured in the same study (p = 0.66, Supplementary Figure 5).

Meta-regression on standard outcomes: IS/AAR

Univariable meta-regression revealed multiple correlating variables with all our outcomes (Tables 2–4), which were subsequently used for multivariable analyses.

Multivariable meta-regression (p < 0.001) for the outcome IS/AAR revealed that infarct size was smaller when dogs were used (−22% compared to pigs (p < 0.001)). Male animals were also at risk for larger infarcts (−6% for both sexes compared to male (p = 0.040) and −10% for unreported sexes compared to male (p = 0.010)). The use of co-medication was protective (−18% if used (p = 0.01)) and infarct size was also dependent on the type of occlusion (−36% if temporary compared to permanent occlusion (p < 0.001) and −46% if temporary compared to unknown occlusion (p < 0.001)). Occlusion of the LAD leads to larger IS/AAR (+8% compared to LCX (p = 0.008)) and follow-up duration (−0.3% per hour of follow-up (p = 0.011)) also independently influenced the outcome (Table 2). For all temporary occlusion studies (n = 145), ischemia duration was an additional significant influencing variable in multivariable meta-regression of IS/AAR (+0.09%/min ischemia (p = 0.001)) (Table 2).

Table 2 Univariable and multivariable meta-regression for outcome IS/AAR.
Full size table

Meta-regression on standard outcomes: IS/LV

Multivariable meta-regression analysis (p < 0.001) for IS/LV showed that occluded vessel (p = 0.030) and method of quantification (p = 0.01) are of significant influence. For quantification methodology, MRI and planimetry underestimated infarct size compared to tissue staining and other modalities. Furthermore, study quality was associated with a 1.3% difference in IS/LV per quality point (Table 3). The variables species and sex showed only a trend (p = 0.05 and p = 0.08 respectively) for an association, with the same directions for categories as in the IS/AAR analyses (Table 3).

Table 3 Univariable and multivariable meta-regression for outcome IS/LV.
Full size table

Meta-regression on standard outcomes: EF

Multivariable meta-regression for EF showed an effect of species, with a 9% difference in EF for pigs compared to sheep (p = 0.007). Sex also independently influenced EF after MI (−6% for female animals compared to male animals (p = 0.03), −7% for female animals compared to studies using both sexes (p = 0.028) and −6% for female animals compared to animals with unreported sex (p = 0.009)) (Table 4). The choice of occluded vessel also showed an independent effect (+24.2 for only an LAD occlusion (p = 0.014), +26.2 for only an LCX occlusion (p = 0.009) compared to a combined LAD/LCX occlusion); again, this should be interpreted with caution, as the number of comparisons using either the LAD or LCX in the same study is limited (Table 4). Method of quantification had an independent effect on ejection fraction outcome, with echocardiography estimating higher ejection fraction values compared to LV Angio (+6.7%, p = 0.030), SPECT (+7.7%, p = 0.034) and PV loop (+12.6%, p = 0.006).

Table 4 Univariable and multivariable meta-regression for outcome ejection fraction.
Full size table

Mortality

Univariable meta-regression showed no variables investigated correlated with peri-procedural mortality (Table 5). The subsequent multivariable meta-regression was non-significant (p = 0.33), so we did not proceed with further post-hoc testing. A sensitivity analysis, which omitted all animals that were excluded for VF with no attempt to treat the arrhythmia, was performed and also did not show any correlation with the variables of interest, both uni- and multivariably.

Table 5 Univariable and multivariable meta-regression for peri- and post-procedural mortality.
Full size table

Univariable meta-regression for post-procedural mortality showed a correlation with follow-up time, with the addition of 0.002% per hour extra follow-up (p = 0.03). Multivariably, meta-regression was not significant and no further post-hoc analyses were done (p = 0.41). The selected multivariable regression with the addition of ischemia duration (which only applies to temporary occlusion models) was significant (p = 0.047) and post-hoc testing revealed follow-up time as the only significant independent predictor of post-procedural mortality (0.007%/hour, p = 0.001) in studies using a temporary occlusion model.

Post-hoc sensitivity analyses for different pig strains

In a post-hoc analysis we compared the strains ‘regular pigs’, ‘minipigs’ and ‘unknown strains’ within the species group of pigs. For IS/AAR, univariable metaregression was significant (p = 0.025), due to a difference between the unknown group and regular pigs (+13.2% if unknown, p = 0.018) and the unknown group and minipigs (+25.7 if unknown, p = 0.048). There was no univariable difference for pigs vs minipigs (p = 0.32). These significant differences disappeared in multivariable meta-regression (p = 0.15 for the strain variable) (n = 2, 24 and 15 respectively for studies using minipigs, pigs and unknown strains). For the outcomes IS/LV (n = 14, 24 and 14) and ejection fraction (n = 28, 40 and 19) there was no significant difference between minipigs, pigs and unknown strains in both univariable (p = 0.84 and p = 0.91 respectively) and multivariable (p = 0.17 and p = 0.49 respectively) analysis.

Prediction of outcomes in common large animal MI models

Predicted outcomes for predefined commonly used models were generated (Table 6), showing clear differences for all outcomes between these models.

Table 6 Predicted regular outcomes for common large animal MI models.
Full size table

Discussion

The current meta-analysis systematically reveals the effect of methodological choices on primary outcome measurements in large animal MI studies. The identification of the effect of the different experimental setups is of great importance, since it will guide adequate expectations of study results and mortality for specific models. It also enables more adequate and precise power calculations, which are essential when designing any preclinical study. We can now quantify biological differentiating variables for certain effect sizes and more accurately determine if these models resemble human disease. We confirmed some known biological variability within these models, showed effects that can be translated to the human situation and were able to quantify these variations in a meta-analytic manner.

The different disease manifestation across species has been demonstrated in the past17, with canine hearts forming more collaterals than hearts of other species, which we broke down to a ~20% smaller IS/AAR for dog models compared to pig models and lower EF in sheep compared to pigs. Despite the ~20% smaller IS/AAR in dogs, EF does not differ between dogs and pigs. Supplemental Figure 5 shows the absence of a correlation between IS/AAR and EF in our dataset, which could possibly be explained by confounding factors, including follow-up time and occluded vessel. Since infarct size decreases over time18 (Table 6), cardiac remodeling affects ejection fraction by progressive dilatation and systolic dysfunction. Moreover, occlusion of the LAD results in a loss of apical contractility, leading to a more severe decrease of ejection fraction compared to LCx occlusion19,20. As data on age and weight was scarse, we conducted an extra sensitivity analysis to compare minipigs and pigs, as these are considered the same species, but differ substantially in terms of age and weight. In this analysis, we could not find a difference between the two, arguing that MI models in both strains behave similar in terms of regular outcomes.

Conserved within evolution, females seem to show smaller infarcts compared to mixed groups and male counterparts, which is in line with the clinical data on sex influence on infarct size, favoring female subjects21,22,23. Of note, using female animals might leave researchers with a smaller therapeutic window in infarct size, potentially explaining the reduced efficacy of anti-inflammatory compounds in female animals8. Interestingly, pump function seems more decreased in female animals, once again arguing that the different sexes do not respond completely similar to cardiac damage and subsequent remodeling. In this perspective, it is crucial for translational success to include both sexes in future preclinical research, as is also called for by the NIH in preclinical projects24. Furthermore, there seem to be fewer studies using (only) female animals in our dataset, potentially explaining why not all comparisons to the female group always reached statistical significance.

The observed difference of ~9% in IS/AAR for different occlusion sites (LAD vs LCX) is in line with the observed greater loss of regional systolic function for anterior wall ischemia25, but was not observed for the outcome EF.

The observed reduction of infarct size and EF when increasing follow-up time is interesting both from a methodological and biological point of view. Smaller infarct sizes might imply smaller therapeutic windows for new interventions, while a larger reduction in EF might account for the inverse reasoning. Biologically this might be explained by infarct resorption and subsequent myocardial wall thinning, resulting in a decreased attribution of the thinned scar to the total myocardial mass18. Other explanations could be possible regeneration and post-infarction hypertrophy. Hibernating myocardium is not likely to explain this phenomenon, as function should increase after myocardial stunning and hibernation in the early stages of an infarct. Regardless of the cause, a longer follow-up could lead to more clinically relevant conclusions and might need more power to show any true differences. Incorporation of regular MI co-medication also seems to reduce the IS/AAR, which might be crucial for clinically relevant translation to the same poly-pharmaceutical human situation. A limitation of this variable is of course bundling of all studies using one or more of these compounds for power-reasons; we are not able to pinpoint these effects to one single compound. However, for many of these compounds there is either preclinical or even clinical evidence that they can influence infarct size and other outcomes after MI and therefore might be relevant to take into account for future experimental study design26,27.

The addition of quantification method seems crucial to be able to correct for the effects that these have on our different outcomes. For ejection fraction especially, it is known that echocardiography can overestimate cardiac function compared to for example MRI28. Our analyses confirm this, making it crucial to correct for these methods in multivariable analyses.

Interestingly, the composition of the dataset blurred the effect of multiple variables in the univariable analysis for IS/AAR, while our multivariable approach revealed certain effects that would otherwise have gone unnoticed.

No difference in outcome was observed for open versus closed modeling of MI, in contrast to what has been demonstrated in a recent study29. This might mean that conclusions from certain experiments can only be applied to the same setting; in this case an ischemia-reperfusion pig model. On the other hand, it might imply that meta-analyses cannot reveal all subtle differences within MI animal models. The same holds true for other variables in our dataset, like immunosuppression, which theoretically could have an effect on all our outcomes of interest.

Furthermore, we are limited by the data we were able to extract. In preclinical meta-analyses, many ‘known unknowns’ are present; variables that one would like to analyze, but are not reported as such. This is resembled by the unexplained heterogeneity (for multivariable IS/AAR analysis R2 = ~46% and I2 = ~96%) that, in the case of our MI analyses, is potentially influenced by for example the specific occlusion site of the vessel (which directly influences the area at risk), weight of the animal or experience of the surgeon. However, with the variables available, we were able to explain a significant part of the observed heterogeneity, with model-specific differences and human-like variability for sex and co-medication.

Modeling mortality in our study did not result in many explanatory variables, so we can only give summary estimates based on the meta-analysis of the total data. On average, peri-procedural mortality was ~17%, while post-procedural mortality was condensed in a ~5% mortality rate. These are important numbers for future study designs, as power analyses are crucial in the success chance of (pre)clinical trials and the reduction of both type I and type II errors. It is possible that these numbers are incomplete or biased in the current analysis, due to incomplete reporting in prior studies. This might be less of a problem for future similar analyses as the reporting of animal studies will hopefully improve substantially due to the ARRIVE guidelines, EDA application and journals demanding complete reporting30,31.

The need for meta-research on methods and reproducibility has been solicited for by the community and is a crucial process in the self-cleansing ability of research10. This paper untangled a part of the variation observed and generates realistic starting points for well-needed large animal MI models, hopefully adding further insight in disease understanding, accurate modeling of MI and more translational success for new cardiac interventions.

Being able to explain and predict a ‘point of departure’ in large animal MI models will prove useful to tailor experiments and make reasonable power calculations based on the expected damage, mortality and potential experimental effect (example in Fig. 1). This will potentially result in more accurately powered studies, more definite answers to research questions and less waste of animal lives and research money32. Many clinically relevant patient characteristics seem to be of influence in the preclinical setting, and will potentially influence any outcome if not taken into account. In the current era of translational science, all researchers need to take this variation into account when designing new studies to optimize the chance of success of any large animal experiment.