Background

Robust processes of health economic evaluation generally need to jointly consider principles of coverage and comparability given an underlying objective of optimising budget constrained community net benefit that underlies societal decision-making across related research, reimbursement and regulatory decisions (Eckermann 2017; Eckermann and Pekarsky 2014; Eckermann and Coelli 2013; Eckermann et al. 2010; O’Brien 1996). For robust evaluation of health promotion strategies in community settings, satisfying adequate coverage in practice needs to allow for population level impacts in communities and in particular the potential of community network and multiplier effects across community networks (Shiell and Hawe 1996; Shiell et al. 2008; Hawe and Ghali 2008; Hawe et al. 2009; Eckermann et al. 2014; Eckermann and McCaffrey 2017). The public health and health economic potential of health promotion strategies, programs and policies with community ownership of effective strategies are substantial at a population level given community network effects can naturally broaden both the population exposed and their duration of effects without increasing intervention costs. The potential of network effects is particularly large where community ownership of health promotion extends to integrating strategies into practice and policies of community institutions. For example, integration of the Stephanie Alexander Kitchen Garden Program kitchen and garden lessons into school curricula (Eckermann et al. 2014; Yeatman et al. 2014) or the potential for policy integration of early childhood movement guidelines into early childhood care settings (Okely et al. 2017).

This potential for community network impacts and broadening of effects of health promotion strategies and policies with community ownership and integration do not arise with individual focussed strategies (Eckermann and McCaffrey 2017) and point to potential for significant underfunding of health promotion research and strategies in optimising budget constrained health system outcomes (Shiell and Jackson 2018). However, bottom-up methods to map the network and multiplier impacts over time of health promotion strategies with community ownership across populations are not yet well developed. This has frustrated calls for adequate coverage of impacts (effects and downstream cost savings) in evaluating health promotion policies and strategies in practice and associated research pathways and adequate investment in identifying and unlocking the health system potential of successful health promotion policies and strategies (Shiell and Jackson 2018). While health promotion evaluation guidelines generally recognise the key need to triangulate across evidence and levels of analysis in community settings (Zaza et al. 2005), adequate coverage of population level effects allowing for network multiplier impacts of alternative health promotion policies and strategies is required. In practice this would be best achieved if robust methods were available for top-down population level monitoring across countries of the effects of alternative strategies and policies over time. This is particularly the case given inadequacy of current bottom-up methods to capture community network effects (as considered at length in discussion).

Nevertheless, comparability as well as coverage principles need to be satisfied for robust evaluation of health promotion as in any public health evaluation (Shiell and Jackson 2018; Eckermann 2017; Eckermann and McCaffrey 2017). While efficiency analysis with binary data such as meeting or not meeting guidelines are feasible in adequately allowing for population coverage of effects, such analysis faces distinct methods challenges to satisfy comparability and in particular to enable robust consistent analysis with alternative framing of binary effects. This paper’s aim is to address these challenges.

To illustrate these challenges in relative performance efficiency measures and their solution, we consider a public health international comparison in meeting integrated movement guidelines, the International Study of Childhood Obesity, Lifestyle and the Environment (ISCOLE) (Roman-Viñas et al. 2016; Katzmarzyk et al. 2013). The ISCOLE compared school-aged children (age 9–11 years) across 12 socio-economically and culturally diverse countries from all world regions in meeting or not meeting 24-h integrated movement guidelines (IMGs), as shown in Table 1. IMGs jointly consider physical activity, sedentary behaviour and sleep behaviours. Children aged 9–11 years, such as those in the ISCOLE study, meet IMGs if across 24-h periods they undertake ≥1 h of moderate to vigorous intensity physical activity, have ≤2 h of sedentary recreational screen-time and achieve 9 to 11 h of sleep at night.

Table 1 ISCOLE international comparison of 9–11 year-old children meeting or not meeting integrated movement guidelines
Full size table

The extent to which populations of children jointly meet or do not meet IMGs has been shown to be an important population level public health issue given these joint behaviours largely determine long-term habits, lifestyle and risk of non-communicable diseases. International evidence syntheses have shown child populations meeting IMGs have better body composition, cardiorespiratory and musculoskeletal fitness, cardiovascular and metabolic health, academic achievement and cognition, mental health and quality of life, emotional regulation, and pro-social behaviours (Dumuid et al. 2018a, b; Katzmarzyk et al. 2013; Okely et al. 2017; Roman-Viñas et al. 2016; Saunders et al. 2016; Tremblay et al. 2016; Walsh et al. 2018; WHO 2017, 2019). Hence, whether children meet or do not meet IMGs and their joint consideration of sleep, physical activity and sedentary behaviour are key to child health and development and trajectory of their lifestyle and health behaviours more generally.

Nevertheless, when undertaking efficiency comparisons relative to best practice based on binary effects data such as meeting or not meeting IMGs in Table 1, a distinct challenge for robust consistent analysis arises. While binary data such as that from ISCOLE are readily available and make efficiency comparisons relative to observed best practice feasible, they can always be framed as in Table 1 in two alternate ways; meeting or not meeting a guideline, as with any such binary data (success or failure; survival or mortality etc.).

For robust efficiency measures relative to best practice, such binary effects efficiency measures need to be consistent with alternate framing of the binary effect to avoid selection bias or paradoxical results. In other words, to satisfy comparability efficiency measures relative to best practice analysed with the same binary data framed as meeting a guideline or not meeting a guideline, the results should be the same. In this paper, we illustrate the distinct lack of consistency with alternate conventional presentation of data as the probability of meeting or not meeting a guideline. More importantly, we provide methods for an odds solution in methods that is shown in results to: (i) consistently estimates efficiency scores relative to best practice with alternate framing of a binary effect; (ii) provide an intuitive interpretation of relative scores as odds ratios, and; (iii) extend to consistent indirect comparisons, evidence translation and standardisation. Finally, we return to discuss methods required to enable robust bottom-up evaluation of IMGs to aid triangulation, while robust top-down methods for public health efficiency measures are identified, before concluding.

The problem – Inconsistency of relative risk efficiency measures with binary data

For a binary effect such as meeting or not meeting IMGs in Table 1, efficiency measures relative to best practice across countries can be analysed either in terms of trying to maximise population proportion meeting guidelines or minimise the population proportion not meeting guidelines. However, output orientated efficiency analysis comparing the proportion meeting guidelines and input orientated efficiency analysis comparing the proportion not meeting guidelines lead to very different results, as shown in Table 2.

Table 2 Relative risk efficiency measures for countries meeting or not meeting 24-h integrated movement behaviour guidelines (IMGs)
Full size table

Differences between relative performance in efficiency analysis with alternate framing of the same binary data such as meeting (output orientation) or not meeting guidelines (input orientation) arise because:

  1. (i)

    Relative performance measures represent relative risk as the population probability of meeting guidelines (Pi for any country i) compared to the best practice peer (P*) from an output perspective or as the population probability of not meeting guidelines (1-Pi) relative to the best practice peer (1-P*) from an input perspective and;

  2. (ii)

    relative risk is not a symmetric metric; that is, relative risk of meeting guidelines (Pi/P*) is not the reciprocal of not meeting guidelines (Pi/P* ≠ (1-P*)/(1-Pi)).

Hence, inconsistency of efficiency measures with alternative framing of the same binary effect with relative risk arises due to inherent lack of symmetry of relative risk compared to the best practice peer that relative efficiency measures represent. This inconsistency makes interpretation of relative risk efficiency measures for any binary effect highly problematic given that resulting differences arising with alternate framing of the same binary effect either:

  1. (i)

    introduces selection bias where only one framing is considered (for example, meeting or not meeting guidelines alone); or

  2. (ii)

    creates a paradox where both framings are presented making efficiency measures uninterpretable.

To satisfy comparability and avoid either a selection bias or paradox that arises with relative risk efficiency measures requires a method enabling consistent estimation of alternative output (meeting guideline) and input (not meeting guideline) presentation.

Methods

Odds and odds ratio solution

To overcome inconsistency of efficiency measures with binary effects arising with the non-symmetric relative risk measures, use of the symmetric odds and odds ratio metrics are suggested, as with consistent evidence synthesis and evidence translation methods (Eckermann et al. 2009, 2011). The key advantage of consistency arises with odds because odds represent probability divided by 1-probability and hence are symmetric for binary measures such as meeting or not meeting a guideline.

That is, undertaking efficiency analysis for meeting or not meeting the IMGs with odds comparison, performance relative to best practice for each country represents:

  1. (i)

    from an output perspective, odds of meeting guidelines (Pi/(1-Pi) for any country i relative to the best practice peer with highest odds of meeting guideline (P*/(1-P*)) and hence relative efficiency score Pi/(1-Pi)/P*/(1-P*); and

  2. (ii)

    from an input perspective best practice peer with lowest odds of not meeting guideline ((1-P*)/P*) relative to odds of not meeting guidelines ((1-Pi)/Pi for any country i) and hence relative efficiency score ((1-P*)/P*)/((1-Pi)/Pi) = Pi/(1-Pi)/P*/(1-P*).

This novel odds ratio method is now shown in results to overcome the inconsistency of relative risk based efficiency relative to best practice highlighted in Table 1, and can be extended to solutions in any indirect comparison and translation of evidence of relative treatment effects to any jurisdiction of interest, employing results from previously developed odds ratio methods (Eckermann et al. 2009, 2011). These methods are illustrated in empirical analysis comparing school-aged children (age 9–11 years) in meeting or not meeting 24-h IMGs across all countries from the ISCOLE (Roman-Viñas et al. 2016, Katzmarzyk et al. 2013). The 12 countries in the ISCOLE study were selected to be representative of socio-economically and culturally diverse countries in every continent and world region.

Results

Consistent efficiency measures

The odds ratio solution for consistent binary effect efficiency measures with comparison of their odds in methods is illustrated in Table 3 for the international comparison of ISCOLE country’s odds for meeting or not meeting guidelines relative to observed best practice. That is, efficiency measures comparing 9–11 year old children from each of the ISCOLE study countries relative to Australia in maximising odds of meeting IMGs (Australia’s odds 0.175) or minimising odds of not meeting IMGs (Australia’s odds 5.711).

Table 3 Efficiency comparison of IMGs odds in school age children
Full size table

Comparing the population odds of meeting or not meeting guidelines results in the same efficiency measure; a consistent estimate of input and output orientated efficiency relative to best practice. The resulting consistent efficiency measures respectively represent the odds ratios of meeting the guidelines relative to best practice in maximising (output orientated) and the reciprocal of the odds of not meeting the guidelines relative to best practice in minimising (input orientated), as shown in Table 3 for the ISCOLE study. Importantly, this implies efficiency measures are consistent with alternate output and input framing in comparing odds of meeting and not meeting guidelines, overcoming the problematic inconsistency arising with relative risk.

The consistent efficiency result arises with use of odds as there are two inversions that cancel each other with alternate framing for odds of binary effects given:

  1. (1)

    Odds are symmetric, that is, odds of not meeting guidelines are the reciprocal of the odds meeting guidelines, hence:

    • odds of not meeting guidelines

    • = p(not meeting guideline) / (p(meeting guidelines)

    • = 1/(p(meeting guidelines)/p(not meeting guidelines)

    • = 1/odds meeting guidelines;

  2. (2)

    Relative efficiency measures are relative to the best practice country maximum in meeting guidelines and minimum in not meeting guidelines, the same best practice country with reciprocal odds from (1).

For example, consider efficiency for Brazil. From Table 3 Brazil has odds of meeting guidelines of 0.035 relative to Australian best practice odds of meeting guidelines of 0.175. Alternatively, Australia’s odds of not meeting guidelines of 5.711 = 1/0.175 can be compared relative to Brazil’s odds of not meeting guidelines of 28.412 = 1/0.035. Hence, while efficiency relative to best practice for Brazil with odds of meeting guidelines = 0.035/0.175 = 0.201, this is also the efficiency relative to best practice for Brazil with odds of not meeting guidelines (1/0.175)/ (1/0.035) = 5.711/28.412 = 0.201.

For each country, inversion arising from the symmetry of odds between meeting and not meeting guidelines (or any binary effect) cancel with the inversion of relative performance relative to best practice for the same comparator in maximizing versus minimizing in meeting vs not meeting guidelines. In general, comparing odds of meeting or not meeting guidelines leads to the same consistent relative performance efficiency estimate, and in each case are meaningfully interpreted for each country as the odds ratio of meeting or not meeting guidelines relative to best practice.

We now show that the consistent comparison of relative performance with symmetric odds metrics also extend advantages over relative risk to consistent indirect comparison and consistent translation of evidence of treatment effect.

Extension of consistent efficiency odds ratio result to indirect comparison

Given efficiency measures such as those in Table 3 are each measured relative to the same best practice peer, and consistently represent the odds ratio for meeting guidelines with alternate framing, they also enable consistent indirect comparison of relative performance as odds ratios (Eckermann et al. 2009).

To illustrate, consider indirect comparison between the UK and Canada from the odds ratio relative efficiency measures in Table 3 relative to the best practice common comparator, Australia, in each case. From the results in Table 3, efficiency measures are consistently estimated with alternate framing of meeting or not meeting a guideline as 0.735 for the UK and 0.930 for Canada. In each case, this represents the odds ratio for meeting or not meeting guidelines relative to Australia as the common best practice comparator. Now, undertaking indirect comparison, the odds ratio for the UK relative to Canada in meeting or not meeting integrated movement guidelines is consistently calculated from the efficiency measures for the UK relative to Australia and Canada relative to Australia as 0.735/0.930 = 0.790. This is the same calculation as if we had calculated the odds ratio for meeting guidelines directly from odds between the UK and Canada (0.12867/0.16279 = 0.790) in Table 2. That is, 0.12867/0.17509)/ (0.16279/0.17509) = 0.12867/0.16279 = 0.790, with the odds for common best practice comparator Australia effectively cancelling in indirect comparison of odds ratios for the UK and Canada in the same way regardless of the framing of effect as meeting or not meeting guidelines.

Hence, odds ratios for meeting guidelines and relative to the same best practice peer also enable consistent relative performance comparison between any countries in any indirect comparison. Furthermore, note that given the symmetry of odds and odds ratios with alternate framing, the odds ratio of not meeting guidelines for the UK relative to Canada is simply the inverse, 1/(odds ratio of meeting guidelines), and hence 1/0.790 = 1.265. In general, the odds ratio comparison in Table 2 enables consistent and interpretable comparison of relative performance in meeting or not guidelines with any direct or indirect comparison.

In contrast, employing relative risk as in Table 2, lack of symmetry and inconsistency between meeting or not meeting IMGs leads to very different results in indirect comparisons for relative performance such as the UK relative to Canada with alternate framing. Framed as meeting or not meeting guidelines relative performance for the UK relative to Canada with indirect comparison is estimated as 0.765/0.940 = 0.813 and 0.960/0.990 = 0.970, respectively. This inconsistency reflects the same lack of consistency for each of the UK and Canada relative to Australia as a best practice common comparator. Similarly, for any bilateral country comparison while odds ratios (Table 3) will always be consistent, relative risk comparison (Table 2) is expected to be inconsistent. This advantage is again enabled by the symmetry of odds and odds ratios relative to a common best practice comparator and extend previous indirect comparison advantages of odd ratios for treatment effects (Eckermann et al. 2009).

Extension of odds result to evidence translation

Eckermann et al. (2011) show that for any binary effect, translating evidence of a treatment effect to settings with different baseline risk with relative risk leads to inconsistent estimation of absolute effect differences with alternative framing of the same binary outcome, while use of odds ratios is consistent. We now extend this result to demonstrate that odds relative performance comparison also naturally extends to consistently translating relative treatment effect evidence.

To illustrate, consider for example a hypothetical treatment effect found in school-aged populations where integration of IMGs into early childhood centres had an odds ratio (OR) for meeting guidelines of say 1.25 (or 2), while equivalently the OR for not meeting guidelines would be 0.8 = 1/1.25 (or 0.5). These respective ORs can then be applied to prior odds of meeting or not meeting guidelines in any country where that policy had yet to be adopted, to estimate the posterior odds and absolute population risk impact expected with such policy adoption.

For example, Canada with a prior probability of meeting guidelines of 0.14 (Table 1) and associated odds of meeting guidelines of 0.14/0.86 = 0.1628 or not meeting guidelines of 0.86/0.14 = 6.143 (Table 3). Now, applying Bayes Theorem, the odds with changed practice (posterior odds) in Canada is estimated in translating this evidence by simply applying the treatment OR to the odds of current practice.

In terms of meeting guidelines, applying an OR of 1.25 (or 2) with integration of guidelines into early childhood centres to current (prior) odds of 0.1628 in Canada leads to an expected (posterior) odds of 0.2035 (or 0.3256). This posterior odds can then be simply converted back to probabilities to estimate the incremental effect as a difference in absolute probabilities or risk, noting that in converting the relationship between probability (P) and odds (O), P=O/(1 + O) is equivalent to O=P/(1-P). Converting the expected odds back to a probability (P=O/(1 + O)) leads to the expected rate of meeting guidelines with this policy of 0.2035/1.2035 = 0.169 in Canada in translating evidence with an OR of 1.25 (or 0.3256/1.3256 = 0.246 if the OR were 2). Hence, integrating IMGs into early childhood centres with associated OR of meeting guidelines of 1.25 (or 2) applied to prior odds of 0.16279 in Canada leads to a 2.9% (or 10.6%) absolute increase in the proportion of the population meeting guidelines, from 0.140 to 0.169 (or 0.246).

Alternatively, undertaking evidence translation in terms of not meeting guidelines, applying an OR of 0.8 (or 0.5) with integration of guidelines into early childhood centres to current (prior) odds of not meeting guidelines of 6.143 leads to an expected (posterior) odds of 0.8 × 6.143 = 4.914 (or 0.5 × 6.143 = 3.0714). Converting this back to a probability (P=O/(1 + O)) leads to an expected rate of meeting guidelines with this policy of 4.914/5.914 = 0.831 (or 3.0714/4.0714 = 0.754) and hence a 2.9% (or 10.6%) absolute population decrease in not meeting guidelines relative to base rate of 0.860.

Importantly this demonstrates that translating evidence with treatment effect expressed as an OR applied to the odds of meeting or not meeting guidelines results in the same consistent estimation of absolute differences. More generally, Eckermann, Coory and Willan (2011) show that while relative risk does not allow consistent evidence translation with alternative framing of binary effects precisely for the conditions where evidence translation is required (there is a treatment effect and baseline risk differs from that in the trial setting), odds ratios always do.

Consistent absolute effect estimates from evidence translation naturally can also be used to consistently estimate incremental costs associated with incremental effects, noting that appropriate cost consideration allows for downstream cost implications of effects as well as direct costs. Hence, odds methods are required for consistent evidence translation of binary effects in estimating absolute incremental effects, cost or cost effectiveness analysis. This in turn is key to undertaking robust clinical or health economic analysis of the expected net clinical benefit and net benefit of a strategy with binary effects data in translating evidence of effects to any given jurisdiction of interest.

Consequently, for any binary effect data, use of odds and odds ratios has been shown to provide a robust method that solves the lack of consistency of alternative framing with risk or probability presentation and resulting relative risk efficiency measures relative to best practice, but also more generally extends interpretability of relative performance measures to indirect comparison or translating a treatment effect to any jurisdiction of interest. Establishing odds as the appropriate metric to enable consistent analysis in comparison of binary effect data such as meeting or not meeting guidelines is a primary consideration in satisfying comparability for robust decision making.

Discussion

Top-down and bottom-up methods for robust IMGs evaluation

A robust method has been identified in this paper for consistently comparing jurisdiction’s top-down population level efficiency comparisons relative to best practice for any binary effect measure, such as populations meeting or not meeting public health guidelines.

Developing such robust top-down methods is particularly important to evaluating health promotion and disease prevention policies and strategies, such as those aimed at improving 24-h movement behaviours in community settings given current limitations of conventional bottom-up methods. Trial-based bottom-up evaluation methods for estimating effects and downstream costs currently do not allow for network and multiplier effects across populations over time and hence community population level potential of such strategies (Shiell and Hawe 1996; Shiell et al. 2008; Moore et al. 2006; Hawe and Ghali 2008; Hawe et al. 2009; Eckermann et al. 2014; Eckermann and McCaffrey 2017). Therefore, community level potential for expanded scope and duration of population effects with community ownership are not covered or captured by conventional bottom-up methods, but are naturally covered with monitoring of health promotion strategies with top-down population level analysis over time. Such top down analysis can be used to monitor population, community level, effects of alternative health promotion strategies over time as well as associated downstream costs savings in assessing their cost effectiveness.

Consequently, robust top down analysis is key to enabling adequate coverage in identifying effective and cost effective health promotion strategies and addressing chronic underfunding of health promotion strategies internationally (Shiell and Jackson 2018; Eckermann 2017). Such underfunding in turn reflects a lack of profit motivated vested interests in supporting research for health promotion strategies such as IMGs which are either not patentable or have fewer apparent commercialization opportunities (Pekarsky 2012, 2015; Eckermann and Pekarsky 2014).

In evaluation of health promotion strategies in community settings, the robust population-level efficiency comparisons relative to best practice can also consistently, as shown in evidence translation results, combine with bottom-up evaluation methods as they emerge (Zaza et al. 2005). Monitoring compliance with 24-h movement guidelines in early childhood using guidelines launched in Canada and Australia in 2017 (Okely et al. 2017; Tremblay et al. 2016) and by WHO in 2019 (WHO 2019) make these methods particularly timely. These measures are important to identifying and evaluating relative efficiency of health promotion policies and strategies across health systems internationally. They can be used to highlight best relative performance in practice, the policies and practices associated with best practice, and the potential for, and factors associated with, improving policy and practice across suboptimal jurisdictions.

Ideally this would coincide with developing robust methods for bottom-up evaluation of health promotion approaches, environmental and whole of government strategies, and policies and practices aimed at improving compliance with guidelines. These strategies include, but are not limited to: the integration of guidelines into educational settings; building networks of walking and bike paths; creating public gardens, green spaces and public places; whole of government policy support across education, transport, parks and recreation, sport, and urban planning; and the running of culturally appropriate activities and programs (WHO 2017; Okely et al. 2017; Tremblay et al. 2016).

Nevertheless, distinct challenges remain for robust bottom-up evaluations that have yet to be jointly addressed. Those challenges relate to the interrelated time constrained nature of 24-h movement behaviours across physical activity, sedentary behaviour and sleep domains but also multiple domains of health effects that integrated movement behaviours map to and across community populations with multiple alternative health promotion and environmental strategies, modalities and pathways. Robust evaluation methods need to jointly consider physical activity, sedentary and sleep movement behaviours constrained by the 24 h day in identifying optimal population behaviours with compositional data analysis methods (Dumuid et al. 2018a, b) in mapping to multiple health effects. Such mapping to multiple health effects then need to combine with best evaluation methods for multiple strategy and joint multiple domain of effect cost effectiveness and efficiency comparisons (Eckermann et al. 2008; Eckermann and Willan 2011; Eckermann and Coelli; McCaffrey et al. 2015).

Finally, in evaluating expected population level effects and cost implications across alternative health promotion and environmental strategies, methods should allow for community ownership, network and multiplier effects (Eckermann et al. 2014; Shiell et al. 2008; Hawe et al. 2009). In this respect, it should be noted that environmental and whole of government strategies (e.g. improving community walkability and access to public recreation spaces) are not restricted to single target populations in their scope of expected community-level effects. For example, the same types of environmental and community accessible policies aimed at supporting children meeting movement guidelines have been shown to have health and lifestyle benefits for aged populations in being physically, mentally and socially active (WHO 2002, 2007; Kalache 2013; Eckermann 2017 chapter 12; Dyer et al. 2018; Eckermann et al. 2019). Such analyses point to whole of government environmental strategies for active communities providing potential multiplier benefits in all community populations across the lifecycle. Importantly, this implies that environmental strategies to promote movement behaviours for children must ensure community ownership to have high potential multiplier effects. This potential is both in terms of population scope with network effects across populations of all ages in the community as well as duration of health effects with improving trajectories of integrated movement behaviours, lifestyle and habits.

Developing and applying robust methods for bottom-up as well as the robust top-down evaluation of population level effects with consistent odds method for evaluating the movement guidelines identified in this paper are important to supporting appropriate research and investment in health promotion strategies such as those for IMGs. However, the consistent odds ratio method for monitoring populations IMG compliance and comparing countries strategies in practice identified in this paper is key to robustly enabling adequate coverage in assessing effectiveness or cost effectiveness of strategies and policies both currently and in combining with such robust bottom-up methods as they develop.

More generally, the finding of consistent odds ratio relative performance efficiency comparison for binary effects in this paper have also been illustrated to extend to avoiding inconsistency with evidence translation and indirect comparison that arise with relative risk comparison.

Conclusion

We have shown that the use of odds overcomes inconsistency between input and output population level efficiency measures relative to best practice that arises for binary effects such as meeting or not meeting guidelines with use of probability or proportions. This finding for any binary effect results from the symmetrical nature of odds and odds ratios and their inversion with relative efficiency measures in maximising output (e.g. meeting guidelines) or minimising input (e.g. not meeting guidelines) relative to the same best practice comparator. Noting that binary effects can always be expressed in one of two ways (e.g. meeting or not meeting guidelines; success or failure; survival or mortality) this finding is key to consistent comparisons of population evidence with any binary effect data.

Furthermore, by employing odds for comparison, efficiency measures have been shown to represent odds ratios for meeting guidelines relative to best practice and as such are also highly interpretable. These findings overcome relative risk inconsistencies and fallacies with alternative framing of binary effects and extend distinct advantages previously shown for odds ratios over relative risk in indirect comparison and evidence translation (Eckermann et al. 2009, Eckermann et al. 2011; Eckermann 2017:65-75).

The odds method identified for consistent efficiency measures with binary effects has been highlighted as particularly important to aid top-down population evaluation in practice of health promotion and disease prevention programs, policies and strategies such as those for meeting IMGs. This is particularly the case given inadequacy of funding for such non-patentable or not easily commercialisable public health strategies (Pekarsky 2012, 2015; Eckermann and Pekarsky 2014; Eckermann 2017; Shiell and Jackson 2018) and bottom-up methods which currently fail to cover the network and multiplier potential over time of such community-level strategies where they have community ownership (Shiell and Hawe 1996; Shiell et al. 2008; Hawe and Ghali 2008; Hawe et al. 2009; Eckermann et al. 2014; Eckermann and McCaffrey 2017).