Background

Globally, there is a growing interest in using the research evidence in public health policy-making [1, 2]. Public health systems are diverse and complex, and health policymakers face many challenges in developing and implementing policies and programs that are required to be efficient [1, 3]. The use of scientific evidence is considered to be an effective approach in the decision-making process [3,4,5]. Due to the lack of sufficient resources, evidence-based decision-making (EBDM) is regarded as a way to optimize costs and prevent wastes [6]. At the same time, the direct consequence of ignoring evidence is poorer health for the community [7].

Evidence suggests that health systems often fail to exploit research evidence properly, leading to inefficiencies, death or reduced quality of citizens’ lives, and a decline in productivity [8]. Decision-making in the health sector without sufficient attention to evidence may lead to a lack of effectiveness, efficiency, and fairness in health systems [9]. Instead, the advantages of EBDM include adopting cost-effective interventions, making optimal use of limited resources, increasing customer satisfaction, minimizing harm to individuals and society, achieving better health outcomes for individuals and society [10, 11], as well as increasing the effectiveness and efficiency of public health programs [12].

Using the evidence in health systems’ policymaking is a considerable challenging issue that many developed and developing countries are facing nowadays. This is particularly important in the latter, where their health systems are in a rapid transition [13]. For instance, although in 2012, a study in European Union countries showed that health policymakers rarely had necessary structures, processes, and tools to exploit research evidence in the policy cycle [14], the condition can be worse among the developing and the underdeveloped ones. For example, evidence-based policy-making in developing countries like those located in the Middle East can have more significant impacts [15, 16]. In such countries resources are generally scarce, so the policymakers' awareness of research evidence becomes more important [17]. In general, low and middle-income countries have fewer resources to deal with health issues and need quality evidence for efficient use of these resources [7].

Since the use of EBDM is fraught with the dilemma of most pressing needs and having the least capacity for implementation especially in developing countries [16], efforts have been made to create more comprehensive guidelines for EBDM in healthcare settings, in recent years [18]. Stakeholders are significantly interested in supporting evidence-based projects that can quickly prioritize funding allocated to health sectors to ensure the effective use of their financial resources [19,20,21]. However, it is unlikely that the implementation of EBDM in Health System Management (HSM) will follow the evidence-based medicine model [10, 22]. On the other hand, the capacity of organizations to facilitate evidence utilization is complex and not well understood [22], and the EBDM process is not usually institutionalized within the organizational processes [10]. A study in 2005 found that few organizations support the use of research evidence in health-related decisions, globally [23]. Weis et al. (2012) also reported there is insufficient information on EBDM in local health sectors [12]. In general, it can be emphasized that relatively few organizations hold themselves accountable for using research evidence in developing health policies [24]. To the best of our knowledge, there isn`t any comprehensive global and practical model developed for EBDM in health systems/organizations management. Accordingly, the present study aimed to develop a comprehensive framework for EBDM in health system management. It can shed the light on policymakers to access a detailed practical model and enable them to apply the model in actual conditions.

Methods

This was a Best Fit Framework (BFF) synthesis conducted in 2020 to develop a comprehensive framework for EBDM in HSM. Such a framework synthesis is achieved as a combination of the relevant framework, theory, or conceptual models and particularly is applied for developing a priori framework based on deductive reasoning [25]. The BFF approach is appropriate to create conceptual models to describe or express the decisions and behaviors of individuals and groups in a particular domain. This is distinct from other methods of evidence synthesis because it employs a systematic approach to create an initial framework for synthesis based on existing frameworks, models, or theories [25] for identifying and adapting theories systematically with the rapid synthesis of evidence [25, 26]. The initial framework can be derived from a relatively well-known model in the target field, or be formed by the integration of several existing models. The initial framework is then reduced to its key components that have shaped its concepts [25]. Indeed, the initial framework considers as the basis and it can be rebuilt, extended, or reduced based on its dimensions [26]. New concepts also emerge based on the researchers' interpretation of the evidence and ongoing comparisons of these concepts across studies [25]. This approach of synthesis possesses both positivist and interpretative perspectives; it provides the simultaneous use of the well-known strengths of both framework and evidence synthesis [27].

In order to achieve this aim the following methodological steps were conducted as follows:

Searching and selection of studies

In this step, we aimed to look for the relevant models and frameworks related to evidence-based decision-making in health systems management. The main research question was “what is the best framework for EBDM in health systems?” after defining the research question, the researchers searched for published studies on EBDM in HSM in different scientific databases with relevant keywords and constraints as inclusion and exclusion criteria from 01.01.2000 to 12.31.2020 (Table 1).

Table 1 Search strategy for the review
Full size table

Inclusion and exclusion criteria

Inclusion criteria were determined as the studies that identify the components or develop a model or framework of EBDM in health organization in the form of original or review articles or dissertations, which were published in English and had a full text. The studies like book reviews, opinion articles, and commentaries that lacked a specific framework for conducting our review were excluded. During the search phase of the study, we attempted as much as possible to access studies that were not included in the search process or gray literature by reviewing the references lists of the retrieved studies or by contacting the authors of the articles or experts and querying them, as well as manually searching the related sites (Fig. 1).

Fig. 1
figure 1

The PRISMA flowchart for selection of the studies in scoping review

Full size image

Quality appraisal

The quality of the obtained studies was investigated using three tools for assessing the quality of various types of studies considering types and methods of the final include studies in systematic review. These tools were including Critical Appraisal Skills Program (CASP) for assessing the quality of qualitative researches [28], Scale for the Assessment of Narrative Review Articles (SANRA) [29], and The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers [30] (Table 3-Appendix).

Data extraction

After searching the studies from all databases and removing duplicates, the studies were independently reviewed and screened by two members (TS and MRAM) of the research team in three phases by the title, abstract, and then the full text of the articles. At each stage of the study, the final decision to enter the study to the next stage was based on agreement and, in case of disagreement, the opinion of the third person from the research team was asked (PB). Mendeley reference manager software was used to systematically search and screen relevant studies. The data from the included studies were extracted based on the study questions and accordingly, a form of the studies’ profile including the author's name, publication year, country, study title, type of study, and its conditions were prepared in Microsoft Excel software (Table 4-Appendix).

Synthesis and the conceptual model

In this step, a thematic analysis approach was applied to extract and analyze the data. For this purpose, first, the texts of the selected studies were read several times, and the initial qualitative codes or thematic concepts, according to the determined keywords and based on the research question, were found and labeled. Then these initial thematic codes were reviewed to achieve the final codes and they were integrated and categorized to achieve the final main themes and sub-themes, eventually. The main and the sub-themes are representative of the main and sub-steps of EBDM. At the last stage of the synthesis, the thematic analysis was finalized with 8 main themes and all the main and the sub-themes were tabulated (Table 5-Appendix).

Creation of a new conceptual framework

For BFF synthesis in the present study, we compared the existing models and tried to find a model that fits the best. Three related models that appeared to be relatively well-suited to the purpose of this study to provide a complete, comprehensive, and practical EBDM model in HSM were found. According to the BFF instruction in Carroll et al. (2013) study [25], we decided to use all three models as the basis for the best fit because any of those models were not complete enough and we could give no one an advantage over others. Consequently, the initial model or the BFF basis was formed and the related thematic codes were classified according to the category of this basis as the main themes/steps of EBDM in HSM (Table 5-Appendix). Then, the additional founded thematic codes were added and incorporated to this basis as the other main steps and the sub-steps of the EBDM in HSM according to the research team and some details in the form of sub-steps were added by the research team to complete the synthesized framework. Eventually, a comprehensive practical framework consisting of 10 main steps and 47 sub-steps was created with the potentiality of applying and implementing EDBM in HSM that we categorized them into four main phases (Table 6-Appendix).

Testing the synthesis: comparison with the a priori models, dissonance and sensitivity

In order to assess the differences between the priori framework and the new conceptual framework, the authors tried to ask some experts’ opinions about the validity of the synthesized results. The group of experts has included eight specialists in the field of health system management or health policy-making. These experts have been chosen considering their previous research or experience in evidence-based decision/policy making performance/management (Table 2). This panel lasted in two three-hour sessions. The finalized themes and sub-themes (Table 6-Appendix) and the new generated framework (Fig. 3) were provided to them before each session so that they could think and then in each meeting they discussed them. Finally, all the synthesized themes and sub-themes resulted were reviewed and confirmed by the experts.

Table 2 The demographic characteristic of the experts that participated in the synthesis
Full size table

Ethical considerations

To prevent bias, two individuals carried out all stages of the study such as screening, data extraction, and data analysis. The overall research project related to this manuscript was approved by the medical ethics conceal of the research deputy of Shiraz University of Medical Sciences with approval number IR.SUMS.REC.1396–01-07–14184, too.

Results

The initial search across six electronic databases and the Cochrane library yielded 4613 studies. After removing duplicates, 2416 studies were assessed based on their titles. According to the abstract screening of the 1066 studies that remained after removing the irrelevant titles, 291 studies were selected and were entered into the full-text screening phase. Due to full-text screening of the studies, 17 final studies were selected for extracting the components and steps of EBDM in HSM (Fig. 1). The features of these studies were summarized in Table 4-Appendix (see supplementary data). Furthermore, according to the quality appraisal of the included studies, the majority of them had an acceptable level of quality. These results have been shown in Table 3-Appendix.

Results of the thematic analysis of the evidence (Table 5-Appendix) along with the concepts proposed and added by the research team according to the focus-group discussion of the experts were shown in Table 6-Appendix. Accordingly, the main steps and related sub-steps of the EBDM process in HSM were defined and categorized.

After collecting, synthesizing, and categorizing thematic concepts, incorporating them with the initial models, and adding the additional main steps and sub-steps to the basic models, the final synthesized framework as a best-fit framework for EBDM in HSM was developed in the form of four general phases of inquiring, inspecting, implementing, and integrating and 10 main steps (Fig. 2). For better illustration, this framework with all the main steps and 47 sub-steps has been shown in Fig. 3, completely.

Fig. 2
figure 2

The final synthesized framework of evidence-based decision-making in health system management

Full size image
Fig. 3
figure 3

The main steps and sub-steps of the framework of EBDM in health system management

Full size image

Discussion

In the present study, a comprehensive framework for EBDM in HSM was developed. This model has different distinguishing characteristics than the formers. First of all, this is a comprehensive practical model that combined the strengths and the crucial components of the limited number of previous models; second, the model includes more details and complementary steps and sub-steps for full implementation of EBDM in health organizations and finally, the model is benefitted from a cyclic nature that has a priority than the linear models. Concerning the differences between the present framework and other previous models in this field, it must be said that most of the previous models related to EBDM were presented in the scope of medicine (that they were excluded from our SR according to the study objectives and exclusion criteria). A significant number of those models were proposed for the scope of public health and evidence-based practice, and only a limited number of them focused exactly on the scope of management and policy/decision making in health system organizations.

Given that the designed model is a comprehensive 10-step model, it can be used in some way at all levels of the health system and even in different countries. However, there will be a difference here, given that this framework provides a practical guide and a comprehensive guideline for applying evidence-based decision-making approach in health systems organizations, at each level of the health system in each country, this management approach can be applied depending on their existing infrastructure and the processes that are already underway (such as capacity building, planning, data collection, etc.), and at the same time, with a general guide, they can provide other infrastructure as well as the prerequisites and processes needed to make this approach much more possible and applicable.

It is true that evidence-based management is different from evidence-based medicine and even more challenging (due to lack of relevant data, greater sensitivity in data collection and their accuracy, lack of consistency and lack of transparency in the implementation of evidence-based decision-making in management rather than evidence-based medicine, etc.). Still, the general framework provided in this article can be used to help organizations that really want to act and move forward through this approach.

Furthermore, based on the findings, most of the previous studies only referred to some parts of the components and steps of the EBDM in health organizations and neglected the other parts or they were not sufficiently comprehensive [31,32,33,34,35,36,37,38,39,40]. Most of the previous models did not mention the necessary sub-steps, tools, and practical details for accurate and complete implementation of the EBDM, which causes the organizations that want to use these models, will be confused and cannot fully implement and complete the EBDM cycle. Among the studies that have provided a partly complete model than the other studies, were the studies by Brownson (2009), Yost (2014), and Janati (2018) [3, 41, 42]. Consequently, the combination of these three studies has been used as the initial framework for the best-fit synthesis in the present study.

Likewise, the models presented by Brownson (2009) and Janati (2018) were only limited to the six or seven key steps of the EBDM process, and they did not mention the details required for doing in each step, too [3, 4, 42]. Also, the model presented in the study of Janati (2018) was linear, and the relationships between the EBDM components were not well considered [42, 43]; however, the model presented in this study was recursive. Also, in Yost's study (2014), despite the 7 main steps of EBDM and some details of each of the steps, the proposed process was not schematically drawn in the form of a framework and therefore the relationships between steps and sub-steps were not clear [41]. According to what was discussed, the best-fit framework makes the possibility of concentrating the fragmented models to a comprehensive one that can be fully applied and evaluated by the health systems policymakers and managers.

In the present study, the framework of EBDM in HSM was developed in the form of four general scopes of inquiring, inspecting, implementing, and integrating including 10 main steps and 47 sub-steps. These scopes were discussed as follows:

Inquiring

In the first step, “situation analysis and priority setting”, the most frequently cited sub-step was identifying and prioritizing the problem. Accordingly, Falzer (2009), emphasized the importance of identifying the decision-making conditions and the relevant institutions and determining their dependencies as the first steps of EBDM [44]. Aas (2012) has also cited the assessment of individuals and problem status and problem-finding as the first steps of EBDM [34]. Moreover, the necessity of identifying the existing situation and issues and prioritizing them has been emphasized as the initial steps in most management models such as environmental analysis in strategic planning [45].

Despite considering the opinions and experience of experts and managers as one of the important sources of evidence for decision-making [42, 46,47,48,49,50], many studies did not mention this sub-step in the EBDM framework. Hence, the present authors added the acquisition of experts’ opinions as a sub-step of the first step because of its important role in achieving a comprehensive view of the overall situation.

In the second step, “quantifying the issue and developing a statement”, “Developing the conceptual model for the issue” was more addressed [37, 41, 47]. In addition, the authors to complete this step added the fourth sub-step, “Defining the main statement of issue”. This is because that most of the problems in health settings may have a similar value for managers and decision-makers and quantifying them can be used as a criterion for more attention or selecting the problem as the main issue to solve.

The third step, “Capacity building and setting objectives”, was not seen in many other included studies as a main step in EBDM, however, the present authors include this as a main step because without considering the appropriate objectives and preparing necessary capacities and infrastructures, entering to the next steps may become problematic. Moreover, in numerous studies, factors such as knowledge and skills of human resources, training, and the availability of the essential structures and infrastructures have been identified as facilitators of EBDM [51,52,53,54,55]. According to this justification, they are included in the present framework as sub-steps of the third step.

Considering the third step and based on the knowledge extracted from the previous studies, the three sub-steps of “understanding context and Building Culture” [56, 57], “gaining the support and commitment of leaders” [39, 57, 58], and “identifying the capabilities required by employees and their skills weaknesses” [58,59,60] were the most important sub-steps in this step of EBDM framework. In this regard, Dobrow (2004) has also stated that the two essential components of any EBDM are the evidence and context of its use [32]. Furthermore, Isfeedvajani (2018) stated that to overcome barriers and persuade hospital managers and committees to apply evidence-based management and decision-making, first and foremost, creating and promoting a culture of "learning through research" was important [61].

The present findings showed that in the fourth main step, “evidence acquisition and integration”, the most important sub-step was “finding the sources for seeking the evidence” [39,40,41, 60, 62, 63]. Concerning the sources for the use of evidence in decision-making in HSM, studies have cited numerous sources, most notably scientific and specialized evidence such as research, articles, academic reports, published texts, books, and clinical guidelines [39, 64, 65]. After scientific evidence, using the opinions and experiences of experts, colleagues, and managers [42, 46, 49, 66] as well as the use of census and local level data [49, 66, 67], and other sources such as financial [67], political [42, 49] and evaluations [49, 68] data were cited.

Inspecting

The fifth step of the present framework, “evidence appraising”, was emphasized by previous literature; for instance, Pierson (2012) pointed to the use of library services in EBDM [69]. Appraising and selecting the evidence according to appropriate appraisal tools/methods was cited the most. International and local evidence is confirmed that ignoring these criteria can lead to serious faults in the process of decision and policy-making [70, 71].

Furthermore, the sixth step, “analysis, synthesis, and interpretation of data”, was mentioned in many included studies [36, 39, 41, 42, 57, 59, 72]. This step emphasized the role of analysis and synthesis of data in the process of generation applied and useful information. It is obvious that the local interpretation according to different contexts may lead to achieving such kind of knowledge that can be used as a basis for local EBDM in HSM.

Implementing

The third scope consisted of the seventh and eighth steps of the EBDM process in HSM. In the seventh step, “developing evidence-based alternatives”, the issue of involving stakeholders in decision-making and subsequently, planning to design and implementation of the process and evaluation strategies had been focused by the previous studies [58, 60, 62, 63, 73]. Studies by Belay (2009) and Armstrong (2014) had also emphasized the need to use stakeholder and public opinion as well as local and demographic data in decision-making [49, 67].

“Pilot-implementation of selected alternatives” was the eighth step of the framework. Some key sub-steps of this step were resources allocation [58], Pre-implementation and pilot change in practice and assessing barriers and enablers for implementation [40] that indicated the significance of testing the strategies in a pilot stage as a pre- requisition of implementing the whole alternatives. It is obvious that without attention to the pilot stage, adverse and unpleasant outcomes may occur that their correction process imposes many financial, organizational, and human costs on the originations. In addition, a study explained that one of the strategies of the decision-makers to measure the feasibility of the policy options was piloting them, which had a higher chance of being approved by the policymakers. Also, pilot implementation in smaller scales has been recommended in public health in cases of lack of sufficient evidence [74].

Integrating

This last scope consists of the ninth and tenth steps. The main sub-step of the ninth step, “evaluating alternatives”, was to evaluating process and outcomes and revise. After a successful implementation of the pilot, this step can be assured that the probable outcomes may be achieved and this evaluation will help the decision and policymakers to control the outcomes, effectively. Also, it impacts the whole target program and proposes some correcting plans through an accurate feedback process, too. Pagoto (2007) explained that a facilitator for EBDM would be an efficient and user-friendly system to assess utilization, outcomes, and perceived benefits [55].

Also, the tenth step, “integrating and maintaining change in practice”, was not considered as a major step in previous models, too, while it is important to maintain and sustain positive changes in organizational performance. In this regard, Ward (2011) also suggested several steps to maintain and sustain the widespread changes in the organization, including increasing the urgency and speed of action, forming a team, getting the right vision, negotiating for buy-in, empowerment, short-term success, not giving up and help to make a change stick [35]. Finally, the most important sub-steps that could be mentioned in this step were the dissemination of evidence results to decision-makers and the integration of changes made to existing standards and performance guidelines. Liang (2012) had also emphasized the importance of translating existing evidence into useful practices as well as disseminating them [47]. In addition, the final sub-step, “feedback and feedforward towards the EBDM framework”, was explained by the authors to complete the framework.

Some previous findings showed that about half and two-thirds of organizations do not regularly collect related data about the use of evidence, and they do not systematically evaluate the usefulness or impact of evidence use on interventions and decisions [75]. The results of a study conducted on healthcare managers at the various levels of an Iranian largest medical university showed that the status of EBDM is not appropriate. This problem was more evident among physicians who have been appointed as managers and who have less managerial and systemic attitudes [76]. Such studies, by concerning the shortcomings of current models for EBDM in HSM or even lack of a suitable and usable one, have confirmed the necessity of developing a comprehensive framework or model as a practical guide in this field. Consequently, existing and presenting such a framework can help to institutionalize the concept of EBDM in health organizations.

In contrast, results of Lavis study (2008) on organizations that supported the use of research evidence in decision-making reported that more than half of the organizations (especially institutions of health technology assessment agencies) may use the evidence in their process of decision-making [75], so applying the present framework for these organizations can be recommended, too.

Limitations

One of the limitations of the present study was the lack of access to some studies (especially gray literature) related to the subject in question that we tried to access them by manual searching and asking from some articles’ authors and experts. In addition, most of the existing studies on EBDM were limited to examining and presenting results on influencing, facilitating, or hindering factors or they only mentioned a few components in this area. Consequently, we tried to search for studies from various databases and carefully review and screen them to make sure that we did not lose any relevant data and thematic code. Also, instead of one model, we used four existing models as a basis in the BFF synthesis so that we can finally, by adding additional codes and themes obtained from other studies as well as expert opinions, provide a comprehensive model taking into account all the required steps and details. Also, the framework developed in this study is a complete conceptual model made by BFF synthesis; however, it may need some localization, according to the status and structure of each health system, for applying it.

Conclusions

The present framework provides a comprehensive guideline that can be well adapted for implementing EBDM in health systems and organizations especially in underdeveloped and developing countries where there is usually a lag in updating and applying evidence in their decision-making process. In addition, this framework by providing a complete, well-detailed, sequential and practical process including 10 steps and 56 sub-steps that did not exist in the incomplete related models, can be tested in the organizational decision-making process or managerial tasks by developed countries to improve their EBDM cycle, too.