Background

Acceptance of health technologies in health systems within the last decades has experienced some intensive changes. The belief in abiding with the acceptance of health technologies within 1960s and 1970s gradually was replaced by an increasing suspicion of society towards the credibility of advanced health technologies. Nowadays, politicians and the public have an ambiguous opinion about the use of a new health technology [1].

Within recent years, the use of health technologies for diagnosis and treatment of diseases experienced considerable and accelerated growth. The proper use of these technologies may considerably help in the diagnosis and treatment of diseases. On the other hand, unlimited and unrestricted entry of these technologies may result in induced demand by service providers [2].

Given, the number of health technologies which required assessment was more than available resources; so all health technology assessment organizations must prioritize their research projects [3]. Various studies have been applied for this purpose worldwide until now, such as Canadian Drug and Health Technologies Organization (CADTH) which utilized the analytical hierarchy process (AHP) for priority setting of health technologies [4]. Another existing model in relation to priority and selection of health technologies was EVIDEM (Evidence and Value: Impact on Decision Making) model in Canada [5]. Nonetheless, there is no consensus on appropriate priority setting between health technology assessment organizations [3]. The main goal of the present study was to describe the designated pilot MCDM model for priority setting of health technology assessment in Iran.

Methods

In order to achieve the aforementioned goal, relevant articles until March, 2015, based on three separate strategies using MESH and free text were sought for using the Cochrane Library, Pub Med and Scopus. The criteria and sub criteria (in pre assessment and assessment phase, at this stage, the criteria of assessment phase were also included, on the assumption that there may be some of these criteria which can be used in pre assessment phase in Iranian health system context) obtained from systematic review were questioned by experts of health technology assessment context via questionnaire (these expert were selected from official committee of HTA in Iranian ministry of health and medical education and during of this research just 11 ones of them were available for inclusion). At first, to determine the content validity as pilot, experts of health technology assessment were provided with questionnaire in three forms, including (a.) determination of face validity for each item (as two-options: it is clear and demonstrative or it is not clear and demonstrative), (b.) determination of content validity rate (CVR) for each item (as three-options: it is necessary, useful but not necessary or not necessary) and (c.) determination of content validity index (CVI) for each item (as five-options: completely irrelevant, irrelevant, relatively irrelevant, relevant or completely relevant). Sub criteria from which the minimum content validity was obtained was also included in the final questionnaire phase. In this questionnaire, the importance of each of these sub criteria was questioned by experts in these five options context (very slightly important, slightly important, moderately important, important, and very important). For this purpose, a 75 % cut point was used. This implied that for each one of these sub criteria, 75 % of experts who voted important or very important in the process of health technology assessment priority setting were included in the analytic hierarchy process (AHP) phase for extraction of their relative weight through paired wise comparisons. In the analytical hierarchy process, relative weight was obtained by paired comparison matrix. For a consistent paired comparison matrix, weight was calculated through normalization of elements of each column. However, if the matrix was inconsistent, to obtain the weight, four major methods were introduced, including 1 - least squares, 2 - least logarithm squares, 3 - Eigen vector, 4 - approximate methods [6]. The first three methods involve tedious calculations, thus some approximate methods have been proposed which are of lower accuracy (that is acceptable) and involve less and simpler calculations. In this study, geometric mean method was used. The geometric mean of elements of each row was calculated; later, the obtained vector was normalized to obtain the weight vector. Inconsistency rate of each matrix A was calculated based on the following steps:

  1. 1.

    Formation of the paired comparison matrix A;

  2. 2.

    Specification of the weight vector (W);

  3. 3.

    Is the maximum Eigen value of matrix A (λ max) specified? If yes, go to forth step, otherwise (according to following steps) estimate its value.

    1. 3-1:

      Obtaining a well-suited estimation of λ maxW by multiplying vector W by matrix A (AW = λ maxW);

    2. 3-2:

      Upon dividing the obtained values for λ maxW by relative W, estimation of of λ max is calculated;

    3. 3-3:

      The obtained mean λ max is calculated;

  4. 4.

    Calculation of inconsistency index (I.I) based on following equation:Formula (1)

    $$ I.I=\frac{\lambda_{\max }-n}{n-1} $$
  5. 5.

    Calculation of inconsistency ratio (I.R.) based on following formula:Formula (2)

    $$ I.R=\frac{I.I}{I.I.R.} $$

In this formula I.I.R. denotes the random matrix inconsistency index. This matrix was calculated for matrices whose numbers were completely randomly selected. In each matrix, the product of dividing the inconsistency index by random matrix inconsistency matrix with the same dimension will be a suitable criterion for judgment about matrix inconsistency which is called inconsistency ratio. If this number is smaller or equal to 0.1, system consistency is acceptable; otherwise the judgments must be revised [6].

After extraction of relative weights based on the above procedure, TOPSIS priority setting model was designed. Essential principles of this method are as follows:

  1. a.

    Appropriateness of each index must be uniform in ascending (descending) order (whatever rij is higher, appropriateness also higher or vice versa) so that the best existing value of an index denotes its ideal and the worst existing value thereof shows negative ideal for it.

  2. b.

    The interval of an option from ideal (or negative ideal) may be calculated as Euclidean distance (from second exponent) or as the sum total of absolute value of linear distances (known as block gaps) that is dependent on the exchange and replacement rate among indices.

First step: this involves the conversion of existing decision making matrix to a matrix descaled using following formula:

Formula (3):

$$ {n}_{\mathrm{i}\mathrm{j}}=\frac{{\mathrm{r}}_{ij}}{\sqrt{{\displaystyle {\sum}_{\mathrm{i}=1}^{\mathrm{m}}{\mathrm{r}}_{\mathrm{i}\mathrm{j}}^2}}} $$

Second step: this involves creating weighted descaled matrix by assumption of vector W as input to the algorithm.

Formula (4):

$$ \begin{array}{c}\hfill W=\left\{{w}_1,{w}_2,\dots {w}_n\right\}\approx \left(\mathrm{supposed}\ \mathrm{of}\kern0.5em DM\right)\hfill \\ {}\hfill \mathrm{V}={\mathrm{N}}_{\mathrm{D}}.\kern0.5em {\mathrm{W}}_{\mathrm{n}\times \mathrm{n}}=\begin{array}{ccc}\hfill {\mathrm{V}}_{11\dots}\hfill & \hfill {\mathrm{V}}_{1\mathrm{j}\dots}\hfill & \hfill {\mathrm{V}}_{1\mathrm{n}\dots}\hfill \\ {}\hfill \vdots \hfill & \hfill \vdots \hfill & \hfill \vdots \hfill \\ {}\hfill {\mathrm{V}}_{\mathrm{m}1\cdots}\hfill & \hfill {\mathrm{V}}_{\mathrm{m}\mathrm{j}\cdots}\hfill & \hfill {\mathrm{V}}_{\mathrm{m}\mathrm{n}\cdots}\hfill \end{array}\hfill \end{array} $$

Where ND is a matrix having the score of indices therein descaled and comparable and Wn × n is a diametrical matrix having only its main elements as nonzero.

Third step: This involves determination of ideal solution and negative ideal solution

For ideal option A+ and negative ideal A-, the following definition is provided:

Formula (5):

$$ \begin{array}{c}\hfill {\mathrm{A}}^{+}=\left\{\left(\begin{array}{c}\hfill max\hfill \\ {}\hfill i\hfill \end{array}{V}_{ij}\left|j\in J\right.\right),\left(\begin{array}{c}\hfill min\hfill \\ {}\hfill i\hfill \end{array}{V}_{ij}\left|j\in J\right.\right)\left|i=1,2,\dots m\right.\right\}=\left\{{V}_1^{+},{V}_2^{+},\dots, {V}_j^{+},\dots, {V}_n^{+}\right\}\hfill \\ {}\hfill {\mathrm{A}}^{\hbox{-} }=\left\{\left(\begin{array}{c}\hfill min\hfill \\ {}\hfill i\hfill \end{array}{V}_{ij}\left|j\in J\right.\right),\left(\begin{array}{c}\hfill max\hfill \\ {}\hfill i\hfill \end{array}{V}_{ij}\left|j\in J\right.\right)\left|i=1,2,\dots m\right.\right\}=\left\{{V}_1^{-},{V}_2^{-},\dots, {V}_j^{-},\dots, {V}_n^{-}\right\}\hfill \\ {}\hfill J=\left\{j=1,2,\dots, n\left|\mathrm{j}\mathrm{s}\right.\kern0.5em \mathrm{referred}\kern0.5em \mathrm{t}\mathrm{o}\kern0.5em \mathrm{t}\mathrm{he}\kern0.5em \mathrm{profit}\right\}\hfill \\ {}\hfill J=\left\{j=1,2,\dots, n\left|\mathrm{j}\mathrm{s}\right.\kern0.5em \mathrm{referred}\ \mathrm{t}\mathrm{o}\kern0.5em \mathrm{t}\mathrm{he}\kern0.5em \mathrm{cost}\right\}\hfill \end{array} $$

Forth step: This involves calculation of separation size (distance).

The distance between option i and ideals based on Euclidean method is as follows:

Formula (6)

$$ \begin{array}{c}\hfill {\mathrm{d}}_{\mathrm{i}+}={\left\{{\displaystyle \sum_{\mathrm{j}=1}^{\mathrm{n}}{\left({\mathrm{V}}_{\mathrm{i}\mathrm{j}}\hbox{-} {\mathrm{V}}_{\mathrm{j}}^{+}\right)}^2}\right\}}^{0.5},\mathrm{i}=1,2,\dots, \mathrm{m}\hfill \\ {}\hfill {\mathrm{d}}_{\mathrm{i}\hbox{-} }={\left\{{\displaystyle \sum_{\mathrm{j}=1}^{\mathrm{n}}{\left({\mathrm{V}}_{\mathrm{i}\mathrm{j}}\hbox{-} {\mathrm{V}}_{\mathrm{j}}^{\hbox{-}}\right)}^2}\right\}}^{0.5},\mathrm{i}=1,2,\dots, \mathrm{m}\hfill \end{array} $$

Fifth step: This involves calculation of relative closeness of Ai to the ideal solution. This relative closeness is defined as follows:

Formula (7):

$$ c{l}_{i+}=\frac{d_{i-}}{\left({d}_{i+}+{d}_{i-}\right)};0\le c{l}_{i+}\le 1;i=1,2,\dots, m $$

As observed, if

$$ {A}_i={A}^{\mp } $$

Then:

$$ {d}_{i+}=0 $$

And will have:

$$ c{l}_{i+}=1 $$

And in case:

$$ {A}_i={A}^{-} $$

Then:

$$ {d}_{i-}=0 $$

And

$$ c{l}_{i+}=0 $$

Therefore, whatever Ai is closer to the ideal solution A+, value of cli will be closer to the unit.

Sixth step: This involves ranking of the options. The existing options of supposed problem can be ranked based on the descending order cl i + [7, 8].

Ultimately, the first designed model was examined by pilot test using the last health technology assessment projects call of The Iranian Department of Health Technology Assessment, including 3 technologies (Adenosine, Tissue Plasminogen Activator (tPA), Mechanical Thrombectomy) and the results were provided to the experts for judgment.

Results

Systematic review

After screening of retrieved papers via PRISMA framework, from 7012 papers, 40 were left included in the final phase. Criteria for selection of health technologies (in pre-assessment and also in assessment phase) were categorized into the six main themes such as 1 - health outcomes, 2 - disease and target population, 3 - technology alternatives, 4 -economic aspects, 5 - evidence and 6 - other factors. Maximum frequency in health outcomes related to health effects/benefits (8 studies), maximum frequency in disease and target population was related to disease severity (12 studies), maximum frequency in alternatives was related to number of alternatives (2 studies), maximum frequency in economic aspects was related to cost-effectiveness (15 studies), maximum frequency in evidence was related to quality of evidence (4 studies) and in other factors, max frequency was related to issues concerning the health system (11 studies) (Table 1).

Table 1 Classification of main criteria and sub criteria based on frequency in reviewed Studies (in pre-Assessment and also in Assessment phase)
Full size table

Phase 1: Content validity verification

Sub criteria obtained from the systematic review phase that were classified into six dimensions were designed as three questionnaires, including 54 sub criteria and one question for extraction of extra criteria. To determine the content validity of 11 health experts, technology assessment were questioned in three ways: (a.) determination of face validity for each item (as two-option: it is clear and demonstrative or it is not clear and demonstrative), (b.) determination of content validity ratio (CVR) for each item (as three-option: it is necessary, useful but not necessary or not necessary) and (c.) determination of content validity index (CVI) for each item (as five-option: completely irrelevant, irrelevant, relatively irrelevant, relevant or completely relevant). Firstly, content validity index (CVI) was calculated for each item such that out of 55 items, 21 items were obtained following a minimum of 0.79 which included health benefits, clinical impacts, efficacy/effectiveness, potential to increase the life, safety, population size, the number of potential beneficiaries, the impact on target groups, the size of vulnerable populations, the availability of alternative technologies, cost-effectiveness, costs, budget impact, financial protection, the quality of evidence, number of evidence, the issues related to the health system, the issues related to the availability of expertise, equity in better access to health care and the question of additional criteria. Shortly following this, content validity ratio was calculated for each item such that, out of 54 sub criteria, all questioned experts had consensus on the necessity of only 9 sub criteria (health benefits, clinical benefits, efficacy/effectiveness, safety, cost-effectiveness, costs, technology price, quality of the evidence and issues related to rate availability of expertise) in this framework and obtained a score higher than score of Lawshe table for 11 experts. Whereas, Lawshe clarified that each item with more than half the total population of people had consensus on its necessity, and had a level of content validity (to the same rate of people that voted to its necessity) [9]. Thus, in addition to 9 sub criteria that had obtained the required score of CVR, a sub criteria greater than 50 % of people (including 50 % itself and higher) voted to its necessity and a CVI greater than 0.79 was obtained, kept and included into the final questionnaire (for assessment of their importance). These sub criteria were as follows: clinical impacts, efficiency/effectiveness, individual health benefits, safety, population size, number of potential beneficiaries, effect on targeted group, vulnerable population size, availability of alternative technologies, quality of evidence, number of evidence, budget impact, cost effectiveness, financial protection, issues related to health system and availability rate of experience and specialty and additional question for other criteria. Based on face validity, according to the opinion of experts, 4 sub criteria including individual health benefits, clinical impacts, efficiency/effectiveness, and effect on targeted group due to their overlap were all combined in the efficiency/effectiveness sub criteria. In addition, in the same manner, sub criteria of number of potential beneficiaries was combined with the population size, costs combined with cost effectiveness while number of evidence was renamed as the adequacy of the evidence. Thus, in the final questionnaire, 13 sub criteria were included based on 6 dimensions of conceptual model of study (Table 2).

Table 2 Sub criteria obtained after first phase of question from experts
Full size table

Phase 2: Determining importance of each sub criteria

Thirteen obtained sub criteria in previous phase were included into the final questionnaire phase based on the same process, 9 sub criteria, including Efficiency/effectiveness, safety, Population size, vulnerable population size, availability of alternative technologies, budget Impact, cost effectiveness of technology in other countries, financial protection, quality of evidence that obtained a greater than 75 % importance as viewpoint of questioned experts, were included in the paired wise comparisons questionnaire as the final 9 effective criteria in the priority setting of health technology assessment and this finalized the extraction of relative weight (Table 3). To assess the reliability of the final questionnaire, data obtained from each sub criterion were entered into SPSS software as Likert spectrum and the Cronbach’s alpha was calculated with its value at 0.697. This demonstrated an acceptable reliability of the final questionnaire. Moreover, no consensus was made between questioned experts in relation to the additional criteria individually mentioned by the experts. These criteria include Ethical issues, burden of disease, information value, the elasticity of demand for technology, procedures and methods of diagnosis and treatment, the external dependence of the use of technology, the development and application of technology in the world and the indication of technology.

Table 3 Final sub criteria obtained after two phases questioning from experts
Full size table

Phase 3: Identification of relative weights of final criteria

Nine final effective criteria selected based on the opinion of experts were designed during the first and second phases as paired comparisons questionnaire and in another case they were presented to nine questioned experts in the first and second phases (at this phase, two experts of the previous phases were not available) in order to extract the relative weight of each one of the 9 final criteria. After filling the 9 questionnaires aforementioned, the information of these questionnaires was entered into an excel sheet as 9 matrixes (9*9) and to extract the final weight and equalizing the data of these 9 matrixes, a final matrix was formed such that in each one of its entries was included the geometric mean of its corresponding entries in 9 matrixes (9*9). The relative weigh of each criterion was extracted from the final formed matrix through normalization of geometric mean of each line thereof. Later, the obtained inconsistency ratio for the system was calculated, which was less than the range of 0.1 (IR = 0.03) and demonstrated consistency between extracted weights (Table 4).

Table 4 Relative weight obtained from normalization of geometric mean of each row of final matrix
Full size table

Phase 4: Piloting the priority setting model

According to the inquiry from health technology assessment department of ministry of health, and medical education, 3 health technologies that were requested for health technology assessment, were selected and TOPSIS priority setting pilot model was implemented on them. These technologies included the following:

  1. a.

    Adenosine

  2. b.

    Tissue Plasminogen Activator (tPA)

  3. c.

    Mechanical Thrombectomy

Furthermore, to fill the decision matrix for execution of model pilot, values 1 to 5 were designed for each one of 9 final criteria, and data related thereto were extracted from available evidence for the three pilot technologies: (Tables 5, 6, 7, 8, 9, 10, 11, 12 and 13).

Table 5 Coding of efficiency/effectiveness criteria [10]
Full size table
Table 6 Coding of Safety Criteria (as qualitative estimation or using quantitative indicators available in the safety context): Improvement rate in reduction of risk or incidence of side effects arising out of utilization of technology in the target population
Full size table
Table 7 Coding of population size criteria: number of people under effect of disease or health problem that will be treated or prevented by technology [10]
Full size table
Table 8 Coding of vulnerable population criteria: share of children below 5 years old or women in fertility ages or elders over 65 years old or patients suffering from special diseases (including MS, dialysis, hemophilia and thalassemia) or patients suffering from neuropsychological diseases in the target population
Full size table
Table 9 Coding of Availability rate of Alternative technologies (qualitative estimation): Availability of other alternative methods for technology in target population
Full size table
Table 10 Coding of quality of evidence: quality of evidences of health technology assessment, systematic review and clinical trials on technology
Full size table
Table 11 Coding of cost effectiveness: status of cost effectiveness of technology in other countries
Full size table
Table 12 Coding of budget impact (as qualitative estimation): net impact of covering the technology on the budget of health system
Full size table
Table 13 Coding of financial protection: the probability that the technology covered by the basic health insurance package and thus prevent the occurrence of catastrophic health expenditures
Full size table

The data obtained from evidence about three above-mentioned technologies

Adenosine for the treatment of supraventricular tachycardia [11]

Status of efficiency/effectiveness criteria: This medication with a dose of 12 mg quickly and effectively terminated proximal episodes of the supraventricular tachycardia (relative code = 5) [12]. Status of safety criteria: The use of this medication resulted in a slightly bad side effect such as confusion, nausea, redness of face etc. (relative code = 4) [13]. Status of population size criteria: Supraventricular tachycardia prevalence was 2.25 cases in a general 1000-persons population. Therefore, the size of target population of this medication in Iran, considering its 80 million population was about 180 thousand peoples (relative code = 4) [11]. Status of vulnerable population size criteria: The probability of supraventriclar tachycardia in women was 2 times more than men, and majorly in pregnancy and fertility ages (within 15–30 years) (relative code = 4) [11]. Status of alternative technologies availability criteria: Verapamile is the alternative medication that has equal efficiency and relatively equal safety and mostly available in the health centers (relative code = 2) [11, 13]. Status of quality of evidence criteria: Concerning this medication, there is a systematic review study of randomized controlled studies of meta-analysis type with a high quality (relative code = 5) [13]. Status of cost effectiveness criteria: The data about this criterion was unspecified (relative code = 1). Status of budget impact criterion: The data about this criterion was unspecified (relative code = 1). Status of financial protection criterion: The data about this criterion was unspecified (relative code = 1).

Tissue plasminogen activator (tPA) for the treatment of stroke [14]

Status of efficiency/effectiveness criteria: The use of this medication resulted in a 0.07 increase of lifetime together with 0.24 increase of QALY (relative code = 3) [15]. Status of safety status: In comparison to placebo, the use of this technology had higher risk of intracranial hemorrhage (relative code = 3) [16]. Status of population size criteria: prevalence of stroke in west countries was about 12000 per million peoples [17]. The prevalence and incidence of stroke in Iran was significantly higher than western countries [18]. Thus, in Iranian population, assuming 80 million people, the population size under impact of this disease will be higher than one million people (relative code = 5). Status of vulnerable population size criteria: The incident of stroke was mostly prevalent among Iranian women (within 51–53 %) and majorly occurred within seventh decade of life (relative code = 4) [19]. Status of alternative technologies availability criteria: This technology was the first and the only effective therapeutic method for patients with stroke 3 times after its first occurrence (relative code = 5) [16]. Status of quality of evidence criteria: In relation to this technology, there was a high quality randomized multicenter controlled trial (elative code = 3) [16]. Status of cost effectiveness criteria: The incremental cost effectiveness ratio for all patients consuming this medication was about 6255 Dollars per QALY. This value was obtained for USA. Considering her per capital GDP in 2014 as equal to 53000$. This value is rather highly cost effective (relative code = 5) [15]. Status of budget impact criteria: The data in relation to this criterion was unspecified (relative code = 1). Status of financial protection criteria: The data in relation to this criterion was unspecified (relative code = 1).

Mechanical thrombectomy for the treatment of stroke [20]

Status of efficiency/effectiveness criteria: This technology resulted in a 0.68 increase of QALY (relative code = 1) [21]. Status of safety criteria: This technology was safe which could demonstrate higher rate of symptomatic bleeding than medicinal methods (relative code = 3) [20]. Status of population size criteria: Prevalence of stroke in west countries was about 12000 per million peoples [17]. The Prevalence and incidence of stroke in Iran was significantly higher than western countries [18]. Thus, in Iranian population, assuming 80 million people, the population size under impact of this disease will be higher than one million people (relative code = 5). Status of vulnerable population size criteria: The incident of stroke was mostly prevalent among Iranian women (within 51–53 %) and majorly occurred within seventh decade of life (relative code = 4) [19]. Status of alternative technologies availability criteria: This method was the only therapeutic method for patients with incident of stroke who were not treated by clot solving medications, since the use of this method resulted in failure with respect to them (during first 3–4 h after this occurrence) (relative code = 5) [20]. Status of quality of evidence criteria: In relation to this technology, there was health technology report from randomized r controlled trial with high quality (elative code = 5) [20]. Status of cost effectiveness criteria: The incremental cost effectiveness ratio for all patients taking this medication was about 16000 Dollars per QALY. This value was obtained for USA and considering per capital GDP of this country in 2014, which equaled 53000$, this value is rather highly cost effective (relative code = 5) [21]. Status of budget impact criteria: The data in relation to this criteria was unspecified (relative code = 1). Status of financial protection criterion: The data in relation to this criterion was unspecified (relative code = 1).

Final TOPSIS pilot model

Finally TOPSIS pilot model was designed by 3 health technologies and relative weights. The results showed that tissue plasminogen activator treatment was the first priority, adenosine was second and mechanical thrombectomy was third in performing health technology assessment by Iranian ministry of health and medical education (Table 14). Finally, the pilot model was presented to an internal panel which was made up of 6 experts of health technology assessment and was approved.

Table 14 Final TOPSIS pilot model
Full size table

Discussion

Generally, this study which was applied based on the combined model of analytic hierarchy process and TOPSIS, was presented for the first time in the world, in the health technology assessment priority setting context. The differences and similarities between this model and main model of Canadian Agency for Drugs and Technologies in Health are explained in terms of finalized criteria in this model. The criteria used in this model which extracted using a systematic review which included a number of alternatives and access such as, budget impact, clinical impact, controversial nature of technology, disease burden, economic impact, moral and legal issues, quality and level of evidences, expected level of interests, time period of assessment and its effect on health and clinical medicine policies, variety in direction for use and application of technology. The criteria used in the model of this study included efficiency/effectiveness, safety, population size, vulnerable population size, availability of alternative technologies, cost effectiveness in other countries, budget impact, financial protection, and quality of evidences. Efficiency/effectiveness, availability of alternative technologies, quality of evidences and budget impact as common items and controversial nature of technology, disease burden, economic impact, moral and legal issues, expected level of interests, time period of assessment and its effect on health and clinical medicine policies, variety in direction for use and application of technology in Canadian model and safety, population size, vulnerable population size, cost effectiveness in other countries and financial protection in the model of this study were assumed as differences. In relation to multi criteria decision making model used in this study, it is noteworthy that in Canadian model, overall relative weight extraction, as well as, final priority setting were performed based on analytic hierarchy process, whilst in the model of this study, only relative weight was extracted for determination of exact weight using final purpose which implied that, health technology assessment priority setting in Iran, through paired comparisons and no judgment of experts, were applied on the final model. This subject was designed in order to adhere to evidence based medicine (EBM) and its evidence pyramid using TOPSIS model and ascertain the use of codes extracted from evidence related to the technology, which can prevent the fully and unlimited interventions of medical beneficiary groups for selection of health technologies that are assumed as the stewardship scope. In fact, out of 11 criteria acquired from systematic review phase of Canadian model and mentioned earlier, 6 criteria including disease burden, clinical impact, alternative technologies, budget impact, economic impact and available evidence were selected as the main criteria by health technology assessment experts of that country. In relation to weight extraction phase, the following values were obtained: disease burden (0.216), clinical impact (0.258), alternative technologies (0.081), budget impact (0.143), economic impact (0.167), and available evidence (0.135) [4]. Furthermore, these values were comparable with the values obtained from the present, study including efficiency/effectiveness (0.12), safety (0.2), population size (0.06), vulnerable population size (0.08), availability of alternative technologies (0.08), cost effectiveness in other countries (0.13), budget impact (0.08), financial protection (0.09), and quality of evidence (0.15). The model designed in this study may also be compared to the health technology assessment priority setting model in Latvia. In this model, research of literature review as well as qualitative study was used in the model design, followed by a test for health technology assessment priority setting. To test the model and method, 3 round Delphi model was implemented using electronic questionnaire. Accordingly, Delphi technique was invented as a method for making consensus in the health technology assessment priority setting for Latvia. Criteria affecting this model included health benefit, evidence, assessment scheduling, expected level of policy makers’ benefits, social, legal and moral concepts [22]. Two criteria of health benefit and evidence were common criteria in the model designed in the present study. The designed model of this study may be compared to the health technology assessment priority setting model in Netherlands. In this model, various methods were used for classification, scoring and weighting the policy making criteria, including actual disease burden, potential health benefit, number of patients, direct intervention costs for each patient, financial consequences of intervention implementation over time and its impact on the health system policies [23]. Three criteria, including health benefit, number of patients and financial consequences were assumed as the three common criteria in the model designed for this study. Overall, the model designed in the present study in comparison to previous models in this context had innovation in the light of effective criteria because of its additional safety, vulnerable population size, cost effectiveness of technology in other countries and financial protection to the previous models. In addition, as mentioned before, technically this is the first time that a combined model of analytic hierarchy process and TOPSIS has been used in this relation.

Conclusion

According to the results of this study, this model with nine effective criteria and their relative weights and in combination with TOPSIS approach could be used with suitable applicability by health technology assessment department in deputy of curative affairs and food and drug organization for determination of research priorities in health technology assessment. Considering the increasing growth of health technologies, ministry of health and medical education, using this model, can allocate its research budgets to health technologies assessment which are of priority and importance to the health system and society. It may promote the final impact of health technology assessment reports on the macro health policies and optimize the allocation of financial resources of health sector.

Limitations

This model is not applicable for prioritization of orphan drugs.