Background

In Chinese public hospitals, the Pharmacist Management Committee (PMC) makes medicine purchasing decisions based on evidence which made for assessing drug value [1, 2]. With the approval of new drugs and the adjustment of the medical insurance drug catalogue, the PMC of Chinese public Hospital is facing increasing challenges regarding the choice of drug varieties. So far there has not been a canonical method to estimate the value of medicine. The common method to assess the value of drug is to make a systematic review of three dimensions: effectiveness, safety, and economics. However, this method is time consuming and PMC in Chinese public hospitals was asked to make quick decisions. Therefore, evidence sent to PMC is secondary literature or reviews made by rapid assessment which is not systematic or explicit [3]. Meanwhile the assessment mainly focuses on cost-effectiveness analysis and does not systematically consider the multi-dimension value of a drug [3, 4]. The decision-making process is complex, subjective, and requires both systematic and explicit methods to ensure transparency and consistency of the participating factors. There is a need for a systematic and transparent assessment method that meets the requirements for assessment drug in Chinese public hospitals.

Multicriteria decision analysis (MCDA) is a method used to appraise alternatives for the individual that often have conflicting criteria by combining them into a single appraisal [5]. Currently MCDA is applied to health care decision making such as hospital purchasing [6, 7], and helps increase the consistency, transparency and legitimacy of decision making [8]. The Evidence and Value: Impact on Decision Making (EVIDEM) collaboration developed an EVIDEM framework, which aims to estimate medicine systematically, and includes a core model containing 13 criteria that are adaptable to a contextual tool [9] (Tables 1 and 2). EVIDEM framework bridges the MCDA with health technology assessment (HTA) [10]. This study used the 10th edition which is based on 10 years of open source development and its roots are in real-life deliberation and decision.

Table 1 EVIDEM Core Model
Full size table
Table 2 EVIDEM Contextual tool
Full size table

In July 2019, the PMC in a public hospital in Gansu China was asked to make a decision whether to purchase new Dipeptidyl Peptidase-4 (DPP-4) inhibitors to replace the existing ones within a short period of time. The hospital health technology assessment(HTA) group plan to provide evidence by using EVIDEM framework and MCDA to assess the drug. There are currently five DPP-4 inhibitors that are marketed in China: Sitagliptin (approved in 2006), Linagliptin (approved in 2011), Vildagliptin (approved in 2007), Alogliptin (approved in 2013), and Saxagliptin(approved in 2009, [11]). The aim of this study was to appraise DPP-4 inhibitors with the goal of piloting the EVIDEM framework and MCDA to identify the most optimal medication.

Methods

Methodology of the EVIDEM framework was followed for MCDA [8]. MCDA was performed following reports published by International Society for Pharmacoeconomics and Outcomes Research (ISPOR) [12, 13]. The estimating program is shown in Fig. 1.

Fig. 1
figure 1

The estimating program

Full size image

First, in order to build the EVIDEM evidence matrix of DPP-4 inhibitors, PubMed, Embase, The Cochrane Library and China Biology Medicine disc (CBM) were searched until July 2019. Search terms included the five DPP-4 inhibitors, type 2 diabetes, efficacy, safety, quality of life, epidemiol*/prevalence/incidence, mortality, guidelines, recommendations, clinical practice, patient-reported outcome*/PRO, cost*, econom*, meta-analysis and net meta-analysis (Additional file 2). For the factors that are country-specific (e.g. cost-effectiveness and clinical guidelines), data was only included based on Chinese settings. Placebo was chosen as the comparator. To similarly compare the five DPP-4 inhibitors, meta-analysis comparing the five DPP-4 inhibitors with placebo was first performed. The HTA group analyzed and synthesized the data, meanwhile completed the different criteria of evidence matrix with suitable data.

Second, building an appraisal committee which were comprising two doctors, one nurse, one pharmacist, one health economist and one decision-maker. All six members were already staff at the hospital. Both the doctor and nurse worked in the Endocrinology Institute.

Next, following EVIDEM methodology, the committee assigned weight individually for each criterion in EVIDEM framework according to their importance before it was known which medicines needed to be evaluated. Criterion scored as 5 were considered the most important and those scored as 1 were considered the least important.

Based on the evidence matrix synthesized by the HTA group (Additional file 1), experts assigned a score for each criterion in evidence matrix, individually. The scores for the absolute criteria (those that were not compared to placebo) ranged from 0 to 5, where 0 was the lowest value and 5 was the highest. For relative criteria (those compared to the placebo), the scale ranged from - 5 to 5 to reflect the full range of comparative effects. Meanwhile, criteria in the contextual tool were discussed by committee.

Finally, a value estimate (V) of the intervention was calculated by Excel 2019 based on a linear additive model as a sum of the value contributions (Vx) [or combined normalized weights (Wx) and standardized scores (Sx)] of all (n) criteria of the quantitative EVIDEM Core Model. Each estimated value was transformed to a 0-1 scale.

$$ \mathrm{V}=\sum \limits_{x=\mathrm{l}}^{\mathrm{n}}{\mathrm{V}}_x=\sum \limits_{x=\mathrm{l}}^{\mathrm{n}}\left({\mathrm{W}}_x\times {\mathrm{S}}_x\right) $$

Results

Criterion weights

Members of the panel committee decided on a mean weight for every criterion as shown in Table 3. Every criterion was of high relevance besides the comparative cost consequences-non-medical cost. The panelists decided the most important criterion to be the quality of evidence. Interestingly, six of the criteria included disease severity, size of the affected population, comparative effectiveness, type of therapeutic/preventive benefit and cost of intervention, which were all assigned the same weight. The type of therapeutic/preventive benefit and non-medical cost showed high variability.

Table 3 The mean weight for every criterion
Full size table

Scores for the five DPP-4 inhibitors

Mean scores for the five DPP-4 inhibitors are presented in Table 4. Criteria including disease severity, size of the affected population, unmet need, type of therapeutic/preventive benefit, quality of evidence and expert consensus/clinical practice guidelines were evaluated using absolute terms. All of these criteria were given the same scores for all five medicines. The size of affected population showed the highest score and lowest variability in absolute criteria. Disease severity and type of therapeutic benefit followed affected population size. In contrast, the committee did not estimate the type of preventive benefit since there was no data available on this topic. Quality of evidence showed the lowest score based on three meta-analyses evaluated with high bias by the ROBIS tool. Expert consensus/clinical practice guidelines showed the highest variability.

Table 4 Mean scores of criterions of five DPP-4 inhibitors
Full size table

Comparing the interventions to placebo, five DPP-4 inhibitors showed greater added value for effectiveness. Sitagliptin showed the highest score for effectiveness. Alogliptin and Linagliptin both had the same scores. Vildagliptin and Saxagliptin both had scores that were less than those of Alogliptin and Linagliptin. In contrast, medicine showed a lower score and higher variability when analyzing comparative safety. Comparative patient-perceived outcomes/PRO received a negative score for all medicines, meaning that the PRO for the medications was worse when compared to the placebo.

In terms of cost, direct costs for Saxagliptin, Alogliptin and Sitagliptin which were purchased by hospital were presented at the hospital price. Linagliptin and Vildagliptin were not located on the hospital purchased list and the mean price used in Chinese nine provinces were calculated. The costs of intervention received a negative score indicating that medicine cost was higher than placebo. There were no data available for other medical costs and thus the committee did not assign scores to these. The committee believed that the medication showed greater non-medical costs with higher variability.

Estimated value for five DPP-4 inhibitors

The overall estimated value that integrated the weights and scores for five DPP-4 inhibitors all showed that the medications had higher added value compared to the placebo (Fig. 2).

Fig. 2
figure 2

Estimated value for five DPP-4 inhibitors

Full size image

Discussion

In this pilot study Sitagliptin showed the highest value and Saxagliptin received the lowest score. The hospital HTA group gave a value order for decision makers on the hospital PMC, therefore it is feasible to use the EVIDEM framework and MCDA in purchasing medicine for Chinese public hospitals.

To our knowledge, this is the first time that MCDA and EVIDEM frameworks were used in drug purchasing in a Chinese hospital. The MCDA is used in China for the decision-making framework for medical insurance reimbursement and medical device assessment [14, 15]. Our study makes it possible for estimating medicine value with quantitative calculation and includes more comprehensive considerations for the value of medicines by using the EVIDEM framework. Prior to this, the valuation of drug value was qualitative. An HTA group in a hospital in Beijing evaluated DPP-4 inhibitors by using a systematic review that qualitatively described effectiveness, safety and economics [16,17,18].

The application of EVIDEM and MCDA is still an indirect comparison of drugs. Although there were head-to-head meta-analysis results in terms of safety and effectiveness, no direct comparison data was there for other dimensions of criteria, such as cost. The drug value range was similar with other studies estimating the value by EVIDEM-MCDA. For example, the obinutuzumab for rituximabrefractory indolent non-Hodgkin lymphoma obtained a value of 0.45 in Italy [19]. Interestingly, even though comparative data in the net-meta analysis showed no significant differences between the drugs, value differences still existed [20,21,22,23,24]. Since the weights and scores generated by the committee would inevitably revolve around their own perceptions, experience, training and value judgments, it is critical to have a multi-disciplinary vision.

In applying EVIDEM-MCDA, the participating experts noted that this work was meaningful and inspiring, and the exchange of opinions between them enriched the analysis. However, they also emphasized that the evaluation process requires a good understanding of the amount of evidence provided, and the concepts and terminologies used. In this regard, it is important to emphasize the importance of providing the best understanding. On the other hand, it is worth noting the difference between the specialties of experts that make up the committee. Their weights and scores were inevitably influenced by their own opinions, experience, training and value judgments, thus showing the importance of having such a multidisciplinary vision.

Although this study provides guidance for decision makers, there are still some limitations. Firstly, DPP-4 inhibitors were used a lot in treatment programs of clinical trials such as a DPP-4 inhibitor versus placebo/metformin/other anti-diabetes drugs or a DPP-4 inhibitor/anti-diabetes drugs versus other anti-diabetes drugs [25, 26]. Based on time limitation and feasibility, we chose the placebo as the comparator. There was no consideration from other comparators due to only comparing to the placebo. Secondly, in terms of costs, sufficient data in China was not available since there was little research on Pharmacoeconomics from a Chinese social perspective, especially for the cost of drugs. Due to this, the panelist did not give a score for other-medical costs. Thirdly, although the EVIDEM contained 13 widely identified criteria, other relevant criteria might have been excluded. Fourthly, members of the committee and drug administration in this study limited representation to only one hospital. Thus, the estimated value of the MCDA needs additional interpretation. The MCDA used for complementing pharmacoeconomic analyses lacks the broader perspective, meanwhile it also lacks the discussion basis of ordering health technology [27, 28]. Lastly, we didn’t test the internal consistency of the result, which took the insufficient of the result. This needs to be improved in future research.

Conclusion

We found that the overall value of five DPP-4 inhibitors was similar. This pilot study assessed the value of the inhibitors and took the value order to help the hospital make decisions regarding drug purchasing using EVIDEM-MCDA.