Introduction

Access to medicines is a multifaceted concept in that it is informed or influenced by different access dimensions, such as the availability of medicines within markets and the affordability of the healthcare system, among others. The Word Health Organisation (WHO) states that access to medicines is achieved when access is affordable and the medicines are safe, of high quality and effective [1]. The European Parliament (EP) has suggested that Europe should “guarantee the right of patients to universal, affordable, effective, safe and timely access to essential and innovative therapies” [2]. Even though better access to medicines might be a shared goal amongst healthcare stakeholders, its achievement has proven complicated. In Europe, a plethora of evidence showcases variability in access to medicines across countries [3,4,5,6,7,8,9,10,11]. These variations can be attributed to a variety of factors: some are associated with broader-level features such as (i) the general country characteristics, including gross domestic product (GDP) per capita and the epidemiological profile; and (ii) the country’s healthcare system characteristics, including healthcare expenditure, organisation of the healthcare system and clinical practices. Others are associated with more specific features such as (iii) the pharmaceutical market characteristics, including regulatory frameworks and the policies medicines undergo to become available and publicly funded in a given market [3]. Regulatory frameworks and policies are of particular interest to policymakers because they are amenable to policy changes. However, they can still be further complicated by the need to find a balance across different perspectives and objectives of involved stakeholders. For instance, whilst healthcare payers are seeking ways to optimise costs and ensure the sustainability of the healthcare system, patients seek timely access to medicines without considering the likely burden on local budgets.

In recent years, health technology assessment (HTA) has become one of the most important stages for efficacious and cost-effective medicines to become available and accessible to patients [12]. HTA recommendations play a crucial role in informing pricing and reimbursement decisions, facilitating negotiations, and updating national clinical guidance on disease treatment protocols, which can further impact the diffusion and uptake of new technologies [13,14,15,16,17,18]. Nowadays, HTA is used across all European countries, at least to some extent [13]. However, discrepancies are seen in the way HTA systems are set-up, the processes that are employed, the way assessment is performed, and the extent to which HTA recommendations inform reimbursement decisions, all of which can have an impact on access to medicines [5, 6, 8,9,10,11, 13, 15, 17, 19,20,21,22,23,24,25,26,27,28,29,30].

Within the European context and to alleviate access inequalities occurring due to variations in the conduct of HTA, numerous efforts have been made at both EU and national levels to harmonise, simplify, and expedite HTA processes [31,32,33]. Furthermore, efforts to establish collaborations between regulatory agencies and HTA bodies, such as parallel review processes and early scientific advice, are taking place to ensure that some alignment exists between what regulators and HTA agencies want, ultimately impacting patients’ access to the right treatment in a timely manner [16, 17, 25, 34]. However, evidence is scarce on what features of HTA, from the way it is set-up within the healthcare system to its role in funding decisions, are more likely to positively impact access to medicines beyond the details of submissions by manufacturers, including the clinical and economic evidence and their respective quality [8, 9, 17, 35,36,37,38]. Additionally, it is not clear whether current efforts aiming to improve HTA systems and processes, such as the harmonisation of clinical assessments through the new EU HTA regulation [31], are welcomed by both Western and Eastern European countries given differences in how well developed HTA processes are. And whether these efforts are considered as successful means to optimise access to innovative medicines by relevant stakeholders. Finally, evidence is scarce on what dimensions of access (e.g., availability, time to patient access, affordability) are targeted and, potentially, improved by different HTA features and components. In a nutshell, there is a gap in the literature on how HTA can be improved in a holistic way (i.e.: from its set-up to its uptake in funding decisions) to facilitate access to medicines across Europe and in light of the implementation of the new EU HTA regulation [31].

In addressing the above gaps, the objectives of this study are twofold: First, to explore how can HTA be improved to optimise access to medicines. And second, to assess levels of agreement between stakeholders from different geographic jurisdictions and/or different stakeholder groups on what features of HTA are more likely to have the most positive impact on access. To engage and elicit the views of European stakeholders, a Delphi exercise was conducted to develop an expert-based judgment [39]. Contrary to simple surveys and interviews, the Delphi method structures and organises group communications while allowing for controlled feedback [40,41,42].

While there are studies in the literature which use the Delphi method to elicit opinions on subjects such as value assessment of medical devices [43, 44], population health [45] and digital health technologies [46], to our knowledge there is only one study similar to ours in remit. This study explores how HTA for medicines can be improved across Europe, but with a different focus on the value assessment of oncology and haematology products, and the recent EU HTA regulation [33]. In our study, we aimed to validate HTA features that existing studies found to have an impact on access to medicines, and explored how a better understanding of these features through expert views can help improve HTA at national, regional and supranational levels in a holistic way (i.e.: from its set-up to its uptake in funding decisions) in order to facilitate access to clinically- and cost- effective medicines.

Methods

Both primary and secondary evidence were used. Secondary data collection was conducted through a scoping review of the literature to identify, first, a list of HTA features that have shown to have an impact on access or features that could be improved. And second, to identify relevant access dimensions. Primary evidence was collected through a web-based Delphi panel in European stakeholders from both Western and Eastern European countries to validate the findings of the literature.

Scoping review: HTA features and access dimensions

A scoping review was selected over a systematic literature review, as the scope of our search and the inclusion criteria were broader than the ones usually used in a systematic literature review. Generally, scoping reviews can help identify and map available evidence that is still unclear and cannot yet be addressed through a more precise systematic review [47].

HTA features

To identify recent peer-reviewed literature on HTA features and areas for improvement, we searched the MEDLINE via the PubMed database from January 2011 to December 2021 using the keywords (’health technology assessment’ OR ‘HTA’ OR ‘value assessment’) AND ‘Europe’. A detailed description of the scoping review strategy including the screening process and the exclusion and inclusion criteria used is outlined in detail in Appendix 1. The titles and abstracts of the resulting papers were screened by the first two authors in a double-blind fashion. Any disputes were resolved between first two authors. Papers considered relevant to our study objectives were downloaded and screened by the first author. An additional search was conducted by the first author on the websites of the European Commission and EUnetHTA to identify relevant grey literature using ‘Health technology assessment’ OR ‘HTA’ as key terms. Reports published from 2017 and onwards were included to capture recent developments and the current landscape of HTA in Europe. Figure 1 outlines the different steps and respective search results of the scoping review.

Fig. 1
figure 1

Flow diagram of the scoping review process

Full size image

Relevant evidence was recorded and grouped into four main categories/endpoints, following an iterative process. The identified HTA features and components related to: (i) HTA system set-up; (ii) HTA procedures; (iii) HTA evaluation processes; and (iv) HTA and funding. An additional endpoint was created to record evidence on the access dimensions used in the relevant studies. The results of the scoping review on HTA features are summarised in Appendix 2. Table 1 presents the list of HTA features considered relevant in having an impact on access to medicines in the European region (Table 1).

Table 1 Features related to HTA as shown in the web Delphi panel
Full size table

Access dimensions

To provide a comprehensive definition of access to medicines, the different dimensions of access used in the resulting papers of the scoping review (described above) were explored, when available. Additional searches were conducted on the websites of international organisations such as the WHO, the United Nations and the European Commission, using the key term “access to medicines” OR “patient access” OR “access”.

Three relevant dimensions of access were identified and included in this study. The dimensions and definitions of access are used for the sole purpose of this study are as follows:

  • Availability of medicines: whether clinically- and cost–effective medicines are available and marketed in a given market;

  • Time to patient access (timeliness): the timely access of patients to publicly reimbursed medicines, and;

  • Affordability: whether the prices of clinically- and cost- effective medicines are in line with the purchasing ability of healthcare systems and of patients.

The Delphi process

The Delphi method can be used to fulfil a variety of research objectives such as reaching participant consensus on a complex topic, prioritisation of policies, and generation of debate among participants who might not share a common vision [48, 49]. The Delphi method can also be used when current knowledge is incomplete, uncertain or lacking [50]. During a series of rounds (surveys), panel participants can first respond to a set of questions and, in subsequent rounds, are given the opportunity to re-consider and re-assess their initial opinions after seeing the aggregate responses of other participants [40, 45, 48, 51,52,53,54,55]. Hence, the Delphi method is an iterative process that avoids intentional and unintentional noise, such as irrelevant and non-productive communication among the participants [42, 48]. Panel responses are always anonymous allowing participants to express their opinion freely without introducing potential bias due to peer pressure or the presence of potentially dominant or more vocal experts [40, 45, 48, 51,52,53,54,55,56].

No set minimum or maximum participant number for Delphi panels exists, with Delphi panels being conducted from five up to thousands of participants [39, 50, 57,58,59,60,61,62]; an appropriate number is most likely dictated by the objectives and nature of the research, though methodological advice, and Delphi panels in practice, often range between 10 and 20 participants [54, 60,61,62,63].

Even though Delphi panels may usually include three or more rounds to reach consensus amongst participants, in this study we deemed that two rounds were sufficient to ensure desirable completion rates, in line with other studies in the literature [39, 57, 58, 64]. This is because we had already compiled a list of HTA features likely to have an impact on access, thus an initial round soliciting experts’ opinions was deemed unnecessary.

Stakeholder sample

A list of stakeholders was compiled from the authors’ network, considering their knowledge and areas of expertise, country of origin/work, and affiliation. Overall, our sample followed a purposive and snowball sampling strategy targeting experts in HTA from all European Union Member States, Norway, Switzerland, and the United Kingdom. Invited experts (n = 128) were either from academic or health policy research institutions, the pharmaceutical industry, decision-making/payer bodies, or patient organisations to capture the views of relevant stakeholders. To ensure a representative sample of European stakeholders, we invited a minimum of four experts, one of each stakeholder group, across all study countries. A limitation of this study is that healthcare professionals were not included in the sample as the authors were unable to identify clinicians that were familiar with and/or involved in HTA through either their network or the sampling strategies used.

Study design and administration

The survey was piloted with five health economists from our institution to reflect on the structure and content prior to dissemination to external participants.

All stakeholders were invited via a personal email sent by the authors inviting them to participate in a two-round Delphi panel. Experts who indicated they were unable to participate were asked to identify a team member or colleague with similar expertise as a replacement. Where an alternate expert was identified, the original invitee was asked to provide the name, email and job title of their suggested colleague to ensure that their expertise was relevant to the research objectives of this study.

The study utilised a web platform, Welphi®, for the Delphi process. The platform ensures all experts received an automated email with a unique URL link. Participation was anonymised by Welphi® and each participant had a unique identifier containing an alphanumeric string (e.g.: 079AB). These identifiers allowed the authors to track whether the same individual participated in both Delphi rounds. Each round remained open for a month to accommodate schedules and availability. Automated reminders were sent every week to participants who had not started the survey and participants who had not yet completed their responses.

Participants were requested to complete an informed consent form to be able to continue with the Delphi process. All participants were asked to respond to demographic questions including the country they live and work in, their organisation affiliation, and their perspective selected from a list of pre-defined categories: research and policy, patient/patient organisation, industry, or decision-maker/payer. Participants were given clear definitions of all three access dimensions, and were able to rank their agreement using a five-point Likert scale (‘strongly agree’ (SA), agree’ (A), ‘neither agree nor disagree’, ‘disagree’ (D), ‘strongly disagree’ (SD)) on the positive impact of the HTA features on the three access dimensions. To ensure reliability of the panel’s outcomes, participants were given the option to select ‘do not know’ for instances where they did not feel confident about their response and a ‘not applicable’ option was also given to allow participants to indicate HTA features they felt might not be relevant to an access dimension. A single, open-ended question was available to the participants in the first round only to provide the opportunity to add any factor or HTA feature that, in their opinion, might have a positive impact on access and was not identified through our scoping review. However, these responses were used only as contextual information and were not included as statements in the second and final round of the Delphi panel for two reasons: first, the objective of the study was to validate the results of the scoping review and; second, if these new statements would have been included in the second round, the participants’ ability to engage with the statements would have been limited as they would have not been able to see the aggregate responses of the participants and potentially revise or keep their initial responses in an additional round, which is a main feature of the Delphi method.

In round 2, participants were asked to rank again the value statements. In this round, participants were able to see the aggregate responses of all the participants from round 1 as percentages. Participants had the option to revise or keep their initial responses from round 1. The study received ethics approval by our institution.

Data analysis

The analytical methods employed were chosen considering the ordinal scale nature of our data, our study objectives and the results of a thorough search of the literature on Delphi panel methodologies [48, 51, 58, 59, 65,66,67] and other studies using the Delphi method [45, 60, 68,69,70,71,72,73,74,75,76,77]. Quantitative methods were used, including both descriptive and inferential statistics, to explore (i) what features of HTA had the most positive impact against different access dimensions in the final round; (ii) the level of agreement between stakeholders about the impact and rank of different HTA features across access dimensions in both rounds, and (iii) how stable their responses were across rounds. The open-ended responses provided by the participants in the first round were used only as contextual information and were excluded from the data analysis.

Different measures and methods were used to explore the aforementioned points which are outlined in detail in Table 2. For points (i) and (iii), additional analyses were performed using more than one commonly used method to validate the robustness of our results, recognising that there is limited to no evidence on which exact method is the most suitable to use in specific circumstances, or how results can change when using different methodologies. All analysis was conducted for 39 value statements (13 HTA features across the three access dimensions).

Table 2 Summary of definitions and methods used in this study
Full size table

Strongly agree (SA) and agree (A) and strongly disagree (SD) and disagree (D) responses were grouped, respectively, for the percentage agreement analysis. Median and interquartile ranges, rather than mean and standard deviation, were used for measuring central tendency and level of dispersion to avoid skewed results due to outliers. Gwet’s kappa coefficient was selected to test inter-rater agreement on each round over other kappa coefficients as it allows for multiple participants and any level of measurement by applying relevant weights for the ordinal scale, and missing values due to the selection of the ‘do not know’ or ‘not applicable’ options [78,79,80,81]. The ‘do not know’ and ‘not applicable’ responses were excluded from the quantitative analysis to limit analysis of agreement to participants who were confident in their responses.

Finally, since consensus is a term poorly and ambiguously defined in the literature [48] while its measurement greatly varies across studies [48, 50, 82], in this study, we differentiate between agreement and consensus. For consensus, stricter criteria were applied compared to group agreement to avoid inconclusive results. However, given that consensus is based on subjective criteria, it was only used for discussion purposes. All analyses were conducted in Stata SE 16.1 and SPSS Version 27.

Results

Participation rate

A total of 128 participants across Europe were approached for involvement in the Delphi panel. Of these, 27 participants from 16 European countries took part in round 1. From the 27 participants in round 1, 19 participants from 14 countries completed round 2. Figure 2 illustrates the characteristics of the stakeholders from round 2.

Fig. 2
figure 2

Expertise and geographic setting for participants in round 2

Full size image

Delphi panel results

We present the results of all statistical analyses across 39 value dimensions (13 HTA features across three access dimensions). Appendix 3 provides the results for both rounds.

Agreement

Group agreement on value statements with the most positive impact on access dimensions in round 2

Table 3 summarises the group agreement on the positive impact of each HTA feature on each access dimension in round 2.

Table 3 Results of agreement in round 2 using percentage agreement, central tendency and median, and inter-rater agreement
Full size table

Percentage agreement: From a total of 39 value statements in round 2, 18 (46.2%) were approved by qualified majority (i.e.: SA + A > 75%), including one statement (‘harmonisation of rules for HTA methodologies, evidentiary requirements, and procedures across HTA bodies and systems at supranational level’ on availability of clinically- and cost-effective medicines) which was approved by absolute majority (SA > 50% and SD + D < 33.3%). No value statement was rejected by absolute majority (SD + D > 50%), showing that there was no HTA feature in our list that many participants felt that it cannot have a positive impact on access.


Access dimensions: Most HTA features were found to have a positive impact on time to patient access (9 out of 13 HTA features). Seven HTA features were considered to have a positive impact on availability of medicines within markets while only two features were believed to have a positive impact on affordability for patients and healthcare systems.


HTA features: One HTA feature, ‘scientific advice provided to manufacturers by HTA bodies ahead of the initiation of the HTA process’, was considered to have a positive impact by qualified majority across all three access dimensions (89% on availability and 79% on both timeliness and affordability, respectively). ‘Reliance on real-world (RWE) evidence in cases of limited clinical data’ was the only HTA feature that all stakeholders (100%) believed to have a positive impact on time to patient access.


Central tendency and level of dispersion: 22 (out of 39) value statements reached agreement across participants, with a median of 1 or 2 and IQR ≤ 1 (56.4%) in round 2.


Access dimensions: Participants agreed (median:2 and IQR ≤ 1) that most HTA features had a positive impact on time to patient access (10 out of 13 HTA features with a median of 1 and 2 and IQR ≤ 1), while six HTA features resulted in agreement on their positive impact on availability. Six HTA features were found to have a positive impact on affordability.


HTA features: Participants strongly agreed (median:1; IQR:1) on the positive impact on availability of ‘harmonisation of rules for HTA methodologies, evidentiary requirements, and procedures across HTA bodies and systems’.

Factors not captured by the scoping review that might have an impact on access as suggested by participants in round 1

Open-ended question: Only three participants provided factors that could potentially have an impact on access not identified through the scoping review through a response to the open-ended question in round 1. These suggested factors included: (i) choosing a cost-effectiveness approach rather than comparative clinical benefit assessment; (ii) having pre-defined criteria for which stakeholders should be involved during HTA processes (for impact on availability, not necessarily time to access), and; (iii) having a linkage between horizon scanning, budgeting and HTA. These statements were not validated by the Delphi participants in the second round.

Overall group agreement per value statement in rounds 1 and 2

Inter-rater reliability (IRR), Gwet’s kappa coefficient: In round 1, low levels of agreement were observed across participants. Participants had fair or moderate agreement for 30.8% (12 out of 39) and 38.5% (15 out of 39) value statements, respectively. Substantial agreement was reached in only 15.4% (6 out of 39) of value statements, two of which had been approved by qualified majority and reached agreement through central tendency and low level of dispersion in round 1. There was no value statement with almost perfect agreement.

In round 2, agreement levels changed; 33.3% of value statements (13 of 39) resulted in substantial agreement and one value statement (on the positive impact of ‘establishing procedures to deal with clinical and economic uncertainties’ on availability) reached an almost perfect agreement. Six of the value statements showcasing substantial agreement amongst participants and the one value statement with almost perfect agreement were also approved by qualified majority and showcased a median of 1 or 2 with IQR ≤ 1 in round 2.

Table 3 summarises the overall group agreement per value statement in round 2.

Stability and consistency of responses between rounds

Non-parametric Wilcoxon matched-pairs signed-ranks test: 94.9% of the value statements were stable between rounds (i.e.: not significantly changed). Only two value dimensions had a p-value less than 0.05 which indicated that they were statistically significant, thus unstable: these two were the positive impact of ‘agreed timelines for the conduct of HTA processes’ on time to patient access, and the positive impact on the ‘use of established procedures to handle uncertainty’ on affordability.


Spearman's rank-order correlation coefficient (Spearman's rho): Participants’ opinions had a statistically significant high degree of concordance in 69.2% (27 out of 39) of the value statements.

Table 4 presents the results of stability between rounds 1 and 2.

Table 4 Results of stability between rounds 1 and 2
Full size table

Discussion

Using the Delphi method, we explored how HTA systems, procedures and processes can be improved to optimise access to medicines by canvassing opinions and perspectives of European HTA experts. Our results have several implications for both the HTA features and the access dimensions. However, they should be interpreted with caution due to the inherit limitations of the Delphi method, such as low participation and high dropout rates. In our study, a small number of experts participated in both rounds, and responses were predominately received from research and policy makers, with no opinions from healthcare professionals and decision-makers captured.

With regards to HTA features, 11 out of the 13 showed a positive impact on at least one of the three access dimensions suggesting that participants’ views are broadly aligned with current efforts and discussions on how HTA can be designed or adjusted at regional, national and supranational levels to optimise access to medicines. ‘Early scientific advice’ and ‘clarity in evidentiary requirements’ reached consensus on their positive impact on all access dimensions. Interestingly, even though many well-established HTA bodies in Europe currently provide early scientific advice to manufacturers and have published guidelines for evidence requirements, a call to action for some HTA bodies to (i) emphasise more the provision of early support to manufacturers before HTA initiation, (ii) provide more clarity on the evidence required for evaluation, and (iii) be more transparent and systematic on the way they deal with uncertainty if it arises, was identified.

‘Established ways to deal with potential uncertainty occurring during HTA assessments’ reached consensus on its positive impact on both availability and time to patient access. This HTA feature was also identified by a recent study [33] which highlighted that the management of uncertainty is one of the challenges that need to be addressed to provide an ‘additional benefit’ to a European HTA process.

‘Reliance on RWE in HTA’ reached 100% agreement among participants in the second round on its positive impact on timeliness, emphasising the importance of the use of new types of evidence beyond strict clinical studies which do not test for the clinical benefit of a medicine in a real-world setting. This has been extensively discussed across Europe, especially at regulatory and HTA levels for instances where clinical evidence might still be incomplete or of low quality. However, the use of RWE varies across countries with some HTA bodies accepting RWE while others do not [85], and with other access implications arising due to a lack of systematic ways to collect, interpret and use these data during assessments [86].

Looking at the results of all the analytical methods used and recognising that different methods can lead to different conclusions, we can only conclude confidently (SA + A > 75%, median:1 or 2 and IQR ≤ 1, substantial or almost perfect agreement and high degree of concordance and stable responses between rounds) that participants agreed on the positive impact of ‘reliance on RWE’ on availability of medicines and of ‘provision of scientific advice’ on both availability and affordability and; of ‘clarity of evidentiary requirements for value assessment’ on timeliness.

Table 5 summarises the HTA features with the most positive impact on the respective access dimensions.

Table 5 HTA features with the most positive impact on access dimensions
Full size table

Across the grouping of HTA features presented in Table 1, all features targeting evaluation processes reached consensus on their positive impact on at least two access dimensions: participants agreed access to medicines could be ameliorated by having clear guidance on what evidence is required, on ways to deal with uncertainty, and on the incorporation of additional dimensions of value beyond clinical and cost-effectiveness, together with general coordination and harmonisation of evaluation processes at regional, national and supernational levels. The new HTA regulation of the European Commission on joint clinical assessments across European Member States, to be officially implemented in 2025, aims to address access issues arising due to discrepancies in the evaluation processes of national/regional HTA bodies. The importance of this is also reiterated in our findings, as ‘harmonisation of rules for HTA methodologies, evidentiary requirements, and evaluation procedures across HTA bodies and systems at supranational level’ was approved by absolute majority in both rounds for its positive impact on the availability of medicines and reached consensus on its positive impact on availability and time to patient access. Therefore, standardising HTA evaluation processes and creating coherent and consistent scientific evidence collection, generation and interpretation across Europe could achieve better and more controlled access to medicines within countries. On the other hand, HTA features related to procedures and set-up reached consensus mainly on their positive impact on time to patient access and affordability, rather than the availability of medicines. As both of these are more relevant to the specificities of each setting, they should still remain a country competence taking into account country-specific characteristics, objectives and values and further reflect the way the healthcare system is organised [13, 16, 34].

With regards to the access dimensions, Delphi participants believed that the included HTA features mostly had a positive impact on timely access to publicly funded medicines which is in line with broader HTA objectives as a tool informing reimbursement decisions within nations or healthcare systems to streamline national/regional accessibility to medicines after receiving marketing authorisation. However, a number of concerns have been raised previously that HTA processes can hinder timeliness due to assessment delays and the presence of an additional regulatory step to medicines’ availability within markets [8, 87, 88]. More HTA features were expected to have a positive impact on affordability of the healthcare system, as HTA processes are implemented in an effort to allocate resources efficiently considering evidence-based information, the sustainability of the healthcare system, and the finite budgets available. On the contrary, HTA features with the lowest percentage agreement on their positive impact were identified on the affordability dimension. And interestingly, ‘legally binding HTA recommendations for reimbursement decisions and/or negotiations’ did not reach agreement or consensus amongst participants in round 2, in having a favourable effect on affordability, even though required translation of HTA recommendations into funding would mean that the most cost-effective medicine would be covered using publicly available budgets.

Our findings on affordability, however, are not conclusive because of the lack of representation of decision-makers/payers in our sample. Yet, these findings can still observe what other HTA experts believe: For instance, the ‘presence of an independent body’ reached consensus on its positive impact on affordability (and not on any other access dimension), highlighting that transparency and conflict of interest concerns may remain when HTA processes are integrated within national/regional healthcare payers/-decision-makers opposed to taking place independently at arm’s length [13]. Therefore, more transparency might be needed to better understand how HTA recommendations are used during negotiations and price setting within jurisdictions. However, this may only apply in some cases, as HTA systems for medicines integrated to governmental institutions are rarely seen in Europe [13].

Overall, even though HTA is an essential instrument to streamline and monitor access to medicines across settings, it is important to highlight that any action to achieve better and faster patient access should be complemented by other appropriate and effective regulatory policies and procedures, which are equally important. Targeted efforts and interventions in HTA alone will not necessarily translate to better patient access without adjustments in other areas: for example, if reimbursement policies are not adjusted to align to, or at least take into consideration, HTA recommendations which promote the most cost-effective therapeutic option. Not only should each stage of the access pathway aim to maximise the effects on improving access, it may also benefit from synergies between these stages. For instance, the ‘introduction of parallel review processes’ reached consensus on its positive impact on time to patient access, highlighting that collaboration between marketing authorisation and HTA bodies could improve timeliness.

Study limitations

Our study is not without limitations. First, our results should be interpreted with caution due to the small sample size caused by low participation and high dropout rates, limited or lack of representation of some stakeholder groups (i.e., healthcare professionals and decision-makers), and limited geographical representation. Additionally, participant representation which was skewed towards policy and research experts could have introduced bias in our results. Nevertheless, the findings of this study can still be considered informative in (i) identifying how different HTA features target different access dimensions, (ii) understanding (dis-) agreement on whether current efforts to improve HTA are successful according to experts from different geographic settings, and (iii) identifying areas of HTA that might need improvement, as long as, this limitation is acknowledged in the interpretation of these three conclusions. Second, while a scoping review was conducted to create a list of HTA features that might have an impact on access, this list may not be exhaustive. To address this, participants had the opportunity to respond to an open-ended question in round 1 to share additional HTA features that might have not been included in our list. Third, our Delphi panel included two rounds rather than three rounds. However, we deemed that two rounds were sufficient as we had already conducted a scoping review and compiled a list of HTA features that were likely to have an impact on access. Finally, there are numerous definitions for agreement, stability and consensus in the literature, which are often unclear, and each of these can rely on several different methodologies for results analysis. To address this, our study defined the relevant terms in detail and conducted analysis using more than one method, when applicable and appropriate.

Conclusion

Using the Delphi method, this study found that improved HTA processes and procedures were shown to have a predominantly positive impact on timeliness, and a less clear impact on affordability despite HTA’s remit to ensure efficient allocation of finite resources. The most positive impact on all three access dimensions was seen on HTA features related to more clear, consistent and harmonised evaluation processes within and across countries, which is in line with current European efforts targeting the harmonisation of clinical assessment processes. Even though our results might not be conclusive, they reiterate the following overarching themes: increased transparency during HTA and decision-making processes is essential, use of and reliance on RWE can optimise availability of medicines, while better collaborations between regulatory institutions within and between countries are paramount for better access to medicines.