FormalPara Key Points

European Union regulators highly value the input received from patients and healthcare professionals at their first ever public hearing and dedicated stakeholder meeting on valproate birth defects; however, they have identified crucial gaps in input and research into risk minimisation effectiveness by using a new approach presented here based on implementation science.

Derived proposals for regulators to remedy these gaps aim at catalysing stakeholder agreement and healthcare leadership for the implementation of risk minimisation measures, building up more comprehensive stakeholder input, and collaborating with all stakeholders for monitoring implementation and evaluative research.

Pilotable and practicable actions are presented to enhance the dialogue between regulators, patients, and healthcare professionals and to lead to risk minimisation measures that are implementable in real-world healthcare for patient safety.

1 Introduction

1.1 Objective of Pharmacovigilance

Pharmacovigilance comprises the science and activities relating to the detection, assessment, understanding, and prevention of adverse effects of medicines or any other medicine-related problem [1]. Legal pharmacovigilance obligations apply to regulatory bodies and marketing authorisation holders (MAHs). Measuring the impact of regulatory decisions as well as identifying enablers and barriers for effective risk management are fundamental for the success of regulatory medicinal product-specific risk minimisation measures (RMM). Such RMM are imposed on MAHs, and patients and healthcare professionals (HCPs) are central stakeholders. Effectiveness evaluations of RMM may trigger adjusted or new RMM and thus serve pharmacovigilance quality management [2]. However, the effects of RMM are often poorly quantified because of limitations in data sources and the choice of methods used for evaluation [3,4,5,6]. Considering RMM as planned health interventions calls for formative research [7] into likely RMM effectiveness before regulatory decision making, to find the most adequate RMM. The established product management cycle for ensuring that the benefit-risk balance remains positive can hence be expanded with implementation of RMM in healthcare as part of the cycle, formative research into RMM options and implementability, and feedback from patients and HCPs on experiences with using the medicine and RMM in healthcare for the evaluation step (Fig. 1).

Fig. 1
figure 1

Life cycle management of medicinal products for improving the benefit-risk balance developed from [17] and expanded with implementation of risk minimisation in healthcare, formative research into risk minimisation options and implementability, and feedback from patients and healthcare professionals on using the medicine and risk minimisation in healthcare for evaluation

Full size image

1.2 European Union Initiative for Pharmacovigilance Impact

In the European Union (EU), routine RMM include the authorised product information, and additional RMM may be imposed as needed, such as controlled access, educational materials or pregnancy prevention programmes (PPPs). Monitoring their effectiveness is a legal requirement for MAHs in the context of risk management plans [8]. Considering options for RMM and monitoring outcomes of regulatory actions are within the legal mandate of the EU regulatory network [9, 10]. The Pharmacovigilance Risk Assessment Committee (PRAC) at the European Medicines Agency (EMA) therefore has a ‘Strategy for Measuring the Impact of Pharmacovigilance Activities’ [11], which aims at further developing proactive pharmacovigilance, including through active engagement and capacity building with patient communities and HCP bodies [12, 13].

1.3 Mechanisms of Pharmacovigilance Risk Assessment Committee Engagement with Patients and Healthcare Professionals

Within the existing EMA stakeholder engagement framework [14], the risk assessment process of PRAC contains various options for interacting with patients and HCPs. These consist of written consultations (targeted to organisations and individuals of the EMA’s stakeholder network) and meetings, i.e. dedicated meetings (with invited participants from the EMA’s stakeholder network and expert pool), scientific advisory group (SAG) meetings (with invited HCPs, academics and patients from the EMA’s expert pool) and public hearings (open to all EU citizens and organisations, including MAHs, as well as parties outside the EU). Since PRAC was established in 2012, all these engagement mechanisms have been used. The public hearing [15] is the most recently added tool to obtain input from a wide range of stakeholders.

1.4 Current Uncertainties Regarding Engagement

However, uncertainty remains about the optimal approach for engaging with patients and HCPs. Public hearings held by the US Food and Drug Administration have shown that contributions from patients and HCPs may be qualitatively different from what the data regulators usually review, and some argue that regulators need a specific process for soliciting and integrating the views from patients and HCPs for regulatory decision making [16]. International multi-stakeholder experts have also recognised the need for optimising this integration for strengthening benefit-risk assessment, communication/risk minimisation and evaluation (BRACE) [17].

2 Objectives and Methods

The aim of this study was to further develop active engagement and capacity building with patient communities and HCP bodies regarding effective risk minimisation through the use of a case study with the following objectives:

  1. 1

    Analyse the input from patient and HCP representatives regarding RMM during a public hearing and dedicated meeting as major face-to-face interactions, including comparisons between these two engagement mechanisms and between the two stakeholder groups; and

  2. 2

    Draw, based on the findings of (1), proposals to be considered by PRAC for enhancing their engagement process with patients and HCPs for effective risk minimisation.

The following methods were applied:

2.1 Case Study on the 2017–18 European Union Procedure on the Teratogenic Risk of Valproate

A public hearing was used for the first time by PRAC and followed by a dedicated meeting when assessing the teratogenic risk of valproate in 2017–18. Valproate is an active substance used for treating epilepsy and bipolar disorders as well as, in some EU member states, for preventing migraine headaches. However, it may be teratogenic when taken during pregnancy and cause malformations and lifelong neurodevelopmental harm in the child. The evidence for these effects has been accumulating over time and been subject to assessments at the EU level in 2002–04, 2014 and 2017–18 (see [18, 19] for the references to the evidence assessed by PRAC, i.e. the new committee established in 2012) (see Table 1).

Table 1 Pharmacovigilance Risk Assessment Committee (PRAC) assessments of the teratogenic risk of valproate
Full size table

2.2 Recording Input from Patient and Healthcare Professional Representatives

Stakeholder input to PRAC is provided within public hearings as written submissions and in speech (published as submitted/video recording on the EMA website). Input within dedicated meetings is provided in speech only (reflected in the public PRAC assessment reports on the EMA website). For their decision-making, PRAC uses the written submissions and, for speech input, summaries of responses and discussions prepared by EMA. The case study used the written submissions and the video recording as the records to be analysed for the public hearing and verbatim transcripts (specifically made for the study as exhaustive as possible in handwriting or typing by EMA staff members) of the speech input as records to be analysed for the dedicated meeting.

2.3 Developing a Content Analysis Tool Based on Implementation Theories

For analysing the input from patients and HCPs, a content analysis tool was developed prior to the public hearing and dedicated meeting, and independent from the sets of PRAC questions used at the hearing/meeting (for the PRAC questions, see Appendix 1 of the ESM). Content analyses are qualitative research and can be used to describe phenomena not yet well understood and to be explored by organising and unifying content items by themes [22, 23]. Such studies apply an interpretative approach to content and are concerned with understanding the meanings that persons attach to actions, decisions, beliefs and values as well as with understanding the mental processes persons use to make sense of the world around them [24]. Qualitative research is still rare in regulatory pharmacovigilance contexts but has previously been undertaken within the EU regulatory network when conducting media monitoring and has proven utility for preparing for vaccine safety communication with journalists, members of parliaments and the public [25].

For this case study, the themes were developed with a deductive approach based on existing implementation theories (see Table 2). Implementation science is the study of methods to promote the adoption and integration of evidence-based practices, interventions and policies into routine healthcare and public health settings [26] as complex systems [27]. This includes studying why implementation succeeds or fails [28]. Implementation hence covers all processes from disseminating knowledge and skills to practicing medical care for positive patient and public health outcomes. Implementation theories were chosen as the basis for the content analysis tool because the 2017–18 EU procedure was triggered by concerns that the 2014 RMM were not sufficiently effective and the study therefore assumed that strengthened RMM implementability would be a target outcome of the procedure. It was also considered that implementation is key to RMM effectiveness for any product and that this approach might be generalisable for future application.

Table 2 Implementation theories underpinning the content analysis tool
Full size table

The content analysis tool was finalised with 21 themes in six domains, providing for structure and granularity of the analysis and taking into account that different actors may be responsible for different steps of implementation (for details on the development of the content analysis tool, see Appendix 2 of the ESM). The tool was named the ‘Analysing Stakeholder Safety Engagement Tool’ (ASSET) (see Table 3).

Table 3 Analysing Stakeholder Safety Engagement Tool (ASSET)
Full size table

2.4 Analysing Input from Patient and Healthcare Professional Representatives

2.4.1 Inclusion of Input

Only RMM-related content of the recorded input was analysed. Given the scope of implementation covering all processes from disseminating knowledge and skills to practicing medical care [26], any input from patient and HCP representatives relating to regulatory or healthcare action that may help to reduce harm was considered RMM-related content, which went beyond the regulatory meaning of RMM.

2.4.2 Exclusion of Input

Content not relating to RMM was not analysed, nor the input from the MAH speaker at the public hearing. Body language was also not part of the analysis (body language includes posture, movement, gestures, facial expression, eye contact, and tone of voice [36] and may hence add content or express agreement or disagreement with statements of others).

2.4.3 Content Analysis of Included Input

The content was summarised and organised into a tabulated format of the ASSET. Regulatory terminology was applied consistently in these summaries, but some verbatim quotes were included for illustration (e.g. ‘contraindication’ as the regulatory term and in addition “ban” [in certain clinical circumstances] as a quote). This terminological consistency facilitated the comparisons between stakeholder groups and engagement mechanisms. Additionally, it was analysed as to which implementation-relevant aspects were provided in response to which question, to gain insight into how far PRAC questions provided for focus (e.g. if all representatives provided input categorised into a certain ASSET theme in response to only one question, the question would be considered specific and focused from an implementation perspective).

The analysis was not conducted quantitatively in terms of how much discussion time was taken for each aspect, or how often certain content was repeated, or how many representatives provided similar content or agreed explicitly with a statement made. This methodological choice was made to treat each statement made by a stakeholder as equally important to be heard and considered for the decision making on RMM, keeping it with the qualitative research approach.

2.4.4 Identification of Input Gaps

As part of the content analysis, ASSET themes for which only limited stakeholder input had been obtained or the discussion did not lead to agreement on RMM and convergence on solutions for overcoming potential barriers to RMM implementation were identified as input gaps.

3 Results

3.1 Content Analysis of the Patient and Healthcare Professional Input for the 2017–18 European Union Procedure on Valproate Teratogenicity

The content analysis yielded the following key findings (for details on the analysis, see Table 4 and Appendix 3 of the ESM).

Table 4 Summary results of the content analysis using the Analysing Stakeholder Safety Engagement Tool (ASSET) with identification of gaps of the patient and healthcare professional (HCP) input to the public hearing on 26 September, 2017 (PH) and dedicated stakeholder meeting on 13 October, 2017 (DM) for the 2017/18 European Union procedure on valproate teratogenicity
Full size table

3.1.1 Input by Themes and Input Gaps

  • The input covered all themes relevant to RMM implementation, albeit to a variable extent. Stakeholders detailed severe shortcomings in the implementation of the 2014 RMM with evidence.

  • A wide range of future RMM was proposed, but without presenting much evidence gained for valproate or other medicinal products from previous RMM experiences and evaluations on the capacity to reduce harm when fully implemented. Overall, the favoured RMM objectives were: to limit exposure to valproate in epileptic women of childbearing potential; to reduce risks of birth defects due to valproate exposure in utero; and to facilitate an informed choice for women regarding pregnancy. Many RMM proposals went beyond the regulation of medicines and related to healthcare (theme 1.1). Ethical questions relevant to applying RMM in healthcare were also raised (theme 1.2).

  • A clear call was made to not deprive patients from the benefits of valproate where needed (theme 2.1) and to not jeopardise patient adherence to treatment in these cases (theme 2.2).

  • A number of specific needs for tailoring RMM for patients and HCPs were highlighted to overcome current shortcomings in knowledge, attitudes and practices (themes 3.1 and 3.2) and address patients’ expectations for information and counselling (themes 3.4 and 3.6) and increase HCP skills to communicate with patients and deliver RMM (themes 3.3 and 3.5). Input gaps were left regarding who should develop and provide HCP training on risk management and communication skills (theme 3.3); which information items precisely should be conveyed to patients and how to frame them (theme 3.4); and who should provide leadership for developing and disseminating tailored RMM to various HCP types via all available channels (theme 3.5).

  • Burden for patients associated with RMM, resource constraints and contraception reimbursement policies were discussed as barriers to implementing RMM. However, the discussion did not try to outline RMM that could be bearable for patients (theme 4.3), or how RMM would be compatible with the existing healthcare structures (theme 4.1) and reimbursement policies (theme 4.4). The need for incorporating RMM into clinical guidelines was highlighted, and some, but not all, HCPs voiced commitment for advocating RMM within their respective professional communities (theme 4.2).

  • There was some, but no resolving discussion on how proposed RMM could actually be best delivered to HCPs (theme 5.1) and be integrated into healthcare processes, and who will be responsible for healthcare leadership to ensure behavioural change for the benefit of patients. Only the patient card was discussed in the context of the dispensing process (theme 5.2). Patient and HCP representatives called for RMM to be driven by clear objectives and followed up through auditing and effectiveness measurement. They provided good suggestions for evaluative methods, but some uncertainty remained about how to improve survey methods. A responsible actor for the proposed healthcare audits was not identified (theme 5.3).

  • Timebound requirements were highlighted with a view to an urgent outreach to patients for immediate effectiveness (theme 6.1) as well as to sustainability of RMM effectiveness over time (theme 6.2), but without defining actors and processes to achieve these goals.

3.1.2 Input by Stakeholder Groups

  • Between patient and HCP representatives, there was a remarkable convergence in views and favoured RMM objectives overall.

  • Divergence occurred among HCP representatives in relation to capacity of some RMM for harm reduction (theme 1.1); the informed choice objective questioned by one HCP representative (theme 1.2); the therapeutic place of valproate and the underlying evidence (theme 2.1); and the delivery of RMM to HCPs (theme 5.2) [see theme 1.3 focussing on agreement of HCP representatives overall].

3.1.3 Input in Relation to the Pharmacovigilance Risk Assessment Committee Questions

  • In each EU procedure, questions posed by PRAC to stakeholders are specific to the given medicinal product and safety concern, but at the same time are kept broad to allow stakeholders to elaborate on aspects that may be new to regulators. In the case of valproate, the input did not always match the question asked at the given moment, and most themes were brought up multiple times in response to different questions. This was to be expected during the public hearing where the PRAC questions were high level and posed with the option for stakeholders to address them in an integrated response. The partial disconnect between questions and answers was more unexpected during the dedicated meeting, where the PRAC questions were more specific on RMM implementation to provide structure to the meeting agenda. However, discussing one issue sparked further ideas from stakeholders, and it was the spirit of the dedicated meeting to not interrupt such flows of thought. Furthermore, specific implementation-relevant aspects sometimes matched more than one question. This indicates that the PRAC questions were still rather broad in comparison to the specificity of themes critical to implementation.

3.1.4 Analysis of the Input by Engagement Mechanisms

  • The dedicated meeting was deliberately held after the public hearing to allow more detailed discussions. When comparing these two engagement mechanisms, input was overall similar in terms of the breadth of items and their content.

  • The public hearing yielded more (in the qualitative sense) input on: benefit deprivation due to RMM (theme 2.1); prior knowledge, attitudes, and practice of HCPs and patients (themes 3.1 and 3.2); HCP tool preferences (theme 3.5); delivery of RMM to HCPs (theme 5.1); and immediate implementation (theme 6.1). At the end of the public hearing, there was time left to speak for both speakers and participants who had attended the hearing but had not been a speaker. Their remarks re-enforced or expressed agreement with previous statements and did not contain new content.

  • The dedicated meeting yielded more (in the qualitative sense) input on: patient tool preferences (theme 3.6); patient burden of RMM (theme 4.3); barriers to RMM implementation due to reimbursement issues (theme 4.4); and RMM effectiveness measurement (theme 5.3).

  • These differences between the public hearing and the dedicated meeting may reflect the differences in the questions posed.

3.2 Proposals for Enhancing Patient and Healthcare Professional Engagement for Effective Risk Minimisation

From the key findings, three proposals were drawn for enhancing the PRAC process of engagement with patients and HCPs in relation to RMM and will be presented to PRAC, to be considered for piloting in 2021.

3.2.1 Proposal A—Agreement on Risk Minimisation Measures

3.2.1.1 Content Analysis Finding

The content analysis identified shortcomings in convergence among HCPs on which RMM are appropriate, i.e. capable to reduce harm when fully implemented and ethically acceptable, as well as on some other themes of RMM implementation.

3.2.1.2 Proposal: Agreeing on Appropriate Risk Minimisation Measures with Stakeholders and Catalysing Healthcare Leadership for Implementation

While discussing divergent views on RMM is vital and valuable, the lack of achieving convergence is detrimental, given that implementation science has identified agreement between stakeholders as a key success factor for health interventions [29, 30]. The responsibility for agreeing and deciding on RMM as part of the marketing authorisation lies with the regulatory body. Actions A.1, 2 and 3 are proposed to PRAC for achieving agreement across stakeholders on which RMM are appropriate prior to regulatory decision making and catalysing leadership of HCPs for RMM implementation in healthcare. For a start, this might improve dissemination in terms of consistent and synchronised messages on new RMM from both regulators and HCPs, which would correspond to a need expressed by patients. Further, clinical guidelines might be updated more quickly, and risk communication protocols and training could be introduced for strengthening skills and confidence of HCPs to tackle sensitive communication with patients, corresponding to a need expressed by HCPs. Ultimately, full integration of RMM in healthcare processes is necessary for RMM effectiveness. It must be recognised that agreement of all stakeholders may not always be possible, owing to uncertainties about the given risks or RMM implementation, persisting divergency among stakeholders, or differences in healthcare systems and situations between EU member states. Therefore, transparency towards stakeholders and the public about the agreements reached, but also about dilemmas and causes of divergence might increase the understanding of stakeholders for PRAC decisions and increase the readiness for implementation, even if a stakeholder does not fully agree.

  • Pilot action A.1: Conduct meetings with stakeholders that facilitate creation of common ground, shared problem ownership, and exchange of expertise and result in stakeholder agreement on appropriate RMM

  • Pilot action A.2: Motivate leadership in HCP organisations and obtain their commitment for taking responsibilities for immediate and sustainable RMM implementation in healthcare, including updates of clinical guidelines and HCP training

  • Pilot action A.3: Increase transparency on agreements reached with stakeholders or dilemmas faced by PRAC as part of already published rationales of PRAC decisions on RMM

3.2.2 Proposal B—Build-up Towards Implementable Risk Minimisation Measures

3.2.2.1 Content Analysis Finding

The content analysis demonstrated that although the PRAC questions at the dedicated meeting addressed RMM implementation, stakeholder input left gaps regarding some implementation-relevant themes and how to overcome barriers to implementation. Further, responses relating to certain RMM implementation themes were spread over different questions. Hence, the discussion seemed not to have been focused and covered the complete implementation process from the regulatory beginning to the end in healthcare.

3.2.2.2 Proposal: Building-up Stakeholder Input with a Structured Focus on all Elements Critical to Risk Minimisation Measures Implementation

Build-up of comprehensive stakeholder input on all elements critical to implementation of RMM is proposed for decision making and design of RMM. Actions B.1, 2 and 3 include using and refining the ASSET for the new purpose of soliciting input on the implementability of RMM comprehensively at dedicated meetings. This proposal complements the EMA’s lesson learnt [37] to keep questions simple, focused and limited in number at public hearings.

  • Pilot action B.1: Use ASSET as a checklist for PRAC questions and as a discussion guide for chairpersons of dedicated meetings, to ensure soliciting stakeholder input on all elements critical to RMM implementation

  • Pilot action B.2: Identify through stakeholder discussions barriers and enablers of successful RMM implementation in healthcare

  • Pilot action B.3: Focus discussion outcomes on proactive solutions to overcome foreseeable barriers, integrate RMM in existing healthcare processes and allocate responsibilities for delivery of RMM to HCPs and patients

3.2.3 Proposal C—Collaborative Risk Minimisation Measures Evaluation

3.2.3.1 Content Analysis Finding

Patient and HCP representatives described a lack of implementation of the 2014 RMM and called for an effectiveness evaluation of future RMM. The content analysis also identified a lack of existing research and evidence on the predictable effectiveness in terms of RMM objectives as well as some doubts of stakeholders over the validity of current methods for effectiveness research, in particular surveys.

3.2.3.2 Proposal: Collaborating with all Stakeholders for Monitoring of Implementation and Evaluation of Risk Minimisation Measures Effectiveness

Actions C.1, 2 and 3 are proposed to strengthen implementation monitoring and high-quality RMM effectiveness research in collaboration with all stakeholders. In addition to EMA monitoring of MAH compliance with RMM requirements and results from RMM effectiveness studies imposed on MAHs, real-time feedback on receipt of RMM and further implementation could be obtained from patients and HCPs. For this purpose, PRAC could intensify their ongoing engagement with patient and HCP members of PRAC, the EMA Patients’ and Consumers’ Working Party, the EMA Healthcare Professionals’ Working Party and the wider EMA stakeholder network. Furthermore, studies undertaken by the EMA or academia to investigate intended and unintended RMM outcomes could be planned in collaboration with patient and HCP organisations. More active roles of patients and HCPs in research, such as defining research topics or collecting data, may be developed by applying participatory research approaches proven beneficial in other areas such as health development [38, 39]. For example, research is needed on the effectiveness of pictograms, as patient representatives called for this at the public hearing, but preliminary evidence has raised doubts over potential risks of misinterpretation of teratogenicity pictograms. The quality of RMM effectiveness studies will be supported with EU good pharmacovigilance practices currently under revision by PRAC. A standard to ensure high-quality reporting of such studies, the RIMES statement, has already become available [40] and its use will be encouraged in the revised EU good pharmacovigilance practices.

  • Pilot action C.1: Agree with MAH timetables for their RMM delivery to patients and HCPs and RMM effectiveness studies, and publish timetables and updates from compliance monitoring on the EMA website

  • Pilot action C.2: Investigate together with patient and HCP representatives whether and how RMM has reached them and been implemented in healthcare by means of a methodologically sound survey, focus group, interview and pharmacoepidemiological studies

  • Pilot action C.3: Promote conduct and high-quality research into RMM effectiveness through calls for research and methods development

4 Discussion

4.1 Novel Tool ‘ASSET’

The 2017–18 EU procedure for valproate was triggered by concerns about insufficient implementation of the 2014 RMM, and therefore the ASSET for analysing stakeholder input was developed based on implementation theories and pharmacovigilance experience. It successfully structured stakeholder input to the public hearing and dedicated meeting, identified input gaps and allowed comparisons. The ASSET might have utility for supporting regulatory decision making on implementable RMM (see below). Further, the ASSET is suggested for piloting with a new purpose, i.e. as a checklist and discussion guide for ensuring stakeholder input on all elements of implementation. Future application may lead to revisions of the tool, allowing wider use for different products, safety concerns, jurisdictions and purposes.

4.2 Limitations of the Content Analysis

Only RMM-related stakeholder input was analysed, not other statements such as those on the teratogenic risk itself or the lives of affected children and mothers, and not questions on accountability. This was in line with the study objectives. However, a content analysis of the other input could clarify stakeholder expectations towards governments and inform initiatives for meeting these expectations.

Regarding the analysed input, it cannot be excluded that the real divergence among stakeholders was larger than that found in the analysis. This is because only explicitly voiced or otherwise obvious (e.g. due to other statements) disagreement was recorded; and where no explicit disagreement was voiced, this was interpreted as convergence. Furthermore, body language, which may express agreement or disagreement with the statements of others, was not part of the analysis.

4.3 Validity of the Content Analysis

A content analysis needs to be valid to serve as evidence for policy changes. Validity means that study results truthfully reflect the phenomena studied [41] and is mainly achieved through the reliability of the analytical methods. This refers to how well the analytical method accounts for the information inherent in the data and how sensitive it is to distinctions in the data for correctly representing the meanings of the content [42]. For theory-based studies, reliability is established through a coding scheme adherent to its underlying theory and through adherence of the analysis to the coding rules [43]. In this case study, the ASSET adhered to implementation theories (see Table 2 of Appendix 2 of the ESM). During its development, the draft tool was consulted with two co-authors and reviewed by another co-author because peer examination of research design is a further means to ensure the truth value of qualitative research [44]. The draft tool was then tested, which helped finalise the coding rules through clarifying the precise scope of each theme. Specific attention was paid to adhering to these scopes during the case study. Hence, a range of actions was taken to ensure valid results.

4.4 Utility of the Content Analysis for the 2017–18 European Union Procedure on Valproate Teratogenicity and Predictive Value in Relation to Ethical Aspects

The content analysis of the stakeholder input was started while the 2017–18 EU procedure for valproate was ongoing, and its preliminary results were shared with PRAC in January 2018 prior to their decision making. The preliminary results predicted that ethical aspects (theme 1.2) would become highly relevant for the final discussions and decision making on a PPP: there was overall stakeholder convergence on risk and family planning counselling to support informed choice for women. HCP representatives raised questions about how to act if a woman without an alternative to valproate is or plans to become pregnant, or does not need contraception because a pregnancy is excluded otherwise, or if there are doubts over adherence to a PPP. Most HCP representatives felt that these women, and their unborn children, cannot be deprived from life-saving treatment because of regulatory requirements and that they need to be supported by the healthcare system regardless of their choices. These HCP representatives experienced a dilemma between the medical needs of their patients and wanting to adhere to safe use requirements and regulation. These preliminary results from the content analysis, in addition to the regular records of the stakeholder input (see Sect. 2.2), were appreciated by the PRAC rapporteurs, as the stakeholder feedback was very extensive and listening by any individual can naturally be selective. They stated that the preliminary results reassured them in their overall understanding of the stakeholders’ views and offered an additional perspective on the stakeholder input that broadened their awareness of potential issues.

4.5 Added Value of a Public Hearing for Regulatory Decision Making

The public hearing provided PRAC with multi-perspective insights into problems with real-world healthcare. This was instrumental for PRAC deciding on new RMM, including a new PPP and a visual warning on the teratogenic risk on the outer packaging. A survey and a lessons learnt analysis conducted by EMA reported that the majority of PRAC members said that “they learnt something new” and 79% specifically said that it would “make a difference to the assessment of valproate”. Overall, EMA concluded that the hearing improved the quality of the assessment [37]. In particular, the PRAC rapporteurs were able to confirm that the 2014 RMM were considered by patients as an “excellent toolkit”, but that more needed to be done for the delivery of RMM to patients and HCPs.

4.6 Added Value of a Public Hearing for the Texture of Engagement

Different from written consultations, face-to-face interactions provide a higher degree of what has been named ‘texture’ in a conceptualisation of pharmacovigilance engagement developed for the PRAC Impact Strategy. There, texture describes “the interactive dynamics of what engagement feels like, what it means to people, and how this shapes motivations to engage and change behaviour—based on values, emotions, (mis)trust and rationales” [45]. The EMA survey after the public hearing demonstrated the added value of the hearing in terms of texture. One hundred percent of the stakeholder respondents felt that the hearing was a “positive experience” and 88% felt that it “would make a difference to the Committee recommendations” [37]. An individual patient representative sees in public hearings a major sign of regulatory commitment and trustworthiness and describes them as a chance for patients to make impactful presentations that have visibility to the world (François Houÿez, European Organisation for Rare Diseases [EURORDIS], personal communication, 3 March, 2020). As public hearings are broadcast live by the EMA and the recording remains available on the EMA website, the EMA provides transparency of the texture of the interactions, beyond publishing the written submissions to the public hearing.

Although the case study was not a discourse analysis on the use of language nor a sentiment analysis, the following reflections are offered here regarding the texture of the public hearing on the basis of the content analysis and related observations. The face-to-face interaction allowed for the expression of mutual thanks for the invitations and the input in a personal manner. The chairperson could create a common ground among all stakeholders at the beginning of the hearing by highlighting its significance and objectives. Some patient representatives shared personal experiences of the medical, practical, and emotional impact of the valproate teratogenicity for the affected children and families. Furthermore, individual or others’ complaints were conveyed about not having received the 2014 RMM and delays in risk identification and warnings; questions were raised regarding trustworthiness, responsibilities and MAHs’ accountability; references were made to interactions with parliaments and political contexts; and clear demands were directed at the authorities. Patient representatives brought samples of valproate tablets dispensed in plastic bags without package leaflets or in blister packages without the name of the active substance printed on the blister, and also samples of patient cards, which they described as “small and flimsy”. Seeing such physical samples might have been more impactful than photos. Moreover, there was the opportunity for PRAC to ask for details and confirm understanding with the speakers directly. Some speakers chose to present in their own language if this was not English and have their input translated by a trusted person into English as the working language of the hearing. Regulators verbally expressed empathy and appreciation of the time taken, the altruism and the courage of those affected to share personal experiences in the public domain. They also acknowledged the seriousness of the problem to solve and confirmed to apply the regulatory tools in the best way possible “to get this right”. Although body language was not part of the analysis, postures, facial expressions, and tone of voice of regulators corresponding to attention, seriousness, and empathy were observed by the authors and have probably contributed to the engaging texture of the interaction. Likewise, there was verbal and bodily expression of sympathy and commitment to change from some HCP representatives vis-à-vis the patients.

4.7 Broad Scope of Risk Minimisation in Healthcare

The content analysis highlighted that stakeholders contributed experiences, expectations and suggestions for risk minimisation beyond the remit of regulatory bodies. These related to healthcare, such as family planning counselling, healthcare software and nationwide risk awareness campaigns. Hence, a view on risk minimisation broader than what can be demanded through regulatory action had to be applied in this case study, and the term RMM had to be used with this broader understanding beyond its regulatory definition. Even for RMM in the regulatory sense, their implementation requires dissemination and changes in knowledge, attitude and practice in healthcare, which are not subject to the regulation of medicines. It is unlikely that patients and HCPs can have the in-depth knowledge of medicine regulation for differentiating between the respective remits of regulatory bodies and healthcare supervision. Most importantly, the interactions between patients, HCPs and regulators are meant to be an open dialogue about RMM and their implementation in the context of other measures for patient care. This broader insight may allow regulators to design RMM that are implementable in real-world healthcare.

5 Conclusions

Enhancing engagement between regulators, patients and HCPs is considered one of the most important areas of progress for ensuring a positive impact of pharmacovigilance [46]. This case study analysed the stakeholder input at the first ever EMA public hearing and subsequent dedicated meeting. For this analysis, the novel ASSET was developed on the basis of implementation theories and could be used for further purposes. The analysis identified several important gaps in stakeholder input and research relating to critical elements of RMM implementation. From the results, three proposals with practical actions were derived to be considered by PRAC for piloting, to establish how these actions can most efficiently (i.e. in a prioritised risk-proportionate manner) contribute to improving regulatory stakeholder engagement and RMM decision making. Among the three proposals, achieving stakeholder agreement on RMM and catalysing healthcare leadership for RMM implementation may be the most challenging for regulators because this concerns connecting more with patients and HCPs, their real-world experiences and potential for behavioural change within healthcare systems. Inspired by one of the widely acclaimed principles for being effective in life, “Begin with the end in mind” [47], the approach of applying implementation theories to RMM may be coined as ‘Begin with the implementation-end in mind’. This is expected to enhance mutual engagement, trust building and respective leadership of all stakeholders through jointly envisioning the intended positive RMM outcomes upfront, and then designing together RMM that can change medicine use for patient safety and health.