Introduction and objectives

Engagement of stakeholders is emphasised by researchers and policymakers as a fundamental tool for enhancing pharmacovigilance processes [1,2,3,4,5,6,7]. Engaging patients and other medicines users (MUs), and healthcare professionals (HCPs)—as the stakeholders closest to the prescribing and use of medicines—can support pharmacovigilance systems [8, 9] across various settings, e.g. regulatory bodies, pharmaceutical companies and healthcare. This has been argued for the European case [10], the USA [1] and more internationally via CIOMS [8]. In practice, pharmacovigilance is composed of structured processes, including aggregating adverse drug reaction (ADR) case reports [1, 11,12,13,14,15], reviewing pharmacoepidemiological and other study data, identifying and assessing new risks and making decisions [11, 16,17,18,19,20,21] on risk management strategies [22] and their implementation in healthcare [23,24,25,26]. While engagement is increasingly pursued in practice [27, 28] and receives global attention [8], empirical research [16, 29] into pharmacovigilance engagement remains in its infancy [7]. Conceptual precision (construct validity) is vital for the internal validity of research [30]. An explicit and common definition of engagement is therefore necessary if a body of empirically grounded studies is to emerge which can be built upon, critiqued and refined [31, 32]—informing engagement interventions more generally [33].

Among the many academic articles referring to engagement in relation to pharmacovigilance and risk with medicines, very few explicitly state what engagement means. This has been evidenced by preparatory work for this article by reviewing publications in Drug Safety, one of the leading journals on pharmacovigilance. Between 2010 and 2017, more than 40 articles in this journal refer to ‘engagement’, either in depth or in passing (see Annex 1 for a summary). Often the terms ‘engagement’, ‘participation’ or ‘involvement’ are used interchangeably and ambiguously in recent literature, with their meaning seemingly self-evident. Despite calls to develop evidence based engagement [1, 26], the current lack of clear conceptualisations and a definition of engagement in the literature is a glaring impediment to valid measurement and analysis.

This conceptual ambiguity is the starting point for this article which has three objectives, to:

  1. (1)

    Develop a widely applicable conceptualisation of engagement;

  2. (2)

    Consider various methodological challenges to researching engagement alongside solutions; and

  3. (3)

    Outline a basis for converting our conceptualisation into specific measures and indicators (operationalisation [30]) of engagement among pharmacovigilance stakeholders.

Conceptualising and defining engagement

Amid the general lack of definitions, some articles regarding risk management of medicinal products [22, 34] nevertheless discuss the quantity and quality of (patient) engagement, involvement or participation, or add understandings of ‘activation’ and ‘empowerment’. The authors of one more sophisticated empirical study have gone further to consider engagement for pharmacovigilance via a model of ‘motivation, incentives, activation and behaviour’ (MIAB) [29]. The precise relationship between MIAB and engagement is not specified, though the original social science study on which these authors draw refers to MIAB as an ‘engagement mechanism’ [35]. This raises questions, however, as to whether engagement should be seen as an independent variable that may lead to behavioural outcomes (e.g. changes in prescribing or using medicines), or whether certain behaviours (e.g. prescribing or using medicines or ADR reporting) are intrinsic parts of engagement itself.

One solution to such conceptual questions is to look to existing work regarding patient engagement in the wider field of health and medicine [33, 36]. This explores how to capture the effectiveness of ‘engagement networks’ [37], or how qualitative research among patients, combined with patient-reported outcomes, can inform our understandings of which risks are relevant for MUs [38]. Moreover, insights from the social sciences can greatly enhance the risk management of medicinal products [26, 34]. Studies of risk governance—how institutions manage risk together with different stakeholders [39]—and of public understandings of science have developed expansive literatures on engagement regarding (novel) technologies and their risks and benefits [40,41,42,43,44,45,46,47].

Social scientists typically discuss engagement in normative, instrumental or substantive terms [44]: ‘normative’ refers to the ethical rightness of engaging different publics; from a more practical perspective, ‘instrumental’ approaches consider engagement as a means towards legitimate and effective governance; ‘substantive’ approaches explore engaging different stakeholders or acquiring input for better understandings of risks and risk perceptions [44, 45]. These approaches usefully illuminate existing engagement activities in pharmacovigilance and risk minimisation, which are often beset by limitations in the impact of, and compliance with, safe use and risk minimisation advice, alongside underreporting of ADRs. Interrelated challenges of motivation, legitimacy [1, 16, 48,49,50] and accuracy of risk management–related processes [9, 11, 17, 18, 20] are central here.

Influential studies of risk governance have gone further to show how the instrumental aspects and substantive aspects of engagement relate to one another [46]. Including more diverse stakeholders and risk perspectives may enhance legitimacy and outcomes among stakeholder groups [44,45,46,47], but this depends on the context. Where hazards of technology are able to be modelled probabilistically via relatively straightforward linear models [51]—as may be the case where an ADR is pharmacologically predictable and dose-dependent [15]—then a more limited form of engagement may usually suffice [46]. However, in contexts where there is more uncertainty, complexity (involving causal sequences) or ambiguity (where there may be multiple values and definitions of an ‘unwanted outcome’), deeper levels of participatory engagement are required to enhance knowledge, legitimacy and thus impact [44, 46]. Examples of these latter contexts include actions to minimise teratogenic risks of medicines [52].

These key distinctions made in the risk governance literature between uncertainty, complexity and ambiguity of risk, and the related requirements for different interventions for engagement [46] suggest two key lessons for regulators: first, differentiating engagement interventions is necessary and important for effectively fulfilling the legal mandates of a regulatory authority; second, pursuing deeper levels of inclusive engagement has important implications for who governs risk governance (greater inclusion of MUs, for example, would, in full consequence, also mean their greater input into shaping priorities regarding which risks to assess and which risk minimisation goals to set [45]).

Responding to these lessons from existing work and returning to influential studies within the public understandings of science literature [42], we can conceptualise different models or depths of engagement interventions—information, consultation and participation (see Table 1), and their respective implications. Examples from health promotion research consider a similar spectrum of ‘community engagement’ ranging from outreach, consultation, involvement to collaboration and ‘shared leadership’ [55].

Table 1 Different depths of engagement interventions*
Full size table

This basic schema assists us descriptively in categorising engagement interventions. The schema is also useful analytically—in illuminating inherent weaknesses in many existing attempts at pharmacovigilance engagement. For example, tensions often exist between a ‘participation’ model—whereby MUs and HCPs are expected to also take the initiative and maintain participation over prolonged periods—and features of a consultation model—where there are strict confines around the scope of what can be said, with limited feedback and a lack of discussion. Such tensions are, for example, present in many ADR reporting systems which usually provide very limited, often only automised or generic feedback to reporters and are not experienced as two-way participation by the reporter. Another example consists of the warnings and safe use advice added to the product information and supported by regulators’ websites. These are rarely disseminated on the basis of a strategic plan agreed with all stakeholders. The recognition of such tensions helps us understand the shortcomings in the effectiveness of these engagement interventions. More successful engagement, as seen within some empirical studies, can be understood in terms of a fuller and more consistent embracing of participatory models [29, 47].

Glimpses of such dynamic, interactive two-way approaches are apparent among interventions by some regulators, where various efforts to involve MUs and HCPs, both directly and via their respective organisations, reflect a more participatory approach [27] (Note 1). Each of these approaches aims, explicitly or implicitly, at different depths of engagement and this will also vary in practice across different stakeholders.

Campaigns by national bodies, for example to increase ADR reporting or appropriate prescribing, are also engagement interventions. Evidence on the depth, dynamics and behavioural impact of engagement is so far available from a small number of pharmacovigilance studies: for example, HCPs describe themselves as being more likely to report ADRs within a therapeutic advice service where they could receive advice in return [56]; or the meaning and motivations of engagement, not least in relation to community belonging, positive feedback (from an online platform) and related feelings of altruism, were deemed fundamental for MUs’ usage of safety information and ADR reporting [29, 57]. Here we see not only that the depth of engagement is pertinent but, moreover, the interactive dynamics, emotions and meaningful experience—the texture [58]—of engagement processes are also fundamental to their effectiveness [47]. Texture thus relates to what engagement feels like, what it means to people and how this shapes motivations based on values, emotions, trust and rationales.

Drawing on these multiple insights from literatures on risk governance and public understandings of science, as well as from empirical and policy articles in the pharmacovigilance field, we define and conceptualise engagement in pharmacovigilance along three dimensions (see Table 2).

Table 2 Conceptualisation and definition of engagement in pharmacovigilance
Full size table

Above we raised the question—should engagement be seen as an independent variable which may lead to behavioural outcomes or are behaviours such as, prescribing, medicines use and ADR reporting intrinsic parts of engagement itself? The conceptual framework we have developed above leads us to distinguish between engagement interventions—information, consultation and/or participation, as outlined in Table 1—and engagement outcomes, such as (changing) behaviours. We thus see engagement interventions as part of the pharmacovigilance processes and their outcomes as manifestations of the design of engagement intervention (see Fig. 1).

Fig. 1
figure 1

Conceptualising engagement in breadth, depth and texture

Full size image

For the pharmacovigilance processes run by regulatory bodies, there are two critical entry points for input from MUs and HCPs during the continuous benefit-risk assessment of a medicinal product throughout its life-cycle: First, MUs and HCPs can contribute to the ‘life-cycle management’ step of generating scientific evidence about risks and risk management, e.g. with ADR reports or results from PASS (including surveys on knowledge, attitudes and practices regarding risk minimisation), or other real-world data and research into healthcare practices. Second, MUs and HCPs can contribute to decision-making on product-related action and policy, assisting the integrating of evidence with considerations of MU perspectives and preferences [22]. Engagement interventions can enhance this input (through knowledge and legitimation), and this can be seen as an outcome of engagement.

If, as noted above, deeper engagement is a two-way process, then we must also question whether there is one engager or multiple engagers. For while the legal responsibility for engagement activities in regulatory pharmacovigilance lies with the regulatory body, MUs and HCPs can take the initiative through advocacy and can also conduct their own engagement interventions, e.g. a campaign for ADR reporting. Likewise, the outcomes of engagement will relate to all stakeholders involved, including those initiating the engagement.

Measuring engagement and its impact along the dimensions of breadth, depth and texture

We have delineated engagement in dimensions of breadth, depth and texture, with these three dimensions pertinent in specifying both the design and evaluation of engagement programmes. These dimensions form a useful basis for operationalising engagement as a measurable phenomenon for cross-sectional and longitudinal research across pharmacovigilance activities. The evaluation of engagement can relate to the process itself (preparation and conduct, assessing how does it work and under what conditions) or its outcomes (the actual changes in mutual understanding, and its impact on behaviours, resulting health outcomes and pharmacovigilance policies).

Breadth: methods and measurement

Measuring the number and diversity of those engaged is ostensibly straightforward but, as implied by our three-dimensional approach, measuring breadth cannot be separated from considerations of depth and texture [58].

Where avoidable pharmacological type-A risks are well established among older medicines and are not controversial within the HCP communities, a risk information-based campaign (without deeper levels of engagement) targeting the practices of the wider HCP community (or only those most frequently applying this medicine) may be appropriate (as suggested above, following 45). Awareness surveys to identify specific target audiences, or to evaluate the reach of the campaign, would be important starting points here [36, 56]. Studies should evaluate whether the intended breadth has actually been achieved and whether this has been sufficient to achieve the ultimate goals of patient safety and public health. Therefore studies of breadth should be accompanied by data that captures how ‘awareness’ of risk information translates into behaviour change, i.e. what happens to knowledge once it has been communicated [59], investigating the depth of the engagement outcome (see ‘Depth: method and measurement’ section).

For serious risks, survey research has demonstrated the contrast between ‘awareness’ of boxed warnings in the product information of medicines, which may be widespread, and ‘adherence’ to the related risk management protocols, which may be comparatively limited. Observational research, meanwhile, has drawn attention to the gap between what people say they do, in surveys and interviews, and what they do in practice [60, 61]. Hence, many experts are sceptical of surveys to measure engagement outcomes [36]. Particular problems have been described for social network analysis due to the tendency to neglect ‘weak ties’ (less central relationships and interactions) [62]. This form of recall bias, alongside other weaknesses (social desirability, for example), indicates that survey data should be interpreted with caution [36].

One way around this problem would be to survey different stakeholders involved. Triangulating data from these different surveys illuminates different stakeholders’ experiences of outcomes, providing a more complete understanding. For example, compliance with a training programme for prescribers was reported as satisfactory within a pharmacy survey. However, the relative ineffectiveness of the same intervention in terms of onward risk communication to patients was made apparent through an MU survey [63].

The validity and reliability of survey findings have been further questioned in light of low response rates [63]. This common feature of studies in this field raises important concerns regarding self-selection bias [64, 65] when inferring the number and, especially, the diversity of those being engaged. Combining surveys with findings from other analyses (e.g. of healthcare data and/or qualitative data) can help mitigate these weaknesses [66]. Multi-method design, longitudinally employing various sources of primary and secondary data, enables exploration of variations in ‘breadth’ of the outcome of the engagement intervention (numbers of those successfully impacted by the engagement intervention in terms of their behaviours) [67]. The extent of ‘breadth’ as an outcome of engagement will thus vary (often narrowing) as we go deeper beyond mere risk awareness to consider risk-related behaviours.

Depth: methods and measurement

As noted above, we distinguish the ‘depth of engagement intervention’ (the design of format, see Table 1 [42]) from ‘depth of the engagement outcome’ (impact of intervention—see Fig. 1). One might expect that ‘broader and deeper’ interventions result in likewise ‘broader and deeper’ outcomes, but that might not necessarily be the case. For a non-controversial issue, a targeted information campaign might lead to a broad and deep behavioural change. In the absence of deeper engagement, however, outcomes resulting from information campaigns may be harder to ensure and possibilities for misinterpretation, unintended effects and mistrust are greater, particularly in scenarios of potential controversy.

Remembering that one dimension cannot be measured independently of the others, one way to measure engagement is to categorise different depths and then to measure breadth (n) across each depth (see examples in Table 3). Surveys could be useful here, but this is contingent upon the effective use of preparatory research to generate questions which are sensitive to mapping deeper processes where communication translates into behaviour (and back again). A study of cough medicine use following a change in advice from the US Food and Drug Administration (US FDA), surveying both parents and HCPs, found that a fifth of parents would be more likely to request antibiotics [68], an unintended and undesirable outcome. The survey included questions on antibiotics because the researchers’ drew on their own experiences working in General Practice, but there may be other unexpected consequences of engagement which survey studies unwittingly omit. Preparatory qualitative research with information-rich experts or focus groups will help overcome this problem.

Table 3 Methods for measuring and indicators relating to engagement processes and outcomes by depth and breadth across three levels of depth of engagement interventions
Full size table

Within models of engagement, where the depth increases and texture of knowledge sharing becomes more open and dynamic [42], other measurement tools are useful [75, 76].

Qualitative research is particularly suited to assessing engagement processes [79], elucidating (unexpected) understandings and mechanisms which can then be explored through larger quantitative studies. Qualitative data furthermore facilitate the necessary grasp of everyday practices to assist in interpreting survey and secondary data in context [80]. The influence of health literacy on how people (do not) act on risk knowledge [2, 13], legitimacy on HCPs’ heeding of regulatory advice [69] and mistrust in industry and declining trust in regulators on trust-building strategies [23, 47, 81, 82] are all subtle features of texture, noted within the pharmacovigilance literature, whereby the relationship between knowledge and behaviour is mediated by norms, emotions and what is meaningful for stakeholders (see ‘Texture: methods and ‘measurements’’ section and wider literature e.g. [47, 58, 78]).

Finally, our conceptualisation of depth of engagement in relation to its outcomes also involves a consideration of how the understandings and application of knowledge become adopted as ‘normal’ over time [72]. Recent studies of engaging HCPs in ADR reporting, for example, pay attention to whether the impact of interventions endures over months and years [70, 74]. The lasting impact of educational interventions towards safer prescribing practices can similarly be tracked over time through experimental [75] or interrupted time series designs [7, 69]. Such designs provide the possibility for measuring the varying depths of engagement outcomes resulting from different depths of the intervention. Behaviour change becomes embedded as knowledge and motivations underpinning these learned behaviours (such as safer prescribing practices or better quality ADR reporting) gradually develop. This points to the value of tracking the textures of MU and HCP engagement processes over time (see ‘Texture: methods and ‘measurement’’ section), as well as the outcomes. Use of secondary data—e.g. on prescribing rates, prescriptions themselves [75], ADR reporting and pregnancy-exposure rates [67]—can be usefully employed alongside surveys to indicate changes in behaviour [7, 76].

Such longitudinal research, as with designs including qualitative components, is more elaborate and therefore potentially more costly. Trends towards smaller cohort studies within pharmacovigilance, however, indicate an efficient research design format for bringing several different data sources together to study the same intervention [7, 71, 76]. Drawing small samples from larger databases enables the combining of primary data on small cohorts with secondary data on wider populations.

We consider various approaches to operationalising evaluation of breadth alongside different levels of depth in Table 3.

Texture: methods and ‘measurement’

Considerations of breadth and depth are fundamental to describing or designing engagement interventions and to capturing the extent of engagement of various publics as outcomes of an intervention. Texture, meanwhile, relates more to process [80], with researchers able to use approaches such as ‘interpretative policy analysis’ to explore and evaluate the process and social dynamics of how engagement is experienced [79]. If we are to move beyond describing the relative success or failings (i.e. outcomes) of engagement interventions, towards understanding and explaining their immediate and sustainable effectiveness, then texture is everything.

One of the ‘deepest’ engagement interventions [29] we came across in our literature review noted the motivating role of feelings of recognition and empathy within the interactional dynamics of an online community–based ‘outreach’. These were understood to impact significantly on ADR reporting and use of safety information [29]. The key instrumental parts of this intervention—a time-saving online reporting app—could be employed in similar future interventions, but these may or may not prove successful in facilitating the same impact if the social-interactional dynamics and related emotions are not also reflected in the future intervention approach. In this sense, researching texture contributes to both the internal and external validity of engagement analyses.

Texture—what engagement feels like, what it means to people and how these shape motivations—is much more complex and challenging to capture and measure than the other two dimensions. Myriad feelings and meanings pertinent to engagement are described in various studies concerning safety of medicines—trust, mistrust, confidence, diffidence, altruism, partnership and indifference [11, 23, 48, 56, 77, 78, 83]—and many of these are operationalised fairly robustly within existing social science research [84, 85]. The first difficulty emerges, however, in delineating which emotions and meanings are pertinent to modelling values and motivations within a particular engagement intervention. Different emotions and meanings will be pertinent, and interact differently, across different settings and as shaped by different previous experiences. The second difficulty is that although tools for measuring meanings, emotions and related expectations (trust for example) exist, these will need to be reworked and revalidated for measurement within pharmacovigilance engagement processes.

The literatures on public understandings of science and risk governance considered earlier, alongside further insights from ‘interpretative policy analysis’, provide a number of important conceptual and analytical starting points for investigating textures of engagement [47, 86, 87]. Qualitative research will also be important in developing understandings of the complex causal processes and the conceptualisations of emotions, values, meanings and motivations. These will form a thorough basis for developing and validating quantitative measurement tools (and inferences based on these) as the field develops [79]. Existing approaches to process-evaluation research provide an important methodological basis, as well as analytical foci, such as context (e.g. underlying barriers, such as mistrust, and facilitators, such as high health literacy), implementation (e.g. how are prescribers trained or how a risk minimisation measure fits in the established healthcare process [88]) and key mechanisms (e.g. reactions to the intervention, such as how medicines safety officers [5] in hospitals are perceived by other HCPs and patients; mediating factors in their success or failure, such as time, resources and trust; unintended consequences or barriers) [80, 89, 90]. Recent methodological research provides guidance on how to integrate process evaluation (for example within randomized controlled trials), in particular how process-oriented analyses of textures can feed into quantitative measurement and analyses of outcomes, and how this, in turn, can inform future intervention designs [80].

Conclusion

We began by noting the problematic lack of a definition of, and conceptual work on, engagement in relation to pharmacovigilance and risk minimisation activities. The conceptual lessons we have drawn from wider social science literatures, alongside more empirical work in pharmacovigilance, including risk minimisation studies, lead us to stress that ‘engagement’ is not a homogenous entity. Processes of knowledge exchange and resulting behavioural change will vary depending on context—be this HCPs changing their prescribing and communication with their patients, regulators changing the terms of what is considered an effective regulatory action in response to newly identified risks, or patients changing how they accept or reject medicines, ideally in the context of shared therapeutic decision-making. Our definition and three-dimensional conceptualisation in terms of breadth, depth and texture are intended to be sufficiently generic to capture varied forms of engagement in many clinical and policy contexts. Interpreting and specifying individual engagement interventions in terms of their breadth, depth and texture will assist in comparing and sharing findings, thus stimulating research and debate for optimising engagement.

Regarding pharmacovigilance and risk minimisation activities, the conceptualisation is intended for regulators as they seek to tailor formats of engagement to different medicines (and related risks), with the aim to optimise the safe and effective use of medicines, including the design and application of risk minimisation activities [73]. For researchers seeking to measure engagement, we have noted some basic ways of operationalising depth and breadth (summarised in Table 3), as these are understood in light of one another, underpinned by texture. We have also considered several methodological challenges which require sophisticated designs for combining data and methods. Research designs which combine qualitative, survey and secondary data analyses are likely to be more valid and reliable in capturing engagement in its three dimensions, therefore more effectively informing the (re)design of engagement interventions.

Acknowledgements and declarations

The authors thank Valerie Straßmann, Peter Arlett, Thomas Goedecke, Xavier Kurz, Maria Mavris and Gerben Moerman for their comments and input on earlier versions of the manuscript. This work has been conducted in the framework of the Strategy for Measuring Impact of Pharmacovigilance Activities of the Pharmacovigilance Risk Assessment Committee (PRAC) of the EMA.