Advertisement

European Journal of Clinical Pharmacology

, Volume 75, Issue 9, pp 1181–1192 | Cite as

‘Engagement’ of patients and healthcare professionals in regulatory pharmacovigilance: establishing a conceptual and methodological framework

  • Patrick BrownEmail author
  • Priya Bahri
Open Access
Review

Abstract

Background

Engagement of patients and healthcare professionals is increasingly considered as fundamental to pharmacovigilance and risk minimisation activities. Few empirical studies of engagement exist and a lack of explicit conceptualisations impedes effective measurement, research and the development of evidence-based engagement interventions.

Aims

This article (1) develops a widely applicable conceptualisation, (2) considers various methodological challenges to researching engagement, proposing some solutions, and (3) outlines a basis for converting the conceptualisation into specific measures and indicators of engagement among stakeholders.

Method

We synthesise social science work on risk governance and public understandings of science with insights from studies in the pharmacovigilance field.

Findings

This leads us to define engagement as an ongoing process of knowledge exchange among stakeholders, with the adoption of this knowledge as the outcome which may feed back into engagement processes over time. We conceptualise this process via three dimensions; breadth, depth and texture. In addressing challenges to capturing each dimension, we emphasise the importance of combining survey approaches with qualitative studies and secondary data on medicines use, prescribing, adverse reaction reporting and health outcomes. A framework for evaluating engagement intervention processes and outcomes is proposed. Alongside measuring engagement via breadth and depth, we highlight the need to research the engagement process through attentiveness to texture—what engagement feels like, what it means to people, and how this shapes motivations based on values, emotions, trust and rationales.

Conclusion

Capturing all three dimensions of engagement is vital to develop valid understandings of what works and why, thus informing engagement interventions of patients and healthcare professionals to given regulatory pharmacovigilance scenarios.

Keywords

Stakeholder engagement Patient engagement Pharmacovigilance Regulation Risk governance 

Introduction and objectives

Engagement of stakeholders is emphasised by researchers and policymakers as a fundamental tool for enhancing pharmacovigilance processes [1, 2, 3, 4, 5, 6, 7]. Engaging patients and other medicines users (MUs), and healthcare professionals (HCPs)—as the stakeholders closest to the prescribing and use of medicines—can support pharmacovigilance systems [8, 9] across various settings, e.g. regulatory bodies, pharmaceutical companies and healthcare. This has been argued for the European case [10], the USA [1] and more internationally via CIOMS [8]. In practice, pharmacovigilance is composed of structured processes, including aggregating adverse drug reaction (ADR) case reports [1, 11, 12, 13, 14, 15], reviewing pharmacoepidemiological and other study data, identifying and assessing new risks and making decisions [11, 16, 17, 18, 19, 20, 21] on risk management strategies [22] and their implementation in healthcare [23, 24, 25, 26]. While engagement is increasingly pursued in practice [27, 28] and receives global attention [8], empirical research [16, 29] into pharmacovigilance engagement remains in its infancy [7]. Conceptual precision (construct validity) is vital for the internal validity of research [30]. An explicit and common definition of engagement is therefore necessary if a body of empirically grounded studies is to emerge which can be built upon, critiqued and refined [31, 32]—informing engagement interventions more generally [33].

Among the many academic articles referring to engagement in relation to pharmacovigilance and risk with medicines, very few explicitly state what engagement means. This has been evidenced by preparatory work for this article by reviewing publications in Drug Safety, one of the leading journals on pharmacovigilance. Between 2010 and 2017, more than 40 articles in this journal refer to ‘engagement’, either in depth or in passing (see Annex 1 for a summary). Often the terms ‘engagement’, ‘participation’ or ‘involvement’ are used interchangeably and ambiguously in recent literature, with their meaning seemingly self-evident. Despite calls to develop evidence based engagement [1, 26], the current lack of clear conceptualisations and a definition of engagement in the literature is a glaring impediment to valid measurement and analysis.

This conceptual ambiguity is the starting point for this article which has three objectives, to:
  1. (1)

    Develop a widely applicable conceptualisation of engagement;

     
  2. (2)

    Consider various methodological challenges to researching engagement alongside solutions; and

     
  3. (3)

    Outline a basis for converting our conceptualisation into specific measures and indicators (operationalisation [30]) of engagement among pharmacovigilance stakeholders.

     

Conceptualising and defining engagement

Amid the general lack of definitions, some articles regarding risk management of medicinal products [22, 34] nevertheless discuss the quantity and quality of (patient) engagement, involvement or participation, or add understandings of ‘activation’ and ‘empowerment’. The authors of one more sophisticated empirical study have gone further to consider engagement for pharmacovigilance via a model of ‘motivation, incentives, activation and behaviour’ (MIAB) [29]. The precise relationship between MIAB and engagement is not specified, though the original social science study on which these authors draw refers to MIAB as an ‘engagement mechanism’ [35]. This raises questions, however, as to whether engagement should be seen as an independent variable that may lead to behavioural outcomes (e.g. changes in prescribing or using medicines), or whether certain behaviours (e.g. prescribing or using medicines or ADR reporting) are intrinsic parts of engagement itself.

One solution to such conceptual questions is to look to existing work regarding patient engagement in the wider field of health and medicine [33, 36]. This explores how to capture the effectiveness of ‘engagement networks’ [37], or how qualitative research among patients, combined with patient-reported outcomes, can inform our understandings of which risks are relevant for MUs [38]. Moreover, insights from the social sciences can greatly enhance the risk management of medicinal products [26, 34]. Studies of risk governance—how institutions manage risk together with different stakeholders [39]—and of public understandings of science have developed expansive literatures on engagement regarding (novel) technologies and their risks and benefits [40, 41, 42, 43, 44, 45, 46, 47].

Social scientists typically discuss engagement in normative, instrumental or substantive terms [44]: ‘normative’ refers to the ethical rightness of engaging different publics; from a more practical perspective, ‘instrumental’ approaches consider engagement as a means towards legitimate and effective governance; ‘substantive’ approaches explore engaging different stakeholders or acquiring input for better understandings of risks and risk perceptions [44, 45]. These approaches usefully illuminate existing engagement activities in pharmacovigilance and risk minimisation, which are often beset by limitations in the impact of, and compliance with, safe use and risk minimisation advice, alongside underreporting of ADRs. Interrelated challenges of motivation, legitimacy [1, 16, 48, 49, 50] and accuracy of risk management–related processes [9, 11, 17, 18, 20] are central here.

Influential studies of risk governance have gone further to show how the instrumental aspects and substantive aspects of engagement relate to one another [46]. Including more diverse stakeholders and risk perspectives may enhance legitimacy and outcomes among stakeholder groups [44, 45, 46, 47], but this depends on the context. Where hazards of technology are able to be modelled probabilistically via relatively straightforward linear models [51]—as may be the case where an ADR is pharmacologically predictable and dose-dependent [15]—then a more limited form of engagement may usually suffice [46]. However, in contexts where there is more uncertainty, complexity (involving causal sequences) or ambiguity (where there may be multiple values and definitions of an ‘unwanted outcome’), deeper levels of participatory engagement are required to enhance knowledge, legitimacy and thus impact [44, 46]. Examples of these latter contexts include actions to minimise teratogenic risks of medicines [52].

These key distinctions made in the risk governance literature between uncertainty, complexity and ambiguity of risk, and the related requirements for different interventions for engagement [46] suggest two key lessons for regulators: first, differentiating engagement interventions is necessary and important for effectively fulfilling the legal mandates of a regulatory authority; second, pursuing deeper levels of inclusive engagement has important implications for who governs risk governance (greater inclusion of MUs, for example, would, in full consequence, also mean their greater input into shaping priorities regarding which risks to assess and which risk minimisation goals to set [45]).

Responding to these lessons from existing work and returning to influential studies within the public understandings of science literature [42], we can conceptualise different models or depths of engagement interventions—information, consultation and participation (see Table 1), and their respective implications. Examples from health promotion research consider a similar spectrum of ‘community engagement’ ranging from outreach, consultation, involvement to collaboration and ‘shared leadership’ [55].
Table 1

Different depths of engagement interventions*

Depth of intervention

Format

Information (less deep)

A one-way form of information giving

Consultation (deeper)

A two-way but nevertheless limited and asymmetric engagement at specific moments in time within a remit designed by the engager-organisation

Participation (deepest)

A two-way and more open flexible approach where engaged persons and groups have more initiative and input in the timing and nature of the discussions

*Drawing upon definitions from the Rowe and Frewer’s typology [42]. EMA policy [27, 53] applies this categorisation already but has not yet referred to them as levels of depth. The US FDA currently uses different terms with similar, yet nuanced meanings, i.e. ‘respond and publish’, ‘solicit input’ and ‘inform regulatory decisions’ [54]

This basic schema assists us descriptively in categorising engagement interventions. The schema is also useful analytically—in illuminating inherent weaknesses in many existing attempts at pharmacovigilance engagement. For example, tensions often exist between a ‘participation’ model—whereby MUs and HCPs are expected to also take the initiative and maintain participation over prolonged periods—and features of a consultation model—where there are strict confines around the scope of what can be said, with limited feedback and a lack of discussion. Such tensions are, for example, present in many ADR reporting systems which usually provide very limited, often only automised or generic feedback to reporters and are not experienced as two-way participation by the reporter. Another example consists of the warnings and safe use advice added to the product information and supported by regulators’ websites. These are rarely disseminated on the basis of a strategic plan agreed with all stakeholders. The recognition of such tensions helps us understand the shortcomings in the effectiveness of these engagement interventions. More successful engagement, as seen within some empirical studies, can be understood in terms of a fuller and more consistent embracing of participatory models [29, 47].

Glimpses of such dynamic, interactive two-way approaches are apparent among interventions by some regulators, where various efforts to involve MUs and HCPs, both directly and via their respective organisations, reflect a more participatory approach [27] (Note 1). Each of these approaches aims, explicitly or implicitly, at different depths of engagement and this will also vary in practice across different stakeholders.

Campaigns by national bodies, for example to increase ADR reporting or appropriate prescribing, are also engagement interventions. Evidence on the depth, dynamics and behavioural impact of engagement is so far available from a small number of pharmacovigilance studies: for example, HCPs describe themselves as being more likely to report ADRs within a therapeutic advice service where they could receive advice in return [56]; or the meaning and motivations of engagement, not least in relation to community belonging, positive feedback (from an online platform) and related feelings of altruism, were deemed fundamental for MUs’ usage of safety information and ADR reporting [29, 57]. Here we see not only that the depth of engagement is pertinent but, moreover, the interactive dynamics, emotions and meaningful experience—the texture [58]—of engagement processes are also fundamental to their effectiveness [47]. Texture thus relates to what engagement feels like, what it means to people and how this shapes motivations based on values, emotions, trust and rationales.

Drawing on these multiple insights from literatures on risk governance and public understandings of science, as well as from empirical and policy articles in the pharmacovigilance field, we define and conceptualise engagement in pharmacovigilance along three dimensions (see Table 2).
Table 2

Conceptualisation and definition of engagement in pharmacovigilance

Engagement: An ongoing process of knowledge exchange among stakeholders

This process is enacted through engagement interventions, with the steps of preparing (e.g. advocating for, initiating, organising an intervention), conducting, and evaluating the intervention (e.g. its preparatory phase, the actual conduct of knowledge exchange, adoption of knowledge and changes in mutual understanding, its impact on behaviours and resulting health outcomes).

The actors in this process include all pharmacovigilance stakeholders such as MUs, HCPs, carers, wider publics, regulatory bodies, academics, industry, national health boards and healthcare institutions.

We use ‘knowledge exchange’ in its broader sense, referring to sharing of perspectives, norms, values and meanings, as well as scientific knowledge [58]. Adoption of this knowledge - through mutual understanding, changed behaviour or policy - is the main direct outcome, which will often feed back into engagement processes over time.

Dimensions of engagement

Definition

Breadth

The quantity and diversity of stakeholders being engaged

Depth

The extent of knowledge exchanged between stakeholders

Texture

Interactive dynamics of what engagement feels like, what it means to people, and how this shapes motivations to engage and change behaviour—based on values, emotions, (mis)trust and rationales

Above we raised the question—should engagement be seen as an independent variable which may lead to behavioural outcomes or are behaviours such as, prescribing, medicines use and ADR reporting intrinsic parts of engagement itself? The conceptual framework we have developed above leads us to distinguish between engagement interventions—information, consultation and/or participation, as outlined in Table 1—and engagement outcomes, such as (changing) behaviours. We thus see engagement interventions as part of the pharmacovigilance processes and their outcomes as manifestations of the design of engagement intervention (see Fig. 1).
Fig. 1

Conceptualising engagement in breadth, depth and texture

For the pharmacovigilance processes run by regulatory bodies, there are two critical entry points for input from MUs and HCPs during the continuous benefit-risk assessment of a medicinal product throughout its life-cycle: First, MUs and HCPs can contribute to the ‘life-cycle management’ step of generating scientific evidence about risks and risk management, e.g. with ADR reports or results from PASS (including surveys on knowledge, attitudes and practices regarding risk minimisation), or other real-world data and research into healthcare practices. Second, MUs and HCPs can contribute to decision-making on product-related action and policy, assisting the integrating of evidence with considerations of MU perspectives and preferences [22]. Engagement interventions can enhance this input (through knowledge and legitimation), and this can be seen as an outcome of engagement.

If, as noted above, deeper engagement is a two-way process, then we must also question whether there is one engager or multiple engagers. For while the legal responsibility for engagement activities in regulatory pharmacovigilance lies with the regulatory body, MUs and HCPs can take the initiative through advocacy and can also conduct their own engagement interventions, e.g. a campaign for ADR reporting. Likewise, the outcomes of engagement will relate to all stakeholders involved, including those initiating the engagement.

Measuring engagement and its impact along the dimensions of breadth, depth and texture

We have delineated engagement in dimensions of breadth, depth and texture, with these three dimensions pertinent in specifying both the design and evaluation of engagement programmes. These dimensions form a useful basis for operationalising engagement as a measurable phenomenon for cross-sectional and longitudinal research across pharmacovigilance activities. The evaluation of engagement can relate to the process itself (preparation and conduct, assessing how does it work and under what conditions) or its outcomes (the actual changes in mutual understanding, and its impact on behaviours, resulting health outcomes and pharmacovigilance policies).

Breadth: methods and measurement

Measuring the number and diversity of those engaged is ostensibly straightforward but, as implied by our three-dimensional approach, measuring breadth cannot be separated from considerations of depth and texture [58].

Where avoidable pharmacological type-A risks are well established among older medicines and are not controversial within the HCP communities, a risk information-based campaign (without deeper levels of engagement) targeting the practices of the wider HCP community (or only those most frequently applying this medicine) may be appropriate (as suggested above, following 45). Awareness surveys to identify specific target audiences, or to evaluate the reach of the campaign, would be important starting points here [36, 56]. Studies should evaluate whether the intended breadth has actually been achieved and whether this has been sufficient to achieve the ultimate goals of patient safety and public health. Therefore studies of breadth should be accompanied by data that captures how ‘awareness’ of risk information translates into behaviour change, i.e. what happens to knowledge once it has been communicated [59], investigating the depth of the engagement outcome (see ‘Depth: method and measurement’ section).

For serious risks, survey research has demonstrated the contrast between ‘awareness’ of boxed warnings in the product information of medicines, which may be widespread, and ‘adherence’ to the related risk management protocols, which may be comparatively limited. Observational research, meanwhile, has drawn attention to the gap between what people say they do, in surveys and interviews, and what they do in practice [60, 61]. Hence, many experts are sceptical of surveys to measure engagement outcomes [36]. Particular problems have been described for social network analysis due to the tendency to neglect ‘weak ties’ (less central relationships and interactions) [62]. This form of recall bias, alongside other weaknesses (social desirability, for example), indicates that survey data should be interpreted with caution [36].

One way around this problem would be to survey different stakeholders involved. Triangulating data from these different surveys illuminates different stakeholders’ experiences of outcomes, providing a more complete understanding. For example, compliance with a training programme for prescribers was reported as satisfactory within a pharmacy survey. However, the relative ineffectiveness of the same intervention in terms of onward risk communication to patients was made apparent through an MU survey [63].

The validity and reliability of survey findings have been further questioned in light of low response rates [63]. This common feature of studies in this field raises important concerns regarding self-selection bias [64, 65] when inferring the number and, especially, the diversity of those being engaged. Combining surveys with findings from other analyses (e.g. of healthcare data and/or qualitative data) can help mitigate these weaknesses [66]. Multi-method design, longitudinally employing various sources of primary and secondary data, enables exploration of variations in ‘breadth’ of the outcome of the engagement intervention (numbers of those successfully impacted by the engagement intervention in terms of their behaviours) [67]. The extent of ‘breadth’ as an outcome of engagement will thus vary (often narrowing) as we go deeper beyond mere risk awareness to consider risk-related behaviours.

Depth: methods and measurement

As noted above, we distinguish the ‘depth of engagement intervention’ (the design of format, see Table 1 [42]) from ‘depth of the engagement outcome’ (impact of intervention—see Fig. 1). One might expect that ‘broader and deeper’ interventions result in likewise ‘broader and deeper’ outcomes, but that might not necessarily be the case. For a non-controversial issue, a targeted information campaign might lead to a broad and deep behavioural change. In the absence of deeper engagement, however, outcomes resulting from information campaigns may be harder to ensure and possibilities for misinterpretation, unintended effects and mistrust are greater, particularly in scenarios of potential controversy.

Remembering that one dimension cannot be measured independently of the others, one way to measure engagement is to categorise different depths and then to measure breadth (n) across each depth (see examples in Table 3). Surveys could be useful here, but this is contingent upon the effective use of preparatory research to generate questions which are sensitive to mapping deeper processes where communication translates into behaviour (and back again). A study of cough medicine use following a change in advice from the US Food and Drug Administration (US FDA), surveying both parents and HCPs, found that a fifth of parents would be more likely to request antibiotics [68], an unintended and undesirable outcome. The survey included questions on antibiotics because the researchers’ drew on their own experiences working in General Practice, but there may be other unexpected consequences of engagement which survey studies unwittingly omit. Preparatory qualitative research with information-rich experts or focus groups will help overcome this problem.
Table 3

Methods for measuring and indicators relating to engagement processes and outcomes by depth and breadth across three levels of depth of engagement interventions

Design of engagement intervention by depth

Engagement outcomes

Engagement process and outcomes

Breadth of engagement measured at different depths: (indicators)

Useful methods and indicators using primary data (including references to relevant studies for specific examples)

Useful methods and indicators using secondary data * (including references to relevant studies for specific examples)

Texture

(indicators of texture and qualitative research approaches for grasping texture as a complex process)

Information—one-way from regulatory body (sometimes via others—e.g. HCPs)

Percentage of target group(s)#, with:

basic awareness

more complete understanding; changed attitudes

changed behaviour

Survey questions tapping level of awareness of risk or safe practices (e.g. safe prescribing, medicines use or ADR reporting) [49, 56, 63]; reading of package leaflets [50]

Survey questions tapping extent of understanding, and attitudinal changes, in relation to risk or safe practices [49, 68]

Survey questions addressing relative adoption of knowledge and application in healthcare practice; impact on medicines use, health(care) behaviours or health outcomes [59, 68, 69]; sustainability of these behaviour changes

Volume of coverage in news media (content analyses—volume)

Content analysis of news media and social media (content analyses—by volume and theme to assess changing levels and forms of understanding)

Short-term changes in patterns of medicines use, handling, prescribing, dispensing, administration, ADR reporting and risk management [7, 16, 63]; long-term changes in patterns of medicines use, handling, prescribing, dispensing, administration, ADR reporting [70], changes in risk management policy

health outcomes, e.g. ADR occurrence, reduced ADR severity/seriousness/sequelae; pregnancy rate; children affected by teratogenic effects (e.g. valproate, isotretinoin) [63, 67, 71]

Survey questions regarding experience, emotions and related interpretation of information, trust in provider of information, motivation

Focus groups and interviews involving diverse range of stakeholders# and researching these social processes as these develop and change over time, and as these are connected to outcomes (depth of engagement, attitudes towards future engagement)

Open-ended nature of qualitative research would help identify other important features of texture (thus informing survey design)

Consultation

Percentage of target group(s)#, with:

basic awareness

fuller relevant understanding and changed beliefs

active knowledge sharing and changed behaviour

incorporation of knowledge (scientific knowledge, experiential knowledge, attitudinal understandings) into decision-making and policy

Survey questions tapping awareness of risks and safe practices; awareness among different stakeholders of various possibilities for input /sharing their knowledge [16, 56]; awareness of pharmacovigilance or existence of regulatory bodies [48, 49, 56]

Survey questions—level of (mutual) understandings, shifts in understanding [49] (e.g. safe use guidance), or sharing information (e.g. ADR reporting) [72], collection of data on public consultation in terms of number, variety of respondents, content [41, 73]

Survey questions addressing individuals’ attentiveness towards, and likelihood to, report ADRs and their familiarity with this process [72]; safe use behaviours in healthcare

Survey questions eliciting perceptions and experiences of different stakeholders regarding opportunities for input, level of input, level of perceived mutual understanding

Use of information sources (e.g. hits on public consultation webpage); ADR reporting data – quantity within target groups [29, 70, 74]

Changes in behaviour and/or communication over time [75, 76]—e.g. patterns of prescribing or quantity and quality of ADR reporting [70, 74]

Sustained changes in behaviour, communication or health outcomes over time—e.g. medicines use, handling, prescribing, dispensing, administration, ADR reporting [70, 74]; ADR occurrence, reduced ADR severity/seriousness/sequelae; pregnancy rate; children affected from teratogenic effects (e.g. valproate, isotretinoin) [63, 67, 71]

Documentary analysis to measure the extent of input of different stakeholders into policy and decisions (and changes in this input over time)

Survey questions tapping stakeholders’ expectations and whether these were met, satisfaction with engagement process, whether they felt ‘listened to’ and that their opinion counted, regulators’ satisfaction with knowledge gained from stakeholders, trust in regulatory and other stakeholders involved, motivation to take part in the future, e.g. [77]

Useful indicators may be drawn from existing studies of HCPs’ (non-) reporting of ADRs which consider emotions—such as diffidence (‘fear of appearing ridiculous for reporting ADRs merely on the ground of some suspicion’) and confidence (‘will it make a difference?’)—as key motivating factors, e.g. [48]

Qualitative data gathered via focus groups, interviews and via open comment boxes in feedback surveys

Observations of stakeholder meetings (using ethnographic methods) to further assist in capturing interaction dynamics, emotions, trust processes

Participation

Percentage of target group(s)# who are: involved

collaborating—active knowledge sharing and giving wide-ranging input relevant to decisions and policies, deep, wide and sustained adoption of changed behaviour

deeply participating—taking ownership of engagement process and outcomes, and taking initiative or even leadership in implementation or measures for deep, wide and sustained adoption of changed behaviour

Several indicators outlined above are relevant, plus:

Survey questions regarding the frequency and extent of contact with, and participation within, a particular network/activity [72], see also [33, 36, 57, 62],++

Collecting observed measures of activity (e.g. online discussions and contributions)—e.g. [78]

Survey questions addressing initiative and openness of communication within the organisation/network, alongside impact of involvement on attitudes and behaviour [56, 57]

Survey data addressing the extent of collaboration, involvement and ownership in network; online ethnography involving observations of involvement and interactions [29]

Use of information sources (e.g. webpage hits) and possibilities for communicating (ADR reporting data—quantity within target groups [29, 70, 74])

Changes in behaviour and/or communication over time—e.g. patterns of prescribing or quantity and quality of ADR reporting [29, 70, 74]; or where answers to survey questions shift over time indicating changing understandings (of risks), needs and interests; documentary analysis to measure the extent of input of different stakeholders into policy and decisions (and changes in this input over time)

Sustained changes in behaviour, communication or health outcomes over time—e.g. medicines use, handling, prescribing, dispensing, administration, ADR reporting [70, 74]; ADR occurrence, reduced ADR severity/seriousness/sequelae; pregnancy rate; children affected from teratogenic effects (e.g. valproate, isotretinoin) [63, 67, 71]

Survey questions for participation would be similar to those used for consultation see (above). Given the less structured and ongoing nature of participation, these data should be collected at different time points.

Again, survey designs (see above) would be informed by qualitative data collected through focus groups, interviews and observations. For participation, these would be more oriented to detecting changes over time and to textures of power, input and ownership.

*Preferably collected at multiple time-points within prospective, comparative and randomised study design—see [19, 37]

#Important to segment target groups to also consider diversity (demographics, educational background, disease types, different professionals, etc.) and thus representativeness [3]

++Many experts are sceptical of using surveys to measure engagement [36]. Particular problems exist for social network analysis due to the tendency to neglect ‘weak ties’ (less central relationships and interactions) [62]. Other data collection techniques can overcome this

Within models of engagement, where the depth increases and texture of knowledge sharing becomes more open and dynamic [42], other measurement tools are useful [75, 76].

Qualitative research is particularly suited to assessing engagement processes [79], elucidating (unexpected) understandings and mechanisms which can then be explored through larger quantitative studies. Qualitative data furthermore facilitate the necessary grasp of everyday practices to assist in interpreting survey and secondary data in context [80]. The influence of health literacy on how people (do not) act on risk knowledge [2, 13], legitimacy on HCPs’ heeding of regulatory advice [69] and mistrust in industry and declining trust in regulators on trust-building strategies [23, 47, 81, 82] are all subtle features of texture, noted within the pharmacovigilance literature, whereby the relationship between knowledge and behaviour is mediated by norms, emotions and what is meaningful for stakeholders (see ‘Texture: methods and ‘measurements’’ section and wider literature e.g. [47, 58, 78]).

Finally, our conceptualisation of depth of engagement in relation to its outcomes also involves a consideration of how the understandings and application of knowledge become adopted as ‘normal’ over time [72]. Recent studies of engaging HCPs in ADR reporting, for example, pay attention to whether the impact of interventions endures over months and years [70, 74]. The lasting impact of educational interventions towards safer prescribing practices can similarly be tracked over time through experimental [75] or interrupted time series designs [7, 69]. Such designs provide the possibility for measuring the varying depths of engagement outcomes resulting from different depths of the intervention. Behaviour change becomes embedded as knowledge and motivations underpinning these learned behaviours (such as safer prescribing practices or better quality ADR reporting) gradually develop. This points to the value of tracking the textures of MU and HCP engagement processes over time (see ‘Texture: methods and ‘measurement’’ section), as well as the outcomes. Use of secondary data—e.g. on prescribing rates, prescriptions themselves [75], ADR reporting and pregnancy-exposure rates [67]—can be usefully employed alongside surveys to indicate changes in behaviour [7, 76].

Such longitudinal research, as with designs including qualitative components, is more elaborate and therefore potentially more costly. Trends towards smaller cohort studies within pharmacovigilance, however, indicate an efficient research design format for bringing several different data sources together to study the same intervention [7, 71, 76]. Drawing small samples from larger databases enables the combining of primary data on small cohorts with secondary data on wider populations.

We consider various approaches to operationalising evaluation of breadth alongside different levels of depth in Table 3.

Texture: methods and ‘measurement’

Considerations of breadth and depth are fundamental to describing or designing engagement interventions and to capturing the extent of engagement of various publics as outcomes of an intervention. Texture, meanwhile, relates more to process [80], with researchers able to use approaches such as ‘interpretative policy analysis’ to explore and evaluate the process and social dynamics of how engagement is experienced [79]. If we are to move beyond describing the relative success or failings (i.e. outcomes) of engagement interventions, towards understanding and explaining their immediate and sustainable effectiveness, then texture is everything.

One of the ‘deepest’ engagement interventions [29] we came across in our literature review noted the motivating role of feelings of recognition and empathy within the interactional dynamics of an online community–based ‘outreach’. These were understood to impact significantly on ADR reporting and use of safety information [29]. The key instrumental parts of this intervention—a time-saving online reporting app—could be employed in similar future interventions, but these may or may not prove successful in facilitating the same impact if the social-interactional dynamics and related emotions are not also reflected in the future intervention approach. In this sense, researching texture contributes to both the internal and external validity of engagement analyses.

Texture—what engagement feels like, what it means to people and how these shape motivations—is much more complex and challenging to capture and measure than the other two dimensions. Myriad feelings and meanings pertinent to engagement are described in various studies concerning safety of medicines—trust, mistrust, confidence, diffidence, altruism, partnership and indifference [11, 23, 48, 56, 77, 78, 83]—and many of these are operationalised fairly robustly within existing social science research [84, 85]. The first difficulty emerges, however, in delineating which emotions and meanings are pertinent to modelling values and motivations within a particular engagement intervention. Different emotions and meanings will be pertinent, and interact differently, across different settings and as shaped by different previous experiences. The second difficulty is that although tools for measuring meanings, emotions and related expectations (trust for example) exist, these will need to be reworked and revalidated for measurement within pharmacovigilance engagement processes.

The literatures on public understandings of science and risk governance considered earlier, alongside further insights from ‘interpretative policy analysis’, provide a number of important conceptual and analytical starting points for investigating textures of engagement [47, 86, 87]. Qualitative research will also be important in developing understandings of the complex causal processes and the conceptualisations of emotions, values, meanings and motivations. These will form a thorough basis for developing and validating quantitative measurement tools (and inferences based on these) as the field develops [79]. Existing approaches to process-evaluation research provide an important methodological basis, as well as analytical foci, such as context (e.g. underlying barriers, such as mistrust, and facilitators, such as high health literacy), implementation (e.g. how are prescribers trained or how a risk minimisation measure fits in the established healthcare process [88]) and key mechanisms (e.g. reactions to the intervention, such as how medicines safety officers [5] in hospitals are perceived by other HCPs and patients; mediating factors in their success or failure, such as time, resources and trust; unintended consequences or barriers) [80, 89, 90]. Recent methodological research provides guidance on how to integrate process evaluation (for example within randomized controlled trials), in particular how process-oriented analyses of textures can feed into quantitative measurement and analyses of outcomes, and how this, in turn, can inform future intervention designs [80].

Conclusion

We began by noting the problematic lack of a definition of, and conceptual work on, engagement in relation to pharmacovigilance and risk minimisation activities. The conceptual lessons we have drawn from wider social science literatures, alongside more empirical work in pharmacovigilance, including risk minimisation studies, lead us to stress that ‘engagement’ is not a homogenous entity. Processes of knowledge exchange and resulting behavioural change will vary depending on context—be this HCPs changing their prescribing and communication with their patients, regulators changing the terms of what is considered an effective regulatory action in response to newly identified risks, or patients changing how they accept or reject medicines, ideally in the context of shared therapeutic decision-making. Our definition and three-dimensional conceptualisation in terms of breadth, depth and texture are intended to be sufficiently generic to capture varied forms of engagement in many clinical and policy contexts. Interpreting and specifying individual engagement interventions in terms of their breadth, depth and texture will assist in comparing and sharing findings, thus stimulating research and debate for optimising engagement.

Regarding pharmacovigilance and risk minimisation activities, the conceptualisation is intended for regulators as they seek to tailor formats of engagement to different medicines (and related risks), with the aim to optimise the safe and effective use of medicines, including the design and application of risk minimisation activities [73]. For researchers seeking to measure engagement, we have noted some basic ways of operationalising depth and breadth (summarised in Table 3), as these are understood in light of one another, underpinned by texture. We have also considered several methodological challenges which require sophisticated designs for combining data and methods. Research designs which combine qualitative, survey and secondary data analyses are likely to be more valid and reliable in capturing engagement in its three dimensions, therefore more effectively informing the (re)design of engagement interventions.

Acknowledgements and declarations

The authors thank Valerie Straßmann, Peter Arlett, Thomas Goedecke, Xavier Kurz, Maria Mavris and Gerben Moerman for their comments and input on earlier versions of the manuscript. This work has been conducted in the framework of the Strategy for Measuring Impact of Pharmacovigilance Activities of the Pharmacovigilance Risk Assessment Committee (PRAC) of the EMA.

Notes

Contributions

Both authors have contributed to the researching, planning, drafting, writing and editing of this article.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Disclaimer

The views expressed in this article are the authors’ personal views and may not be understood or quoted as being made on behalf of or reflect the position of the European Medicines Agency (EMA) or one of its committees or working parties.

Supplementary material

228_2019_2705_MOESM1_ESM.docx (38 kb)
ESM 1 (DOCX 38 kb)
228_2019_2705_MOESM2_ESM.docx (24 kb)
ESM 2 (DOCX 23 kb)

References

  1. 1.
    Dal Pan G (2014) Ongoing challenges in pharmacovigilance. Drug Saf 37(1):1–8Google Scholar
  2. 2.
    Dodoo A, Hugman B (2012) Risk perception and communication in sub-Saharan Africa. Drug Saf 35(11):1041–1052Google Scholar
  3. 3.
    Hugman B (2017) Shaking up culture and communication in PV. In: Edwards R, Lindquist M (eds) Pharmacovigilance: Critique and ways forward, 1st edn. Adis - Springer, Heidelberg, pp 81–97Google Scholar
  4. 4.
    Donelan R, Walker S, Salek S (2015) Factors influencing quality decision-making: regulatory and pharmaceutical industry perspectives. Pharmacoepidemiol Drug Saf 24(3):319–328Google Scholar
  5. 5.
    Goedecke T, Ord K, Newbould V, Brosch S, Arlett P (2016) Medication errors: new EU good practice guide on risk minimisation and error prevention. Drug Saf 39(6):491–500Google Scholar
  6. 6.
    Leviano F, Scarazzini L, Shen F, Duhig J, Jokinen J (2017) The future of safety science is happening now: the modernization of the benefit-risk paradigm. Pharmacoepidemiol Drug Saf 26(8):869–874Google Scholar
  7. 7.
    Cluxton R, Li Z, Heaton P et al (2005) Impact of regulatory labeling for troglitazone and rosiglitazone on hepatic enzyme monitoring compliance: findings from the state of Ohio Medicaid program. Pharmacoepidemiol Drug Saf 14(1):1–9Google Scholar
  8. 8.
    COUNCIL FOR INTERNATIONAL ORGANIZATIONS OF MEDICAL SCIENCES. Working group XI – patient involvement. https://cioms.ch/working_groups/working-group-xi-patient-involvement/. Updated 2019
  9. 9.
    Edwards I (2017) A new Erice report considering the safety of medicines in the 21st century. Drug Saf 40(10):845–849Google Scholar
  10. 10.
    Santoro A, Genov G, Spooner A, Rainer J, Arlett P (2017) Promoting and protecting public health: how the European Union pharmacovigilance system works. Drug Saf 40:855–869Google Scholar
  11. 11.
    Basch E (2013) Systematic collection of patient-reported adverse drug reactions: a path to patient-centred pharmacovigilance. Drug Saf 36(4):277–278Google Scholar
  12. 12.
    van Hunsel F, Talsma A, van Puijenbroek E, de Jong-van den Berg L, van Grootheest K (2011) The proportion of patient reports of suspected ADRs to signal detection in the Netherlands: case-control study. Pharmacoepidemiol Drug Saf 20(3):286–291Google Scholar
  13. 13.
    van Hunsel F, Härmark L, Pal S, Olsson S, van Grootheest K (2012) Experiences with adverse drug reaction reporting by patients. Drug Saf 35(1):45–60Google Scholar
  14. 14.
    Dal Pan G, Arlett P (2015) The US Food and Drug Administration-European Medicines Agency collaboration in pharmacovigilance: common objectives and common challenges. Drug Saf 38(1):13–15Google Scholar
  15. 15.
    Waller P, Harrison-Woolrych M (2017) An introduction to pharmacovigilance, 2nd edn. Wiley Blackwell, OxfordGoogle Scholar
  16. 16.
    Leone R, Moretti U, D’Incau P et al (2011) Effect of pharmacist involvement on patient reporting of adverse drug reactions: first Italian study. Drug Saf 36(4):267–276Google Scholar
  17. 17.
    Norén N (2014) Pharmacovigilance for a revolving world: prospects of patient-generated data on the internet. Drug Saf 37(10):761–764Google Scholar
  18. 18.
    Härmark L, Raine J, Leufkens H, Edwards IR, Moretti U, Sarinic VM, Kant A (2016) Patient-reported safety information: a renaissance of pharmacovigilance? Drug Saf 39(10):883–890Google Scholar
  19. 19.
    Edwards R (2017) Causality assessment in pharmacovigilance: still a challenge. Drug Saf 40:365–372Google Scholar
  20. 20.
    Smith M, Benattia I (2016) The patient’s voice in pharmacovigilance: pragmatic approaches to building a patient-centric drug safety organization. Drug Saf 39(9):779–785Google Scholar
  21. 21.
    Callréus T (2005) The precautionary principle and pharmaceutical risk management. Drug Saf 28(6):465–471Google Scholar
  22. 22.
    Radawski C, Morrato E, Hornbuckle K, Bahri P, Smith M, Juhaeri J, Mol P, Levitan B, Huang HY, Coplan P, Li H, on behalf of the BRACE Special Interest Group (2015) Benefit–risk assessment, communication, and evaluation (BRACE) throughout the life cycle of therapeutic products: overall perspective and role of the pharmacoepidemiologist. Pharmacoepidemiol Drug Saf 24:1233–1240Google Scholar
  23. 23.
    Bahri P (2010) Public pharmacovigilance communication: a process calling for evidence-based, objective-driven strategies. Drug Saf 33(12):1065–1079Google Scholar
  24. 24.
    Chakraborty S, Bouder F (2013) The future of risk communication and the role of the pharmaceutical industry. Curr Drug Saf 8(1):4–10Google Scholar
  25. 25.
    Hoos A, Anderson J, Boutin M, Dewulf L, Geissler J, Johnston G, Joos A, Metcalf M, Regnante J, Sargeant I, Schneider RF, Todaro V, Tougas G (2015) Partnering with patients in the development and lifecycle of medicines: a call for action. Ther Innov Regul Sci 49(6):929–939Google Scholar
  26. 26.
    Way D, Blazsin H, Löfstedt R, Bouder F (2017) Pharmaceutical benefit–risk communication tools: a review of the literature. Drug Saf 40(1):15–36Google Scholar
  27. 27.
    European Medicines Agency (2014) Revised framework for interaction between the European Medicines Agency and patients and consumers and their organisations. EMA/637573/2014Google Scholar
  28. 28.
    European Medicines Agency (2016) Revised framework for interaction between the European Medicines Agency and healthcare professionals and their organisations. EMA/89918/2016Google Scholar
  29. 29.
    Bahk C, Goshgarian M, Donahue K et al (2015) Increasing patient engagement in pharmacovigilance through online community outreach and mobile reporting applications: an analysis of adverse event reporting for the Essure device in the US. Pharm Med 29(6):331–340Google Scholar
  30. 30.
    Trochim W, Donnelly J, Arora K (2015) Research methods: the essential knowledge base. Cengage, BostonGoogle Scholar
  31. 31.
    Kuhn T (1962) The structure of scientific revolutions. University of Chicago Press, ChicagoGoogle Scholar
  32. 32.
    Lindquist M (2007) The need for definitions in pharmacovigilance. Drug Saf 30(10):825–830Google Scholar
  33. 33.
    Dukhanin V, Topazian R, DeCamp M (2018) Metrics and evaluation tools for patient engagement in healthcare organization- and system-level decision-making: a systematic review. Int J Health Policy Manag 7(10):889–903Google Scholar
  34. 34.
    Smith M, Morrato E (2014) Advancing the field of pharmaceutical risk minimization through application of implementation science best practices. Drug Saf 37(8):569–580Google Scholar
  35. 35.
    Keating M, Furberg R (2013) A methodological framework for crowdsourcing in research. Federal Committee on Statistical Methodology Research ConferenceGoogle Scholar
  36. 36.
    Volpp K, Motha N (2018) Patient engagement survey: improved engagement leads to better outcomes, but better tools are needed. NEJM CatalystGoogle Scholar
  37. 37.
    Pronovost P, Jha A (2014) Did hospital engagement networks actually improve care? N Engl J Med 371:691–693Google Scholar
  38. 38.
    Basch E (2013) Toward patient-centered drug development in oncology. N Engl J Med 369:397–400Google Scholar
  39. 39.
    Renn O (2006) Risk governance: towards an integrative approach. International Risk Governance Council, GenevaGoogle Scholar
  40. 40.
    Fiorino D (1990) Citizen participation and environmental risk: a survey of institutional mechanisms. Sci Technol Hum Values 15(2):226–243Google Scholar
  41. 41.
    Rowe G, Horlick-Jones T, Walls J, Pidgeon N (2005) Difficulties in evaluating public engagement initiatives: reflections on an evaluation of the UK GM nation? Public debate about transgenic crops. Public Underst Sci 14(4):331–352Google Scholar
  42. 42.
    Rowe G, Frewer L (2005) A typology of public engagement mechanisms. Sci Technol Hum Values 30(2):251–290Google Scholar
  43. 43.
    Hagendijk R, Irwin A (2006) Public deliberation and governance: engaging with science and technology in contemporary Europe. Minerva. 44(2):167–184Google Scholar
  44. 44.
    Pidgeon N, Rogers-Hayden T (2007) Opening up nanotechnology dialogue with the publics: risk communication or ‘upstream engagement’? Health Risk Soc 9(2):191–210Google Scholar
  45. 45.
    Van Asselt M, Renn O (2011) Risk governance. J Risk Res 14(4):431–449Google Scholar
  46. 46.
    Renn O, Klinke A, van Asselt M (2012) Coping with complexity, uncertainty and ambiguity in risk goverance: a synthesis. Ambio. 40:231Google Scholar
  47. 47.
    Engdahl E, Lidskog R (2014) Risk, communication and trust: towards an emotional understanding of trust. Public Underst Sci 23(6):703–717Google Scholar
  48. 48.
    Lopez-Gonzalez E, Herdeiro M, Figueiras A (2009) Determinants of under-reporting of adverse drug reactions: a systematic review. Drug Saf 32(1):19–31Google Scholar
  49. 49.
    Himmelstein M, Miron-Shatz T, Hanoch Y, Gummerum M (2011) Over-the-counter cough and cold medicines for children: a comparison of UK and US parents’ parental usage, perception and trust in governmental health organisation. Health Risk Soc 13(5):451–468Google Scholar
  50. 50.
    Leemans L, Heylen N, Quanten A, Deferme S (2011) Étude sur l’utilisation des notices destinées aux patients. J Pharm Belg 2011(4):109–116Google Scholar
  51. 51.
    Aven T (2013) The concepts of risk and probability: an editorial. Health Risk Soc 15(2):117–122Google Scholar
  52. 52.
    Gadot Y, Koren G (2015) Medication use in pregnancy: treating the mother, protecting the unborn. In: Harrison-Woolrych M (ed) Medicines for women. Springer, Dordrecht, pp 97–137Google Scholar
  53. 53.
    European Medicines Agency (2016) European Medicines Agency (EMA) stakeholder relations management framework. EMA/48651/2016Google Scholar
  54. 54.
    Food and Drug Administration of the United States (US FDA). Enhancing FDA's approach to patient engagement: current state analysis and recommendations. Silver Spring, MD: US FDA; 2017.Google Scholar
  55. 55.
    McCloskey D, McDonald M, Cook J et al (2011) Community engagement: Definitions and organizing concepts from the literature. In: CTSA, Community Engagement Key Function Committee Task Force on the Principles of Community Engagement (ed) Principles of community engagement, 2nd edn. Department of Health and Human Services, Rockville, MD, pp 3–41Google Scholar
  56. 56.
    Hasford J, Goettlera M, Munterb K, Müller-Oerlinghausen B (2002) Physicians’ knowledge and attitudes regarding the spontaneous reporting system for adverse drug reactions. J Clin Epidemiol 55(9):945–950Google Scholar
  57. 57.
    de Vries S, Wong L, Sutcliffe A, Houÿez F, Ruiz C, Mol P (2017) Factors influencing the use of a mobile app for reporting adverse drug reactions and receiving safety information: a qualitative study. Drug Saf 40(5):433–455Google Scholar
  58. 58.
    Brown P (2016) Using medicines in the midst of uncertainty – developing a Habermasian understanding of medicines lifeworlds. In: Chamberlain J (ed) Medicine, risk, discourse and power, 1st edn. Routledge, London, pp 76–99Google Scholar
  59. 59.
    Ceilley R, Eisenthal A (2009) The unintended effects of a boxed warning. J Clin Aesthet Dermatol 2(9):33–39Google Scholar
  60. 60.
    Makoul G, Arntson P, Schofield T (1995) Health promotion in primary care: physician-patient communication and decision making about prescription medications. Soc Sci Med 41(9):1241–1254Google Scholar
  61. 61.
    Jerolmack C, Khan S (2014) Talk is cheap: ethnography and the attitudinal fallacy. Sociol Methods Res 43(2):178–209Google Scholar
  62. 62.
    Bruggeman J (2008) Social networks: an introduction, 1st edn. Routledge, LondonGoogle Scholar
  63. 63.
    Brinker A, Kornegay C, Nourjah P (2005) Trends in adherence to a revised risk management program designed to decrease or eliminate isotretinoin-exposed pregnancies evaluation of the accutane SMART program. Arch Dermatol 141:563–569Google Scholar
  64. 64.
    Baruch Y, Holtom B (2008) Survey response rate levels and trends in organizational research. Hum Relat 61(8):1139–1160Google Scholar
  65. 65.
    Alqvist-Radstad J, Andric A, Baldelli I, et al (2016) SCOPE work package 6 - healthcare professional survey. Medicines safety communications and their effectiveness. Work Package 6. http://www.scopejointaction.eu/_assets/files/WP6-HCP-Survey-Report(1).pdf
  66. 66.
    Bryman A (2016) Integrating quantitative and qualitative research: how is it done? Qual Res 6(1):97–113Google Scholar
  67. 67.
    Raine J (2016) Why measure the impact of regulatory action? Plenary presentation at European Medicines Agency 'Workshop on measuring the impact of pharmacovigilance activities’. London 5th Dec 2016Google Scholar
  68. 68.
    Garbutt J, Sterkel R, Banister R, Walbert C, Strunk R (2010) Physician and parental response to the FDA advisory about use of over the counter cough and cold medications. Acad Pediatr 10(1):64–69Google Scholar
  69. 69.
    Habib A, Gan T (2008) The use of droperidol before and after the food and drug administration black box warning: a survey of the members of the society of ambulatory anesthesia. J Clin Anesth 20:35–39Google Scholar
  70. 70.
    Leporini C, Marrazzo G, Mumoli L et al (2017) Adverse drug reactions reporting in Calabria (southern Italy) in the four-year period 2011-2014: impact of a regional pharmacovigilance project in light of the new European legislation. Expert Opin Drug Saf 16(5):51515515–51515522Google Scholar
  71. 71.
    Layton D, Shakir S (2015) Specialist cohort event monitoring studies: a new study method for risk management in pharmacovigilance. Drug Saf 38(2):153–163Google Scholar
  72. 72.
    Scott HD, Thacher-Renshaw A, Rosenbaum S, Waters WJ Jr, Green M, Andrews LG, Faich GA (1990) Physician reporting of adverse drug reactions: results of the Rhode Island adverse drug reaction reporting project. JAMA. 263(13):1785–1788Google Scholar
  73. 73.
    European Medicines Agency PRAC (2017) Pharmacovigilance risk assessment committee (PRAC) strategy for measuring impact of pharmacovigilance activities. EMA, LondonGoogle Scholar
  74. 74.
    Figueiras A, Herdeiro M, Polonia J (2006) An educational intervention to improve physician reporting of adverse drug reactions: a cluster-randomized controlled trial. JAMA. 296(9):1086–1093Google Scholar
  75. 75.
    Manning PR, Lee P, Clintworth W, Denson T, Oppenheimer P, Gilman N (1986) Changing prescribing practices through individual continuing education. JAMA. 256(2):230–232Google Scholar
  76. 76.
    Graham D, Drinkard C, Shatin D, Tsong Y, Burgess M (2001) Liver enzyme monitoring in patients treated with troglitazone. JAMA. 286(7):831–833Google Scholar
  77. 77.
    van Hunsel F, van der Welle C, Passier A, van Puijenbroek E, van Grootheest K (2010) Motives for reporting adverse drug reactions by patient-reporters in the Netherlands. Eur J Clin Pharmacol 66(11):1143–1150Google Scholar
  78. 78.
    Butt T, Cox A, Oyebode J, Ferner R (2012) Internet accounts of serious adverse drug reactions: a study of experiences of Stevens-Johnson syndrome and toxic epidermal necrolysis. Drug Saf 35(12):1159–1170Google Scholar
  79. 79.
    Yanow D (1999) Conducting interpretive policy analysis. Sage, LondonGoogle Scholar
  80. 80.
    Oakley A, Strange V, Bonell C, Allen E, Stephenson J (2006) Process evaluation in randomised controlled trials of complex interventions. Br Med J 332(7538):413–416Google Scholar
  81. 81.
    Olsen A, Whalen M (2009) Public perceptions of the pharmaceutical industry and drug safety implications for the pharmacovigilance professional and the culture of safety. Drug Saf 32(10):805–810Google Scholar
  82. 82.
    Chakraborty S (2010) Ex-post pharmacovigilance and trust: a perspective. Eur J Risk Regul 1(1):83–85Google Scholar
  83. 83.
    Rodrigues C (2016) Medicines and therapeutic pluralism in Maputo: exploring modalities of trust and the (un)certainties of everyday users. Health Risk Soc 18(7–8):385–406Google Scholar
  84. 84.
    Thom D, Ribisl K, Stewart A, Luke D (1999) Further validation and reliability testing of the trust in physician scale. Med Care 37(5):510–517Google Scholar
  85. 85.
    Sawyer J (1966) The altruism scale: a measure of co-operative, individualistic, and competitive interpersonal orientation. Am J Sociol 71(4):407–416Google Scholar
  86. 86.
    Earle T, Siegrist M, Gutscher H (2007) Trust, risk perception and the TCC model of cooperation. In: Earle T, Siegrist M, Gutscher H (eds) Trust in cooperative risk management, 1st edn. Routledge - Earthscan, London, pp 1–50Google Scholar
  87. 87.
    Brown P (2016) Trust and risk. In: Burgess A, Alemanno A, Zinn J (eds) Routledge handbook of risk studies, 1st edn. Routledge, London, pp 49–57Google Scholar
  88. 88.
    Bahri P, Smith M, Leufkens H, Morrato E (2018) Implementation science: the art nouveau of pharmacoepidemiology for improving risk mitigation in health care. Pharmacoepidemiology and Drug Safety 27(S2):23-Abstract 45Google Scholar
  89. 89.
    Moore G, Audrey S, Barker M et al (2015) Process evaluation of complex interventions: Medical Research Council guidance. Br Med J 350:h1258Google Scholar
  90. 90.
    Farre A, Shaw K, Heath G, Cummins C (2017) On doing ‘risk work’ in the context of successful outcomes: exploring how medication safety is brought into action through health professionals’ everyday working practices. Health Risk Soc 19(3–4):209–225Google Scholar

Copyright information

© The Author(s) 2019

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.University of AmsterdamAmsterdamThe Netherlands
  2. 2.European Medicines Agency (EMA)AmsterdamThe Netherlands

Personalised recommendations