Pharmaceutical Medicine

, Volume 31, Issue 2, pp 101–112 | Cite as

Additional Risk Minimisation Measures for Medicinal Products in the European Union: A Review of the Implementation and Effectiveness of Measures in the United Kingdom by One Marketing Authorisation Holder

  • Elaine AgyemangEmail author
  • Lorna Bailey
  • John Talbot
Open Access
Original Research Article



Additional risk minimisation measures (aRMMs) for medicinal products are necessary to address specific important safety issues which may not be practically achieved through routine risk management measures alone. The implementation and determination of effectiveness for aRMMs can be a challenge as it involves multiple stakeholders. It is therefore important to have concise objectives to avoid undue burden on patients, healthcare professionals and the healthcare system.


The aim of this study was to examine how aRMMs are implemented and how effectiveness is assessed in the European Union (EU) using practical examples from Roche Products Limited in the United Kingdom (UK) (referred to as the ‘Company’).


Three centrally authorised products were selected from the Company’s portfolio, each of which had aRMMs to address important safety concerns; specifically, teratogenicity, medication error and infections. The implementation of EU aRMMs, effectiveness checks and specific UK activities were analysed. Hard copy folders and electronic sites for Company aRMMs were used to assess process indicators. Periodic benefit-risk evaluation reports for specified time intervals and the Company safety database was used in checking safety outcomes for the selected products. For each product, the effectiveness of aRMMs was analysed based on specific process indicators and the subsequent safety outcomes. Literature searches were performed on scientific databases for the purposes of the broader study.


The main process indicators in measuring effectiveness of Company aRMMs were distribution metrics for educational materials, assessment of awareness and clinical actions among healthcare professionals (HCPs). Case reports of pregnancy, medication errors and progressive multifocal leukoencephalopathy (PML) were the outcome indicators for Erivedge®▼, Kadcyla®▼ and MabThera® (the latter specifically in autoimmune indications: rheumatoid arthritis, granulomatosis with polyangiitis and microscopic polyangiitis) respectively. No pregnancy, one medication error and 10 confirmed PML cases were reported for Erivedge®▼, Kadcyla®▼ and MabThera® respectively.


For the chosen products, a reasonable awareness of aRMMs amongst HCPs is a positive indicator of success in the use of educational materials. However, low response rates from surveys indicate that voluntary feedback may not always achieve the desired level of response in measuring effectiveness. There is a challenge in determining overall effectiveness of aRMMs due to a lack of defined success thresholds. Further regulatory guidance to outline the elements and desired outcomes of aRMMs will be useful for consistency in achieving successful outcomes.


European Union Medication Error Progressive Multifocal Leukoencephalopathy Educational Material Herceptin 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Key Points

This study examines the implementation of additional risk minimisation measures (aRMMs) and effectiveness assessment on an EU level with practical examples from the UK. Low response rates to questionnaires indicate a limitation of voluntary feedback. Pharmacists and patients should be actively involved in measuring effectiveness of aRMMs.

Despite detailed legislation, the implementation and determination of effectiveness of aRMMs can be a challenge.

There is the need for additional regulatory guidance in defining the threshold for success in relation to aRMMs.

1 Introduction

European Union (EU) pharmacovigilance (PV) legislations in July 2012 imposed new challenges on pharmaceutical companies to be proactive in minimising risks of adverse drug reactions (ADRs) for all medicinal products throughout their life cycle [1, 2]. Marketing authorisation holders (MAHs) are required to put risk minimisation measures (RMMs) in place and evaluate their effectiveness to ensure that the benefits of a medicinal product outweigh the risks by the greatest achievable margin. RMMs are useful for consistency in identifying safety issues across generic and non-generic companies [3, 4]. The majority of safety concerns may be adequately addressed by routine RMMs such as pack size restrictions, suitable wordings in patient information leaflets (PILs) and summary of product characteristics (SmPC). Additional risk minimisation measures (aRMMs) may be necessary to address specific safety issues which may not be practically achieved through routine RMMs alone including potential for medication errors, off-label use and safety concerns in a special population such as the elderly. Examples of aRMMs include controlled access programmes and educational materials for healthcare professionals (HCPs) and patients [3, 4, 5, 6].

Periodic effectiveness evaluation of aRMMs is relevant to analyse whether, or not, the objective has been met and to identify gaps and/or opportunities for continuous improvement. There may be differences in perceptions on drug safety issues between MAHs and regulatory authorities as well as issues with implementation and effectiveness evaluation due to differences in national legislation [7, 8, 9, 10, 11, 12, 13, 14, 15].The inclusion of process and outcome indicators as risk minimisation tools in measuring effectiveness of aRMMs are encouraged [4]. Process indicators are the measures taken to implement aRMMs and analysis of variations in delivery (e.g. distribution metrics and surveys to assess clinical knowledge). Outcome indicators are the overall measures of the level of risk control achieved as a result of aRMMs which include impact on the frequency of ADR occurrence. The implementation and effectiveness measurements of aRMMs involve multiple stakeholders including patients, HCPs, regulatory authorities and MAHs. Therefore, it is important that the MAH has a clear identifiable plan, carefully coordinated to avoid undue burden on stakeholders [3, 4].

The aim of this study was to examine the implementation and effectiveness of aRMMs using practical examples from Roche Products Limited. Selected Company medicinal products used in this research were: Erivedge®▼ (vismodegib), Kadcyla®▼ (trastuzumab emtansine) and MabThera® (rituximab).

Erivedge®▼ is a Hedgehog (Hh) pathway inhibitor for the management of metastatic basal cell carcinoma and locally advanced basal cell carcinoma which has recurred following surgery and for patients who cannot be treated with surgery or radiation therapy due to medical reasons. Patients prescribed Erivedge®▼ are predominantly male adults and the elderly. The median duration of treatment has been reported as 10.2 months depending on the type of carcinoma being treated [16, 17, 18, 19, 20]. Erivedge®▼ has a targeted mechanism of action and interrupts the signal within cells of the Hh pathway that causes basal cell carcinoma cells to grow. This results in the shrinkage, reduction in growth rate and death of basal cell carcinoma cells.

Hh pathway inhibitors have been associated with teratogenicity in animal species especially in the first trimester of pregnancy. Teratogenic effects include severe structural abnormalities and altered growth in the foetus [21, 22, 23].

Erivedge®▼ was granted conditional approval in the EU (2013) with a legal obligation for the MAH to implement a pregnancy prevention programme as an aRMM to report any pregnancies that occur during treatment and monitor all pregnancy outcomes. Other risks with Erivedge®▼ are addressed with routine RMM only. The aim of the pregnancy prevention programme was to emphasise the safe and appropriate use of Erivedge®▼ to prevent foetal exposure by providing relevant education to HCPs and patients.

Kadcyla®▼ is an antibody drug conjugate for the treatment of human epidermal growth factor receptor-2-positive advanced breast cancer [24]. It is administered as an intravenous (IV) infusion. Duration of use may be unpredictable depending on clinical response or occurrence of unacceptable toxicity. Kadcyla®▼ has a targeted mechanism of action; trastuzumab attaches to human epidermal growth factor receptor-2 protein that makes cancer cells grow. This attachment stops the growth of cancer cells and allows migration of the emtansine moiety into the cancer cell, which eventually kills them [24, 25].

Medication error is considered an important potential risk with Kadcyla®▼ by the Company due to the potential for name confusion between its generic name, trastuzumab emtansine, and that of another marketed Company product, Herceptin®, which contains trastuzumab and is administered via IV and subcutaneous (SC) routes. Although Kadcyla®▼ and Herceptin® are both licensed for use in breast cancer, their dosing and treatment schedules are different. Potential confusion during drug administration could lead to dosing errors and serious ADRs in patients. As a condition of Kadcyla®▼ marketing authorisation in the EU (2013), educational materials with the aim of minimising the risk of medication error from name confusion were distributed in EU countries pre-launch.

MabThera® is a monoclonal antibody for the indications of Non-Hodgkin’s lymphoma and chronic lymphocytic leukaemia. It is also approved in the following autoimmune indications: rheumatoid arthritis (RA) (2006), granulomatosis with polyangiitis (GPA) (2013) and microscopic polyangiitis (MPA) (2013). MabThera® binds to CD20 leading to B-cell depletion [26, 27].

MabThera® has several known safety concerns, including an increased risk of infections. A very rare infection known as progressive multifocal leukoencephalopathy (PML) has been observed in patients with autoimmune diseases and is also a known risk observed following MabThera® treatment (less than 1/10,000) [28]. PML is a serious viral disease of the brain caused by the John Cunningham virus. It damages brain nerves and may lead to weakness, impaired speech, visual symptoms, cognitive problems and death if not detected early or managed appropriately. The risks of PML with MabThera® in autoimmune indications are mainly related to its mechanism of action, leading to neutropenia and increased risk of infections and infection-related adverse events. The exact influence of MabThera® with regards to infections remains controversial due to the use of concomitant immunosuppressive medications plus underlying diseases in patients [29].

The Company introduced educational materials in the EU to educate physicians and patients about the possible risks of infections, especially PML with the use of MabThera®. The distribution of MabThera® educational materials to prescribing physicians in EU affiliates was initiated in May 2009 prior to the new EU legislations.

2 Methodology

2.1 Selection Process for Company Medicines

The Company portfolio for centrally authorised products with aRMMs were grouped into three major programmes under pregnancy prevention, medication error and infection control in consultation with the Company’s Risk Minimisation Coordinator for UK and Malta. One product was selected from each of these categories to analyse the implementation and measures of effectiveness at an EU level alongside specific UK activities. Two newly approved products with mandatory aRMMs for different safety concerns (Erivedge®▼ and Kadcyla®▼) were selected. In addition, a more established product (MabThera®) with aRMMs implemented prior to the 2012 legislation was selected.

2.2 Data Review and Sources

Review of Company data was performed from 01 May to 30 September 2014; aRMMs from product launches to 30 September 2014 were reviewed for all three Company medicines. Hard copy folders containing educational materials were provided by the Company’s Risk Minimisation Coordinator for UK and Malta. The Company’s aRMM tracking tool was assessed for additional information for process indicators including distribution metrics, Company surveys and results.

Periodic benefit-risk evaluation reports (PBRERs) available at the time of the review and the Company safety database were used to check case reports and safety outcomes. The reporting intervals were as follows: Erivedge®▼: 30 January 2014 to 29 July 2014 (inclusive), Kadcyla®▼: 22 February 2014 to 21 August 2014 (inclusive) and Mabthera®: 18 November 2012 to 17 November 2013 (inclusive).

2.3 Literature Review

A literature review was performed with the following scientific search engines: Medical Literature Analysis and Retrieval System Online via PubMed, Cumulative Index to Nursing and Allied Health Literature, Cochrane Library and EMBASE.

Search terms included risk management systems, medication errors, basal cell carcinoma, teratogens, pregnancy exposure, risk minimisation measures, progressive multifocal leukoencephalopathy and pharmacovigilance. The search terms were used independently or in combination using “AND” to get a wide range of relevant hits on RMMs, tools used in the implementation of aRMMs and how the effectiveness of these measures have been assessed especially in the EU.

3 Results

3.1 Components of Product Specific aRMMs

Table 1 shows the specific elements of aRMMs for Erivedge®▼, Kadcyla®▼ and MabThera®. Elements of the Erivedge®▼ pregnancy prevention programme were Direct Healthcare Professional Communication letters (DHPCs) and pregnancy prevention programme folders containing educational materials. The educational materials included an HCP educational brochure, HCP reminder card, Verification of Counselling Form and SmPC. Table 2 shows a description of Erivedge®▼ educational materials.
Table 1

Elements of additional risk minimisation measures for Erivedge®▼, Kadcyla®▼ and MabThera®a

Elements of the Erivedge®▼ pregnancy prevention programme

Elements of the Kadcyla®▼ medication error educational programme

Elements of the MabThera® progressive multifocal leukoencephalopathy educational materials

Direct Healthcare Professional Communication letters (subject ‘Erivedge®▼ (vismodegib): Important information to support safe use including pregnancy prevention programme’

European Union healthcare professional information booklet

Healthcare professional education leaflet

Healthcare professional educational brochure


 Healthcare professional reminder card

 Company pregnancy report form

Healthcare professional key points to remember leaflet

Patient alert cards

Verification of Counselling Form

Specific United Kingdom measures:  

 ‘Confirmation of Healthcare Professionals’ role’ form

 Company pre-paid envelopes

Specific United Kingdom measures:

 Take-care poster

 Feedback questionnaire

 Company pre-paid envelopes

Patient education leaflets

Patient educational brochure containing a patient reminder card

Summary of product characteristics

Summary of product characteristics

Healthcare professional web portal

Patient information leaflet


Summary of product characteristics


Patient information leaflet


aThe MabThera® analysis was restricted to autoimmune indications

Table 2

Elements of the Erivedge®▼ Pregnancy Prevention Programme

Educational material


Healthcare professional educational brochure

Describe the role of the prescriber in the Erivedge®▼ pregnancy prevention programme

Healthcare professional reminder card

Contained a summary of pregnancy prevention measures and contact information for healthcare professionals to report suspected adverse drug reactions to the Company

Verification of Counselling Form

Confirmation checklist for prescriber and patient after the prescriber had educated the patient on the teratogenic risk of Erivedge®

Summary of product characteristics

Teratogenicity of Erivedge®▼ incorporated under section ‘4.4 Special warnings and precautions for use’; also highlighted that the Company provided educational materials for the pregnancy prevention programme for awareness

Educational materials focused on a wide range of topics such as embryo/foetal death, effects on post-natal development, criteria for a woman of childbearing potential, counselling and contraception for women of childbearing potential and men, pregnancy testing, prescribing and dispensing restrictions, blood and semen donation.

To complement the Erivedge®▼ pregnancy prevention programme in the EU, the Company added a HCP web portal as an electronic site, to assess the compliance and functionality of the EU pregnancy prevention programme in the post-marketing setting. This portal included a brief, anonymous questionnaire to capture patient demographic data (age groups and gender) for patients to be prescribed Erivedge®▼ and also provided a mechanism for HCPs to re-order or download copies of educational materials.

Kadcyla®▼ educational materials included the SmPC, key points to remember leaflet and an EU HCP information booklet. They emphasised the similarity between the generic names, trastuzumab emtansine for Kadcyla®▼ and trastuzumab for Herceptin® and reinforced the differences between Kadcyla®▼ and Herceptin® IV/SC focusing on trademarks/trade names, indications, mode of action, international non-proprietary names, doses and forms, vial content, vial size and colours. HCP key points to remember included ways to identify Kadcyla®▼ from Herceptin® IV/SC to avoid confusion. In the UK only, an additional ‘take-care poster’ was added to the suite of educational materials. Also UK specific was a hard copy questionnaire included to seek feedback on the quality and utility of educational materials for Kadcyla®▼ from HCPs. This form could be completed online via The link was provided on the back cover of the EU HCP booklet.

MabThera® educational materials included an HCP education leaflet, SmPC, PILs, patient alert cards and patient education leaflets. Key safety information on PML including risk factors, early recognition, signs and symptoms, diagnosis etc. were provided on educational materials for MabThera®.

3.2 Implementation of aRMMs

Erivedge®▼ DHPCs and pregnancy prevention programme folders containing educational materials were distributed to medical and clinical oncologists, plastic surgeons, dermatologists, skin cancer nurse specialists and oncology pharmacists. In the UK, educational materials were mailed to 1962 HCPs as part of product pre-launch activities in August 2013. Table 3 shows a breakdown of HCP specialities and the number of pregnancy prevention programme educational packs distributed in the UK. In February 2014, the original criteria used to produce the HCP mailing list were re-run and educational packs were sent to an additional 51 HCPs who had been identified as potential new prescribers. During this period of research, only a small number of centres (38) placed an order for Erivedge®▼ in the UK.
Table 3

Distribution of Erivedge®▼ pregnancy prevention programme educational materials in the United Kingdom

Healthcare professionals in United Kingdom

Number of pregnancy prevention programme packs distributed in August 2013

Consultant medical and clinical oncologists


Consultant plastic surgeons


Consultant dermatologists


Skin cancer nurse specialists


Oncology pharmacists




Patient materials were provided to HCPs rather than to patients directly. Company hospital sales specialists and medical science liaisons were trained on the details of the Erivedge®▼ pregnancy prevention programme. Educational materials were made available for re-ordering by the Company via Company representatives, e-mails, Medical Information department, re-mailings, telephone and the HCP web portal. The flow of pregnancy prevention programme events and HCP process for informing patients about the teratogenic effects of Erivedge®▼ was as follows:

Prior to prescribing, HCPs to read their educational brochure and keep reminder cards at their disposal, inform and counsel patient about the pregnancy prevention programme and risks associated with Erivedge®▼, then provide patient with educational brochure containing the patient reminder card.

Prescribers were also expected to sign the ‘Verification of Counselling Form’ with their patient prior to treatment, keep ‘Verification of Counselling Form’ in patient’s records and submit confirmation on the HCP web portal about a patient’s education and signed ‘Verification of Counselling Form’. At this point, Erivedge®▼ could then be prescribed.

In addition to signing the ‘Verification of Counselling Form’, the Company also expected patients to play a role in the Erivedge®▼ pregnancy prevention programme by reading the patient educational brochures given to them by HCPs and keeping their patient reminder cards on them at all times.

Kadcyla®▼ educational materials were mailed to all HCPs who could potentially prescribe, administer or dispense either Kadcyla®▼ or Herceptin® IV/SC or both drugs, including prescribing physicians, general oncologists, breast cancer specialists, pharmacists, hospital nurses who specialised in metastatic gastric cancer (an indication for Herceptin® IV/SC) and oncology nurses who treated patients in the private sector or areas where a breast cancer specialist was not available.

MabThera® patient alert cards were first distributed in 2009 in EU affiliates via country-specific distribution channels including sales representatives or direct mailing to all HCPs (prescribing physicians and nurses) treating patients with MabThera® for autoimmune indications. Patient alert cards were designed to a reasonable size to ensure that patients carried them at all times. In accordance with the SmPC, it was the responsibility of physicians to provide the patient alert card along with a patient educational leaflet to patients prior to initiating treatment and also during each patient treatment cycle with MabThera®. To emphasise this, the wording on the PIL also informed patients that an alert card should be provided to them by their HCP prior to their treatment with MabThera®.

3.3 Effectiveness of aRMMs

Effectiveness and compliance of the Erivedge®▼ pregnancy prevention programme in the EU was measured using process indicators including metrics on dispatched educational materials in reaching the target population, assessment of clinical actions via the HCP web portal and a survey to assess clinical knowledge and resulting behaviour. Figure 1 shows the flow of effectiveness measurement for the Erivedge®▼ pregnancy prevention programme.
Fig. 1

Effectiveness measurements for the Erivedge®▼ (vismodegib) pregnancy prevention programme. *Process indicator = specific to the UK. +Market Research = not conducted in all EU countries. HCP = healthcare professional

In the UK only, the Company also monitored awareness of the pregnancy prevention programme through the return of completed ‘Confirmation of Healthcare Professionals’ role’ forms as a process indicator to confirm whether educational materials had reached the target HCP population and that HCPs had read them. The aim of this form was to gain signed confirmation that each prescribing centre had at least one HCP who had read and understood their obligations with the Erivedge®▼ pregnancy prevention programme. The Company requested all HCPs to complete, sign and return the ‘Confirmation of Healthcare Professionals’ role’ forms to the Company using pre-paid envelopes provided.

When an institution placed an order for Erivedge®▼, the Company checked whether the institution had returned a signed ‘Confirmation of Healthcare Professionals’ role’ form. If not, the Company’s Medical Information department followed up on this. In UK, 69 of 2013 (3.4%) ‘Confirmation of Healthcare Professionals’ role’ forms were returned by HCPs and 37 patient details had been entered on the HCP web portal. ‘Verification of Counselling Forms’ were signed by 81.1% (30 out of 37) of patients who had been registered by their HCPs.

A market research study was also conducted in the UK using an anonymised online questionnaire in May 2014. The aims of this market research were to provide compliance and effectiveness data on the Erivedge®▼ pregnancy prevention programme and evaluate the impact of educational materials on the level of clinical knowledge or awareness of HCPs and resulting behaviours. The market research obtained feedback from qualified and practising oncologists and dermatologists (n = 31).

Results from this survey are presented in Table 4. Overall, 61.3% of respondents (n = 19) were aware of the Erivedge®▼ pregnancy prevention programme and 38.7% (n = 12) not aware. There were mixed responses to the questions asked. When questioned on actions required to be taken by HCPs upon awareness of the pregnancy prevention programme, the responses were as follows: 89.5% (17 out of 19) to educate patients; 84.2% (n = 16) to perform pregnancy testing and ensure use of contraception in a woman of childbearing potential; 78.9% (n = 15) to counsel patients on contraception and educate male patients; 73.7% (n = 14) to complete ‘Verification of Counselling Forms’, report pregnancies and refer patient to a specialist in case pregnancy occurred; 68.4% (n = 13) to give patients educational brochure and reminder card; 57.9% (n = 11) to complete survey for patients on HCP web portal and adhere to prescription restrictions [30].
Table 4

United Kingdom awareness survey for the Erivedge®▼ pregnancy prevention programme

Action by healthcare professional

No of respondents (n = 19)

% of respondents

Educate patients on teratogenic risks



Ensure pregnancy testing in women of childbearing potential



Ensure compliance with contraception in women of childbearing potential



Ensure contraceptive counselling to patients



Educate male patients on the use of condoms



Ensure all patients complete and sign a Verification of Counselling Form



Report pregnancies to the Company



Refer patient to a specialist obstetrician in the event of pregnancy



Provide patient with educational brochure and a ‘patient reminder card’



Complete the survey for this patient in the healthcare professional web portal



Limit prescriptions to 28 days of treatment. Continuation of treatment should require a new prescription



The outcome indicator in measuring the effectiveness of the EU Erivedge®▼ pregnancy prevention programme was the number of pregnancy cases in the Company global safety database. An estimated total of 1500 patients had received Erivedge®▼ globally from marketing experience in the PBRER reporting period 30 January 2014 to 29 July 2014.

No pregnancy cases had been reported to the Company in patients exposed to Erivedge®▼ either from maternal or paternal exposures [31].

The process indicator for measuring the effectiveness of the Kadcyla®▼ educational materials for medication error prevention in the EU was determination of awareness of the potential for medication errors with Kadcyla®▼ and Herceptin®. A description of how effectiveness was measured for the Kadcyla®▼ medication error prevention programme is presented in Fig. 2.
Fig. 2

Effectiveness measurements for Kadcyla®▼ (trastuzumab emtansine) medication error prevention. *Process indicator = UK specific; HCP = healthcare professional

In the UK, responses from HCP feedback questionnaires were used as an additional process indicator in measuring effectiveness. Approximately 15,000 feedback forms were mailed to HCPs in the UK. Sixty-eight out of 15,000 (0.5%) forms were returned to the Company. The results of this feedback are presented in Table 5.
Table 5

Feedback Questions on Kadcyla®▼ (trastuzumab emtansine) educational materials in the United Kingdom

Feedback question


Resulta n (%)

Q1: These educational materials helped me differentiate between the medicines: Herceptin® (trastuzumab), Herceptin® subcutaneous (trastuzumab) and Kadcyla®▼ (trastuzumab emtansine)

4 or 5

58 (85.3)

Q2: I understand the risk which may occur whilst prescribing any of Herceptin® (trastuzumab), Herceptin® subcutaneous (trastuzumab) and Kadcyla®▼ (trastuzumab emtansine)

4 or 5

63 (92.6)

Q3: I understand the mitigation measure described which will help to prevent such medication errors

4 or 5

54 (79.4)

Q4: These educational materials have, and will in the future, help to minimise risk in the prescription, preparation, or administration of Herceptin® (trastuzumab), Herceptin® subcutaneous (trastuzumab) and Kadcyla®▼ (trastuzumab emtansine)

4 or 5

52 (76.5)

Score Key: 1 = strongly disagree, 2 = disagree, 3 = neither agree or disagree, 4 = agree, 5 = strongly agree

a68 forms were received in total

The outcome indicator for the Kadcyla®▼ medication error prevention educational materials was the number of medication error cases from name confusion between Kadcyla®▼ and Herceptin® reported to the Company. The estimated global cumulative market exposure to Kadcyla®▼ was 6519 patients in the period 22 February 2014 to 21 August 2014 [32]. There were 24 medication error reports in this period for which 8 were from the EU, but none from UK. Although various adverse event terms with medication errors were reported, product confusion with Herceptin® was not reported for any. Further follow-up was not always possible as some reporters did not consent to be contacted.

The Company monitored the effectiveness and compliance of MabThera® PML educational programme by the use of distribution metrics for educational materials as a process indicator. A description of how effectiveness was measured for the MabThera® PML educational programme is presented in Fig. 3.
Fig. 3

Effectiveness measurements for MabThera® (rituximab) PML educational materials. PML = progressive multifocal leukoencephalopathy

Details of the distribution of educational materials in each EU country were recorded to provide reports on the implementation of MabThera® educational materials. The number of physicians prescribing MabThera® were evaluated against the number of individual patient alert cards distributed.

In the UK, patient alert cards were sent out in packs of 25 for re-ordering requests to physicians and nurses. A total of 209,270 patient alert cards were mailed to 3959 potential prescribers [33].

PML ADR reports were used as the outcome indicator for MabThera® PML educational materials. The global estimated market exposure to MabThera® was 105,876 patients for autoimmune indications (RA/GPA/MPA) for the reporting period, 18 Nov 2012 to 17 Nov 2013 (7374 for GPA/MPA).There were 65 PML cases in autoimmune indications (RA/GPA/MPA). Upon review, 10 of these cases were confirmed by the Company as PML from further analyses (8 in RA; 2 in GPA/MPA). Four cases were from the EU but none from UK. All patients with confirmed PML had significant risk factors and concomitant immunosuppressive therapy including disease-modifying antirheumatic drugs and tumour necrosis factor (TNF) inhibitors [32, 33].

Follow-up was performed by the Company with HCPs to verify whether all 10 patients received PML patient alert cards. In 2010 a Company patient alert card questionnaire was implemented and sent to respective reporters for EU confirmed PML cases. Responses were received for two cases which indicated that these patients did not receive patient alert cards prior to treatment [34].

4 Discussion

4.1 Implications of Study Findings

The study findings indicated that aRMMs are drug-specific and depend on the important safety issue being addressed. Educational materials for HCPs are often used as a means to raise awareness about important specific safety concerns with medicinal products. Possibly, this is because they are easier for MAHs to produce and implement, with less complexities across the EU, than other types of aRMMs. Educational materials may have been sent to many HCPs, including nurses and pharmacists, who have perhaps never prescribed Company medicines. The voluntary nature, in terms of documentation of process indicator-related actions by HCPs, could have contributed to low compliance [15].

The outcome measure for Erivedge®▼ was encouraging (0 reported pregnancy exposures) and may be a positive indicator of educational materials. The UK market research result showed a high level of awareness of the pregnancy prevention programme, whereas the process indicator results were not equally encouraging. HCPs may have read educational materials but not returned ‘Confirmation of Healthcare Professionals’ role’ forms. Similarly, they may counsel patients but not necessarily complete a Verification of Counselling Form or enter details into the HCP web portal. The low number of entries with the HCP web portal could be due to ignorance that the web portal is part of the pregnancy prevention programme despite clinician awareness. Reasons for variable responses to questions are unknown but could be possibly due to how HCPs perceived the questions and their relevance. There could also have been the factor of accidental omission in responses.

The Erivedge®▼ patient population being male and elderly in most cases may be a contributing factor to no pregnancies. The process indicators for Erivedge®▼ were targeted at increasing HCPs’ and patients’ awareness about the drug to minimise its teratogenic risks. HCPs may use the pregnancy prevention programme as a means of Erivedge®▼ ADR reporting but none were reported. From the Company’s global safety database, no pregnancies were reported for Erivedge®▼ patients or their partners. Reliance on HCPs alone to report ADRs is a limitation for the pregnancy prevention programme. Overall, considering the relatively large number of patients who received Erivedge®▼ globally with no maternal or paternal pregnancy exposures, the pregnancy prevention programme can be seen as effective for the study period.

Scores from Kadcyla®▼ feedback questions in UK shows that the majority of HCPs who returned the questionnaire understood the risk of medication errors with Herceptin® IV/SC and that educational materials help HCPs to differentiate between the two medicines. However, the low response rate for the Kadcyla®▼ survey in the UK is an indication that voluntary feedback may not always achieve a significant response. From this study, no medication errors from name confusion suggest that the educational materials were effective or could be a possible indication of lower medication error reporting in the EU to the Company [35, 36, 37].

Educational materials and patient alert cards could have contributed to the awareness of PML symptoms and subsequent reporting to the Company. Educational materials are therefore significant awareness tools to inform physicians and patients about the risk of PML with MabThera® and should be encouraged.

Patient alert cards were implemented for MabThera® in 2009. However, effectiveness check using follow-up questionnaires was introduced in 2010. It is therefore a challenge to determine effectiveness of educational materials with such mixed periods for implementation and effectiveness measurements.

Generally, all the MabThera® exposed patients since 2009 should have received patient alert cards but this could not be confirmed at the time of the study due to lack of data. Follow-up details to confirm if patients received alert cards for eight cases in RA could not be determined. This highlights the need to put in place effectiveness measures in a timely manner with the design and implementation of aRMMs. Current mandatory legislation for measuring effectiveness of aRMMs is useful for MAHs [4]. Follow-up confirmation that none of the two patients with confirmed PML in GPA/MPA received a patient alert card is an indication that the follow-up questionnaire as a process indicator is a useful effectiveness assessment tool and should be continued and reviewed periodically. The Company has been encouraged by usefulness of this tool and the need to put more stringent process indicators in place rather than rely on distribution metrics for educational materials and was in the planning phase of a drug utilisation study and patient card evaluation.

4.2 Potential Ways to Improve Specific aRMMs

There is a lack of clarity from results of the market research as to whether clinicians prescribing Erivedge®▼ without awareness on the pregnancy prevention programme were unaware of the teratogenic risk of Erivedge®▼ or its pregnancy prevention programme. Possibly, these clinicians were aware of the teratogenic risk with Erivedge®▼ from the SmPC but did not know about the pregnancy prevention programme or educational materials. However, the pregnancy prevention programme is mentioned specifically in the SmPC, as well as information on educational materials, which could be an indication that aRMMs may not necessarily be noticed when using the SmPC as the source of communication. These inconsistencies could have stemmed from a lack of clarity in the wording of the market research question that prompted these answers. There could have been potential confusion with names and forms identification, reminders and educational materials within the pregnancy prevention programme pack. Considering that the initial mailing of the Erivedge®▼ pregnancy prevention programme educational materials to HCPs was done at product launch (August 2013), which was approximately 9 months before market research was conducted (May 2014) to seek feedback on these educational materials, the gap between these activities could have had an impact on information recall. The market research could have been done sooner for a better recall although the timing of conducting a market research does not guarantee better results.

For the Kadcyla®▼ process indicators, a minority (7.4%) of respondents in the UK disagreed or were neutral in their understanding of the possible risk of medication errors when prescribing Herceptin® IV/SC and Kadcyla®▼. This question could have been asked in a different way to ascertain HCPs knowledge about the similarities and differences between these similar drugs and to increase awareness about possible medication errors related to these two drugs and how to prevent such occurrences.

Proactive patient follow-up in those who have developed PML after MabThera® exposure may help to identify loop holes in determining receipt of educational materials. MabThera® questionnaires may have been sent to HCPs who reported PML cases but not necessarily the patient’s prescriber. Also, genuineness of answers is unpredictable because it is only patients treated with MabThera® or their carers who can really confirm whether they had received patient alert cards or not.

4.3 Potential Ways to Improve Measuring the Success of Company aRMMs in The Future

Mailing of educational materials may not be sufficient to ensure HCPs have the relevant information. The Company cannot assume that HCPs who are sent educational materials had received, read or retained the knowledge. In this regard, the Company needs to consider different methods, including medical science liaison-targeted education of key prescribers or HCPs, and mandatory process indicators to collect and enforce more specific feedback face-to-face or via telephone appropriately.

The Company needs to include pharmacists, nurses, patients and carers in measuring effectiveness of aRMMs to ensure that data collected are fully reflective of clinical practice [38]. To determine whether or not patients have been adequately counselled and understand important safety information, patients, carers, pharmacists and nurses could be utilised as an essential verification step to close any loop holes. For example, patients could have access to an electronic site to verify that they have been counselled or partake in a survey to provide feedback on awareness and understanding of safety risks. Pharmacy data can be useful in the provision of an insight into the effectiveness of safety warnings and clinical behaviour of HCPs [39, 40]. Pharmacists can be engaged in a registration process to be able to order and dispense medicines with potential safety concerns and certification upon educational courses to check that patients understand contents of all patient educational materials. They could also document if a patient has been adequately counselled by signing prescriptions or using annotations in the patient’s drug history or medication chart. The Company could simplify aRMMs by reducing the number of educational materials and using simple, well-targeted open and closed questions to solicit more information.

It would also be helpful for regulatory authorities to provide specific guidance on the content of aRMMs especially across similar safety concerns. The Pharmacovigilance Risk Assessment Committee could act as gatekeepers of aRMMs to work with pharmaceutical companies in setting thresholds for success to ensure safety outcomes in the implementation and effectiveness of aRMMs.

4.4 Strengths and Limitations

These research findings may also apply to non-EU countries where these products are marketed by the Company subject to local pharmacovigilance regulations. Some of the specific measures discussed were UK-specific and not necessarily representative of what is happening across Company branches in the EU and beyond. However, this pinpoints the challenges faced by a global Company in trying to fit aRMMs across different countries and healthcare systems. Some details of the aRMMs could have been missed, for example, proposals by the Company to ensure effectiveness of aRMMs and feedback from regulatory authorities due to multiple repositories for document storage.

Additional risk minimisation measures described therein, along with the way they are implemented are dynamic. The author is aware, for example, that changes have been made to the implementation of the Erivedge®▼ pregnancy prevention programme in the UK to improve HCP awareness and compliance since this research was conducted.

5 Conclusions

aRMM programmes need to be tailored to the product and the specific important safety issue to be addressed. This research shows that, typically, the Company implements aRMMs through postal distribution of educational materials to HCPs. For the chosen products, a reasonable awareness of aRMMs amongst HCPs is a positive indicator of success in the use of educational materials. However, low response rates from surveys indicate that voluntary feedback may not always achieve the desired level of response and therefore may not be the most appropriate method to use as process indicators in measuring effectiveness. For each product there were additional mitigating factors to be considered. It is likely that the elderly patient population for Erivedge®▼ was a contributing factor to the lack of reported pregnancies. For Kadcyla®▼, although no name confusion medication error cases were reported, the known challenge of low rates of spontaneous adverse event reporting and lack of detailed information in individual case safety reports could also have an impact on evaluating safety outcomes. The disease history and concomitant medications of patients on MabThera® were also likely alternative explanations/risk factors for the reported cases of PML infections. Pharmacists and patients should be actively involved in measuring effectiveness of aRMMs. There is the need for regulatory guidance to further define the elements and desired outcomes of aRMMs for consistency.



This work was conducted by Elaine Agyemang in partial fulfilment of the regulations governing the award of a Master’s degree in Pharmacovigilance at the University of Hertfordshire. Elaine Agyemang was a student at the University of Hertfordshire at the time of the research in 2014 and an employee of Roche Products Limited, UK.

Dr. Lorna Bailey, also an employee of Roche Products Limited as a Risk Minimisation Coordinator (UK & Malta), supervised this research together with Dr John Talbot, Senior Lecturer at the School of Life and Medical Sciences, University of Hertfordshire.

In compliance with European Union pharmacovigilance legislation, periodic benefit-risk evaluation reports (PBRERs) and individual case safety reports are submitted to regulatory agencies. Additionally, for Erivedge®▼, an annual report is submitted to the UK Medicines and Healthcare products Regulatory Agency (MHRA). Market research results and distribution metrics are not available publicly, but provided to regulatory authorities as discussed in the PBRER and annual report.

Compliance with Ethical Standards


Elaine Agyemang received partial funding from Roche Products Limited, UK towards her Master’s degree in Pharmacovigilance at the University of Hertfordshire.

Conflict of interest

Elaine Agyemang has been an Employee of Roche Products Limited (Welwyn Garden City, UK) since 2011. Roche Products Limited conducted a factual accuracy check on the final article and manuscript but any decision to incorporate comments was made solely at the discretion of the author.

Dr. Lorna Bailey has been an Employee of Roche Products Limited (Welwyn Garden City, UK) since 2005 and received stock options as per Roche employment.

Dr. John Talbot declares that he has no conflict of interest.

Ethical Approval

Ethical approval was not required for this study.


  1. 1.
    European Commission. Directive 2010/84/EU of the European parliament and of the council. 2010. Retrieved August, 22, 2014, from
  2. 2.
    European Commission. Regulation (EU) no 1235/2010 of the European parliament and of the council. 2010. Retrieved August, 22, 2014, from
  3. 3.
    European Medicines Agency Guideline on good pharmacovigilance practices Module V: risk management systems. 2012. Retrieved March, 12, 2014, from
  4. 4.
    European Medicines Agency. Guideline on good pharmacovigilance practices (GVP) Module XVI, risk minimisation measures: selection of tools and effectiveness indicators. 2014. Retrieved March, 12, 2014, from
  5. 5.
    European Medicines Agency. Position paper on potential medication errors in the context of benefit-risk balance and risk minimisation measures. 2013. Retrieved April, 11, 2014, from
  6. 6.
    Uhl K, Trontell A, Kennedy D. Risk minimization practices for pregnancy prevention: Understanding risk, selecting tools. Pharmacoepidemiol Drug Saf. 2007;16(3):337–48.CrossRefPubMedGoogle Scholar
  7. 7.
    European Medicines Agency. Guideline on good pharmacovigilance practices (GVP) Module IX: signal management. 2012. Retrieved August, 28, 2014, from
  8. 8.
    Talbot JCC, Aronson JK. Stephen’s detection and evaluation of adverse drug reactions. 6th ed. Chichester: Wiley-Blackwell; 2012.Google Scholar
  9. 9.
    Aydinkarahaliloglu ND, Aykac E, Kasap Y, Durmus N, Babacanoglu C, Basgut CE, et al. Pharmaceutical risk management in Turkey: the first national overview. Regul Toxicol Pharmacol. 2013;67(3):344–50.CrossRefPubMedGoogle Scholar
  10. 10.
    Rychetnik L, Frommer M, Hawe P, Shiell A. Criteria for evaluating evidence on public health interventions. J Epidemiol Community Health. 2002;56(2):119–27.CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Prieto L, Spooner A, Hidalgo Simon A, Rubino A, Kurz X, Arlett P. Evaluation of the effectiveness of risk minimization measures. Pharmacoepidemiol Drug Saf. 2012;21(8):896–9.CrossRefPubMedGoogle Scholar
  12. 12.
    Banerjee AK, Zomerdijk IM, Wooder S, Ingate S, Mayall SJ. Post-approval evaluation of effectiveness of risk minimisation: methods, challenges and interpretation. Drug Saf. 2014;37:33–42.CrossRefPubMedGoogle Scholar
  13. 13.
    Zomerdijk IM, Trifirò G, Sayed Tabatabaei FA, Sturkenboom MC, Straus SM. Additional risk minimisation measures in the EU: are they eligible for assessment? Pharmacoepidemiol Drug Saf. 2013;22(10):1046–53.PubMedGoogle Scholar
  14. 14.
    Perera G, Holbrook A, Thabane L, Foster G, Willison DJ. Views on health information sharing and privacy from primary care practices using electronic medical records. Int J Med Inform. 2011;80(2):94–101.CrossRefPubMedGoogle Scholar
  15. 15.
    Crijns I, Zomerdijk I, Sturkenboom M, de jong van den Berg L, Straus S. A comparison of pregnancy prevention programmes in Europe. Expert Opin Drug Saf. 2014;13(4):411–20.CrossRefPubMedGoogle Scholar
  16. 16.
    Roche Genentech. Erivedg (vismodegib), prescribing information. 2012. Retrieved October, 17, 2014, from
  17. 17.
    Amin SH, Motamedi KK, Ochsner MC, Song TE, Hybarger CP. Mechanisms and efficacy of vismodegib in the treatment of basal cell carcinoma. Discov Med. 2013;16(89):229–32.PubMedGoogle Scholar
  18. 18.
    Lyons TG, O’Kane GM, Kelly CM. Efficacy and safety of vismodegib: a new therapeutic agent in the treatment of basal cell carcinoma. Expert Opin Drug Saf. 2014;13(8):1125–32.CrossRefPubMedGoogle Scholar
  19. 19.
    Lear JT, Corner C, Dziewulski P, Fife K, Ross G, Varma S, et al. Challenges and new horizons in the management of advanced basal cell carcinoma: a UK perspective. Br J Cancer. 2014;10:1038.Google Scholar
  20. 20.
    Lomas A, Leonardi Bee J, Bath Hextall F. A systematic review of worldwide incidence of non-melanoma skin cancer. Br J Dermatol. 2012;166(5):1069–80.CrossRefPubMedGoogle Scholar
  21. 21.
    Gliklich RE, Dreyer NA, Leavy MB. Registries for evaluating patient outcomes: a user’s guide. 3rd ed. Rockville: U.S. Department of Health and Human Services; 2014.Google Scholar
  22. 22.
    Lipinski RJ, Hutson PR, Hannam PW, Nydza RJ, Washington IM, Moore RW, et al. Dose- and route-dependent teratogenicity, toxicity, and pharmacokinetic profiles of the hedgehog signalling antagonist cyclopamine in the mouse. Toxicol Sci. 2008;104(1):189–97.CrossRefPubMedPubMedCentralGoogle Scholar
  23. 23.
    Lipinski RJ, Song C, Sulik KK, Everson JL, Gipp JJ, Yan D, et al. Cleft lip and palate results from hedgehog signaling antagonism in the mouse: phenotypic characterization and clinical implications. Birth Defects Res. 2010;88(4):232–40.Google Scholar
  24. 24.
    Oostra DR, Macrae ER. Role of trastuzumab emtansine in the treatment of human epidermal growth factor receptor-2 -positive breast cancer. Breast Cancer. 2014;6:103–13.PubMedPubMedCentralGoogle Scholar
  25. 25.
    Electronic Medicines Compendium. Kadcyla 100 mg & 160 mg Powder for Concentrate for solution for infusion. Summary of product characteristics. 2014. Retrieved October, 15, 2014, from
  26. 26.
    Korhonen R, Moilanen E. Anti-CD20 antibody rituximab in the treatment of rheumatoid arthritis. Basic Clin Pharmacol Toxicol. 2010;106(1):13–21.PubMedGoogle Scholar
  27. 27.
    Dreyfus DH, Na CR, Randolph CC, Kearney D, Price C, Podell D. Successful rituximab B lymphocyte depletion therapy for angioedema due to acquired C1 inhibitor protein deficiency: association with reduced C1 inhibitor protein autoantibody titers. Isr Med Assoc J. 2014;16(5):315–6.PubMedGoogle Scholar
  28. 28.
    Electronic Medicines Compendium. MabThera 100 mg and 500 mg concentrate for solution for infusion: Summary of product characteristics. 2014. Retrieved October, 15, 2014, from
  29. 29.
    Nissen JC, Hummel M, Brade J, Kruth J, Hofmann WK, Buchheidt D, et al. The risk of infections in hematologic patients treated with rituximab is not influenced by cumulative rituximab dosage: a single center experience. BMC Infect Dis. 2014;14:364.CrossRefPubMedPubMedCentralGoogle Scholar
  30. 30.
    Roche Products Limited. First annual report on the compliance and effectiveness of the Erivedge®▼ (vismodegib) pregnancy prevention programme. 2014.Google Scholar
  31. 31.
    Roche Products Limited. Periodic safety update report 1057983: vismodegib/Erivedge®▼. Roche Products Limited. Company Literature. 2014.Google Scholar
  32. 32.
    Roche Products Limited. Periodic safety update report 1061128: trastuzumab emtansine/Kadcyla®▼. Roche Products Limited. Company Literature. 2014.Google Scholar
  33. 33.
    Roche Products Limited. Periodic safety update report 1053866: rituximab/MabThera®/Rituxan®. Roche Products Limited. Company Literature. 2014.Google Scholar
  34. 34.
    Roche Products Limited. PML Drug Safety Report 1058316: rituximab/MabThera®/Rituxan®. Roche Products Limited. Company Literature. 2014.Google Scholar
  35. 35.
    Lesar TS, Briceland L, Stein DS. Factors related to errors in medication prescribing. J Am Med Assoc. 1997;277(4):312–7.CrossRefGoogle Scholar
  36. 36.
    Waller P. An introduction to pharmacovigilance. Chichester: Wiley-Blackwell; 2010.Google Scholar
  37. 37.
    Waller PC, Evans SJ. A model for the future conduct of pharmacovigilance. Pharmacoepidemiol Drug Saf. 2003;12(1):17–29.CrossRefPubMedGoogle Scholar
  38. 38.
    Aronson JK. Medication errors: what they are, how they happen, and how to avoid them. Q J Med. 2009;102(8):513–21.CrossRefGoogle Scholar
  39. 39.
    Schwappach DLB, Mülders V, Simic D, Wilm S, Petra A, Thürmann PA. Is less more?: patients’ preferences for drug information leaflets. Pharmacoepidemiol Drug Saf. 2011;20:987–95.PubMedGoogle Scholar
  40. 40.
    Morrato EH, Staffa JA. Effectiveness of risk management plans: a case study of pemoline using pharmacy claims data. Pharmacoepidemiol Drug Saf. 2007;16:104–12.CrossRefPubMedGoogle Scholar

Copyright information

© The Author(s) 2017

Open AccessThis article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (, which permits any noncommercial use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.Roche Products LimitedWelwyn Garden CityUK
  2. 2.School of Life and Medical SciencesUniversity of HertfordshireHertfordshireUK

Personalised recommendations