FormalPara Key Points

A mobile application (app) designed for adverse drug reaction (ADR) reporting and product safety alerts can help to augment pharmacovigilance activities and extend a health authority’s reach to patients and healthcare professionals.

Of particular value to app users was the ability to learn about the safety profiles of medicines through user-friendly, interactive graphics within the app as well as privacy and data protection features.

While uptake and use of the app to date seems modest in comparison with other ADR-reporting modalities, it is reasonable to expect that app-based reporting will grow in importance as a younger generation of app-literate patients matures and smartphone owners increasingly use their mobile devices to access the Internet.

1 Introduction

The Innovative Medicines Initiative (IMI) WEB-RADR (Recognising Adverse Drug Reactions) consortium was a public–private partnership supported by the IMI Joint Undertaking (http://www.imi.europa.eu) under Grant Agreement no 115632. Participating members were from European regulatory agencies, the pharmaceutical industry, academia, patient groups and other organisations with an interest in pharmacovigilance (PV). The aim of WEB-RADR was to develop and evaluate two digital tools to support PV activities and promote patient safety, and ultimately provide policy, technical and ethical recommendations on implementing such technology. Full details of IMI WEB-RADR’s objectives and approaches have been detailed elsewhere [1]. The outputs from WEB-RADR arose from four work packages: two work packages undertook original research in social media and mobile application (app) technology; a third evaluated the scientific impact of the original research to determine where it had potential to add value to existing PV methodologies. A fourth work package addressed governance and policy, including personal data protection and ethical and societal considerations related to the use of mobile apps and social media for PV. This paper will focus on recommendations resulting from the work on apps. Recommendations resulting from the research conducted using social media will be the subject of a separate publication.

The profusion of smartphones and other mobile devices offers an opportunity to further engage the public in reporting suspected adverse drug reactions (ADRs) to national competent authorities (NCAs) and to improve communication of emerging safety issues from NCAs to the public and healthcare professionals (HCPs). Together with web-based forms, mobile devices offer a platform for developing real-time PV systems that can enable near-instantaneous transmission of patient safety information at the point of need, potentially improving health outcomes. Data collection in PV typically relies on voluntary reporting of suspected ADRs submitted to NCAs and pharmaceutical companies by HCPs. Over the last 20 years, patient reporting has been increasingly recognised as a valuable addition to safety monitoring activities [2, 3]. This growing interest in the patient perspective has coincided with a proliferation of technology, which in turn has increased access to data, accelerated information exchange, and fostered transparency. It is therefore reasonable to assume that efforts to improve ADR reporting and patient safety communication could benefit from the adoption of new technologies such as mobile apps.

Early experiences with mobile apps for medical product safety have been promising and have demonstrated potential for broad engagement of the public in safety reporting [4]. At the same time, concerns over security and potential for misuse of personal data have grown with several well-publicised breaches such as those at Yahoo, CEX and Facebook [5,6,7]. Aside from developing and evaluating digital tools themselves, the WEB-RADR project also examined the ethical, societal and privacy concerns related to mobile apps, as well as technological and promotional aspects, from a European perspective.

The WEB-RADR mobile apps were designed to facilitate instantaneous reporting of suspected ADRs directly to NCAs by patients and HCPs, as well as timely dissemination of accurate PV information from NCAs to HCPs and the public, thereby allowing potential two-way exchange of safety information. The apps have been made freely available by PV centres in the United Kingdom (UK; Medicines and Healthcare products Regulatory Agency [MHRA]), the Netherlands (NL; Lareb), and Croatia (HR; HALMED) in both Google Android and Apple iOS formats; links to download the apps are available at https://web-radr.eu/mobile-applications-for-adr-submission. A fourth version of the app has been piloted in Burkina Faso and Zambia in collaboration with the World Health Organization (WHO); at the time of writing, this fourth version was still undergoing validation and assessment.

The use of mobile apps for ADR reporting and the downstream use of these data for PV is a relatively new and unexplored concept. The anticipated impact of these apps will depend on the degree of uptake by the public and HCPs, the quality and volume of ADRs reported, and the utility of the data in safety signal detection and evaluation. In WEB-RADR, each NCA devised its own strategy for mobile app promotion best suited to its national context and providing a range of engagement models for assessment. Focus group discussions and face-to-face interviews were conducted with potential app users from multiple countries [8]. Enthusiasm for the app was high and the feedback received was constructive, targeted and generally encouraging. Comments from the participants have been used to inform further app development to improve uptake and encourage continued use of the app. As of 31 December 2017, there have been over 17,000 app downloads and 838 suspect ADR reports received since the first app was launched on 14 July 2015 (Table 1).

Table 1 Breakdown of app downloads and reported ADRs by country to 31 December 2017a
Full size table

To integrate ADR reports submitted via the apps into existing ADR processing workflows at NCAs, a direct connection was made between each app and the respective NCA’s existing electronic gateway. When a user submits an ADR report through the app, the reported data are automatically converted into standard International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) E2B(R2) format, allowing it to be submitted directly to the NCA’s PV portal for triage, assessment and signal detection alongside ADR reports received through other reporting channels. The apps also provide a means for NCAs to broadcast to users personalised safety alerts and news about user-selected drugs. App users can also explore aggregate ADR report information in interactive graphs that display volumes of reports received by the NCA (at the national level in the UK and NL and at the global or EU level for HR) for medicines of interest according to Medical Dictionary for Regulatory Activities (MedDRA®; https://www.meddra.org) system organ class and, where available, by patient demographics such as age and sex. NCAs submitted suspected ADRs from the mobile apps to EudraVigilance by batching app reports with reports submitted through other channels, in accordance with usual reporting requirements [9].

The following recommendationsFootnote 1 relate to policy considerations and original research pertaining to the development and utility of a mobile app platform for PV conducted under the auspices of the IMI WEB-RADR project between September 2014 and September 2017. In total, we present 27 separate recommendations with the intention of informing PV professionals, particularly those with an interest in the development of digital technologies to enable PV.

2 Summary of the Research and Recommendations

The recommendations outlined in this paper have been developed based on evidence from the activities undertaken through IMI WEB-RADR. For a full description of the studies, refer to the cited original publication or technical report. For the sake of brevity, unless explicitly stated to the contrary, the terms ‘mobile app’ (or simply ‘app’) are used to cover software designed for mobile devices, irrespective of the means of presentation, i.e. via a smartphone or a tablet. Recommendations were drafted by the authors of the various outputs that collectively described the outcomes of the app work package. Draft recommendations were subjected to several rounds of review by other work package members, resulting in either their revision or rejection. Recommendations were typically rejected on the grounds that they were too general in nature or not adequately supported by insights arising from the work of WEB-RADR or if they were considered not implementable in practice. Final-draft recommendations were circulated to the WEB-RADR General Advisory BoardFootnote 2 for review and comment. Finally, all consortium participants were invited to review a mature draft of this paper.

2.1 Drivers for Mobile Application Development and Other Pre-development Considerations

Prior to committing the effort and investment needed to develop any mobile app, it is important to gain an understanding of the setting in which the app is to be used and the intended target group. Prior knowledge of technical and regulatory limitations and likely barriers to uptake and usage can inform app design and implementation. The recommendations provided in this section could be deemed good practice for app development in any setting; however, it is pertinent to share the specific experiences from WEB-RADR as a means of emphasising the value of undertaking a scoping exercise prior to full-scale app development. The value versus opportunity cost of app development has not been factored into these recommendations since it is likely to vary from country to country given differences such as availability of established means of ADR reporting, infrastructure maturity (internet, mobile networks, etc.), education and awareness of healthcare systems generally and PV more specifically.

Consideration of the intended use of the technology is also important. For example, the key benefits in the EU settings evaluated were related to easy access to trusted regulatory information and the impact in terms of reporting considered additive to the suite of tools available to reporters [8]. However, early feedback from the pilots in Burkina Faso and Zambia points to the value of off-line functionality (reporting and news) independent of access to a website.

Furthermore, there are efficiencies in the costs associated with standardised/shared platforms/code, with significantly reduced maintenance costs associated with the app version used in Burkina Faso and Zambia developed towards the end of the WEB-RADR project. Technology now enables the use of common code to deliver the same functionality or multiple modalities; this offers a more flexible and cost-effective framework to deliver on user needs independent of the end user device. See Table 2 for app development recommendations.

Table 2 Recommendations for application (app) development
Full size table

2.2 Mobile Application Design

Traditional reporting mechanisms require significant time and effort to complete and submit, inhibiting ADR report submission to Marketing Authorisation Holders (MAHs) and regulators [14]. Mobile app design for PV offers the opportunity to improve upon traditional ADR-reporting forms for patients and HCPs alike. The app design process in WEB-RADR focused on developing an uncluttered visual design incorporating feedback from patient and HCP groups, while maintaining enough consistency to enable comparison with suspected ADRs obtained through other mechanisms. Since the reports submitted through the app would be passed along to NCAs, the app screens had to include the minimum necessary data fields to comply with ICH E2B(R2) standards and requirements in line with the guideline on good pharmacovigilance practices (GVP) Module VI, which served as one of the starting points for their design [15]. The four mandatory reporting elements were the focus of data collection: (1) patient information, (2) reporter information, (3) suspected medicinal product and (4) suspected adverse reaction.

The digital platform afforded opportunities to have adaptive forms. Since ADR reports contain information about medical products and ADRs, structured report fields may require knowledge of medical terminology; however, it is unlikely that all patients will be familiar with this type of vocabulary. In the UK, work was done as part of WEB-RADR to investigate the use of patient-friendly MedDRA® terms for reporting ADRs, and a list of patient-friendly terms is available on the MedDRA Maintenance and Support Services Organization (MSSO) website [16].

There are also several features that could be integrated into an app to facilitate processing and analysis by NCAs, including structured and hierarchical event ontologies. Autofill features, an adaptive suggestion that appears in a field automatically when a user types a few letters, for symptoms and product names may expedite the report submission process and ensure that these data points are spelled correctly and adhere to standardised terminologies.

While structured text fields such as drop-down menus or radio buttons promote data standardisation, these may not be ideal in every situation. For example, patients may not comprehend ontology-based medical terms suggested in a drop-down menu or they may find that the options do not best describe their experiences. From a cognitive design perspective, overly structured fields may negate useful nuance in the patient’s experience, reducing the usefulness of the report. Early testing suggested that patients may find that completing free-text fields to describe ADRs in their own words preferable to selecting options from drop-down menus with multiple choice and radio buttons [17]. Such design considerations can conceivably impact report completeness and accuracy.

It is important to ensure that a mobile PV app can be used by as wide a population as possible. Technology industry standards can provide guidance on design elements such as preferred font size for legibility, preferred colours or shades for text and image visibility and button placement for touchscreens. Adhering to such standards as much as possible ensures that a mobile app can be used by patients and HCPs with a variety of abilities. Additionally, when designing an app for ADR reporting, any constraints imposed by regulatory requirements, database or technology limitations and resource implications should be carefully evaluated.

Beyond the visual design of the app, the digital environment permits the creation of a positive feedback loop whereby reporters receive confirmation and positive reinforcement for submitting their report via email or in-app messaging. In early experiences with PV reporting, such feedback mechanisms were believed to have improved the user experience [4]. This also creates a conduit for follow-up reporting to NCAs as well as a mechanism for push notifications when new safety alerts are issued by NCAs. Some elements of this approach to enhancing user engagement were assessed in WEB-RADR.

Since the base WEB-RADR mobile app was to be deployed in multiple countries, yet evaluated centrally by the project, a certain level of design flexibility was required to allow country-specific language and branding. In some countries the NCA’s ADR-reporting programme already has a recognisable identity and the app was therefore designed to be extensible with colours, logos, etc. to mimic extant reporting forms and branding. See Table 3 for design recommendations.

Table 3 Recommendations for application (app) design
Full size table

2.3 Mobile Application Content and Data-Related Considerations

The WEB-RADR app has been developed to improve the two-way exchange of information (i.e. the reporting of ADRs and the provision of patient safety information). Therefore, the app contains an information-sharing component and an ADR-reporting component. The information-sharing component enables app users to receive safety alerts and news, as well as view aggregated statistics on previously reported ADRs.

A qualitative study that involved focus group discussions and face-to-face interviews showed that patients and HCPs were generally positive about the existing content of the app and suggested some additional functions that could also be useful [8]. Perspectives on the content of the app were further assessed in a large online survey among patients and HCPs in Europe [22]. It was shown that potential app users were somewhat more interested in an app for two-way exchange of safety information than in an app that only served one purpose (i.e. either just the information-sharing component or just the ADR-reporting component). The survey provided many suggestions on how to further improve the app to increase ADR reporting to NCAs.

A second online survey was conducted among current users of the WEB-RADR app. The survey asked about their experiences with the app. Most responders had a positive view of the value of the app in general, for both the ADR-reporting feature and the patient safety information provided in the app. Some suggestions were also received on how to further improve the app.

A quantitative survey study found that respondents from different countries preferred access to different types of information, e.g. drug stock levels (Croatia), alternative medicines (the Netherlands), and ADR mitigation strategies (the UK). Information on newly identified drug–drug interactions was of common interest across the three countries surveyed [8, 22]. This echoes our earlier recommendation in the pre-development considerations section (Sect. 2.1): when determining what kind of information to provide to users in the app, consideration should be given to user- and country-specific content. Interests among different user groups could differ, and countries may vary greatly in terms of what safety information may need to be broadcast; for example, local regulatory communication of newly identified drug–drug interactions may vary where there are formulary differences across countries. See Table 4 for a summary of the recommendations regarding content and data-related considerations.

Table 4 Recommendations regarding application (app) content and data-related considerations
Full size table

2.4 Mobile Application Implementation, Awareness and Uptake

NCAs can use various strategies to inform potential users about the existence of the mobile app, but it is most efficient to use those channels and sources that reach the audience with potentially the highest interest in the app.

To assess which patients and HCPs are most interested in an app with capabilities for the two-way exchange of safety information, data from a European-wide online survey study were used [22]. In the survey, patients and HCPs were asked to what extent they were interested in the app. Analyses of the HCP data showed that HCPs who already (at least sometimes) use a health app were very interested in the app. This was also shown among the patients. In addition, patients that were younger were very interested in the app.

See Table 5 for a summary of recommendations on implementation, awareness and uptake.

Table 5 Recommendations on application (app) implementation, awareness and uptake
Full size table

2.5 Mobile Application Impact and Miscellaneous Recommendations

The recommendations presented in Table 6 relate to the assessment of the overall impact of mobile reporting apps for PV and other stand-alone recommendations.

Table 6 Recommendations on assessment of the overall impact of applications (apps) for pharmacovigilance and other stand-alone recommendations
Full size table

3 Discussion

A series of recommendations relevant to the development of mobile apps for the reporting of ADRs are presented with the intention of informing PV professionals and other stakeholders, particularly those with an interest in the development of research methods and digital technologies. Each recommendation is based on the experiences and outcomes of the research conducted under the auspices of IMI WEB-RADR. Collectively, the recommendations should not be considered a comprehensive treatise on the subject, rather they should be considered within the context of the entire corpus of research and regulatory guidance documents that exist outside of IMI WEB-RADR.

Since the start of the WEB-RADR project, mobile ADR-reporting apps have been launched in five separate countries. This group of participating countries has presented both diverse target populations of HCPs and non-HCP users, predominantly patients, as well as widely varying reporting systems and diverse technical resources that have impacted how each app has been configured. It is our hope that, having launched these apps in five unique settings, the recommendations, when taken together with their caveats, will be applicable to many other settings. That said, careful consideration should be given to the generalisability of the recommendations in circumstances that are substantially different from those described in the original research.

Overall, our experience has indicated that a deep understanding of the target app-user population, through local knowledge and potential user outreach prior to embarking on development activities, can facilitate effective design from the start. However, it is equally important to continue improving the app based on user feedback and evaluation of outputs after any public release—particularly following initial launch and major updates. Additionally, preferences and needs for patient safety information and reporting can vary greatly among different groups, and a balance must be struck between providing enough features that are useful to a wide range of users without attempting to create an app that ‘does it all’.

Communication in support of the launch of the app is important, followed by further provision of information at appropriate intervals to keep stakeholders informed about enhancements and new features of the software. Once established, a good place to inform patients about the app is at first prescription or first dispensing of a medicine. Patients are most likely to experience ADRs when starting new medications [32].

It is important to note that even if an app for ADR reporting is well-designed and user-friendly, it will not serve its purpose if the target population is not familiar with that purpose. Patient awareness of reporting ADRs is therefore critical to adoption and utilisation of the app and may be a factor beyond the scope of designing and developing the app itself. It might be necessary to raise public awareness of ADR reporting prior to evaluating the need for an app; in other cases, the existence of a mobile app could serve to drive awareness of ADR reporting among the public, and it may be useful to address underreporting barriers, as part of a publicity strategy. More research is needed to explore ways to encourage patient reporting of ADRs in a balanced way that does not lead to bias or soliciting.

Since release, each app has seen varying degrees of exposure. The Yellow Card mobile app was launched as an extension of MHRA’s Yellow Card Scheme—a safety reporting programme for human medicinal products that has been running for over 50 years. To date, MHRA has received 505 ADR reports through its mobile app, launched in July 2015 (see Table 1). Objectively speaking, this number pales in comparison to the 26,000 ADR reports the MHRA received in 2016 directly from HCPs and the public, and there is a tendency to view this as a shortcoming of the mobile app itself [33]. However, the value identified by users was much broader than just reporting and extended to engagement with the wider PV system and accessing timely safety information. As smartphone users increasingly use their mobile devices to access the Internet (in the UK, for example, 73% of adults browsed the Internet using their mobile phones in 2017, compared to 36% in 2011), so grows the opportunity for health authorities to reach patients through digital channels where they are already seeking information [34].

It is not practical to expect the outputs from novel, patient-centric reporting channels to mimic those from traditional channels, particularly as the value of patient-reported individual case safety reports (ICSRs) has only been recognised relatively recently [3]. There is therefore a need to re-calibrate expectations and perspectives when evaluating mechanisms for patient ADR reporting and establish a methodology that can allow an assessment of these initiatives entirely on their own, and not in the context of traditional reporting channels. For instance, Lareb has found that its Bijwerking app has encouraged participation from new members of the public, specifically HCPs, who have never submitted an ADR report previously, suggesting that the existence of the mobile app has increased the breadth and diversity of reporters [35]. Additionally, evaluation of reports originating from the apps revealed that while the resulting data had lower completeness scores, the app reports demonstrated clinical quality that did not differ significantly from that of reports submitted via conventional reporting channels and contributed to signal detection and safety evaluation activities.

Probably the most important recommendations are those that relate to engagement of users of the app, in particular, establishing the two-way interchange of safety information. Users of the app valued the provision of graphs and charts showing the safety profile of their medicines. This positive reinforcement of the value of transmitting ICSR was a feature of the feedback received during focus groups and interviews. However, a commitment to provide automated feedback should not override the current practices in terms of providing feedback for other reporting routes [22]. In general, provision of more detailed feedback information should be well thought through, as it may introduce questions or concerns relating to the complexity of mandatory ADR reporting requirements that may require time to address.

Equally important was the protection of data privacy, in accordance with national and European laws. Patients wanted assurance that the highest standards of confidentiality were maintained during the data reporting and transfer processes. Patients also found it important to be able to review their own previously submitted reports and to submit corrections or follow-up information later, if needed. Overall, the app and surrounding publicity brought the importance of PV to the attention of various stakeholders in public health. In the case of HCPs, this served as a reminder to report suspected ADRs, but for patients and caregivers, the app provided an alerting mechanism, highlighting the value of the public in recognising potential safety concerns.

The app could be considered an important mechanism to facilitate compliance with the European Union PV framework legislation which became operational in July 2012, obliging all NCAs and MAHs to record and report cases of suspected ADRs received from patients. Careful consultation with a range of patients and patient support groups helped in the design and delivery of a functional app to facilitate patient reporting. An app provides an additional means of reporting suspected ADRs and communication and, furthermore, projects an innovative and progressive image; in turn this fosters engagement with stakeholders. Health authorities might consider providing an app as an additional means of reporting and/or communication; however, the value versus opportunity costs, which were not specifically evaluated in WEB-RADR, are likely to vary from country to country. It is important that regulators are accessible through means that patients and HCPs use in their day-to-day lives; it should not be any more challenging to report an ADR than to use any other service. Furthermore, responsiveness to new technologies projects an innovative and progressive image that in turn fosters trust and improves dialogue with stakeholders.

A maintenance and support programme has been established with the aim of ensuring the sustainability of the app; however, questions remain about its long-term utility. From the patient perspective, there may be no need to repeatedly use the app to submit an ADR report; many patients will not experience multiple ADRs (if at all); thus, most users will not find it necessary to keep the app on their device, let alone use it regularly. Conversely, for HCPs, the app may prove useful on multiple occasions and could be of even greater utility if the app was integrated with other software, such as prescribing records, dispensing systems and other healthcare systems. The WEB-RADR project has established a follow-up programme to develop the app using additional medical ontology mappings and by evaluating additional application programming interfaces for linking to other software.

In the future, the potential value of the app in supporting the notification of product quality issues and/or product complaints should be investigated. This could yield important information about manufacturing issues or batch-related concerns, particularly if used in conjunction with built-in image capture technologies. Similarly, specialist use of the app should be considered if an emerging safety issue arises with a product or class of medicines. Any situation such as this would require the close monitoring of a cohort of patients in a defined setting, and the app could provide a useful adjunct to traditional methods of PV by providing safety data in real time for rapid evaluation.

4 Conclusions

Over a period of 3 years, IMI WEB-RADR has addressed several important research questions relevant to the development of an app for reporting suspected ADRs. The resultant recommendations point to a series of pragmatic steps that those working in the PV community should consider before designing and building an app in support of safety reporting. The recommendations support the idea that the app is an important adjunct to existing ADR reporting pathways; however, engagement of app users is key to successful app development and adoption, in particular, in establishing the two-way interchange of safety information. As a younger generation of app-literate patients matures, it seems likely that app-based reporting will grow in importance and that, at some point, structured electronic reports will supplant the use of traditional paper forms.