FormalPara Key Points

Misreported and misinterpreted disproportionality analyses contribute to poor research quality.

The REporting of A Disproportionality analysis for drUg Safety signal detection using individual case safety reports in PharmacoVigilance (READUS-PV) guidelines will help researchers to adequately report their study and interpret the results.

1 Introduction

Pharmacovigilance is the science and activities relating to the detection, assessment, understanding, and prevention of adverse drug reactions (ADRs) or any other possible drug-related problems [1]. The main data source for detecting new ADRs after drug marketing approval consists of databases of individual case safety reports (ICSRs) recording suspected ADRs—whether spontaneous or generated within active surveillance activities [2,3,4,5]. A range of methods have been developed for mining these databases [6]. Disproportionality analyses are statistical methods that aim at quantifying the association between drug(s) and adverse event(s) comparing the number of observed ICSRs recording both the drug(s) and the event(s) with the number of ICSRs that would be expected in the absence of any association between the drug and the event, based on the underlying drug and event rates within the same ICSR database [7]. These analyses generate a signal of disproportionate reporting (SDR) when the statistic significantly exceeds a predefined threshold [1, 8, 9]. Because of the lack of exposure data and the unquantified under- and selective reporting, SDRs cannot be interpreted in themselves as conclusive scientific evidence of a causal relationship between a drug and an adverse event [1, 10]. Consistently, a signal of suspected causality [11, 12] should only be presented after the SDR has undergone an initial triage including, whenever possible, a case-by-case assessment of the ICSRs and the contextualization within knowledge already accrued from other sources of evidence (e.g., clinical trials, observational studies, case reports/series, literature, and animal experiments) [3, 4, 13, 14].

Nowadays, disproportionality analyses are increasingly used by multiple stakeholders, including pharmaceutical companies, regulatory agencies, and researchers. Their findings are frequently published in both specialized pharmacology journals and general medical journals [15,16,17,18]. However, they are often poorly reported and interpreted [17, 19]. Indeed, over 75% of published disproportionality studies failed to report essential elements needed to understand and reproduce the analyses and results, and more than two thirds of authors over- or misinterpreted their findings, notably in the abstract [17, 19]. Poor reporting and interpretation hamper the use and ability to assess research findings and contribute to research waste (i.e., avoidable inappropriate conduct and dissemination of research) [20,21,22]. Reporting guidelines, representing a minimum standard of items that should be reported in published articles, have therefore been developed for a range of study designs [23,24,25,26]. However, the design of disproportionality analyses has specific features that are not covered by existing reporting guidelines for observational and pharmacoepidemiological studies such as Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) and REporting of studies Conducted using Observational Routinely collected health Data statement for PharmacoEpidemiology (RECORD-PE) [25, 26]. We have therefore developed a guideline specifically tailored to the reporting and interpretation of disproportionality analyses: the REporting of A Disproportionality analysis for drUg Safety signal detection using individual case safety reports in PharmacoVigilance (READUS-PV).

The READUS-PV guidelines are intended as a guidance to support the reporting (and the publication) of disproportionality analyses to ensure that readers can easily determine what was planned (i.e., the research question), done (i.e., the methodology used), and found (i.e., the results and drawn conclusions). They should therefore not be considered as a tool to explicitly assess the quality of a published manuscript or the validity of an SDR as a true safety signal, which also requires considering additional methodological aspects. Nonetheless, these guidelines can indirectly improve the quality of research by pointing to items that should be already addressed during study design and reducing the risk of misinterpretation of results. In this regard, these guidelines should be viewed in conjunction with other recommendations on signal detection practices, such as those promoted by pharmacovigilance experts, the Innovative Medicine Initiative Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium (IMI PROTECT) for methodological considerations on disproportionality analyses, and by regulatory agencies for assessment of safety signals [12, 27,28,29]. The READUS-PV statement comprises a checklist of 14 items recommended for reporting disproportionality analyses, and four additional items for the abstract. Box 1 includes a glossary of terms used throughout the READUS-PV statement. This article is simultaneously published with an “explanation and elaboration” article providing additional reporting guidance for each item, along with examples of good reporting [30].

Box 1 Glossary of terms
Full size table

2 The READUS-PV Checklist

2.1 Conception and Development of the Checklist

We followed the recommendations by the Enhancing the QUality And Transparency Of health Research (EQUATOR) Network for the development of reporting guidelines [32]. The protocol was registered with the EQUATOR Network website on March 7, 2022, and published on a dedicated website (https://readus-statement.org/).

Based on a bibliometric analysis and taking into account expertise and geographical provenance, we established a diverse group of international individuals from universities, the pharmaceutical industry, and regulatory agencies, with expertise in pharmacovigilance, disproportionality analyses, and assessment of safety signals. Of the 70 invitations sent out, 34 experts agreed to take part in the project. We followed a three-step process to develop the checklist: (1) an open-text survey to generate a first list of items to be potentially included in the checklist, (2) a modified Delphi method to select and rephrase items important for the reporting, and (3) a final online consensus meeting.

2.2 Open-Text Survey

We first used an open-text survey in which the participants were asked to suggest items that they thought should be included in articles reporting disproportionality analysis results. To guide participants in their suggestions, we used the same subsection of the STROBE and RECORD-PE checklists (e.g., background/objective/study design/data source). Among the panel members, 32 participants compiled the open-text survey. The steering committee (MF, FS, ER, CK) then summarized the comments retaining all unique ideas (regardless of frequency) and proposed an initial list of 97 items.

2.2.1 The Delphi Method

Panel members were asked to decide whether they believed the proposed items should be included in the READUS-PV reporting guideline, using an online modified Delphi method through Sphinx Online© software version 4.30. We used a logical algorithm to select items to be included in the following steps (Supplementary Figure 1, see the electronic supplementary material). Briefly, items were included in the checklist if more than 80% of participants agreed, with no major comments on wording or concept. In the case of major comments, independently of the agreement rate, items were modified and proposed to the second round, in which the agreement on both the phrasing and the inclusion of the item in the checklist were assessed. Among the panel members, 33 experts responded to round 1 and 30 to round 2 of the Delphi. Overall, 60 recommendations for the main body of the manuscript and 13 recommendations for the abstracts were retained by participants after the Delphi method.

2.2.2 Online Consensus Meeting

Before the meeting, the steering committee merged related items, harmonized terminologies, and shared with all participants the draft version of the checklist. All participants were also asked to review the final list of items and share comments to be discussed in the final meeting through a shared online document. There were 25 attendees from 14 countries participating in the online consensus meeting (https://readus-statement.org/). Each item selected in previous rounds of the Delphi method was discussed, and participants were invited to share their views on the proposal and vote for final decisions about wording of items. In this step, minor comments collected in previous rounds were considered. Decisions were adopted by vote, and items were rediscussed and rephrased, if necessary, until at least 80% of participants agreed. The panel agreed on 14 items for manuscript body, and four items for the abstract.

2.2.3 Post-Meeting Activities

Following the online consensus meeting, the steering committee of READUS-PV prepared the first draft of the two manuscripts: the statement and the explanation and elaboration. The manuscripts were shared with the entire panel for two rounds of revisions.

2.3 Using the READUS-PV Checklist

The READUS-PV checklist templates for the manuscript body (Table 1) and the abstract (Table 2) can be downloaded from either the electronic supplementary material (Supplementary Tables 1 and 2) or the READUS-PV statement website (http://www.readus-statement.org/). We recommend authors refer to READUS-PV already when designing the study to achieve a more complete and transparent documentation. Journals and publishers may impose restriction on word count, manuscript structure, and number of tables and figures allowed. The recommendation for placing information in specific manuscript sections should not be viewed as mandatory, but relevant information should still be presented, if not in the manuscript body, at least in properly referenced supplementary materials, protocols, and/or data repositories (e.g., Open Science Framework, Dryad, Figshare).

Table 1 The READUS-PV checklist
Full size table
Table 2 The READUS-PV checklist for abstracts for standalone studies
Full size table

3 Discussion

The READUS-PV guidelines were designed to improve the reporting of disproportionality analyses using ICSR databases. The completeness and accuracy of research reporting are ethical requirements endorsed by leading international statements and recommendations [33, 34]. Indeed, complete reporting enables readers to assess the appropriateness of methods, and thus the validity of research results. Moreover, improving the transparency of the reporting, these guidelines bear the potential to enhance the reproducibility and replicability of disproportionality analysis.

Reporting guidelines have been shown to improve the completeness and quality of reporting (cf. Consolidated Standards of Reporting Trials [CONSORT] for the reporting of randomized controlled trials) [35, 36]. However, compliance with these guidelines is suboptimal even in journals that actively endorse them [37]. Various strategies to increase the use of reporting guidelines have therefore been proposed, e.g., training, encouraging and checking adherence, providing feedback, or involving experts [38]. Moreover, endorsement by scientific societies and professional networks and inclusion in pharmacovigilance curricula should be envisioned [39]. Most of these strategies have not been evaluated by robust designs, while convincing evidence suggests a pivotal role for the involvement of journal editors [32, 38, 40]. We therefore encourage journal editors and publishers to raise awareness of READUS-PV, endorse its use (for example, by referring to it in the “Instructions for authors” section of the journal), and advise editors and peer reviewers to consider the READUS-PV checklists. Guidance for journal editors for endorsing READUS-PV guidelines is provided on the website. Moreover, we recommend authors submit the reporting template (downloadable from https://readus-statement.org/) along with their submission, indicating where information for each item is reported.

Although these guidelines primarily focus on disproportionality analyses, SDRs alone cannot constitute safety signals and should instead be assessed through a case-by-case analysis and contextualized with evidence from other data sources and methods [13]. The checklist items are partly applicable to other signal detection studies on ICSR databases, such as other observed-to-expected analyses (i.e., where the expected number of ICSRs is defined using epidemiological data), mixed approaches with meta-analyses, or pharmacoepidemiologic studies, but should be used in concomitance with other reporting guidelines when appropriate.

3.1 Limitations

In developing the READUS-PV guidelines, we have set up a large panel of international experts in pharmacovigilance, disproportionality analysis, and assessment of safety signals, including researchers from universities, the pharmaceutical industry, and regulatory agencies. Although we have tried to widely spread our invitations to participate, we may have missed out on people with expertise in the field, particularly in non-European countries, but we hope that more experts will be involved in revising or extending the guidelines. We acknowledge that periodic updates and refinements will be necessary to ensure the READUS-PV guidelines encompass the latest advancements in the field, and item inclusion may be rediscussed (see Supplementary Table 3 in the electronic supplementary material for excluded items). We encourage all users of this checklist to share comments and suggestions for improvement through the dedicated platform in the READUS-PV website. Finally, the actual impact of these guidelines deserves specific studies and will be assessed after relevant implementation and dissemination within the scientific community. Given translation of reporting guideline checklists to scoring systems has been shown to be heterogenous among meta-research studies and poorly reported, the development of a uniform adherence-scoring tool to evaluate adherence to READUS-PV guidelines will be developed [41, 42].

4 Conclusion

The READUS-PV statement is provided to benefit authors, editors, peer-reviewers, and users of disproportionality analyses on ICSR databases. Ultimately, we hope that the uptake of these guidelines will lead to more transparent, complete, and accurate reporting and interpretation of disproportionality analyses, thus facilitating integration of evidence from disproportionality analyses with evidence from other data sources and actual clinical transferability for evidence-based decision-making.