Skip to main content

Development of a quality instrument for assessing the spontaneous reports of ADR/ADE using Delphi method in China


Background and aim

The frequently low quality of submitted spontaneous reports is of an increasing concern; to our knowledge, no validated instrument exists for assessing case reports’ quality comprehensively enough. This work was conducted to develop such a quality instrument for assessing the spontaneous reports of adverse drug reaction (ADR)/adverse drug event (ADE) in China.


Initial evaluation indicators were generated using systematic and literature data analysis. Final indicators and their weights were identified using Delphi method. The final quality instrument was developed by adopting the synthetic scoring method.


A consensus was reached after four rounds of Delphi survey. The developed quality instrument consisted of 6 first-rank indicators, 18 second-rank indicators, and 115 third-rank indicators, and each rank indicator has been weighted. It evaluates the quality of spontaneous reports of ADR/ADE comprehensively and quantitatively on six parameters: authenticity, duplication, regulatory, completeness, vigilance level, and reporting time frame.


The developed instrument was tested with good reliability and validity, which can be used to comprehensively and quantitatively assess the submitted spontaneous reports of ADR/ADE in China.

This is a preview of subscription content, access via your institution.


  1. Patrick CW (2006) Making the most of spontaneous adverse drug reaction reporting. Basic Clin Pharmacol Toxicol 98(3):320–323

    Article  Google Scholar 

  2. Zhang L, Wong LYL, He Y, et al. (2014) Pharmacovigilance in China: current situation, successes and challenges. Drug Saf 37:765–770

    Article  PubMed  Google Scholar 

  3. William NK (2003) The quality of published adverse drug event reports. Ann Pharmacother 37(12):1774–1778

    Article  Google Scholar 

  4. Jun S, Hou-ming X, Xiao-li L (2006) The quality analysis of ADR reports of year 2005 in Jiangsu Province. Chin J Pharmacovigilance 3(4):205–206

    Google Scholar 

  5. Wen C, Yong-chang L, Zheng F, et al. (2008) The quality analysis of adverse drug reaction reports in Shanghai during 2003–2007. Pharm Care Res 8(4):276–280

    Google Scholar 

  6. Ane GD, Pernille H, Hanne SN, et al. (2007) Characteristics and quality of adverse drug reaction reports by pharmacists in Norway. Pharmacoepidemiol Drug Saf 16(9):999–1005

    Article  Google Scholar 

  7. van Grootheest AC, van Puijenbroek EP, de Jong-van den Berg LTW (2002) Contribution of pharmacists to the reporting of adverse drug reactions. Pharmacoepidemiol Drug Saf 11(3):205–210

    Article  PubMed  Google Scholar 

  8. Maintenance of the ICH guideline on clinical safety data management: data elements for transmission of individual case safety reports [online]. Available from URL: Accessed 25 July 2009

  9. Kelly WN, Arellano FM, Barnes J, et al. (2007) Guidelines for submitting adverse event reports for publication. Pharmacoepidemiol Drug Saf 16:581–587

    Article  PubMed  Google Scholar 

  10. Edwards IR, Lindquist M, Wiholm BE, et al. (1990) Quality criteria for early signals of possible adverse drug reactions. Lancet 336(8708):156–158

    CAS  Article  PubMed  Google Scholar 

  11. Lindquist M, Star K, Strandell J (2007) Documentation grading of ICSRs and improved data quality management in VigiBase. Drug Saf 30(10):953

    Google Scholar 

  12. Brown B (1987) Delphi process: a methodology using for the elicitation of opinions of experts. Rand Corp 9:3925

    Google Scholar 

  13. Tan H-z, Ping W-w, Yang T-b, et al. (2006) The synthetic evaluation model for analysis of flooding hazards. Eur J Pub Health 17(2):206–210

    Article  Google Scholar 

  14. Sun Z-q (2006) Medical synthetic evaluation methodology and application, vol 20-28. Chemical Industry Press, Beijing

    Google Scholar 

  15. Lindquist M (2004) Data quality management in pharmacovigilance. Drug Saf 27(12):857–870

    Article  PubMed  Google Scholar 

  16. Tian Y-j, Zhou Y, Feng Q-q, et al. (2009) Study of the evaluation and control for quality of adverse drug reaction reports. Chin J Pharmacovigilance 6(2):90–95

    Google Scholar 

  17. Olsson S (2007) The need for a generic form for spontaneous reporting of drug related problems. Pharma Newsl 7–9

  18. Taofikat BA, Jelena S, Ross H, et al. (2008) The development of a tool to assess the quality of case reports of adverse events. Int J Risk Saf Med 20:123–133

    Google Scholar 

  19. Rehan HS, Chopra D (2009) Guide to reporting adverse drug reactions. Int J Risk Saf Med 21:229–234

    Google Scholar 

  20. Harold AL, Murray T (2002) The Delphi method—techniques and applications, pp. 71–79

    Google Scholar 

  21. Guang Z (1994) Modern epidemiological methods and applications. Beijing Medical University and Peking Union Medical College Press, Beijing, pp. 250–269

    Google Scholar 

  22. Jeremy J, Duncan H (1995) Consensus methods for medical and health services research. BMJ 311:376–380

    Article  Google Scholar 

Download references


The authors thank the following experts for their participation: Le-jie Cao, Fei-hu Chen, Li-ming Chen, Yi-xin Chen, Hong Ding, Wen Fu, Dai-hong Gou, Chuan-hai Huang, Qi-cheng Jiang, Xiao-bin Li, Xiong-wen Lv, Fa-ming Pan, Xiu-qin Su, Jun Sun, Yan-cai Sun, Hai-sun Wang, Da-you Wang, Jing Wang, Li Wang, Yu-hong Wang, Hua-wen Xin, Wen-ke Xu, Mu Yang, Si-yan Zhan, Qin Zhang, Shan-tang Zhang, Wei-dong Zhang.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Lixun Chen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Electronic supplementary material


(DOCX 40 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Chen, L., Jiang, L., Shen, A. et al. Development of a quality instrument for assessing the spontaneous reports of ADR/ADE using Delphi method in China. Eur J Clin Pharmacol 72, 1135–1142 (2016).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI:


  • Adverse drug reaction
  • Adverse drug event
  • Case report
  • Delphi method
  • Quality instrument