Journal of Medical Systems

, 40:37 | Cite as

A Multiagent System for Integrated Detection of Pharmacovigilance Signals

  • Vassilis KoutkiasEmail author
  • Marie-Christine Jaulent
Systems-Level Quality Improvement
Part of the following topical collections:
  1. Systems-Level Quality Improvement


Pharmacovigilance is the scientific discipline that copes with the continuous assessment of the safety profile of marketed drugs. This assessment relies on diverse data sources, which are routinely analysed to identify the so-called “signals”, i.e. potential associations between drugs and adverse effects, that are unknown or incompletely documented. Various computational methods have been proposed to support domain experts in signal detection. However, recent comparative studies illustrated that current methods exhibit high false-positive rates, significantly variable performance across different datasets used for analysis and events of interest, but also complementarity in their outcomes. In this regard, in order to reinforce accurate and timely signal detection, we elaborated through an agent-based approach towards systematic, joint exploitation of multiple heterogeneous signal detection methods, data sources and other drug-related resources under a common, integrated framework. The approach relies on a multiagent system operating based on a collaborative agent interaction protocol, aiming to implement a comprehensive workflow that comprises of method selection and execution, as well as outcomes’ aggregation, filtering, ranking and annotation. This paper presents the design of the proposed multiagent system, discusses implementation issues and demonstrates the applicability of the proposed solution in an example signal detection scenario. This work constitutes a step towards large-scale, integrated and knowledge-intensive computational signal detection.


Pharmacovigilance Computational signal detection methods Heterogeneous data sources Multiagent system Aggregation and reasoning scheme 



This research was supported by a Marie Curie Intra European Fellowship within the 7th European Community Framework Programme FP7/2007-2013 under REA grant agreement no 330422 – the SAFER project.

The authors would like to express their appreciation to the reviewers for their constructive comments and suggestions.

Compliance with ethical standards

Conflict of interests

The authors declare that they have no conflicts of interest.


  1. 1.
    WHO, The Importance of Pharmacovigilance: Safety Monitoring of Medicinal Products. World Health Organization, Geneva, CH (2002)Google Scholar
  2. 2.
    Grootheest, A., and Richesson, R.: Pharmacovigilance. In: Richesson, R., and Andrews, J. (Eds.) In: Clinical Research Informatics, Health Informatics, pp. 367–387. Springer, London (2012), doi: 10.1007/978-1-84882-448-5_19
  3. 3.
    Linguist, M., The need for definitions in pharmacovigilance. Drug Saf 30(10):825–830, 2007. doi: 10.2165/00002018200730100-00001.CrossRefGoogle Scholar
  4. 4.
    Council for International Organizations of Medical Sciences, Practical Aspects of Signal Detection in Pharmacovigilance. Report of CIOMS Working Group VIII. CIOMS, Geneva, CH (2010)Google Scholar
  5. 5.
    WHO, A Practical Handbook on the Pharmacovigilance of Antimalarial Medicines. World Health Organization, Geneva, CH (2008)Google Scholar
  6. 6.
    Kelman, C.W., et al., Evaluating medicines: let’s use all the evidence. Med J Aust 186(5):249–252, 2007.PubMedGoogle Scholar
  7. 7.
    Trifirò, G., et al., Data mining on electronic health record databases for signal detection in pharmacovigilance: which events to monitor. Pharmacoepidemiol Drug Saf 18(12):1176–1184, 2009. doi: 10.1002/pds.1836.CrossRefPubMedGoogle Scholar
  8. 8.
    Gurulingappa, H., Toldo, L., Rajput, A.M., Kors, J.A., Taweel, A., Tayrouz, Y., Automatic detection of adverse events to predict drug label changes using text and data mining techniques. Pharmacoepidemiol Drug Saf 22(11):1189–1194, 2013. doi: 10.1002/pds.3493.CrossRefPubMedGoogle Scholar
  9. 9.
    Sarker, A., et al., Utilizing social media data for pharmacovigilance: a review. J. Biomed. Inform. 54: 202–212, 2015. doi: 10.1016/j.jbi.2015.02.004.CrossRefPubMedGoogle Scholar
  10. 10.
    Harpaz, R., et al., Novel data mining methodologies for adverse drug event discovery and analysis. Clin. Pharmacol. Ther. 91(6):1010–1021, 2012. doi: 10.1038/clpt.2012.50.PubMedCentralCrossRefPubMedGoogle Scholar
  11. 11.
    Hauben, M., and Norén, G.N., A decade of data mining and still counting. Drug Saf 33(7):527–534, 2010. doi: 10.2165/11532430-000000000-00000.CrossRefPubMedGoogle Scholar
  12. 12.
    Shang, N., Xu, H., Rindflesch, T.C., Cohen, T., Identifying plausible adverse drug reactions using knowledge extracted from the literature. J. Biomed. Inform. 52:293–310, 2014. doi: 10.1016/j.jbi.2014.07.011.PubMedCentralCrossRefPubMedGoogle Scholar
  13. 13.
    Koutkias, V.G., and Jaulent, M.-C., Computational approaches for pharmacovigilance signal detection: toward integrated and semantically-enriched frameworks. Drug Saf 38(3):219–232, 2015. doi: 10.1007/s40264-015-0278-8.PubMedCentralCrossRefPubMedGoogle Scholar
  14. 14.
  15. 15.
    EudraVigilance. Accessed 13 September 2015
  16. 16.
    VigiBase®. Accessed 13 September 2015
  17. 17.
    Hauben, M., and Bate, A., Decision support methods for the detection of adverse events in post-marketing data. Drug Discov Today 14(7-8):343–357, 2009. doi: 10.1016/j.drudis.2008.12.012.CrossRefPubMedGoogle Scholar
  18. 18.
    Chazard, E., Ficheur, G., Bernonville, S., Luyckx, M., Beuscart, R., Data mining to generate adverse drug events detection rules. IEEE Trans. Inf. Technol. Biomed. 15 (6): 823–830, 2011. doi: 10.1109/TITB.2011.2165727.CrossRefPubMedGoogle Scholar
  19. 19.
    Reps, J., Garibaldi, J., Aickelin, U., Soria, D., Gibson, J., Hubbard, R., Comparison of algorithms that detect drug side effects using electronic healthcare database. Soft. Comput. 17:2381–2397, 2013. doi: 10.1007/s00500-013-1097-4.CrossRefGoogle Scholar
  20. 20.
    Madigan, D., Schuemie, M.J., Ryan, P.B., Empirical performance of the case-control design: lessons for developing a risk identification and analysis system. Drug Saf 36:S73–S82, 2013. doi: 10.1007/s40264-013-0105-z.CrossRefPubMedGoogle Scholar
  21. 21.
    Ryan, P.B., Schuemie, M.J., Gruber, S., Zorych, I., Madigan, D., Empirical performance of a new user cohort method: lessons for developing a risk identification and analysis system. Drug Saf 36:S59–S72, 2013. doi: 10.1007/s40264-013-0099-6.CrossRefPubMedGoogle Scholar
  22. 22.
    Suchard, M.A., Zorych, I., Simpson, S.E., Schuemie, M.J., Ryan, P.B., Madigan, D., Empirical performance of the self-controlled case series design: lessons for developing a risk identification and analysis system. Drug Saf 36:S83–S93, 2013. doi: 10.1007/s40264-013-0100-4.CrossRefPubMedGoogle Scholar
  23. 23.
    Noreń, N.G., Hopstadius, J., Bate, A., Star, K., Edwards, R.I., Temporal pattern discovery in longitudinal electronic patient records. Data Min. Knowl. Discov. 20:361–387, 2010. doi: 10.1007/s10618-009-0152-3.CrossRefGoogle Scholar
  24. 24.
    The Observational Medical Outcomes Partnership. Accessed 13 September 2015
  25. 25.
    LePendu, P., et al., Pharmacovigilance using clinical notes. Clin Pharmacol Ther 93:547–555, 2013. doi: 10.1038/clpt.2013.47.CrossRefPubMedGoogle Scholar
  26. 26.
    Freifeld, C.C., et al., Digital drug safety surveillance: monitoring pharmaceutical products in Twitter. Drug Saf 37:343–350, 2014. doi: 10.1007/s40264-014-0155-x.PubMedCentralCrossRefPubMedGoogle Scholar
  27. 27.
    Harpaz, R., et al., Text mining for adverse drug events: the promise, challenges, and state of the art. Drug Saf 37:777–790, 2014. doi: 10.1007/s40264-014-0218-z.PubMedCentralCrossRefPubMedGoogle Scholar
  28. 28.
    Harpaz, R., DuMouchel, W., LePendu, P., Bauer-Mehren, A., Ryan, P., Shah, N.H., Performance of pharmacovigilance signal-detection algorithms for the FDA adverse event reporting system. Clin. Pharmacol. Ther. 93:539–546, 2013. doi: 10.1038/clpt.2013.24.CrossRefPubMedGoogle Scholar
  29. 29.
    Liu, M., et al., Comparative analysis of pharmacovigilance methods in the detection of adverse drug reactions using electronic medical records. J. Am. Med. Inform. Assoc. 20:420–426, 2013. doi: 10.1136/amiajnl-2012-001119.PubMedCentralCrossRefPubMedGoogle Scholar
  30. 30.
    Schuemie, M.J., et al., Using electronic health care records for drug safety signal detection: a comparative evaluation of statistical methods. Med. Care. 50:890–897, 2012. doi: 10.1097/MLR.0b013e31825f63bf.CrossRefPubMedGoogle Scholar
  31. 31.
    van Holle, L., and Bauchau, V., Signal detection on spontaneous reports of adverse events following immunisation: a comparison of the performance of a disproportionality-based algorithm and a time-to-onset-based algorithm. Pharmacoepidemiol Drug Saf 23:178–185, 2014. doi: 10.1002/pds.3502.PubMedCentralCrossRefPubMedGoogle Scholar
  32. 32.
    Harpaz, R., et al., Combing signals from spontaneous reports and electronic health records for detection of adverse drug reactions. J. Am. Med. Inform. Assoc. 20(3):413–419, 2013. doi:doi: 10.1136/amiajnl-2012-000930 doi: 10.1136/amiajnl-2012-000930.PubMedCentralCrossRefPubMedGoogle Scholar
  33. 33.
    Stang, P., et al., Variation in choice of study design: findings from the epidemiology design decision inventory and evaluation (EDDIE) survey. Drug Saf 36:S15–S25, 2013. doi: 10.1007/s40264-013-0103-1.CrossRefPubMedGoogle Scholar
  34. 34.
    Koutkias, V.G., and Jaulent, M.-C.: Leveraging post-marketing drug safety research through semantic technologies: the PharmacoVigilance Signal Detectors Ontology.. In: Proceedings of the 7th International Workshop on Semantic Web Applications and Tools for Life Sciences, CEUR Workshop Proceedings, Vol. 1320, Berlin, Germany, December 9–11 (2014)Google Scholar
  35. 35.
    Singh, M.P., and Huhns, M.N., Service-Oriented Computing: Semantics, Processes, Agents: Wiley, 2005.Google Scholar
  36. 36.
    Luck, M., McBurney, P., Preist, C., A manifesto for agent technology: towards next generation computing. Auton. Agent. Multi-Ag. 9(3):203–252, 2004. doi: 10.1023/B:AGNT.0000038027.29035.7c.CrossRefGoogle Scholar
  37. 37.
    Isern, D, Sánchez, D., Moreno, A., Agents applied in health care: a review. Int. J. Med. Inform. 79(3): 145–166, 2010. doi: 10.1016/j.ijmedinf.2010.01.003.CrossRefPubMedGoogle Scholar
  38. 38.
    Klusch, M., and Sycara, K.: Brokering and matchmaking for coordination of agent societies: a survey. In: Omicini, A., Zambonelli, F., Klusch, M., Tolksdorf, R. (Eds.) In: Coordination of Internet Agents, pp. 197–224. Springer-Verlag (2001)Google Scholar
  39. 39.
    Durfee, E.H.: Distributed Problem Solving and Planning. In: Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence. Cambridge, MA, MIT Press, 2000, pp. 121– 164Google Scholar
  40. 40.
    Tran, Q.-N.N., and Low, G., MOBMAS: a methodology for ontology-based multi-agent systems development. Inform. Soft. Tech. 50(7-8):697–722, 2008. doi: 10.1016/j.infsof.2007.07.005.CrossRefGoogle Scholar
  41. 41.
    Lindquist, M., Use of triage strategies in the WHO signal-detection process. Drug Saf 30(7):635–637, 2007. doi: 10.2165/00002018-200730070-00014.CrossRefPubMedGoogle Scholar
  42. 42.
    Kuhn, M., Campillos, M., Letunic, I., Juhl Jensen, L., Bork, P., A side effect resource to capture phenotypic effects of drugs. Mol. Syst. Biol. 6(1), 2010. doi: 10.1038/msb.2009.98.
  43. 43.
    The DrugBank database. Accessed 13 September 2015
  44. 44.
    The ChEMBL database. Accessed 13 September 2015
  45. 45.
    Ahmed, I., and Poncet, A., PhViD: An R package for PharmacoVigilance signal Detection. R package version 1.0.6 (2013)Google Scholar
  46. 46.
    OWL 2 Web Ontology Language: Structural Specification and Functional-Style Syntax, 2nd Ed. Accessed 13 September 2015
  47. 47.
    The openFDA Drug API. Accessed 13 September 2015
  48. 48.
    Europe PubMed Central RESTful Web Service. Accessed 13 September 2015
  49. 49.
    The Twitter REST APIs. Accessed 13 September 2015
  50. 50.
    SPARQL 1.1 Overview, W3C Recommendation, 21 March 2013. Accessed 13 September 2015 (2013).
  51. 51.
    bio2RDF. Accessed 13 September 2015
  52. 52.
    Java Agent DEvelopment framework (JADE). Accessed 13 September 2015
  53. 53.
    The Foundation for Intelligent Physical Agents (FIPA). Accessed 13 September 2015
  54. 54.
    Rserve - Binary R server. Accessed 13 September 2015
  55. 55.
    Foundation for Intelligent Physical Agents, FIPA Abstract Architecture Specification, SC00001L, Accessed 13 September 2015
  56. 56.
    Foundation for Intelligent Physical Agents, FIPA ACL Message Structure Specification, SC00061G, 03/12/2002. Accessed 13 September 2015
  57. 57.
    Foundation for Intelligent Physical Agents, FIPA Communicative Act Library Specification, SC00037J, 03/12/2002. Accessed 13 September 2015
  58. 58.
    Foundation for Intelligent Physical Agents, FIPA SL Content Language Specification, SC00008I, 03/12/2002. Accessed 13 September 2015
  59. 59.
    Caire, G., and Cabanillas, D.: JADE Tutorial: Application-defined Content Languages and Ontologies., 15 April 2010. Accessed 13 September 2015
  60. 60.
    The clinical Text Analysis and Knowledge Extraction System (cTAKES). Accessed 13 September 2015
  61. 61.
    The Unified Medical Language System (UMLS). Accessed 13 September 2015
  62. 62.
    The Unstructured Information Management Architecture (UIMA). Accessed 13 September 2015
  63. 63.
    The OMOP Methods Library. Accessed 13 September 2015
  64. 64.
    Norén, N.G., and Edwards, I.R., Modern methods of pharmacovigilance: detecting adverse effects of drugs. Clinical Medicine 9(5):486–489, 2009. doi: 10.7861/clinmedicine.9-5-486.CrossRefPubMedGoogle Scholar
  65. 65.
    Liu, M., et al., Large-scale prediction of adverse drug reactions using chemical, biological, and phenotypic properties of drugs. J. Am. Med. Inform. Assoc. 19(e1):28–35, 2012. doi:doi: 10.1136/amiajnl-2011-000699 doi: 10.1136/amiajnl-2011-000699.CrossRefGoogle Scholar
  66. 66.
    The Mini-Sentinel project. Accessed 13 September 2015
  67. 67.
    Observational Health Data Sciences and Informatics (OHDSI) program. Accessed 13 September 2015
  68. 68.
    Oliveira, J.L., et al., The EU-ADR web platform: delivering advanced pharmacovigilance tools. Pharmacoepidemiol Drug Saf 22(5):459–467, 2013. doi: 10.1002/pds.3375.CrossRefPubMedGoogle Scholar
  69. 69.
    González-Vélez, H., et al., Healthagents: distributed multi-agent brain tumor diagnosis and prognosis. Appl. Intell. 30(3):191–202, 2009. doi: 10.1007/s10489-007-0085-8.CrossRefGoogle Scholar
  70. 70.
    Bromuri, S., Schumacher, M.I., Stathis, K., Ruiz, J.: Monitoring Gestational Diabetes Mellitus with Cognitive Agents and Agent Environments. In: Proc. of IEEE/WIC/ACM Int. Conf. on Web Intelligence and Intelligent Agent Technology (WI-IAT), Vol. 2, pp. 409–414, Lyon, France, Aug. 22-27 (2011), doi: 10.1109/WI-IAT.2011.37
  71. 71.
    Koutkias, V., Chouvarda, I., Maglaveras, N., A multiagent system enhancing home-care health services for chronic disease management. IEEE Trans. Inf. Technol. Biomed. 9 (4): 528–537, 2005. doi: 10.1007/978-3-662-04401-8_8.CrossRefPubMedGoogle Scholar
  72. 72.
    Kaluža, B., et al., A multi-agent care system to support independent living. Int. J. Artif. Intell. T. 23 (1), 2014. doi: 10.1142/S0218213014400016.
  73. 73.
    Koutkias, V.G., Malousi, A., Maglaveras, N., Engineering agent-mediated integration of bioinformatics analysis tools. Multiagent Grid Syst 3(2):245–258, 2007.Google Scholar
  74. 74.
    Bouzguenda, L., and Turki, M., Dynamic medical cross-organizational workflow management system. J Med Syst 38:32, 2014. doi: 10.1007/s10916-014-0032-2.CrossRefPubMedGoogle Scholar
  75. 75.
    Cortés, U., Vázquez-Salceda, J., López-Navidad, A., Caballero, F., UCTx: a multi-agent system to assist a transplant coordination unit. Appl. Intell. 20(1):59–70, 2004. doi: 10.1023/B:APIN.0000011142.91514.57.CrossRefGoogle Scholar
  76. 76.
    Tian, Y., Zhou, T.-S., Yao, Q., Zhang, M., Li, J.-S., Use of an agent-based simulation model to evaluate a mobile-based system for supporting emergency evacuation decision making. J Med Syst 38:149, 2014. doi: 10.1007/s10916-014-0149-3.CrossRefPubMedGoogle Scholar
  77. 77.
    Ji, Y., et al., A distributed adverse drug reaction detection system using intelligent agents with a fuzzy recognition-primed decision model. Int. J. Intell. Syst. 22(8):827–845, 2007. doi: 10.1002/int.20230.CrossRefGoogle Scholar
  78. 78.
    Nyulas, C.I., O’Connor, M.J., Tu, S.W., Buckeridge, D.L., Okhmatovskaia, A., Musen, M.A.: An ontology-driven framework for deploying JADE agent systems. In: Proc. of IEEE/WIC/ACM Int. Conf. on Web Intelligence and Intelligent Agent Technology (WI-IAT), Vol. 2, pp. 573–577, Sydney, Australia, Dec. 9-12 (2008), doi: 10.1109/WIIAT.2008.25
  79. 79.
    Ferrucci, D., et al., Building Watson: an overview of the DeepQA project. AI Magazine 31(3):59–79, 2010. doi: 10.1609/aimag.v31i3.2303.
  80. 80.
    Santos, E., Wilkinson, J.T., Santos, E., Fusing multiple Bayesian knowledge sources. Int. J. Approx. Reason. 52(7):935–947, 2011. doi: 10.1016/j.ijar.2011.01.008.CrossRefGoogle Scholar
  81. 81.
    He, H., Cao, Y., Wen, J., Cheng, S.: A boost voting strategy for knowledge integration and decision making. In: Sun, F., Zhang, J., Tan, Y., Cao, J., Yu, W. (Eds.) In: Advances in Neural Networks - ISNN 2008, volume 5263 of Lecture Notes in Computer Science, pp. 472–481. Springer, Berlin (2008), doi: 10.1007/978-3-540-87732-5_53
  82. 82.
    DeployR. Accessed 13 September 2015
  83. 83.
    Burian, P., Multi-agent Systems and Cloud Computing for Controlling and Managing Chemical and Food Processes. J. Chem. Chem. Eng. 6:1121–1135, 2012.Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Vassilis Koutkias
    • 1
    • 2
    • 3
    Email author
  • Marie-Christine Jaulent
    • 1
    • 2
    • 3
  1. 1.INSERM, U1142, LIMICSParisFrance
  2. 2.Sorbonne Universités, UPMC University Paris 06, UMR_S 1142, LIMICSParisFrance
  3. 3.Université Paris 13, Sorbonne Paris Cité, LIMICS, UMR_S 1142VilletaneuseFrance

Personalised recommendations