Skip to main content
SpringerLink
Log in

Big Data and Causality

  • Published:
Annals of Data Science Aims and scope Submit manuscript

Abstract

Causality analysis continues to remain one of the fundamental research questions and the ultimate objective for a tremendous amount of scientific studies. In line with the rapid progress of science and technology, the age of big data has significantly influenced the causality analysis on various disciplines especially for the last decade due to the fact that the complexity and difficulty on identifying causality among big data has dramatically increased. Data mining, the process of uncovering hidden information from big data is now an important tool for causality analysis, and has been extensively exploited by scholars around the world. The primary aim of this paper is to provide a concise review of the causality analysis in big data. To this end the paper reviews recent significant applications of data mining techniques in causality analysis covering a substantial quantity of research to date, presented in chronological order with an overview table of data mining applications in causality analysis domain as a reference directory.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Note that this paper focuses on data mining applications in causality analysis only regardless of subjects. It is formed based on a specific aspect of view, therefore it is not comparable with any other reviews of data mining applications. More relevant details, please refer to [14] that focus on time series, [15] for pharmacogenomics, [13] for crime studies, [16] for health informatics, [17] for causality analysis in boimedical informatics, [18] for fraud detection studies, etc.

  2. Note that an application that implemented multiple Data Mining techniques will be categorized into the review subsection of the corresponding technique that was most significantly employed.

  3. It is possible that we can get many different Decision Trees from the same given set of cases. The final choice depends on the research and the individual circumstances.

References

  1. Mayer-Schonberger V, Cukier K (2013) Big data: a revolution that will transform how we live, work, and think. Houghton Mifflin Harcourt, New York

    Google Scholar 

  2. Chen H, Chiang RH, Storey VC (2012) Business intelligence and analytics: from big data to big impact. MIS Q 36(4):1165–1188

    Google Scholar 

  3. Chen H, Chung W, Xu JJ, Wang G, Qin Y, Chau M (2004) Crime data mining: a general framework and some examples. Computer 37(4):50–56

    Article  Google Scholar 

  4. Gupta GK (2006) Introduction to data mining with case studies. PHI Learning Pvt. Ltd, New Delhi

    Google Scholar 

  5. Hassani H, Saporta G, Silva ES (2014) Data mining and official statistics: the past, the present and the future. Big Data 2(1):34–43

    Article  Google Scholar 

  6. Fayyad U, Uthurusamy R (2002) Evolving data into mining solutions for insights. Commun ACM 45(8):28–31

    Article  Google Scholar 

  7. Granger CW (1988) Some recent development in a concept of causality. J Econ 39(1–2):199–211

    Article  Google Scholar 

  8. Soytas U, Sari R (2003) Energy consumption and GDP: causality relationship in G-7 countries and emerging markets. Energy Econ 25(1):33–37

    Article  Google Scholar 

  9. Hassani H, Zhigljavsky A, Patterson K, Soofi A (2010) A comprehensive causality test based on the singular spectrum analysis. In: Causality in Science, 1st edn. Oxford University Press, pp 379–406

  10. Sugihara G, May R, Ye H, Hsieh CH, Deyle E, Fogarty M, Munch S (2012) Detecting causality in complex ecosystems. Science 338(6106):496–500

    Article  Google Scholar 

  11. Hassani H, Huang X, Gupta R, Ghodsi M (2016) Does sunspot numbers cause global temperatures? A reconsideration using non-parametric causality tests. Phys A Stat Mech Appl 460:54–65

    Article  Google Scholar 

  12. Ghodsi Z, Huang X, Hassani H (2017) Causality analysis detects the regulatory role of maternal effect genes in the early Drosophila embryo. Genom Data 11:20–38

    Article  Google Scholar 

  13. Hassani H, Huang X, Silva ES, Ghodsi M (2016) A review of data mining applications in crime. Stat Anal Data Min ASA Data Sci J 9(3):139–154

    Article  Google Scholar 

  14. Fu TC (2011) A review on time series data mining. Eng Appl Artif Intell 24(1):164–181

    Article  Google Scholar 

  15. Hahn U, Cohen KB, Garten Y, Shah NH (2012) Mining the pharmacogenomics literaturea survey of the state of the art. Briefings Bioinform 13(4):460–494

    Article  Google Scholar 

  16. Herland M, Khoshgoftaar TM, Wald R (2014) A review of data mining using big data in health informatics. J Big Data 1(1):2

    Article  Google Scholar 

  17. Kleinberg S, Hripcsak G (2011) A review of causal inference for biomedical informatics. J Biomed Inform 44(6):1102–1112

    Article  Google Scholar 

  18. Sharma A, Panigrahi PK (2012) A review of financial accounting fraud detection based on data mining techniques. Int J Comput Appl 39(1):37–47

    Google Scholar 

  19. Cowie J, Lehnert W (1996) Information extraction. Commun ACM 39(1):80–91

    Article  Google Scholar 

  20. Chinchor NA (1998) Overview of MUC-7/MET-2. In Proceedings of the seventh message understanding conference (MUC-7), April 1998

  21. Chau M, Xu JJ, Chen H (2002) Extracting meaningful entities from police narrative reports. In: Proceedings of the 2002 annual national conference on digital government research, pp 1–5

  22. Girju R, Moldovan DI (2002) Text mining for causal relations. In: FLAIRS conference, pp 360–364

  23. Girju R, Moldovan D (2002) Mining answers for causation questions. In: AAAI symposium on mining answers from texts and knowledge bases

  24. Blanco E, Castell N, Moldovan DI (2008) Causal relation extraction. In: LREC

  25. Radinsky K, Davidovich S, Markovitch S (2012) Learning causality for news events prediction. In: Proceedings of the 21st international conference on World Wide Web, ACM, pp 909–918

  26. Bizer C, Heath T, Berners-Lee T (2009) Linked data-the story so far. Int J Semant Web inf syst 5(3):1–22

    Article  Google Scholar 

  27. Riaz M, Girju R (2013) Toward a better understanding of causality between verbal events: extraction and analysis of the causal power of verb-verb associations. In: Proceedings of the annual SIGdial meeting on discourse and dialogue (SIGDIAL)

  28. Riaz M, Girju R (2010) Another look at causality: discovering scenario-specific contingency relationships with no supervision. In: 2010 IEEE fourth international conference on semantic computing (ICSC), IEEE, pp 361–368

  29. Riaz M, Girju R (2014) Recognizing causality in verb-noun pairs via noun and verb semantics. EACL, p 48

  30. Talmy L (1988) Force dynamics in language and cognition. Cogn Sci 12(1):49–100

    Article  Google Scholar 

  31. Garcia D (1997) COATIS, an NLP system to locate expressions of actions connected by causality links. In: International conference on knowledge engineering and knowledge management. Springer, Berlin Heidelberg, pp 347–352

  32. Al-Saif A, Markert K (2010) The leeds Arabic discourse treebank: annotating discourse connectives for Arabic. In: LREC

  33. Alsaif A, Markert K (2011) Modelling discourse relations for Arabic. In: Proceedings of the conference on empirical methods in natural language processing, Association for Computational Linguistics, pp 736–747

  34. Hashimoto C, Torisawa K, De Saeger S, Oh JH, Kazama JI (2012) Excitatory or inhibitory: a new semantic orientation extracts contradiction and causality from the web. In: Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning, Association for Computational Linguistics, pp 619–630

  35. Hashimoto C, Torisawa K, Kloetzer J, Sano M, Varga I, Oh JH, Kidawara Y (2014) Toward future scenario generation: extracting event causality exploiting semantic relation, context, and association features. ACL 1:987–997

    Google Scholar 

  36. Hashimoto C, Torisawa K, Kloetzer J, Oh JH (2015) Generating event causality hypotheses through semantic relations. In: AAAI, pp 2396–2403

  37. Bögel T, Hautli-Janisz A, Sulger S, Butt M (2014) Automatic detection of causal relations in German multilogs. In: 14th Conference of the European chapter of the association for computational linguistics, pp 20–27

  38. Sadek J, Meziane F (2016) Extracting arabic causal relations using linguistic patterns. ACM Trans Asian Low-Resour Lang Inf Process 15(3):14

    Article  Google Scholar 

  39. Khoo CS, Chan S, Niu Y (2000) Extracting causal knowledge from a medical database using graphical patterns. In: Proceedings of the 38th annual meeting on association for computational linguistics, Association for Computational Linguistics, pp 336–343

  40. Sun Y, Xie K, Liu N, Yan S, Zhang B, Chen Z (2007) Causal relation of queries from temporal logs. In: Proceedings of the 16th international conference on World Wide Web, ACM, pp 1141–1142

  41. Pyysalo S, Ohta T, Kim JD, Tsujii JI (2009) Static relations: a piece in the biomedical information extraction puzzle. In: Proceedings of the workshop on current trends in biomedical natural language processing, Association for Computational Linguistics, pp 1–9

  42. Raja K, Subramani S, Natarajan J (2013) PPInterFindera mining tool for extracting causal relations on human proteins from literature. In: Database: bas052

  43. Bunescu R, Ge R, Kate RJ, Marcotte EM, Mooney RJ, Ramani AK, Wong YW (2005) Comparative experiments on learning information extractors for proteins and their interactions. Artif Intell Med 33(2):139–155

    Article  Google Scholar 

  44. Pyysalo S, Ginter F, Heimonen J, Bjrne J, Boberg J, Jarvinen J, Salakoski T (2007) BioInfer: a corpus for information extraction in the biomedical domain. BMC Bioinform 8(1):50

    Article  Google Scholar 

  45. Fundel K, Kffner R, Zimmer R (2007) RelExRelation extraction using dependency parse trees. Bioinformatics 23(3):365–371

    Article  Google Scholar 

  46. Ding J, Berleant D, Nettleton D, Wurtele E (2002) Mining MEDLINE: abstracts, sentences, or phrases. In: Proceedings of the pacific symposium on biocomputing, vol 7, pp 326–337

  47. Nedellec C (2005) Learning language in logic-genic interaction extraction challenge. In: Proceedings of the 4th learning language in logic workshop (LLL05), vol 7, pp 1–7

  48. Mihăilă C, Ohta T, Pyysalo S, Ananiadou S (2013) BioCause: annotating and analysing causality in the biomedical domain. BMC Bioinform 14(1):2

    Article  Google Scholar 

  49. Mihăilă C, Ananiadou S (2014) Semi-supervised learning of causal relations in biomedical scientific discourse. Biomed Eng Online 13(2):S1

    Google Scholar 

  50. Luo Z, Sha Y, Zhu KQ, Hwang SW, Wang Z (2016, March) Commonsense causal reasoning between short texts. In: KR, pp 421–431

  51. Mahendran D, Nawarathna RD (2016) An automated method to extract information in the biomedical literature about interactions between drugs. In: 2016 Sixteenth international conference on advances in ICT for emerging regions (ICTer), IEEE, pp 155–161

  52. Rinaldi F, Ellendorff TR, Madan S, Clematide S, van der Lek A, Mevissen T, Fluck J (2016) BioCreative V track 4: a shared task for the extraction of causal network information using the Biological Expression Language. In: Database: baw067

  53. Fluck J, Madan S, Ansari S, Kodamullil AT, Karki R, Rastegar-Mojarad M, Catlett NL, Hayes W, Szostak J, Hoeng J, Peitsch M (2016) Training and evaluation corpora for the extraction of causal relationships encoded in biological expression language (BEL). Database: baw113

  54. Casillas A, Pérez A, Oronoz M, Gojenola K, Santiso S (2016) Learning to extract adverse drug reaction events from electronic health records in Spanish. Expert Syst Appl 61:235–245

    Article  Google Scholar 

  55. Prasad R, Miltsakaki E, Dinesh N, Lee A, Joshi A, Robaldo L, Webber BL (2007) The penn discourse treebank 2.0 annotation manual. IRCS Technical Reports Series: 203

  56. Do QX, Chan YS, Roth D (2011) Minimally supervised event causality identification. In: Proceedings of the conference on empirical methods in natural language processing, Association for Computational Linguistics, pp 294–303

  57. Zhao S, Wang Q, Massung S, Qin B, Liu T, Wang B, Zhai C (2017) Constructing and embedding abstract event causality networks from text snippets. In: Proceedings of the tenth ACM international conference on web search and data mining, ACM, pp 335-344

  58. Mirza P, Tonelli S (2014) An analysis of causality between events and its relation to temporal information. In COLING, pp 2097–2106

  59. Mirza P (2014) Extracting temporal and causal relations between events. In: ACL (student research workshop), pp 10–17

  60. Pustejovsky J, Lee K, Bunt H, Romary L (2010) ISO-TimeML: an international standard for semantic annotation. LREC 10:394–397

    Google Scholar 

  61. Mirza P, Tonelli S (2016) CATENA: CAusal and TEmporal relation extraction from NAtural language texts. In: The 26th international conference on computational linguistics, pp 64–75

  62. Lin Z, Ng HT, Kan MY (2014) A PDTB-styled end-to-end discourse parser. Natl Lang Eng 20(02):151–184

    Article  Google Scholar 

  63. Kim JD, Ohta T, Tsujii JI (2008) Corpus annotation for mining biomedical events from literature. BMC Bioinform 9(1):10

    Article  Google Scholar 

  64. Kalpana R, Suresh S, Jeyakumar N (2012) NAGGNERa hybrid named entity tagger for tagging human proteins/genes. In: Proceedings of the tenth Asia Pacific bioinformatics conference, Melbourne, Australia

  65. Suresh S, Kalpana R, Jeyakumar N (2011) ProNormzan automated web server for human proteins and protein kinases normalization. In: Proceedings of the second international conference on bioinformatics and systems biology (INCOBS), Chidambaram, India

  66. Ruppenhofer J, Ellsworth M, Petruck MR, Johnson CR, Scheffczyk J (2006) FrameNet II: extended theory and practice

  67. Rizzolo N, Roth D (2010) Learning based Java for rapid development of NLP systems. LREC 5:313–323

    Google Scholar 

  68. Pang-Ning T, Steinbach M, Kumar V (2006) Introduction to data mining. In: Library of Congress

  69. Xu D, Tian Y (2015) A comprehensive survey of clustering algorithms. Ann Data Sci 2(2):165–193

    Article  Google Scholar 

  70. Matuszewski A (2002) Double clustering: a data mining methodology for discovery of causality. In: Intelligent information systems, Physica-Verlag HD, pp 227–236

  71. Classen JB, Classen DC (2002) Clustering of cases of insulin dependent diabetes (IDDM) occurring three years after hemophilus influenza B (HiB) immunization support causal relationship between immunization and IDDM. Autoimmunity 35(4):247–253

    Article  Google Scholar 

  72. Fujita A, Severino P, Kojima K, Sato JR, Patriota AG, Miyano S (2012) Functional clustering of time series gene expression data by Granger causality. BMC Syst Biol 6(1):137

    Article  Google Scholar 

  73. Wu G, Liao W, Stramaglia S, Chen H, Marinazzo D (2013) Recovering directed networks in neuroimaging datasets using partially conditioned Granger causality. Brain Connect 3(3):294–301

    Article  Google Scholar 

  74. Sato JR, Fujita A, Cardoso EF, Thomaz CE, Brammer MJ, Amaro E (2010) Analyzing the connectivity between regions of interest: an approach based on cluster Granger causality for fMRI data analysis. Neuroimage 52(4):1444–1455

    Article  Google Scholar 

  75. Wismüller A, Nagarajan MB, Witte H, Pester B, Leistritz L (2014) Pair-wise clustering of large scale Granger causality index matrices for revealing communities. In: SPIE Medical Imaging, International Society for Optics and Photonics, pp 90381R–90381R

  76. Wismüller A, Wang X, DSouza AM, Nagarajan MB (2014) A framework for exploring non-linear functional connectivity and causality in the human brain: mutual connectivity analysis (MCA) of resting-state functional mri with convergent cross-mapping and non-metric clustering. arXiv preprint arXiv:1407.3809

  77. Blondel VD, Guillaume JL, Lambiotte R, Lefebvre E (2008) Fast unfolding of communities in large networks. J Stat Mech Theory Exp 2008(10):1–12

  78. Qin X, Lee W (2003) Statistical causality analysis of infosec alert data. International workshop on recent advances in intrusion detection, Springer, Berlin Heidelberg, pp 73–93

  79. Li ST, Kuo SC, Tsai FC (2010) An intelligent decision-support model using FSOM and rule extraction for crime prevention. Expert Syst Appl 37(10):7108–7119

    Article  Google Scholar 

  80. Chow WW, Fung MK (2013) Financial development and growth: a clustering and causality analysis. J Int Trade Econ Dev 22(3):430–453

    Article  Google Scholar 

  81. Wong RK, Chu V, Ghanavati M, Hamzehei A (2015) Trajectory analysis based on clustering and casual structures. In: Workshops at the twenty-ninth AAAI conference on artificial intelligence

  82. Birant D, Kut A (2007) ST-DBSCAN: an algorithm for clustering spatialtemporal data. Data Knowl Eng 60(1):208–221

    Article  Google Scholar 

  83. Zhu JY, Zhang C, Zhi S, Li VO, Han J, Zheng Y (2016) p-causality: identifying spatiotemporal causal pathways for air pollutants with urban big data. arXiv preprint arXiv:1610.07045

  84. Ide D, Ruike A, Kimura M (2015) Extraction of causalities and rules involved in wear of machinery from lubricating oil analysis data. In: the second international conference on digital information processing, data mining, and wireless communications (DIPDMWC2015), p 16

  85. Yuan T, Li G, Zhang Z, Qin S J (2016) Deep causal mining for plant-wide oscillations with multilevel granger causality analysis. In: American control conference (ACC), IEEE, pp 5056–5061

  86. Okada Y, Fukui KI, Moriyama K, Numao M (2015) Cluster sequence mining: causal inference with time and space proximity under uncertainty. In: Pacific-Asia conference on knowledge discovery and data mining, Springer International Publishing, pp 293–304

  87. Ma J, Tang H, Hu X, Bobet A, Zhang M, Zhu T, Song Y, Eldin MAE (2017) Identification of causal factors for the Majiagou landslide using modern data mining methods. Landslides 14(1):311–322

    Article  Google Scholar 

  88. Cai Y (1989) Attribute-oriented induction in relational databases. Doctoral dissertation. Simon Fraser University

  89. Han J, Cai Y, Cercone N (1993) Data-driven discovery of quantitative rules in relational databases. IEEE Trans Knowl Data Eng 5(1):29–40

    Article  Google Scholar 

  90. Porras PA, Fong MW, Valdes A (2002) A mission-impact-based approach to INFOSEC alarm correlation. In: International workshop on recent advances in intrusion detection, Springer, Berlin, Heidelberg, pp 95–114

  91. Teh YW, Jordan MI, Beal MJ, Blei DM (2004) Sharing clusters among related groups: hierarchical Dirichlet processes. In NIPS, pp 1385–1392

  92. De Maesschalck R, Jouan-Rimbaud D, Massart DL (2000) The mahalanobis distance. Chemometr Intell Lab Syst 50(1):1–18

    Article  Google Scholar 

  93. Bishop CM, Svensen M, Williams CK (1998) GTM: the generative topographic mapping. Neural Comput 10(1):215–234

    Article  Google Scholar 

  94. Lee JG, Han J, Whang KY (2007) Trajectory clustering: a partition-and-group framework. In: Proceedings of the 2007 ACM SIGMOD international conference on Management of data, ACM, pp 593–604

  95. Yun H, Ha D, Hwang B, Ryu KH (2003) Mining association rules on significant rare data using relative support. J Syst Softw 67(3):181–191

    Article  Google Scholar 

  96. Agrawal R, Imieliński T, Swami A (1993) Mining association rules between sets of items in large databases. ACM SIGMOD Rec 22:207–216

    Article  Google Scholar 

  97. Agrawal R, Srikant R (1994) Fast algorithms for mining association rules. In: Proceedings of 20th international conference on very large data bases, VLDB ,Vol. 1215, pp. 487–499

  98. Mazlack L J (2008) Considering causality in data mining. In: International conference on software engineering

  99. Cooper GF (1997) A simple constraint-based algorithm for efficiently mining observational databases for causal relationships. Data Min Knowl Disc 1(2):203–224

    Article  Google Scholar 

  100. Silverstein C, Brin S, Motwani R, Ullman J (2000) Scalable techniques for mining causal structures. Data Min Knowl Disc 4(2–3):163–192

    Article  Google Scholar 

  101. Bowes J, Neufeld E, Greer JE, Cooke J (2000) A comparison of association rule discovery and Bayesian network causal inference algorithms to discover relationships in discrete data. In Conference of the Canadian society for computational studies of intelligence, Springer, Berlin, Heidelberg, pp 326–336

  102. Zhang S, Zhang C (2002) Discovering causality in large databases. Appl Artif Intell 16(5):333–358

    Article  Google Scholar 

  103. Hsieh YL, Yang DL, Wu J (2005) Using data mining to study upstream and downstream causal relationship in stock market. Computer 1:F02

    Google Scholar 

  104. Hsieh YL, Yang DL, Hsu FR (2012) An effective mining algorithm for profit mining. In: 2012 International symposium computer, consumer and control (IS3C), IEEE, pp 106–110

  105. Hsieh Y L, Yang D L, Wu J (2014) Effective application of improved profit-mining algorithm for the interday trading model. The Scientific World Journal: ID874825

  106. Hsieh YL, Yang DL, Wu J, Chen YC (2016) Efficient mining of profit rules from closed inter-transaction itemsets. J Inform Sci Eng 32(3):575–595

    Google Scholar 

  107. Li J, Liu L, Le T (2015) Practical approaches to causal relationship exploration. Springer, Berlin

    Book  Google Scholar 

  108. Li J, Le TD, Liu L, Liu J, Jin Z, Sun B (2013) Mining causal association rules. In: 2013 IEEE 13th international conference data mining workshops (ICDMW), IEEE, pp 114–123

  109. Li J, Le TD, Liu L, Liu J, Jin Z, Sun B, Ma S (2016) From observational studies to causal rule mining. ACM Trans Intell Syst Technol (TIST) 7(2):14

    Google Scholar 

  110. Ji Y, Ying H, Dews P, Mansour A, Tran J, Miller RE, Massanari RM (2011) A potential causal association mining algorithm for screening adverse drug reactions in postmarketing surveillance. IEEE Trans Inf Technol Biomed 15(3):428–437

    Article  Google Scholar 

  111. Ji Y, Ying H, Tran J, Dews P, Mansour A, Massanari RM (2013) A method for mining infrequent causal associations and its application in finding adverse drug reaction signal pairs. IEEE Trans Knowl Data Eng 25(4):721–733

    Article  Google Scholar 

  112. Yang CC, Yang H, Jiang L, Zhang M (2012) Social media mining for drug safety signal detection. In: Proceedings of the 2012 international workshop on smart health and wellbeing, ACM, pp 33–40

  113. Yang CC, Yang H, Jiang L (2014) Postmarketing drug safety surveillance using publicly available health-consumer-contributed content in social media. ACM Trans Manag Inf Syst (TMIS) 5(1):2

    Google Scholar 

  114. Yang H, Yang CC (2015) Using health-consumer-contributed data to detect adverse drug reactions by association mining with temporal analysis. ACM Trans Intell Syst Technol (TIST) 6(4):55

    Google Scholar 

  115. Karimi S, Wang C, Metke-Jimenez A, Gaire R, Paris C (2015) Text and data mining techniques in adverse drug reaction detection. ACM Comput Surv (CSUR) 47(4):56

    Article  Google Scholar 

  116. Ibrahim H, Saad A, Abdo A, Eldin AS (2016) Mining association patterns of drug-interactions using post marketing FDAs spontaneous reporting data. J Biomed Inform 60:294–308

    Article  Google Scholar 

  117. Ji Y, Ying H, Tran J, Dews P, Lau SY, Massanari RM (2016) A functional temporal association mining approach for screening potential drugdrug interactions from electronic patient databases. Inform Soc Care 41(4):387–404

    Article  Google Scholar 

  118. Vilar S, Friedman C Hripcsak G (2017) Detection of drugdrug interactions through data mining studies using clinical sources, scientific literature and social media. Briefings in Bioinformatics: bbx010

  119. Jin Z, Li J, Liu L, Le TD, Sun B, Wang R (2012) Discovery of causal rules using partial association. In: 2012 IEEE 12th international conference on data mining (ICDM), IEEE, pp 309–318

  120. Stanton A, Thart A, Jain A, Vyas P, Chatterjee A, Shakarian P (2015) Mining for causal relationships: a data-driven study of the Islamic state. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, ACM, pp 2137–2146

  121. Chen SC, Tsai TH, Chung CH, Li WH (2015) Dynamic association rules for gene expression data analysis. BMC Genom 16(1):786

    Article  Google Scholar 

  122. Yadav P, Prunelli L, Hoff A, Steinbach M, Westra B, Kumar V, Simon G (2016) Causal inference in observational data causal inference in observational data. arXiv preprint arXiv:1611.04660

  123. Sekhon JS (2008) The Neyman-Rubin model of causal inference and estimation via matching methods. In: Box-Steffensmeier JM, Brady HE, Collier D (eds) The oxford handbook of political methodology. Oxford University Press, New York

  124. Scheines R, Spirtes P, Glymour C, Meek C (1994) TETRAD II: users manual and software

  125. Han J, Fu Y, Wang W, Chiang J, Gong W, Koperski K, Xia B (1996) DBMiner: a system for mining knowledge in large relational databases. KDD 96:250–255

    Google Scholar 

  126. Tung AKH, Lu H, Han J, Feng L (2003) Efficient mining of intertransaction association rules. IEEE Trans Knowl Data Eng 15(1):43–56

    Article  Google Scholar 

  127. Wang K, Zhou S, Han J (2002) Profit mining: from patterns to actions. In: International conference on extending database technology, Springer, Berlin, Heidelberg, pp 70–87

  128. Liu B, Hsu W, Ma Y (1998) Integrating classification and association rule mining. In: Proceedings of the 4th international conference on knowledge discovery and data mining. AAAI Press, pp 80–86

  129. Breiman L, Friedman J, Olshen R, Stone C (1984) Classification and regression trees. Wadsworth, Belmont

    Google Scholar 

  130. Quinlan JR (1992) C4.5: program for machine learning. Morgan Kaufmann, Burlington

    Google Scholar 

  131. Quinlan JR (1986) Induction of decision trees. Mach Learn 1(1):81–106

    Google Scholar 

  132. Hunt JW, Szymanski TG (1977) A fast algorithm for computing longest common subsequences. Commun ACM 20(5):350–353

    Article  Google Scholar 

  133. Mehta M, Agrawal R, Rissanen J (1996) SLIQ: A fast scalable classifier for data mining. In: Advances in database technologyEDBT’96, Springer Berlin, Heidelberg, pp 18–32

  134. Shafer JC, Agrawal R, Mehta M (1996) ”SPRINT: a scalable parallel classifier for data mining”. In: Proceedings of the 22th international conference on very large databases, Mumbai (Bombay), India, Sept

  135. Karimi K, Hamilton HJ (2002) TimeSleuth: a tool for discovering causal and temporal rules. In: Proceedings of 14th IEEE international conference on tools with artificial intelligence, (ICTAI 2002), IEEE, pp 375–380

  136. Karimi K, Hamilton HJ (2003) Distinguishing causal and acausal temporal relations. In: Pacific-Asia conference on knowledge discovery and data mining, Springer, Berlin, Heidelberg, pp 234–240

  137. Hamilton HJ, Karimi K (2005) The TIMERS II algorithm for the discovery of causality. In: Pacific-Asia conference on knowledge discovery and data mining, Springer, Berlin Heidelberg, pp 744–750

  138. Girju R (2003) Automatic detection of causal relations for question answering. In: Proceedings of the ACL 2003 workshop on multilingual summarization and question answering, vol. 12, Association for Computational Linguistics, pp 76–83

  139. Kargupta H, Park BH, Pittie S, Liu L, Kushraj D, Sarkar K (2002) MobiMine: monitoring the stock market from a PDA. ACM SIGKDD Explor Newsl 3(2):37–46

    Article  Google Scholar 

  140. Zhang X, Hu Y, Xie K, Wang S, Ngai EWT, Liu M (2014) A causal feature selection algorithm for stock prediction modeling. Neurocomputing 142:48–59

    Article  Google Scholar 

  141. Zhang D, Zhou L (2004) Discovering golden nuggets: data mining in financial application. IEEE Trans Syst Man Cybern Part C Appl Rev 34(4):513–522

    Article  Google Scholar 

  142. Chen M, Zheng AX, Lloyd J, Jordan MI, Brewer E (2004) Failure diagnosis using decision trees. In: Autonomic computing proceedings, IEEE, pp 36–43

  143. Tariq M B, Motiwala M, Feamster N, Ammar M (2009) Detecting network neutrality violations with causal inference. In: Proceedings of the 5th international conference on emerging networking experiments and technologies, ACM, pp 289–300

  144. Ale BJM, Bellamy LJ, Cooke RM, Goossens LHJ, Hale AR, Roelen ALC, Smith E (2006) Towards a causal model for air transport safetyan ongoing research project. Saf Sci 44(8):657–673

    Article  Google Scholar 

  145. Ale BJ, Bellamy LJ, Van der Boom R, Cooper J, Cooke RM, Goossens LH, Spouge J (2009) Further development of a causal model for air transport safety (CATS): building the mathematical heart. Reliab Eng Sys Saf 94(9):1433–1441

    Article  Google Scholar 

  146. Sanmiquel L, Rossell JM, Vintro C (2015) Study of Spanish mining accidents using data mining techniques. Saf Sci 75:49–55

    Article  Google Scholar 

  147. Li J, Ma S, Le T, Liu L, Liu J (2016) Causal decision trees. IEEE Trans Knowl Data Eng 29(2):257–271

    Article  Google Scholar 

  148. Zhang W, Le TD, Liu L, Zhou ZH, Li J (2017) Mining heterogeneous causal effects for personalized cancer treatment. Bioinformatics: btx174

  149. Richard MD, Lippmann RP (1991) Neural network classifiers estimate Bayesian a posteriori probabilities. Neural Comput 3(4):461–483

    Article  Google Scholar 

  150. Zhang GP (2000) Neural networks for classification: a survey. IEEE Trans Syst Man Cybern Part C Appl Rev 30(4):451–462

    Article  Google Scholar 

  151. Gish H (1990) A probabilistic approach to the understanding and training of neural network classifiers. In: 1990 International conference on acoustics, speech, and signal processing, ICASSP-90, IEEE, pp 1361–1364

  152. Shoemaker PA (1991) A note on least-squares learning procedures and classification by neural network models. IEEE Trans Neural Netw 2(1):158–160

    Article  Google Scholar 

  153. Wan EA (1989) Neural network classification: a Bayesian interpretation. IEEE Trans Neural Netw 1(4):303–305

    Article  Google Scholar 

  154. Widrow B, Rumelhart DE, Lehr MA (1994) Neural networks: applications in industry, business and science. Commun ACM 37(3):93–105

    Article  Google Scholar 

  155. Tu JV (1996) Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. J Clin Epidemiol 49(11):1225–1231

    Article  Google Scholar 

  156. Bate A, Lindquist M, Edwards IR, Olsson S, Orre R, Lansner A, De Freitas RM (1998) A Bayesian neural network method for adverse drug reaction signal generation. Eur J Clin Pharmacol 54(4):315–321

    Article  Google Scholar 

  157. Wahde M, Hertz J (2000) Coarse-grained reverse engineering of genetic regulatory networks. Biosystems 55(1):129–136

    Article  Google Scholar 

  158. Vohradský J (2001) Neural network model of gene expression. FASEB J 15(3):846–854

    Article  Google Scholar 

  159. Xu R, Venayagamoorthy GK, Wunsch DC (2007) Modeling of gene regulatory networks with hybrid differential evolution and particle swarm optimization. Neural Netw 20(8):917–927

    Article  Google Scholar 

  160. Noman N, Palafox L, Iba H (2013) Reconstruction of gene regulatory networks from gene expression data using decoupled recurrent neural network model. In: Natural computing and beyond, Springer, Japan, pp 93–103

  161. Kale DC, Che Z, Bahadori MT, Li W, Liu Y, Wetzel R (2015) Causal phenotype discovery via deep networks. In: AMIA annual symposium proceedings, American Medical Informatics Association, p 677

  162. Lagazio M, Russett B (2003) A neural network analysis of militarized disputes, 1885–1992: temporal stability and causal complexity. University of Michigan Press, New Jersey, pp 28–62

    Google Scholar 

  163. Montalto A, Stramaglia S, Faes L, Tessitore G, Prevete R, Marinazzo D (2015) Neural networks with non-uniform embedding and explicit validation phase to assess Granger causality. Neural Netw 71:159–171

    Article  Google Scholar 

  164. Suykens JA, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300

    Article  Google Scholar 

  165. Beamer B, Bhat S, Chee B, Fister A, Rozovskaya A, Girju R (2007) UIUC: A knowledge-rich approach to identifying semantic relations between nominals. In: Proceedings of the 4th international workshop on semantic evaluations, Association for Computational Linguistics, pp 386–389

  166. Chen SH, Sun J, Dimitrov L, Turner AR, Adams TS, Meyers DA, Hsu FC (2008) A support vector machine approach for detecting genegene interaction. Genet Epidemiol 32(2):152–167

    Article  Google Scholar 

  167. Roshan U, Chikkagoudar S, Wei Z, Wang K, Hakonarson H (2011) Ranking causal variants and associated regions in genome-wide association studies by the support vector machine and random forest. Nucleic Acids Res 39(9):e62

    Article  Google Scholar 

  168. Lee S, Ruiz S, Caria A, Veit R, Birbaumer N, Sitaram R (2011) Detection of cerebral reorganization induced by real-time fMRI feedback training of insula activation a multivariate investigation. Neurorehabilit Neural Repair 25(3):259–267

    Article  Google Scholar 

  169. Seol JW, Yi W, Choi J, Lee KS (2017) Causality patterns and machine learning for the extraction of problem–action relations in discharge summaries. Int J Med Inform 98:1–12

    Article  Google Scholar 

  170. Zhang H, Yao DD, Ramakrishnan N (2014) Detection of stealthy malware activities with traffic causality and scalable triggering relation discovery. In: Proceedings of the 9th ACM symposium on information, computer and communications security, ACM, pp 39–50

  171. Sarkar S, Vinay S, Pateshwari V, Maiti J (2016) Study of optimized SVM for incident prediction of a steel plant in India. In: IEEE Annual India conference (INDICON), IEEE, pp 1–6

  172. Langley P, Iba W, Thompson K (1992) An analysis of Bayesian classifiers. AAAI 90:223–228

    Google Scholar 

  173. Kohavi R (1996) Scaling up the accuracy of Naive–Bayes classifiers: a decision-tree hybrid. In KDD, pp 202–207

  174. Zhang H (2004) The optimality of naive Bayes. AA, Vol. 1(2), 3

  175. Chang DS, Choi KS (2004) Causal relation extraction using cue phrase and lexical pair probabilities. In: International conference on natural language processing, Springer, Berlin, Heidelberg, pp 61–70

  176. Sorgente A, Vettigli G, Mele F (2013) Automatic extraction of cause–effect relations in natural language text. DART AI IA, pp 37–48

  177. Zhao S, Liu T, Zhao S, Chen Y, Nie JY (2016) Event causality extraction based on connectives analysis. Neurocomputing 173:1943–1950

    Article  Google Scholar 

  178. Amor NB, Benferhat S, Elouedi Z (2004) Naive bayes versus decision trees in intrusion detection systems. In: Proceedings of the 2004 ACM symposium on applied computing, ACM, pp 420–424

  179. Benferhat S, Kenaza T, Mokhtari A (2008) A naive bayes approach for detecting coordinated attacks. In: 32nd annual IEEE international computer software and applications, COMPSAC’08, IEEE, pp 704–709

  180. Wang L (2015) Mining causal relationships among clinical variables for cancer diagnosis based on Bayesian analysis. BioData Min 8(1):13

    Article  Google Scholar 

  181. Krishna MSG, Singh S (2016) Identification of causal relationships among clinical variables for cancer diagnosis using multi-tenancy. In: 2016 International conference on advances in computing, communications and informatics (ICACCI), IEEE, pp 1511–1516

  182. Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The WEKA data mining software: an update. ACM SIGKDD Explor Newsl 11(1):10–18

    Article  Google Scholar 

  183. Collins M, Duffy N (2001) Convolution kernels for natural language. NIPS 14:625–632

    Google Scholar 

  184. Alcobé JR (2002) Incremental learning of tree augmented naive Bayes classifiers. In: Ibero-American conference on artificial intelligence, Springer, Berlin, Heidelberg, pp 32–41

  185. Chang CC, Lin CJ (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):27

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for International Energy Studies, Tehran, 1967743711, Iran

    Hossein Hassani & Mansi Ghodsi

  2. Faculty of Business and Law, De Montfort University, Leicester, UK

    Xu Huang

Authors
  1. Hossein Hassani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xu Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mansi Ghodsi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hossein Hassani.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hassani, H., Huang, X. & Ghodsi, M. Big Data and Causality. Ann. Data. Sci. 5, 133–156 (2018). https://doi.org/10.1007/s40745-017-0122-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40745-017-0122-3

Keywords

Search

Navigation