Big data analytics in bioinformatics: architectures, techniques, tools and issues

  • Hirak Kashyap
  • Hasin Afzal Ahmed
  • Nazrul Hoque
  • Swarup Roy
  • Dhruba Kumar Bhattacharyya
Review Article


Bioinformatics research is characterized by voluminous and incremental datasets and complex data analytics methods. The machine learning methods used in bioinformatics are iterative and parallel. These methods can be scaled to handle big data using the distributed and parallel computing technologies. Usually big data tools perform computation in batch mode and are not optimized for iterative processing and high data dependency among operations. In the recent years, parallel, incremental, and multi-view machine learning algorithms have been proposed. Similarly, graph-based architectures and in-memory big data tools have been developed to minimize I/O cost and optimize iterative processing. However, standard big data architectures are still lacking. Also appropriate tools are not available for many important bioinformatics problems, such as fast construction of co-expression and regulatory networks and salient module identification, detection of complexes over growing protein-protein interaction data, fast analysis of massive DNA, RNA, and protein sequence data, and fast querying on incremental and heterogeneous disease networks. This paper addresses the issues and challenges posed by several big data problems in bioinformatics, and gives an overview of the state of the art and the future research opportunities.


Big data Bioinformatics Machine learning MapReduce Clustering Gene regulatory network 


  1. Aggarwal CC, Reddy CK (eds)(2013) Data clustering: algorithms and applications. CRC PressGoogle Scholar
  2. Agrawal R, Imieliński T, Swami A (1993) Mining association rules between sets of items in large databases. In: ACM SIGMOD Record, vol 22. ACM, pp 207–216Google Scholar
  3. Agrawal R, Shafer JC (1996) Parallel mining of association rules. IEEE Trans Knowl Data Eng 8(6):962–969CrossRefGoogle Scholar
  4. Agrawal R, Srikant R et al (1994) Fast algorithms for mining association rules. In: Proc. 20th int. conf. very large data bases, VLDB, vol 1215, pp 487–499Google Scholar
  5. Ahmed H, Mahanta P, Bhattacharyya D, Kalita J (2014) Shifting-and-scaling correlation based biclustering algorithm. Comput Biol Bioinf IEEE ACM Trans 11(6):1239–1252CrossRefGoogle Scholar
  6. Ahmed H, Mahanta P, Bhattacharyya D, Kalita J, Ghosh A (2011) Intersected coexpressed subcube miner: an effective triclustering algorithm. In: Information and communication technologies (WICT), 2011 World Congress. IEEE, pp 846–851Google Scholar
  7. Angiuoli SV, Matalka M, Gussman A, Galens K, Vangala M, Riley DR, Arze C, White JR, White O, Fricke WF (2011) CloVR: a virtual machine for automated and portable sequence analysis from the desktop using cloud computing. BMC Bioinf 12(1):356CrossRefGoogle Scholar
  8. Arefin AS, Berretta R, Moscato P (2013) A GPU-based method for computing eigenvector centrality of gene-expression networks. In: Proceedings of the eleventh Australasian symposium on parallel and distributed computing, vol 140. Australian Computer Society, Inc., pp 3–11Google Scholar
  9. Aumann Y, Feldman R, Lipshtat O, Manilla H (1999) Borders: an efficient algorithm for association generation in dynamic databases. J Intell Inf Syst 12(1):61–73CrossRefGoogle Scholar
  10. Bader GD, Hogue CW (2003) An automated method for finding molecular complexes in large protein interaction networks. BMC Bioinf 4(1):2CrossRefGoogle Scholar
  11. Bagyamathi M, Inbarani HH (2015) A novel hybridized rough set and improved harmony search based feature selection for protein sequence classification. In: Hassanien AE, Azar AT, Snasael V, Kacprzyk J, Abawajy JH (eds) Big data in complex systems, vol 9. Springer, pp 173–204Google Scholar
  12. Baraldi A, Bruzzone L, Blonda P (2006) A multiscale expectation-maximization semisupervised classifier suitable for badly posed image classification. Image Process IEEE Trans 15(8):2208–2225CrossRefGoogle Scholar
  13. Barbu A, She Y, Ding L, Gramajo G (2013) Feature selection with annealing for big data learning. arXiv:1310.2880 (preprint)
  14. Barker MS, Dlugosch KM, Dinh L, Challa RS, Kane NC, King MG, Rieseberg LH (2010) EvoPipes. net: bioinformatic tools for ecological and evolutionary genomics. Evol Bioinf Online 6:143CrossRefGoogle Scholar
  15. Ben-Dor A, Chor B, Karp R, Yakhini Z (2003) Discovering local structure in gene expression data: the order-preserving submatrix problem. J Comput Biol 10:373–384CrossRefGoogle Scholar
  16. Berényi Z, Vajk I (2009) Probabilistic model for a distributed feature selection method. In: Soft computing applications, 2009. SOFA’09. 3rd International Workshop. IEEE, pp 27–32Google Scholar
  17. Bergmann S, Ihmels J, Barkai N (2003) Iterative signature algorithm for the analysis of large-scale gene expression data. Phys Rev E 67:031,902–031,919Google Scholar
  18. Berkhin P (2006) A survey of clustering data mining techniques. In: Kogan J, Nicholas C, Teboulle M (eds) Grouping multidimensional data. Springer, Berlin, Heidelberg, pp 25–71Google Scholar
  19. Bhatia S, Prakash P, Pillai G (2008) Svm based decision support system for heart disease classification with integer-coded genetic algorithm to select critical features. In: Proceedings of the world congress on engineering and computer science, WCECS, pp 22–24Google Scholar
  20. Bhattacharyya DK, Kalita JK (2013) Network anomaly detection: a machine learning perspectiveGoogle Scholar
  21. Bishop CM et al (2006) Pattern recognition and machine learning, vol 4. Springer, New YorkzbMATHGoogle Scholar
  22. Blum A (2015) Semi-supervised learning (2015)Google Scholar
  23. Bolón-Canedo V, Sánchez-Maroño N, Alonso-Betanzos A (2015) Distributed feature selection: an application to microarray data classification. Appl Soft Comput 30:136–150CrossRefGoogle Scholar
  24. Bolouri H (2014) Modeling genomic regulatory networks with big data. Trends Genet 30(5):182–191CrossRefGoogle Scholar
  25. Borthakur D (2007) The hadoop distributed file system: architecture and design. Hadoop Project Website 11(2007):21Google Scholar
  26. Bradley PS, Fayyad UM, Reina C et al (1998) Scaling clustering algorithms to large databases. In: KDD, pp 9–15Google Scholar
  27. Brin S, Motwani R, Ullman JD, Tsur S (1997) Dynamic itemset counting and implication rules for market basket data. In: ACM SIGMOD Record, vol 26. ACM, pp 255–264Google Scholar
  28. Cai D, He X, Han J (2008) Srda: an efficient algorithm for large-scale discriminant analysis. Knowl Data Eng IEEE Trans 20(1):1–12CrossRefGoogle Scholar
  29. Calaway R, Edlefsen L, Gong L, Fast S (2016) Big data decision trees with r. RevolutionGoogle Scholar
  30. Cerami EG, Gross BE, Demir E, Rodchenkov I, Babur Ö, Anwar N, Schultz N, Bader GD, Sander C (2011) Pathway commons, a web resource for biological pathway data. Nucleic Acids Res 39(suppl 1):D685–D690CrossRefGoogle Scholar
  31. Chakraborty S, Nagwani N (2011) Analysis and study of incremental k-means clustering algorithm. In: High performance architecture and grid computing. Springer, Berlin, Heidelberg, pp 338–341Google Scholar
  32. Chaudhuri K, Kakade SM, Livescu K, Sridharan K (2009) Multi-view clustering via canonical correlation analysis. In: Proceedings of the 26th annual international conference on machine learning. ACM, pp 129–136Google Scholar
  33. Chen N, Chen AZ, Zhou LX (2002) An incremental grid density-based clustering algorithm. J Softw 13(1):1–7Google Scholar
  34. Cheng Y, Church GM (2000) Biclustering of expression data. Ismb 8:93–103Google Scholar
  35. Cheung DW, Han J, Ng VT, Fu AW, Fu Y (1996) A fast distributed algorithm for mining association rules. In: Parallel and distributed information systems, 1996. Fourth International Conference. IEEE, pp 31–42Google Scholar
  36. Cheung DW, Xiao Y (1998) Effect of data skewness in parallel mining of association rules. In: Research and development in knowledge discovery and data mining. Springer, Berlin, Heidelberg, pp 48–60Google Scholar
  37. Chien BC, Lin ZL, Hong TP (2001) An efficient clustering algorithm for mining fuzzy quantitative association rules. In: IFSA World Congress and 20th NAFIPS International Conference, 2001. Joint 9th, vol 3. IEEE, pp 1306–1311Google Scholar
  38. Choudhury A, Nair PB, Keane AJ et al (2002) A data parallel approach for large-scale gaussian process modeling. In: SDM. SIAM, pp 95–111Google Scholar
  39. Cisco (2015) Cisco visual networking index: global mobile data traffic forecast update, 2014–2019. Cisco Public InformationGoogle Scholar
  40. Croft D, OKelly G, Wu G, Haw R, Gillespie M, Matthews L, Caudy M, Garapati P, Gopinath G, Jassal B et al (2010) Reactome: a database of reactions, pathways and biological processes. Nucleic Acids Res, p gkq1018Google Scholar
  41. Davidich M, Bornholdt S (2008) Boolean network model predicts cell cycle sequence of fission yeast. PLoS One 3(2):e1672CrossRefGoogle Scholar
  42. Day A, Carlson MR, Dong J, O’Connor BD, Nelson SF (2007) Celsius: a community resource for Affymetrix microarray data. Genome Biol 8(6):R112CrossRefGoogle Scholar
  43. Day A, Dong J, Funari VA, Harry B, Strom SP, Cohn DH, Nelson SF (2009) Disease gene characterization through large-scale co-expression analysis. PLoS One 4(12):e8491CrossRefGoogle Scholar
  44. Dean J, Ghemawat S (2005) Mapreduce: simplified data processing on large clusters. In: OSDI\(\backslash \)’04, pp 137–150Google Scholar
  45. Dean J, Ghemawat S (2008) Mapreduce: simplified data processing on large clusters. Commun ACM 51(1):107–113CrossRefGoogle Scholar
  46. Divina F, Pontes B, Giráldez R, Aguilar-Ruiz JS (2011) An effective measure for assessing the quality of biclusters. Comput Biol Med 42(2):245–256CrossRefGoogle Scholar
  47. Jiang D, Pei J, Ramanathan M, Tang C, Zhang A (2004) Mining coherent gene clusters from gene-sample-time microarray data. In: In Proc of the 10 th ACM SIGKDD Conference (KDD’04)Google Scholar
  48. Djuric N (2014) Big data algorithms for visualization and supervised learning. Ph.D. thesis, Temple UniversityGoogle Scholar
  49. Duda RO, Hart PE, Stork DG (2012) Pattern classification. Wiley, New YorkGoogle Scholar
  50. Ecker C, Rocha-Rego V, Johnston P, Mourao-Miranda J, Marquand A, Daly EM, Brammer MJ, Murphy C, Murphy DG, Consortium MA et al (2010) Investigating the predictive value of whole-brain structural mr scans in autism: a pattern classification approach. Neuroimage 49(1):44–56CrossRefGoogle Scholar
  51. Ekanayake J, Li H, Zhang B, Gunarathne T, Bae SH, Qiu J, Fox G (2010) Twister: a runtime for iterative mapreduce. In: Proceedings of the 19th ACM international symposium on high performance distributed computing. ACM, pp 810–818Google Scholar
  52. EMBL-European Bioinformatics Institute (2014) EMBL-EBI annual scientific report 2013Google Scholar
  53. Ester M, Kriegel HP, Sander J, Xu X (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In: Kdd, vol 96, pp 226–231Google Scholar
  54. Faith J, Hayete B, Thaden J, Mogno I, Wierzbowski J, Cottarel G, Kasif S, Collins J, Gardner T (2007) Large-scale mapping and validation of escherichia coli transcriptional regulation from a compendium of expression profiles. PLoS Biol 5(1):e8CrossRefGoogle Scholar
  55. Floridi L (2012) Big data and their epistemological challenge. Philos Technol 25(4):435–437CrossRefGoogle Scholar
  56. Fogel DB (2006) Evolutionary computation: toward a new philosophy of machine intelligence, vol 1. Wiley, New YorkGoogle Scholar
  57. Friedman N, Linial M, Nachman I, Pe’er D (2000) Using bayesian networks to analyze expression data. J Comput Biol 7(3–4):601–620CrossRefGoogle Scholar
  58. Garg A, Mangla A, Gupta N, Bhatnagar V (2006) Pbirch: a scalable parallel clustering algorithm for incremental data. In: Database engineering and applications symposium, 2006. IDEAS’06. 10th International. IEEE, pp 315–316Google Scholar
  59. Gershenfeld N, Krikorian R, Cohen D (2004) The internet of things. Sci Am 291(4):76CrossRefGoogle Scholar
  60. Giveki D, Salimi H, Bahmanyar G, Khademian Y (2012) Automatic detection of diabetes diagnosis using feature weighted support vector machines based on mutual information and modified cuckoo search. arXiv:1201.2173 (preprint)
  61. Goecks J, Nekrutenko A, Taylor J et al (2010) Galaxy: a comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences. Genome Biol 11(8):R86CrossRefGoogle Scholar
  62. Gropp W, Lusk E, Doss N, Skjellum A (1996) A high-performance, portable implementation of the mpi message passing interface standard. Parallel Comput 22(6):789–828zbMATHCrossRefGoogle Scholar
  63. Grosu P, Townsend JP, Hartl DL, Cavalieri D (2002) Pathway Processor: a tool for integrating whole-genome expression results into metabolic networks. Genome Res 12(7):1121–1126CrossRefGoogle Scholar
  64. Guha S, Rastogi R, Shim K (1998) Cure: an efficient clustering algorithm for large databases. In: ACM SIGMOD record, vol 27. ACM, pp 73–84Google Scholar
  65. Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182zbMATHGoogle Scholar
  66. Hall LO, Chawla N, Bowyer KW (1998) Decision tree learning on very large data sets. In: Systems, man, and cybernetics, 1998. 1998 IEEE international conference, vol 3. IEEE, pp 2579–2584Google Scholar
  67. Hall MA, Smith LA (1999) Feature selection for machine learning: comparing a correlation-based filter approach to the wrapper. In: FLAIRS conference, vol 1999, pp 235–239Google Scholar
  68. Haller S, Badoud S, Nguyen D, Garibotto V, Lovblad K, Burkhard P (2012) Individual detection of patients with parkinson disease using support vector machine analysis of diffusion tensor imaging data: initial results. Am J Neuroradiol 33(11):2123–2128CrossRefGoogle Scholar
  69. Han J, Pei J (2000) Mining frequent patterns by pattern-growth: methodology and implications. ACM SIGKDD Explor Newsl 2(2):14–20CrossRefGoogle Scholar
  70. Hinneburg A, Keim DA (1998) An efficient approach to clustering in large multimedia databases with noise. In: KDD, vol 98, pp 58–65Google Scholar
  71. Hinton G, Deng L, Yu D, Dahl GE, Mohamed AR, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath TN et al (2012) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. Signal Process Mag IEEE 29(6):82–97CrossRefGoogle Scholar
  72. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507MathSciNetzbMATHCrossRefGoogle Scholar
  73. Hoi SC, Wang J, Zhao P, Jin R (2012) Online feature selection for mining big data. In: Proceedings of the 1st international workshop on big data, streams and heterogeneous source mining: algorithms, systems, programming models and applications. ACM, pp 93–100Google Scholar
  74. Höppner F (1999) Fuzzy cluster analysis: methods for classification, data analysis and image recognitionGoogle Scholar
  75. Hoque N, Bhattacharyya D, Kalita J (2014) Mifs-nd: a mutual information-based feature selection method. Expert Syst Appl 41(14):6371–6385CrossRefGoogle Scholar
  76. Houtsma M, Swami A (1995) Set-oriented mining for association rules in relational databases. In: Data engineering, 1995. Proceedings of the Eleventh International Conference. IEEE, pp 25–33Google Scholar
  77. Hsieh CJ, Si S, Dhillon IS (2013) A divide-and-conquer solver for kernel support vector machines. arXiv:1311.0914 (preprint)
  78. Huang Z (1998) Extensions to the k-means algorithm for clustering large data sets with categorical values. Data Mining Knowl Discov 2(3):283–304CrossRefGoogle Scholar
  79. Hubert LJ (1974) Some applications of graph theory to clustering. Psychometrika 39(3):283–309MathSciNetzbMATHCrossRefGoogle Scholar
  80. Hughes GP (1968) On the mean accuracy of statistical pattern recognizers. Inf Theory IEEE Trans 14(1):55–63CrossRefGoogle Scholar
  81. Jain A, Zongker D (1997) Feature selection: Evaluation, application, and small sample performance. Pattern Anal Mach Intell IEEE Trans 19(2):153–158CrossRefGoogle Scholar
  82. Jain AK, Murty MN, Flynn PJ (1999) Data clustering: a review. ACM Comput Surv (CSUR) 31(3):264–323CrossRefGoogle Scholar
  83. Janecek A, Gansterer WN, Demel M, Ecker G (2008) On the relationship between feature selection and classification accuracy. In: FSDM, pp 90–105Google Scholar
  84. Jiang H, Zhou S, Guan J, Zheng Y (2006) gtricluster: a more general and effective 3d clustering algorithm for gene-sample-time microarray data. In: BioDM’06, pp 48–59Google Scholar
  85. Judd D, McKinley PK, Jain, AK (1996) Large-scale parallel data clustering. In: Pattern recognition, 1996. Proceedings of the 13th International Conference, vol 4. IEEE, pp 488–493Google Scholar
  86. Kailing K, Kriegel HP, Pryakhin A, Schubert M (2004) Clustering multi-represented objects with noise. In: Advances in knowledge discovery and data mining. Springer, Berlin, Heidelberg, pp 394–403Google Scholar
  87. Kanehisa M, Goto S (2000) KEGG: kyoto encyclopedia of genes and genomes. Nucleic Acids Res 28(1):27–30CrossRefGoogle Scholar
  88. Karypis G, Han EH, Kumar V (1999) Chameleon: hierarchical clustering using dynamic modeling. Computer 32(8):68–75CrossRefGoogle Scholar
  89. Kaufman L, Rousseeuw P (1987) Clustering by means of medoids. North-HollandGoogle Scholar
  90. Kaufman L, Rousseeuw PJ (1990) Finding groups in data. An introduction to cluster analysis. Wiley Series in Probability and Mathematical Statistics. Applied Probability and Statistics, vol 1. Wiley, New YorkGoogle Scholar
  91. Kaufman L, Rousseeuw PJ (2009) Finding groups in data: an introduction to cluster analysis. Wiley, Hoboken, NJGoogle Scholar
  92. Kelley BP, Yuan B, Lewitter F, Sharan R, Stockwell BR, Ideker T (2004) PathBLAST: a tool for alignment of protein interaction networks. Nucleic Acids Res 32(suppl 2):W83–W88CrossRefGoogle Scholar
  93. Kira K, Rendell LA (1992) A practical approach to feature selection. In: Proceedings of the ninth international workshop on Machine learning, pp 249–256Google Scholar
  94. Kluger Y, Basri R, Chang J, Gerstein M (2003) Spectral biclustering of microarray data: coclustering genes and conditions. Genome Res 13(4):703–716CrossRefGoogle Scholar
  95. Kohonen T (1990) The self-organizing map. Proc IEEE 78(9):1464–1480CrossRefGoogle Scholar
  96. Kraska T, Talwalkar A, Duchi JC, Griffith R, Franklin MJ, Jordan MI (2013) Mlbase: a distributed machine-learning system. In: CIDRGoogle Scholar
  97. Kriegel HP, Kröger P, Sander J, Zimek A (2011) Density-based clustering. Wiley Interdiscip Rev Data Mining Knowl Discov 1(3):231–240CrossRefGoogle Scholar
  98. Kumar A, Daumé H (2011) A co-training approach for multi-view spectral clustering. In: Proceedings of the 28th international conference on machine learning (ICML-11), pp 393–400Google Scholar
  99. Kumar S, Nei M, Dudley J, Tamura K (2008) MEGA: a biologist-centric software for evolutionary analysis of DNA and protein sequences. Brief Bioinf 9(4):299–306CrossRefGoogle Scholar
  100. Kurtz S (2003) The vmatch large scale sequence analysis software. Ref Type: Computer Program, pp 4–12Google Scholar
  101. Langfelder P, Horvath S (2008) WGCNA: an R package for weighted correlation network analysis. BMC Bioinf 9(1):559CrossRefGoogle Scholar
  102. Langmead B, Schatz MC, Lin J, Pop M, Salzberg SL (2009) Searching for SNPs with cloud computing. Genome Biol 10(11):R134CrossRefGoogle Scholar
  103. Langmead B, Trapnell C, Pop M, Salzberg SL et al (2009) Ultrafast and memory-efficient alignment of short DNA sequences to the human genome. Genome Biol 10(3):R25CrossRefGoogle Scholar
  104. Lee H, Hsu A, Sajdak J, Qin J, Pavlidis P (2004) Coexpression analysis of human genes across many microarray data sets. Genome Res 14(6):1085–1094CrossRefGoogle Scholar
  105. Li R, Li Y, Fang X, Yang H, Wang J, Kristiansen K, Wang J (2009) SNP detection for massively parallel whole-genome resequencing. Genome Res 19(6):1124–1132CrossRefGoogle Scholar
  106. Li X, Fang Z (1989) Parallel clustering algorithms. Parallel Comput 11(3):275–290MathSciNetzbMATHCrossRefGoogle Scholar
  107. Liang, M., Zhang, F., Jin, G., Zhu, J (2014) FastGCN: a GPU accelerated tool for fast gene co-expression networks. PLoS One 10(1):e0116,776–e0116,776Google Scholar
  108. Lin D, Foster DP, Ungar LH (2011) Vif regression: a fast regression algorithm for large data. J Am Stat Assoc 106(493):232–247MathSciNetzbMATHCrossRefGoogle Scholar
  109. Liu F, Guo W, Fouche JP, Wang Y, Wang W, Ding J, Zeng L, Qiu C, Gong Q, Zhang W et al (2015) Multivariate classification of social anxiety disorder using whole brain functional connectivity. Brain Struct Funct 220(1):101–115CrossRefGoogle Scholar
  110. Liu F, Guo W, Yu D, Gao Q, Gao K, Xue Z, Du H, Zhang J, Tan C, Liu Z et al (2012) Classification of different therapeutic responses of major depressive disorder with multivariate pattern analysis method based on structural MR scans. PLoS One 7(7):e40968CrossRefGoogle Scholar
  111. Liu F, Suk HI, Wee CY, Chen H, Shen D (2013) High-order graph matching based feature selection for Alzheimers disease identification. In: Medical image computing and computer-assisted intervention–MICCAI 2013. Springer, Berlin, Heidelberg, pp 311–318Google Scholar
  112. Liu F, Wee CY, Chen H, Shen D (2014) Inter-modality relationship constrained multi-modality multi-task feature selection for alzheimer’s disease and mild cognitive impairment identification. NeuroImage 84:466–475CrossRefGoogle Scholar
  113. Liu F, Xie B, Wang Y, Guo W, Fouche JP, Long Z, Wang W, Chen H, Li M, Duan X et al (2014) Characterization of post-traumatic stress disorder using resting-state fmri with a multi-level parametric classification approach. Brain Topogr 28(2):221–237CrossRefGoogle Scholar
  114. López M, Still G (2007) Semi-infinite programming. Eur J Oper Res 180(2):491–518MathSciNetzbMATHCrossRefGoogle Scholar
  115. Low Y, Bickson D, Gonzalez J, Guestrin C, Kyrola A, Hellerstein JM (2012) Distributed graphlab: a framework for machine learning and data mining in the cloud. Proc VLDB Endow 5(8):716–727CrossRefGoogle Scholar
  116. Low Y, Gonzalez JE, Kyrola A, Bickson D, Guestrin CE, Hellerstein J (2014) Graphlab: a new framework for parallel machine learning. arXiv:1408.2041 (preprint)
  117. Luo W, Brouwer C (2013) Pathview: an R/Bioconductor package for pathway-based data integration and visualization. Bioinformatics 29(14):1830–1831CrossRefGoogle Scholar
  118. Madhamshettiwar PB, Maetschke SR, Davis MJ, Reverter A, Ragan MA (2012) Gene regulatory network inference: evaluation and application to ovarian cancer allows the prioritization of drug targets. Genome Med 4(5):1–16CrossRefGoogle Scholar
  119. Malewicz G, Austern MH, Bik AJ, Dehnert JC, Horn I, Leiser N, Czajkowski G (2010) Pregel: a system for large-scale graph processing. In: Proceedings of the 2010 ACM SIGMOD international conference on management of data. ACM, pp 135–146Google Scholar
  120. Mannila H, Toivonen H (1997) Levelwise search and borders of theories in knowledge discovery. Data Mining Knowl Discov 1(3):241–258CrossRefGoogle Scholar
  121. Margolin A, Nemenman I, Basso K, Wiggins C, Stolovitzky G, Favera R, Califano A (2006) Aracne: an algorithm for the reconstruction of gene regulatory networks in a mammalian cellular context. BMC Bioinf 7(Suppl 1):S7CrossRefGoogle Scholar
  122. Marx V (2013) Biology: the big challenges of big data. Nature 498(7453):255–260CrossRefGoogle Scholar
  123. Matsunaga A, Tsugawa M, Fortes J (2008) Cloudblast: combining mapreduce and virtualization on distributed resources for bioinformatics applications. In: eScience, 2008. eScience’08. IEEE fourth international conference. IEEE, pp 222–229Google Scholar
  124. McArt DG, Bankhead P, Dunne PD, Salto-Tellez M, Hamilton P, Zhang SD (2013) cudaMap: a GPU accelerated program for gene expression connectivity mapping. BMC Bioinf 14(1):305CrossRefGoogle Scholar
  125. Meyer P, Kontos K, Lafitte F, Bontempi G (2007) Information-theoretic inference of large transcriptional regulatory networks. EURASIP J Bioinf Syst Biol 2007(1):1–9CrossRefGoogle Scholar
  126. Mitchell TM (1997) Machine learning, vol 45. McGraw Hill, Burr RidgezbMATHGoogle Scholar
  127. Moens S, Aksehirli E, Goethals B (2013) Frequent itemset mining for big data. In: Big data, 2013 IEEE international conference. IEEE, pp 111–118Google Scholar
  128. Mohri M, Rostamizadeh A, Talwalkar A (2012) Foundations of machine learning. MIT PressGoogle Scholar
  129. Mosquera J, Sánchez-Pla A (2008) Serbgo: searching for the best go tool. Nucleic Acids Res 36(suppl 2):W368–W371CrossRefGoogle Scholar
  130. Najafabadi MM, Villanustre F, Khoshgoftaar TM, Seliya N, Wald R, Muharemagic E (2015) Deep learning applications and challenges in big data analytics. J Big Data 2(1):1–21CrossRefGoogle Scholar
  131. Nei F, Huang Y, Wang X, Huang H (2014) New primal svm solver with linear computational cost for big data classifications. In: Proceedings of the 31st international conference on machine learning, JMLR, pp 1–9Google Scholar
  132. Nekrutenko A, Taylor J (2012) Next-generation sequencing data interpretation: enhancing reproducibility and accessibility. Nat Rev Genet 13(9):667–672CrossRefGoogle Scholar
  133. Nepusz T, Yu H, Paccanaro A (2012) Detecting overlapping protein complexes in protein-protein interaction networks. Nat Methods 9(5):471–472CrossRefGoogle Scholar
  134. Ng RT, Han J (2002) Clarans: a method for clustering objects for spatial data mining. Knowl Data Eng IEEE Trans 14(5):1003–1016CrossRefGoogle Scholar
  135. Ngiam J, Khosla A, Kim M, Nam J, Lee H, Ng AY (2011) Multimodal deep learning. In: Proceedings of the 28th international conference on machine learning (ICML-11), pp 689–696Google Scholar
  136. Nordberg H, Bhatia K, Wang K, Wang Z (2013) BioPig: a Hadoop-based analytic toolkit for large-scale sequence data. Bioinformatics 29(23):3014–3019CrossRefGoogle Scholar
  137. O’Leary DE (2013) Artificial intelligence and big data. IEEE Intell Syst 28(2):0096–99MathSciNetCrossRefGoogle Scholar
  138. Ordonez C, Omiecinski E (2004) Efficient disk-based k-means clustering for relational databases. Knowl Data Eng IEEE Trans 16(8):909–921CrossRefGoogle Scholar
  139. Ovsiannikov M, Rus S, Reeves D, Sutter P, Rao S, Kelly J (2013) The quantcast file system. Proc VLDB Endow 6(11):1092–1101CrossRefGoogle Scholar
  140. Owen S, Anil R, Dunning T, Friedman E (2011) Mahout in action. Manning, Shelter Island, NYGoogle Scholar
  141. Page M, Molina M, Gordon J (2013) The mobile economy 2013. ATKearney [Online]. pdf. Accessed 09 Feb 2015
  142. Pareto V (1964) Cours d’économie politique. Droz, GenèveGoogle Scholar
  143. Park BH, Kargupta H (2002) Distributed data mining: algorithms, systems, and applications. In: Data mining handbook, pp 341–358Google Scholar
  144. Park JS, Chen MS, Yu PS (1995) An effective hash-based algorithm for mining association rulesGoogle Scholar
  145. Park JS, Chen MS, Yu PS (1995) Efficient parallel data mining for association rules. In: Proceedings of the fourth international conference on Information and knowledge management. ACM, pp 31–36Google Scholar
  146. Park YS, Schmidt M, Martin ER, Pericak-Vance MA, Chung RH (2013) Pathway-PDT: a flexible pathway analysis tool for nuclear families. BMC Bioinf 14(1):267CrossRefGoogle Scholar
  147. Phan JH, Young AN, Wang MD (2013) omniBiomarker: a web-based application for knowledge-driven biomarker identification. Biomed Eng IEEE Trans 60(12):3364–3367CrossRefGoogle Scholar
  148. Pontes B, Giráldez R, Aguilar-Ruiz J (2010) Measuring the quality of shifting and scaling patterns in biclusters. Pattern Recognit Bioinf 6282:242–252CrossRefGoogle Scholar
  149. Prelić A, Bleuler S, Zimmermann P, Wille A, Bühlmann P, Gruissem W, Hennig L, Thiele L, Zitzler E (2006) A systematic comparison and evaluation of biclustering methods for gene expression data. Bioinformatics 22(9):1122–1129CrossRefGoogle Scholar
  150. Raftery AE, Gneiting T, Balabdaoui F, Polakowski M (2005) Using bayesian model averaging to calibrate forecast ensembles. Mon Weather Rev 133(5):1155–1174CrossRefGoogle Scholar
  151. Rana O, Walker D, Li M, Lynden S, Ward M (2000) Paddmas: parallel and distributed data mining application suite. In: Parallel and distributed processing symposium, 2000. IPDPS 2000. Proceedings. 14th International. IEEE, pp 387–392Google Scholar
  152. Reed M, Huang J, Brand R, Graetz I, Neugebauer R, Fireman B, Jaffe M, Ballard DW, Hsu J (2013) Implementation of an outpatient electronic health record and emergency department visits, hospitalizations, and office visits among patients with diabetes. JAMA 310(10):1060–1065CrossRefGoogle Scholar
  153. Rivera CG, Vakil R, Bader JS (2010) NeMo: network module identification in Cytoscape. BMC Bioinf 11(Suppl 1):S61CrossRefGoogle Scholar
  154. Robison RJ (2014) How big is the human genome? Precis MedGoogle Scholar
  155. Rojahn SY (2012) Breaking the genome bottleneck. MIT Technol RevGoogle Scholar
  156. Roy S, Bhattacharyya DK (2008) Opam: an efficient one pass association mining technique without candidate generation. J Convergence Inf Technol 3(3):32–38Google Scholar
  157. Roy S, Bhattacharyya DK, Kalita JK (2014) Reconstruction of gene co-expression network from microarray data using local expression patterns. BMC Bioinf 15(Suppl 7):S10CrossRefGoogle Scholar
  158. Roy S, Bhattacharyya DK, Kalita JK (2015) Analysis of gene expression patterns using biclustering. Methods Mol Biol 1375:91–103. doi: 10.1007/7651_2015_280 CrossRefGoogle Scholar
  159. Savasere A, Omiecinski ER, Navathe SB (1995) An efficient algorithm for mining association rules in large databasesGoogle Scholar
  160. Schumacher A, Pireddu L, Niemenmaa M, Kallio A, Korpelainen E, Zanetti G, Heljanko K (2014) SeqPig: simple and scalable scripting for large sequencing data sets in Hadoop. Bioinformatics 30(1):119–120CrossRefGoogle Scholar
  161. Sheikholeslami G, Chatterjee S, Zhang A (2000) Wavecluster: a wavelet-based clustering approach for spatial data in very large databases. VLDB J 8(3–4):289–304CrossRefGoogle Scholar
  162. Shi W, Guo YF, Jin C, Xue X (2008) An improved generalized discriminant analysis for large-scale data set. In: Machine learning and applications, 2008. ICMLA’08. Seventh International Conference. IEEE, pp 769–772Google Scholar
  163. Shvachko K, Kuang H, Radia S, Chansler R (2010) The hadoop distributed file system. In: Mass storage systems and technologies (MSST), 2010 IEEE 26th Symposium. IEEE, pp 1–10Google Scholar
  164. Son YJ, Kim HG, Kim EH, Choi S, Lee SK (2010) Application of support vector machine for prediction of medication adherence in heart failure patients. Healthc Inf Res 16(4):253–259CrossRefGoogle Scholar
  165. Srikant R, Agrawal R (1996) Mining quantitative association rules in large relational tables. In: ACM SIGMOD record, vol 25. ACM, pp 1–12Google Scholar
  166. Stokes TH, Moffitt RA, Phan JH, Wang MD (2007) chip artifact CORRECTion (caCORRECT): a bioinformatics system for quality assurance of genomics and proteomics array data. Ann Biomed Eng 35(6):1068–1080CrossRefGoogle Scholar
  167. Tan M, Tsang IW, Wang L (2014) Towards ultrahigh dimensional feature selection for big data. J Mach Learn Res 15(1):1371–1429MathSciNetzbMATHGoogle Scholar
  168. Tan PN, Steinbach K, Kumar V (2006) Data mining cluster analysis: basic concepts and algorithmsGoogle Scholar
  169. Tanay A, Sharan R, Kupiec M, Shamir R (2004) Revealing modularity and organization in the yeast molecular network by integrated analysis of highly heterogeneous genome wide data. Proc Natl Acad Sci 101(9):2981–2986CrossRefGoogle Scholar
  170. Thomas S, Bodagala S, Alsabti K, Ranka S (1997) An efficient algorithm for the incremental updation of association rules in large databases. In: KDD, pp 263–266Google Scholar
  171. Thomas SA, Jin Y (2014) Reconstructing biological gene regulatory networks: where optimization meets big data. Evol Intell 7(1):29–47CrossRefGoogle Scholar
  172. Toivonen H et al (1996) Sampling large databases for association rules. VLDB 96:134–145Google Scholar
  173. Tseng GC, Ghosh D, Feingold E (2012) Comprehensive literature review and statistical considerations for microarray meta-analysis. Nucleic Acids Res 40(9):3785–3799. doi: 10.1093/nar/gkr1265 CrossRefGoogle Scholar
  174. Tsiliki G, Vlachakis D, Kossida S (2014) On integrating multi-experiment microarray data. Philos Trans R Soc Lond A Math Phys Eng Sci 372(2016):20130,136Google Scholar
  175. Turner V, Gantz J, Reinsel D, Minton S (2014) The digital universe of opportunities: rich data and the increasing value of the internet of things. International Data Corporation, White Paper, IDC_1672Google Scholar
  176. van Iersel MP, Kelder T, Pico AR, Hanspers K, Coort S, Conklin BR, Evelo C (2008) Presenting and exploring biological pathways with PathVisio. BMC Bioinf 9(1):399CrossRefGoogle Scholar
  177. Widyantoro DH, Ioerger TR, Yen J (2002) An incremental approach to building a cluster hierarchy. In: Data mining, 2002. ICDM 2003. Proceedings. 2002 IEEE International Conference. IEEE, pp 705–708Google Scholar
  178. Wright R, Yang Z (2004) Privacy-preserving bayesian network structure computation on distributed heterogeneous data. In: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 713–718Google Scholar
  179. Xu X, Jäger J, Kriegel HP (2002) A fast parallel clustering algorithm for large spatial databases. In: High performance data mining. Springer, US, pp 263–290Google Scholar
  180. Yang J, Wang H, Wang W, Yu P (2003) Enhanced biclustering on expression data. In: Proceedings of Third IEEE Symposium on Bioinformatics and Bioengineering, pp 321–327Google Scholar
  181. Yang P, Patrick E, Tan SX, Fazakerley DJ, Burchfield J, Gribben C, Prior MJ, James DE, Yang YH (2014) Direction pathway analysis of large-scale proteomics data reveals novel features of the insulin action pathway. Bioinformatics 30(6):808–814CrossRefGoogle Scholar
  182. Yang WH, Dai DQ, Yan H (2011) Finding correlated biclusters from gene expression data. Knowl Data Eng IEEE Trans 23(4):568–584CrossRefGoogle Scholar
  183. Ye J, Chow JH, Chen J, Zheng Z (2009) Stochastic gradient boosted distributed decision trees. In: Proceedings of the 18th ACM conference on Information and knowledge management. ACM, pp 2061–2064Google Scholar
  184. Yoo C, Ramirez L, Liuzzi J (2014) Big data analysis using modern statistical and machine learning methods in medicine. Int Neurourol J 18(2):50–57CrossRefGoogle Scholar
  185. Yuasa T, Urakami S, Yamamoto S, Yonese J, Nakano K, Kodaira M, Takahashi S, Hatake K, Inamura K, Ishikwa Y et al (2011) Tumor size is a potential predictor of response to tyrosine kinase inhibitors in renal cell cancer. Urology 77(4):831–835CrossRefGoogle Scholar
  186. Zaharia M, Chowdhury M, Das T, Dave A, Ma J, McCauley M, Franklin MJ, Shenker S, Stoica I (2012) Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation. USENIX Association, p 2Google Scholar
  187. Zambon AC, Gaj S, Ho I, Hanspers K, Vranizan K, Evelo CT, Conklin BR, Pico AR, Salomonis N (2012) GO-Elite: a flexible solution for pathway and ontology over-representation. Bioinformatics 28(16):2209–2210CrossRefGoogle Scholar
  188. Zeng A, Li T, Liu D, Zhang J, Chen H (2015) A fuzzy rough set approach for incremental feature selection on hybrid information systems. Fuzzy Sets Syst 258:39–60MathSciNetzbMATHCrossRefGoogle Scholar
  189. Zeng HJ, Chen Z, Ma WY (2002) A unified framework for clustering heterogeneous web objects. In: Web information systems engineering, 2002. WISE 2002. In: Proceedings of the third international conference. IEEE, pp 161–170Google Scholar
  190. Zhang S, Wu X, Zhang J, Zhang C (2005) A decremental algorithm for maintaining frequent itemsets in dynamic databases. In: Data warehousing and knowledge discovery. Springer, Berlin, Heidelberg, pp 305–314Google Scholar
  191. Zhang T, Ramakrishnan R, Livny M (1996) Birch: an efficient data clustering method for very large databases. In: ACM SIGMOD record, vol 25. ACM, pp 103–114Google Scholar
  192. Zhao L, Zaki MJ (2005) Tricluster: an effective algorithm for mining coherent clusters in 3D microarray data. ACM, pp 694–705. doi: 10.1145/1066157.1066236
  193. Zhao S, Prenger K, Smith L (2013) Stormbow: a cloud-based tool for reads mapping and expression quantification in large-scale RNA-Seq studies. ISRN Bioinform 2013:481545CrossRefGoogle Scholar
  194. Zhao S, Prenger K, Smith L, Messina T, Fan H, Jaeger E, Stephens S (2013) Rainbow: a tool for large-scale whole-genome sequencing data analysis using cloud computing. BMC Genomics 14(1):425CrossRefGoogle Scholar
  195. Zhao W, Ma H, He Q (2009) Parallel k-means clustering based on mapreduce. In: Cloud computing. Springer, Berlin, Heidelberg, pp 674–679Google Scholar
  196. Zhou Z, Chawla N, Jin Y, Williams G (2014) Big data opportunities and challenges: discussions from data analytics perspectives [discussion forum]. Comput Intell Mag IEEE 9(4):62–74CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Wien 2016

Authors and Affiliations

  • Hirak Kashyap
    • 1
  • Hasin Afzal Ahmed
    • 2
  • Nazrul Hoque
    • 2
  • Swarup Roy
    • 3
  • Dhruba Kumar Bhattacharyya
    • 2
  1. 1.Department of Computer Science, Donald Bren School of Information and Computer SciencesUniversity of California IrvineIrvineUSA
  2. 2.Department of Computer Science and EngineeringTezpur UniversityTezpurIndia
  3. 3.Department of Information TechnologyNorth Eastern Hill UniversityShillongIndia

Personalised recommendations