Abstract
Large-scale feature selection is one of the most important fields in the big data domain that can solve real data problems, such as bioinformatics, where it is necessary to process huge amount of data. The efficiency of existing feature selection algorithms significantly downgrades, if not totally inapplicable, when data size exceeds hundreds of gigabytes, because most feature selection algorithms are designed for centralized computing architecture. For that, distributed computing techniques, such as MapReduce can be applied to handle very large data. Our approach is to scale the existing method for feature selection, Kmeans clustering and Signal to Noise Ratio (SNR) combined with optimization technique as Binary Particle Swarm Optimization (BPSO). The proposed method is divided into two stages. In the first stage, we have used parallel Kmeans on MapReduce for clustering features, and then we have applied iterative MapReduce that implement parallel SNR ranking for each cluster. After, we have selected the top ranked feature from each cluster. The top scored features from each cluster are gathered and a new feature subset is generated. In the second stage, the new feature subset is used as input to the proposed BPSO based on MapReduce which provides an optimized feature subset. The proposed method is implemented in a distributed environment, and its efficiency is illustrated through analyzing practical problems such as biomarker discovery.
Chapter PDF
Similar content being viewed by others
Keywords
References
Jay, S., Hanlee, J.: Next-generation DNA sequencing. Nature Biotechnology 26(10), 1135–1145 (2008)
Yvan, S., Inaki, I., Larranaga, P.: A review of feature selection techniques in bioinformatics. Bioinformatics 23(19), 2507–2517 (2007)
Dean, J., Ghemawat, S.: MapReduce: Simplified Data Processing on Large Clusters. In: OSDI 2004: Sixth Symposium on Operating System Design and Implementation, Sponsored by USENIX, in Cooperation with ACM SIGOPS, pp. 137–150 (2004)
Barnali, S., Debahuti, M.: A Novel Feature Selection Algorithm using Particle Swarm Optimization for Cancer Microarray Data. Procedia Engineering 38, 27–31 (2012)
Azli, A., et al.: Distributed visual enhancement on surveillance video with Hadoop Mapreduce and performance evaluation in pseudo distributed mode. Australian Journal of Basic and Applied Sciences 8(9), 38 (2014)
Kourid, A.: Iterative MapReduce for Feature Selection. International Journal of Engineering Research & Technology 3(7) (2014)
White, T.: Hadoop the definitive guide. O’Reilly Media (2012)
Bekkerman, R., Bilenko, M., Langford, J.: Scaling up Machine learning. Cambridge University Press (2011)
Sean, O., et al.: Mahout in action. Manning Publications (2011)
Gaizhen, Y.: The Application of MapReduce in the Cloud Computing. In: Intelligence Information Processing and Trusted Computing (IPTC), pp. 154–156. IEEE (2011)
Yun, J., Guocheng, X., Na, C., Shan, C.: A New Gene Expression Profiles Classifying Approach Based on Neighborhood Rough Set and Probabilistic Neural Networks Ensembl. In: Lee, M., Hirose, A., Hou, Z.-G., Kil, R.M. (eds.) ICONIP 2013, Part II. LNCS, vol. 8227, pp. 484–489. Springer, Heidelberg (2013)
Sun, X., et al.: Selection of interdependent genes via dynamic relevance analysis for cancer diagnosis. Journal of Biomedical Informatics 46(2), 252–258 (2013)
Tung, C.W., et al.: Identification of Biomarkers for Esophageal Squamous Cell Carcinoma Using Feature Selection and Decision Tree Methods. The ScientificWorld Journal (2013)
Yang, S., Chung, H.C., et al.: Novel biomarker candidates for gastric cancer. Oncology Reports 19(3), 675–680 (2008)
Geetha Ramani, R., Gracia Jacob, S.: Benchmarking Classification Models for Cancer Prediction from Gene Expression Data: A Novel Approach and New Findings. Studies in Informatics and Control 22(2), 133–142 (2013)
Li, X., et al.: SSiCP: a new SVM based Recursive Feature Elimination Algorithm for Multiclass Cancer Classification. Bio-Medical Materials and Engineering 23, S1027–S1038 (2014)
Tuan, T.F., et al.: Putative tumor metastasis-associated genes in human gastric cancer. International Journal of Oncology 41(3), 1068–1084 (2012)
Fröhlich, C., et al.: Molecular Profiling of ADAM12 in Human Bladder Cancer. Clinical Cancer Research 12(24), 7359–7368 (2006)
Hacker, E., et al.: Reduced expression of IL-18 is a marker of ultraviolet radiation-induced melanomas. Int. J. Cancer 123(1), 227–231 (2008)
Mayer, M.: Breast Cancer Prognostic Biomarkers. Accelerating science (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 IFIP International Federation for Information Processing
About this paper
Cite this paper
Kourid, A., Batouche, M. (2015). Biomarker Discovery Based on Large-Scale Feature Selection and MapReduce. In: Amine, A., Bellatreche, L., Elberrichi, Z., Neuhold, E., Wrembel, R. (eds) Computer Science and Its Applications. CIIA 2015. IFIP Advances in Information and Communication Technology, vol 456. Springer, Cham. https://doi.org/10.1007/978-3-319-19578-0_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-19578-0_7
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-19577-3
Online ISBN: 978-3-319-19578-0
eBook Packages: Computer ScienceComputer Science (R0)