Proceedings of 2nd International Conference on Intelligent Computing and Applications pp 571-581 | Cite as
A Survey on Challenges in Integrating Big Data
Abstract
The Big Data is a Buzzword, which is being generated from various sources in and around in our daily life. Big data is the conjunction of big transactional data i.e. relational data base system, users activities huge data e.g. face book, twitter, LinkedIn, web logs, scanned, sensor devices, mails, and big data processing. The four striking characteristics of Big Data are volume, variety, velocity and veracity. Big data analytics refers to the process of gathering, arranging and analyzing huge data set to uncover the hidden knowledge that enables us to take effective and efficient decision making. The source data mostly may contain heterogeneity, noise, outliers, missing values and inconsistency. The poor source data can produce poor quality of analytical results. Traditional data processing system does not resolve these problems. The proposed data integration frame work with NoSQL technology could resolve integration, transformation, inconsistencies, noise challenges in big data.
Keywords
Big data Pre-processing NoSQLReferences
- 1.Wang, M., Nie, T., Shen, D., Kou, Y., Yu, G.: Intelligent similarity joins for big data integration. In: 10th Web Information System and Application Conference (2013)Google Scholar
- 2.Bansal, S.K.: Towards a semantic extract-transform-load (ETL) framework for big data integration. In: IEEE International Congress on Big Data (2014)Google Scholar
- 3.Han, J., Haihong, E., Le, G.: Survey on NoSQL Database, 978-1-4577-0208-2/11/$26.00 ©2011 IEEEGoogle Scholar
- 4.Zhang, D., Hsu, D.F., Wang, Y., Rao, A.R., Zhang, D., Kinsner, W., Pedrycz, W., Berwick, R.C., Zadeh, L.A. (eds.): Inconsistencies in Big Data. 978-1-4799-0783-0/13/$31.00 ©2013 IEEEGoogle Scholar
- 5.Dharmasiri, H.M.L., Goonetillake, M.D.J.S.: A federated approach on heterogeneous NoSQL data stores. In: International Conference on Advances in ICT for Emerging Regions (ICTer), pp. 234–239 (2013)Google Scholar
- 6.Zhao, G., Lin, Q., Li, L., Li, Z.: Schema conversion model of SQL database to NoSQL. In: Ninth International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (2014)Google Scholar
- 7.Kadadi, A., Agrawal, R., Nyamful, C., Atiq, R.: Challenges of data integration and interoperability in big data. In: IEEE International Conference on Big Data (2014)Google Scholar
- 8.Nimmagadda, S.L., Dreher, H.V.: Big-data integration methodologies for effective management and data mining of petroleum digital ecosystems. 978-1-4799-0786-1/13/$31.00 ©2013 IEEE 150Google Scholar
- 9.Hong, X., Rong, C.M.: Cloud Data Integration Sharing and Service. 978-1-4799-3351-8/14/$31.00 ©2014 IEEEGoogle Scholar
- 10.Kaur, K., Rani, R.: Modeling and Querying Data in NoSQL Databases. 978-1-4799-1293-3/13/$31.00 ©2013 IEEEGoogle Scholar
- 11.Gopala Krishnan, S.: Integration of Big Data Technologies into Enterprise Landscape. Co-Chairman, Infosys limited, Bangalore, Big data Spectrum (2012)Google Scholar