Abstract
In heterogeneous data processing, various data model often make analytic task too hard to achieve optimal performance, it is necessary to unify heterogeneous data into the same data model. How to determine the proper intermediate data model and unify the involved heterogeneous data models for the analytical task is an urgent problem need to be solved. In this paper, we proposed a model determination method based on cost estimation. It evaluates the execution cost of query tasks on different data models, which taken as the criterion to measure the data model, and chooses a data model with the least cost as the intermediate representation during data processing. The experimental results of BigBench datasets showed that the proposed cost estimation based method could appropriately determine the data model, which made heterogeneous data processing efficiently.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Wang, L.: Heterogeneous data and big data analytics. Autom. Control Inf. Sci. 3(1), 8–15 (2017)
Xuexian, C., Huiting, J.: Research on heterogeneous data sources integration. Comput. Eng. Sci. 30(8) (2008)
Xiangjiang, K., Yupeng, M., Yingfan, L.: Data type’s conversion at heterogeneous database systems. Appl. Res. Comput. (1), 217–218 (2006)
Jingling, Y., Lili, X., Lianchao, M.: Research on virtual approach about heterogeneous data integration based on XML. Appl. Res. Comput. 26(1) (2009)
Weiwei, W., Qinghong, S.: Integrated platform of distributed heterogeneous data based on XML. J. SE Univ. 36(5) (2006)
Castro, E., Cuadra, D., Velasco, M.: From XML to relational model. INFORMATICA 21(4), 505–519 (2010)
Hui, H., Fu, S., Sui, L.: Data conversion model of heterogeneous database. Comput. Eng. Des. 26(9) (2005)
Derong, S., Ge, Y., Xite, W.: Survey on NoSQL for management of big data. J. Softw. 24(8) (2013)
Shengli, W., Nengbin, W.: Estimation of query cost in object-oriented database systems. Comput. Res. Dev. 35(1) (1998)
Xunbo, S., Xiangguang, Z.: Cost model of query plan in distribute database system. Comput. Syst. Appl. (10) (2007)
OrientDB Document. http://orientdb.com/docs/. Accessed 28 Jan 2018
Ghazal, A., Rabl, T., Hu, M.: BigBench: towards an industry standard benchmark for big data analytics. In: SIGMOD (2013)
Baru, C., Bhandarkar, M., Danisch, M., et al.: Discussion of BigBench: a proposed industry standard performance benchmark for big data. In: TPCTC (2014)
Chowdhury, B., Rabl, T., Saadatpanah, P., et al.: A BigBench implementation in the Hadoop ecosystem. In: WBDB 2013. LNCS, vol. 8585, pp. 3–18 (2014)
Petermann, A., Junghanns, M., Muller, R., et al.: Graph-based data integration and business intelligence with BIIIG. VLDB 7(13) (2014)
De Virgilio, R., Maccioni, A., Torlone, R., et al.: Converting relational to graph databases. In: GRADES (2013)
Acknowledgements
This work was supported by the Fund by The National Natural Science Foundation of China (Grant No. 61462012, No. 61562010, No. U1531246), Guizhou University Graduate Innovation Fund (Grant No. 2017078) and the Innovation Team of the Data Analysis and Cloud Service of Guizhou Province (Grant No. [2015]53).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Zhang, J., Li, H., Zhang, X., Chen, M., Dai, Z., Zhu, M. (2019). Determination of the Data Model for Heterogeneous Data Processing Based on Cost Estimation. In: Silhavy, R. (eds) Artificial Intelligence and Algorithms in Intelligent Systems. CSOC2018 2018. Advances in Intelligent Systems and Computing, vol 764. Springer, Cham. https://doi.org/10.1007/978-3-319-91189-2_37
Download citation
DOI: https://doi.org/10.1007/978-3-319-91189-2_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-91188-5
Online ISBN: 978-3-319-91189-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)