Skip to main content
SpringerLink
Log in

Change Discovery in Heterogeneous Data Sources of a Data Warehouse

  • Conference paper
  • First Online:
Databases and Information Systems (DB&IS 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1243))

Included in the following conference series:

Abstract

Data warehouses have been used to analyze data stored in relational databases for several decades. However, over time, data that are employed in the decision-making process have become so enormous and heterogeneous that traditional data warehousing solutions have become unusable. Therefore, new big data technologies have emerged to deal with large volumes of data. The problem of structural evolution of integrated heterogeneous data sources has become extremely topical due to dynamic and diverse nature of big data. In this paper, we propose an approach to change discovery in data sources of a data warehouse utilized to analyze big data. Our solution incorporates an architecture that allows to perform OLAP operations and other kinds of analysis on integrated big data and is able to detect changes in schemata and other characteristics of structured, semi-structured and unstructured data sources. We discuss the algorithm for change discovery and metadata necessary for its operation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Kaisler, S., Armour, F., Espinosa, J.A., Money, W.: Big data: issues and challenges moving forward. In: 46th International Conference on System Sciences, pp. 995–1004 (2013)

    Google Scholar 

  2. Cuzzocrea, A., Bellatreche, L., Song, I.: Data warehousing and OLAP over big data: current challenges and future research directions. In: 16th International Workshop on Data Warehousing and OLAP, pp. 67–70 (2013)

    Google Scholar 

  3. Holubová, I., Klettke, M., Störl, U.: Evolution management of multi-model data. In: Gadepally, V., et al. (eds.) DMAH/Poly -2019. LNCS, vol. 11721, pp. 139–153. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33752-0_10

    Chapter  Google Scholar 

  4. Solodovnikova, D., Niedrite, L.: Handling evolution in big data architectures. Balt. J. Mod. Comput. 8(1), 21–47 (2020)

    Google Scholar 

  5. Nadal, S., Romero, O., Abelló, A., Vassiliadis, P., Vansummeren, S.: An integration-oriented ontology to govern evolution in big data ecosystems. In: Workshops of the EDBT/ICDT 2017 Joint Conference (2017)

    Google Scholar 

  6. Chen, S.: Cheetah: a high performance, custom data warehouse on top of MapReduce. VLDB Endow. 3(2), 1459–1468 (2010)

    Article  Google Scholar 

  7. Kimball, R., Ross, M.: The Data Warehouse Toolkit: The Definitive Guide to Dimensional Modeling, 3rd edn. Wiley, Indiana (2013)

    Google Scholar 

  8. Sumbaly, R., Kreps, J., Shah, S.: The “big data” ecosystem at LinkedIn. In: ACM SIGMOD International Conference on Management of Data, pp. 1125–1134 (2013)

    Google Scholar 

  9. Solodovnikova, D., Niedrite, L., Niedritis, A.: On metadata support for integrating evolving heterogeneous data sources. In: Welzer, T., et al. (eds.) ADBIS 2019. CCIS, vol. 1064, pp. 378–390. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30278-8_38

    Chapter  Google Scholar 

  10. Quix, C., Hai, R., Vatov, I.: Metadata extraction and management in data lakes with GEMMS. Complex Syst. Inform. Model. Q. 9, 67–83 (2016)

    Article  Google Scholar 

  11. Solodovnikova, D., Niedrite, L.: Towards a data warehouse architecture for managing big data evolution. In: Proceedings of the 7th International Conference on Data Science, Technology and Applications (DATA 2018), Porto, Portugal, pp. 63–70 (2018)

    Google Scholar 

Download references

Acknowledgments

This work has been supported by the European Regional Development Fund (ERDF) project No. 1.1.1.2./VIAA/1/16/057.

Author information

Authors and Affiliations

  1. Faculty of Computing, University of Latvia, Riga, 1050, Latvia

    Darja Solodovnikova & Laila Niedrite

Authors
  1. Darja Solodovnikova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laila Niedrite
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Darja Solodovnikova .

Editor information

Editors and Affiliations

  1. Department of Computer Systems, Tallinn University of Technology, Tallinn, Estonia

    Tarmo Robal

  2. Department of Software Science, Tallinn University of Technology, Tallinn, Estonia

    Hele-Mai Haav

  3. Department of Software Science, Tallinn University of Technology, Tallinn, Estonia

    Jaan Penjam

  4. Institute of Computer Science, University of Tartu, Tartu, Estonia

    Raimundas Matulevičius

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Solodovnikova, D., Niedrite, L. (2020). Change Discovery in Heterogeneous Data Sources of a Data Warehouse. In: Robal, T., Haav, HM., Penjam, J., Matulevičius, R. (eds) Databases and Information Systems. DB&IS 2020. Communications in Computer and Information Science, vol 1243. Springer, Cham. https://doi.org/10.1007/978-3-030-57672-1_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-57672-1_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-57671-4

  • Online ISBN: 978-3-030-57672-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation