A Systematic Approach to Selecting Maintenance Policies in a Data Warehouse Environment

  • Henrik Engström
  • Sharma Chakravarthy
  • Brian Lings
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2287)

Abstract

Most work on data warehousing addresses aspects related to the internal operation of a data warehouse server, such as selection of views to materialise, maintenance of aggregate views and performance of OLAP queries. Issues related to data warehouse maintenance, i.e. how changes to autonomous sources should be detected and propagated to a warehouse, have been addressed in a fragmented manner. Although data propagation policies, source database capabilities, and user requirements have been addressed individually, their co-dependencies and relationships have not been explored. In this paper, we present a comprehensive framework for evaluating data propagation policies against data warehouse requirements and source capabilities. We formalize data warehouse specification along the dimensions of staleness, response time, storage, and computation cost, and classify source databases according to their data propagation capabilities. A detailed cost-model is presented for a representative set of policies. A prototype tool has been developed to allow an exploration of the various trade-offs.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    M.E. Adiba, B.G. Lindsay: Database Snapshots. VLDB (1980)Google Scholar
  2. 2.
    D. Agrawal, A.E. Abbadi, A.K. Singh, T. Yurek: Efficient View Maintenance at Data Warehouses. SIGMOD Conf. (1997)Google Scholar
  3. 3.
    J.A. Blakeley, P.Å. Larson, F.W. Tompa: Efficiently Updating Materialized Views. SIGMOD Conf. (1986)Google Scholar
  4. 4.
    P. Buneman, E.K. Clemons: Efficient Monitoring Relational Databases. ACM Transactions on Database Systems 4(3) (1979)Google Scholar
  5. 5.
    C. Chaudhuri, U. Dayal: An Overview of DataWarehousing and OLAP Technology. SIGMOD Record 26(1) (1997)Google Scholar
  6. 6.
    L.S. Colby, T. Griffn, L. Libkin, I.S. Mumick, H. Trickey: Algorithms for Deferred View Maintenance. SIGMOD Conf. (1996)Google Scholar
  7. 7.
    L.S. Colby, A. Kawaguchi, D.F. Lieuwen, I.S. Mumick, K.A. Ross: Supporting Multiple View Maintenance Policies. SIGMOD Conf. (1997)Google Scholar
  8. 8.
    A. Delis, N. Roussopoulos: Management of Updates in the Enhanced Client-Server DBMS. IEEE-ICDCS (1994)Google Scholar
  9. 9.
    H. Engström, S. Chakravarthy, B. Lings: A Holistic Approach to the Evaluation of Data Warehouse Maintenance Policies. Technical report HS-IDA-TR-00-001, University of Skövde, Sweden (2000)Google Scholar
  10. 10.
    H. Engström, S. Chakravarthy, B. Lings: A User-centric View of Data Warehouse Maintenance Issues. BNCOD (2000)Google Scholar
  11. 11.
    H. Engström, G. Gelati, L. Lings: A Benchmark Comparison of Maintenance Policies in a DataWarehouse Environment. Technical Report HS-IDA-TR-01-005, University of Skövde, Sweden (2001)Google Scholar
  12. 12.
    A. Gupta, I.S. Mumick: Maintenance of Materialized Views: Problems, Techniques, and Applications. IEEE Data Engineering Bulletin 18(2) (1995)Google Scholar
  13. 13.
    H. Gupta, I.S. Mumick: Selection of Views to Materialize Under a Maintenance Cost Constraint. International Conf. on Database Theory (1999)Google Scholar
  14. 14.
    J. Hammer, H. Garcia-Molina, J. Widom, W. Labio, Y. Zhuge: The Stanford Data Warehousing Project. IEEE Data Engineering Bulletin 18(2) (1995)Google Scholar
  15. 15.
    E.N. Hanson: A Performance Analysis of View Materialization Strategies. SIGMOD Conf. (1987)Google Scholar
  16. 16.
    V. Harinarayan, A. Rajaraman, J.D. Ullman: Implementing Data Cubes Efficiently. SIGMOD Conf. (1996)Google Scholar
  17. 17.
    R. Hull, G. Zhou: A Framework for Supporting Data Integration Using the Materialized and Virtual Approaches. SIGMOD Conf. (1996)Google Scholar
  18. 18.
    R. Hull, G. Zhou: Towards the Study of Performance Trade-offs Between Materialized and Virtual Integrated Views. VIEWS’96 (1996)Google Scholar
  19. 19.
    A. Koschel, P.C. Lockemann: Distributed events in active database systems: Letting the genie out of the bottle. DKE 25(1–2) (1998)Google Scholar
  20. 20.
    M. Jarke, Y. Vassiliou: Data Warehouse Quality Design: A Review of the DWQ Project. Conf. on Information Quality, Cambridge (1997)Google Scholar
  21. 21.
    M. Lee, J. Hammer: Speeding Up Warehouse Physical Design Using A Randomized Algorithm. DMDW (1999)Google Scholar
  22. 22.
    B. Lindsay, L. Haas, C. Mohan, H. Pirahesh, P. Wilms: A Snapshot Differential Refresh Algorithm. SIGMOD Conf. (1986)Google Scholar
  23. 23.
    D. Lomet (editor), J. Widom (editor): Special Issue on Materialized Views and Data Warehousing. IEEE Data Engineering Bulletin 18(2) (1995)Google Scholar
  24. 24.
    D. Quass, J. Widom: On-Line Warehouse View Maintenance. SIGMOD Conf. (1997)Google Scholar
  25. 25.
    N. Roussopoulos, H. Kang: Principles and Techniques in the Design of ADMS±. IEEE Computer 19(12) (1986)Google Scholar
  26. 26.
    R. Roussopoulos, C.M. Chen, S. Kelley, A. Delis, Y. Papakonstantinou: The ADMS Project: Views “R” Us. IEEE Data Engineering Bulletin 18(2) (1995)Google Scholar
  27. 27.
    A. Segev, J. Park: Updating Distributed Materialized Views. TKDE 1(2) (1989)Google Scholar
  28. 28.
    A. Segev, W. Fang: Currency-Based Updates to Distributed Materialized Views. ICDE (1990)Google Scholar
  29. 29.
    A. Segev, W. Fang: Optimal Update Policies for Distributed Materialized Views. Management Science 37(7) (1991)Google Scholar
  30. 30.
    J. Srivastava, D. Rotem: Analytical Modeling of Materialized View Maintenance. PODS (1988)Google Scholar
  31. 31.
    D. Theodoratos, M. Bouzeghoub: Data Currency Quality Factors in Data Warehouse Design. DMDW (1999)Google Scholar
  32. 32.
    A. Vavouras, S. Gatziu, K.R. Dittrich: The SIRIUS Approach for Refreshing Data Warehouses Incrementally. BTW’99 (1999)Google Scholar
  33. 33.
    J. Widom: Research Problems in Data Warehousing. CIKM (1995)Google Scholar
  34. 34.
    M.C. Wu, A.P. Buchmann: Research Issues in Data Warehousing. BTW’97 (1997)Google Scholar
  35. 35.
    G. Zhou, R. Hull, R. King, J.C. Franchitti: Data Integration and Warehousing Using H2O. IEEE Data Engineering Bulletin 18(2) (1995)Google Scholar
  36. 36.
    Y Zhuge, H. Garcia-Molina, J. Hammer, J. Widom: View Maintenance in a Warehousing Environment. SIGMOD Conf. (1995)Google Scholar
  37. 37.
    Y. Zhuge, H. Garcia-Molina, J.L. Wiener: The Strobe Algorithms for Multi-Source Warehouse Consistency. PDIS (1996)Google Scholar
  38. 38.
    Y. Zhuge: Incremental Maintenance of Consistent Data Warehouses. PhD Thesis, Stanford University (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Henrik Engström
    • 1
  • Sharma Chakravarthy
    • 2
  • Brian Lings
    • 3
  1. 1.Department of computer scienceUniversity of SkövdeSweden
  2. 2.Computer Science and Engineering DepartmentUniversity of Texas at ArlingtonArlington
  3. 3.Department of Computer ScienceUniversity of ExeterUK

Personalised recommendations