Policy-Regulated Management of ETL Evolution

  • George Papastefanatos
  • Panos Vassiliadis
  • Alkis Simitsis
  • Yannis Vassiliou
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5530)


In this paper, we discuss the problem of performing impact prediction for changes that occur in the schema/structure of the data warehouse sources. We abstract Extract-Transform-Load (ETL) activities as queries and sequences of views. ETL activities and its sources are uniformly modeled as a graph that is annotated with policies for the management of evolution events. Given a change at an element of the graph, our method detects the parts of the graph that are affected by this change and highlights the way they are tuned to respond to it. For many cases of ETL source evolution, we present rules so that both syntactical and semantic correctness of activities are retained. Finally, we experiment with the evaluation of our approach over real-world ETL workflows used in the Greek public sector.


Data Warehouses ETL Evolution Impact of changes 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Banerjee, J., Kim, W., Kim, H.J., Korth, H.F.: Semantics and implementation of schema evolution in object-oriented databases. In: Proc. ACM Special Interest Group on Management of Data, pp. 311–322 (1987)Google Scholar
  2. 2.
    Bellahsene, Z.: Schema evolution in data warehouses. Knowledge and Information Systems 4(3), 283–304 (2002)CrossRefGoogle Scholar
  3. 3.
    Bernstein, P., Levy, A., Pottinger, R.: A Vision for Management of Complex Models. SIGMOD Record 29(4), 55–63 (2000)CrossRefGoogle Scholar
  4. 4.
    Bernstein, P., Rahm, E.: Data warehouse scenarios for model management. In: Laender, A.H.F., Liddle, S.W., Storey, V.C. (eds.) ER 2000. LNCS, vol. 1920, pp. 1–15. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  5. 5.
    Blaschka, M., Sapia, C., Höfling, G.: On Schema Evolution in Multidimensional Databases. In: Mohania, M., Tjoa, A.M. (eds.) DaWaK 1999. LNCS, vol. 1676, pp. 153–164. Springer, Heidelberg (1999)Google Scholar
  6. 6.
    Bouzeghoub, M., Kedad, Z.: A logical model for data warehouse design and evolution. In: Kambayashi, Y., Mohania, M., Tjoa, A.M. (eds.) DaWaK 2000. LNCS, vol. 1874, pp. 178–188. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  7. 7.
    Fan, H., Poulovassilis, A.: Schema Evolution in Data Warehousing Environments – A Schema Transformation-Based Approach. In: Atzeni, P., Chu, W., Lu, H., Zhou, S., Ling, T.-W. (eds.) ER 2004. LNCS, vol. 3288, pp. 639–653. Springer, Heidelberg (2004)Google Scholar
  8. 8.
    Gupta, A., Mumick, I.S., Rao, J., Ross, K.A.: Adapting materialized views after redefinitions: Techniques and a performance study. Information Systems J. 26(5), 323–362 (2001)zbMATHCrossRefGoogle Scholar
  9. 9.
    Golfarelli, M., Lechtenbörger, J., Rizzi, S., Vossen, G.: Schema Versioning in Data Warehouses. In: Atzeni, P., Chu, W., Lu, H., Zhou, S., Ling, T.-W. (eds.) ER 2004. LNCS, vol. 3288, pp. 415–428. Springer, Heidelberg (2004)Google Scholar
  10. 10.
    Kaas, C., Pedersen, T.B., Rasmussen, B.: Schema Evolution for Stars and Snowflakes. In: Sixth Int’l. Conference on Enterprise Information Systems (ICEIS 2004), pp. 425–433 (2004)Google Scholar
  11. 11.
    Liu, C.T., Chrysanthis, P.K., Chang, S.K.: Database schema evolution through the specification and maintenance of changes on entities and relationships. In: Loucopoulos, P. (ed.) ER 1994. LNCS, vol. 881, pp. 132–151. Springer, Heidelberg (1994)Google Scholar
  12. 12.
    Mohania, M., Dong, D.: Algorithms for adapting materialized views in data warehouses. In: Proc. International Symposium on Cooperative Database Systems for Advanced Applications (CODAS 1996), pp. 309–316 (1996)Google Scholar
  13. 13.
    Nica, A., Lee, A.J., Rundensteiner, E.A.: The CSV algorithm for view synchronization in evolvable large-scale information systems. In: Schek, H.-J., Saltor, F., Ramos, I., Alonso, G. (eds.) EDBT 1998. LNCS, vol. 1377, pp. 359–373. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  14. 14.
    Papastefanatos, G., Vassiliadis, P., Vassiliou, Y.: Adaptive Query Formulation to Handle Database Evolution. In: Proc. Forum of the Eighteenth Conference on Advanced Information Systems Engineering (CAISE 2006) (2006)Google Scholar
  15. 15.
    Papastefanatos, G., Vassiliadis, P., Simitsis, A., Vassiliou, Y.: What-if analysis for data warehouse evolution. In: Song, I.-Y., Eder, J., Nguyen, T.M. (eds.) DaWaK 2007. LNCS, vol. 4654, pp. 23–33. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  16. 16.
    Papastefanatos, G., Kyzirakos, K., Vassiliadis, P., Vassiliou, Y.: Hecataeus: A Framework for Representing SQL Constructs as Graphs. In: Proc. Tenth International Workshop on Exploring Modeling Methods in Systems Analysis and Design (held with CAISE) (2005)Google Scholar
  17. 17.
    Papastefanatos, G., Anagnostou, F., Vassiliadis, P., Vassiliou, Y.: Hecataeus: A What-If Analysis Tool for Database Schema Evolution. In: Proc. Twelfth European Conference on Software Maintenance and Reengineering (CSMR 2008) (2008)Google Scholar
  18. 18.
    Papastefanatos, G., Vassiliadis, P., Simitsis, A., Aggistalis, K., Pechlivani, F., Vassiliou, Y.: Language Extensions for the Automation of Database Schema Evolution. In: 10th International Conference on Enterprise Information Systems (ICEIS 2008) (2008)Google Scholar
  19. 19.
    Ra, Y.G., Rundensteiner, E.A.: A transparent object-oriented schema change approach using view evolution. In: Proc. Eleventh International Conference on Data Engineering (ICDE 1995), pp. 165–172 (1995)Google Scholar
  20. 20.
    Roddick, J.F., et al.: Evolution and Change in Data Management - Issues and Directions. SIGMOD Record 29(1), 21–25 (2000)CrossRefGoogle Scholar
  21. 21.
    Roddick, J.F.: A survey of schema versioning Issues for database systems. Information Software Technology J. 37(7) (1995)Google Scholar
  22. 22.
    Simitsis, A., Vassiliadis, P., Terrovitis, M., Skiadopoulos, S.: Graph-based modeling of ETL activities with multi-level transformations and updates. In: Tjoa, A.M., Trujillo, J. (eds.) DaWaK 2005. LNCS, vol. 3589, pp. 43–52. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  23. 23.
    Tsichritzis, D., Klug, A.C.: The ANSI/X3/SPARC DBMS Framework Report of the Study Group on Database Management Systems. Information Systems 3(3), 173–191 (1978)CrossRefGoogle Scholar
  24. 24.
    Velegrakis, Y., Miller, R.J., Popa, L.: Preserving mapping consistency under schema changes. VLDB J. 13(3), 274–293 (2004)CrossRefGoogle Scholar
  25. 25.
    Zicari, R.: A framework for schema update in an object-oriented database system. In: Proc. Seventh International Conference on Data Engineering (ICDE 1991), pp. 2–13 (1991)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • George Papastefanatos
    • 1
  • Panos Vassiliadis
    • 2
  • Alkis Simitsis
    • 3
  • Yannis Vassiliou
    • 1
  1. 1.National Technical University of AthensGreece
  2. 2.University of IoanninaGreece
  3. 3.Stanford UniversityUSA

Personalised recommendations