The VLDB Journal

, Volume 22, Issue 2, pp 203–228 | Cite as

Auditing a database under retention policies

Regular Paper


Auditing the changes to a database is critical for identifying malicious behavior, maintaining data quality, and improving system performance. But an accurate audit log is an historical record of the past that can also pose a serious threat to privacy. Policies that limit data retention conflict with the goal of accurate auditing, and data owners have to carefully balance the need for policy compliance with the goal of accurate auditing. In this paper, we provide a framework for auditing the changes to a database system while respecting data retention policies. Our framework includes an historical data model that supports flexible audit queries, along with a language for retention policies that can hide individual attribute values or remove entire tuples from the history. Under retention policies, the audit history is partially incomplete. Thus, audit queries on the protected history can include imprecise results. We propose two different models (a tuple-independent model and a tuple-correlated model) for formalizing the meaning of audit queries. We implement policy application and query answering efficiently in a standard relational system and characterize the cases where accurate auditing can be achieved under retention restrictions.


Privacy Auditing Retention policy 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Supplementary material

778_2012_282_MOESM1_ESM.tex (0 kb)
ESM (TEX 1 kb)
778_2012_282_MOESM2_ESM.tex (8 kb)
ESM (TEX 9 kb)
778_2012_282_MOESM3_ESM.tex (7 kb)
ESM (TEX 8 kb)
778_2012_282_MOESM4_ESM.eps (425 kb)
ESM (EPS 425 kb)


  1. 1.
    Antova, L., Jansen, T., Koch, C., Olteanu, D.: Fast and simple relational processing of uncertain data. In: ICDE, pp. 983–992 (2008)Google Scholar
  2. 2.
    ARMA Internaltional: Generally Accepted Recordkeeping Principles.
  3. 3.
    Ataullah, A., Aboulnaga, A., Tompa, F.: Records retention in relational database systems. In: Proceeding of the ACM Conference on Information and Knowledge Management (CIKM), pp. 873–882 (2008)Google Scholar
  4. 4.
    Bertino, E., Bettini, C., Samarati, P.: A temporal authorization model. In: ACM Conference on Computer and Communications Security (CCS), pp. 126–135. ACM Press, New York (1994)Google Scholar
  5. 5.
    Biskup J.: A foundation of codd’s relational maybe-operations. ACM Trans. Database Syst. 8, 608–636 (1983)MathSciNetMATHCrossRefGoogle Scholar
  6. 6.
    Blakeley J., Coburn N., Larson P.: Updating derived relations: detecting irrelevant and autonomously computable updates. TODS 14(3), 369–400 (1989)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Blakeley J.A., Larson P.A., Tompa F.W.: Efficiently updating materialized views. SIGMOD Rec. 15(2), 61–71 (1986)CrossRefGoogle Scholar
  8. 8.
    Chomicki, J.: Temporal query languages: a survey. In: Temporal Logic (ICTL’94), vol. 827, pp. 506–534 (1994)Google Scholar
  9. 9.
    EMC Corporation:
  10. 10.
    Fabbri, D., LeFevre, K., Zhu, Q.: PolicyReplay: misconfiguration-response queries for data breach reporting. In: Proceedings of the VLDB Endowment, vol. 3, no. (1–2), pp. 36–47 (2010)Google Scholar
  11. 11.
    Gadia S.K.: A homogeneous relational model and query languages for temporal databases. ACM Trans. Database Syst. 13, 418–448 (1988)MathSciNetMATHCrossRefGoogle Scholar
  12. 12.
    Gadia, S.K., Nair, S.S., Poon, Y.C.: Incomplete information in relational temporal databases. In: 18th VLDB Conference (1992)Google Scholar
  13. 13.
    Garcia-Molina, H., Labio, W., Yang, J.: Expiring data in a warehouse. In: VLDB Conference, pp. 500–511 (1998)Google Scholar
  14. 14.
    Grahne G.: The Problem of Incomplete Information in Relational Databases. Springer, Berlin (1991)MATHCrossRefGoogle Scholar
  15. 15.
  16. 16.
    Guo S., Sun W., Weiss M.: Solving satisfiability and implication problems in database systems. ACM Trans. Database Syst. 21(2), 270–293 (1996)CrossRefGoogle Scholar
  17. 17.
    Hasan, R., Winslett, M.: Trustworthy vacuuming and litigation holds in long-term high-integrity records retention. In: Proceedings of the 13th International Conference on Extending Database Technology, pp. 621–632. ACM (2010)Google Scholar
  18. 18.
    Hasan, R., Winslett, M., Mitra, S.: Efficient Audit-based Compliance for Relational Data Retention. UIUC Dept. of CS Tech Report UIUCDCS-R-2009-3044 (2009)Google Scholar
  19. 19.
    Hochbaum D., Moreno-Centeno E.: The inequality-satisfiability problem. Oper. Res. Lett. 36(2), 229–233 (2008)MathSciNetMATHCrossRefGoogle Scholar
  20. 20.
    Imielinski T., Lipski W.: Incomplete information in relational databases. J. ACM 31(4), 761–791 (1984)MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Jensen C.S., Mark L.: Queries on change in an extended relational model. IEEE TKDE 4, 192–200 (1992)Google Scholar
  22. 22.
    Jensen C.S., Mark L., Roussopoulos N.: Incremental implementation model for relational databases with transaction time. IEEE Trans. Knowl. Data Eng. 3, 461–473 (1991)CrossRefGoogle Scholar
  23. 23.
    Koubarakis M.: Database models for infinite and indefinite temporal information. Inf. Syst. 19, 141 (1994)CrossRefGoogle Scholar
  24. 24.
    Lageweg B., Lenstra J., Kan A.: Minimizing maximum lateness on one machine: computational experience and some applications. Stat. Neerl. 30(1), 25–41 (1976)MATHCrossRefGoogle Scholar
  25. 25.
    LeFevre, K., Agrawal, R., Ercegovac, V., Ramakrishnan, R., Xu, Y., DeWitt, D.: Limiting disclosure in hippocratic databases. In: VLDB Conference, pp. 108–119 (2004)Google Scholar
  26. 26.
    LexisNexis: Document Retention & Destruction Policies for Digital Data.
  27. 27.
    Lomet, D.B., Barga, R.S., Mokbel, M.F., Shegalov, G., Wang, R. Zhu, Y.: Transaction time support inside a database engine. In: ICDE, p. 35 (2006)Google Scholar
  28. 28.
    Lu, W., Miklau, G.: AuditGuard: a system for database auditing under retention restrictions. IN: Proceedings of the VLDB Endowment vol. 1, no. 2, pp. 1484–1487 (2008)Google Scholar
  29. 29.
    Lu, W., Miklau, G.: Auditing a database under retention restrictions. In: IEEE International Conference on Data Engineering (ICDE), pp. 42–53 (2009)Google Scholar
  30. 30.
    Mullins, C.S.: Database Archiving for Long-term Data Retention. (2006)
  31. 31.
    OpenText Corporation:
  32. 32.
    Perez, R.A., Moreau, L.: Securing provenance-based audits. In: International Provenance and Annotation Workshop 2010. Springer, Berlin (2010)Google Scholar
  33. 33.
    RainStor Inc.:
  34. 34.
    Rosenkrantz, D.J., Hunt, H.B.: Processing conjunctive predicates and queries. In: VLDB Conference, p. 72 (1980)Google Scholar
  35. 35.
    SAND Technology:
  36. 36.
    Sarda N.L.: Extensions to sql for historical databases. IEEE Trans. Knowl. Data Eng. 2, 220–230 (1990)CrossRefGoogle Scholar
  37. 37.
    Sarma, A., Benjelloun, O., Halevy, A., Widom, J.: Working models for uncertain data. In: ICDE (2006)Google Scholar
  38. 38.
    Schneier B., Kelsey J.: Secure audit logs to support computer forensics. ACM Trans. Inf. Syst. Secur. 2(2), 159–176 (1999)CrossRefGoogle Scholar
  39. 39.
    Shaull, R., Shrira, L., Xu, H.: Skippy: a new snapshot indexing method for time travel in the storage manager. In: ACM SIGMOD Conference, pp. 637–648 (2008)Google Scholar
  40. 40.
    Simons, B., Sipser, M.: On scheduling unit-length jobs with multiple release time/deadline intervals. Oper. Res. 80–88 (1984)Google Scholar
  41. 41.
    Skyt J., Jensen C., Mark L.: A foundation for vacuuming temporal databases. Data Knowl. Eng. 44(1), 1–29 (2003)MATHCrossRefGoogle Scholar
  42. 42.
    Snodgrass, R., Yao, S., Collberg, C.: Tamper detection in audit logs. In: 13th VLDB Conference, pp. 504–515 (2004)Google Scholar
  43. 43.
    Snodgrass R.T.: The TSQL2 Temporal Query Language. Kluwer Academic Publishers, Norwell (1995)MATHCrossRefGoogle Scholar
  44. 44.
    Snodgrass, R.T.: Developing time-oriented database applications in SQL. Morgan Kaufmann Publishers Inc., San Francisco (1999)Google Scholar
  45. 45.
    Snodgrass, R.T., Collberg, C.S.: The τ-BerkeleyDB Temporal Subsystem. Published: Available at
  46. 46.
    Stahlberg, P., Miklau, G., Levine, B.N.: Threats to privacy in the forensic analysis of database systems. In: SIGMOD Conference, pp. 91–102 (2007)Google Scholar
  47. 47.
    Toman, D.: Expiration of historical databases. In: Symposium on Temporal Representation and Reasoning (TIME), pp. 128–135 (2001)Google Scholar
  48. 48.
    Wang, Q., Yu, T., Li, N., Lobo, J., Bertino, E., Irwin, K., Byun, J.W.: On the correctness criteria of fine-grained access control in relational databases. In: VLDB Conference, pp. 555–566 (2007)Google Scholar
  49. 49.
    Waters, B., Balfanz, D., Durfee, G., Smetters, D.: Building an encrypted and searchable audit log. In: NDSS, vol. 6 (2004)Google Scholar
  50. 50.
  51. 51.
    ZL Technologies, Inc.:
  52. 52.

Copyright information

© Springer-Verlag 2012

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of MassachusettsAmherstUSA

Personalised recommendations