Comprehensive Log Compression with Frequent Patterns
- Cite this paper as:
- Hätönen K., Boulicaut J.F., Klemettinen M., Miettinen M., Masson C. (2003) Comprehensive Log Compression with Frequent Patterns. In: Kambayashi Y., Mohania M., Wöß W. (eds) Data Warehousing and Knowledge Discovery. DaWaK 2003. Lecture Notes in Computer Science, vol 2737. Springer, Berlin, Heidelberg
In this paper we present a comprehensive log compression (CLC) method that uses frequent patterns and their condensed representations to identify repetitive information from large log files generated by communications networks. We also show how the identified information can be used to separate and filter out frequently occurring events that hide other, unique or only a few times occurring events. The identification can be done without any prior knowledge about the domain or the events. For example, no pre-defined patterns or value combinations are needed. This separation makes it easier for a human observer to perceive and analyse large amounts of log data. The applicability of the CLC method is demonstrated with real-world examples from data communication networks.
Unable to display preview. Download preview PDF.