Compressing Semistructured Text Databases
We describe a compression model for semistructured documents, called Structural Contexts Model, which takes advantage of the context information usually implicit in the structure of the text. The idea is to use a separate semiadaptive model to compress the text that lies inside each different structure type (e.g., different XML tag). The intuition behind the idea is that the distribution of all the texts that belong to a given structure type should be similar, and different from that of other structure types. We test our idea using a word-based Huffman coding, which is the standard for compressing large natural language textual databases, and show that our compression method obtains significant improvements in compression ratios. We also analyze the possibility that storing separate models may not pay of if the distribution of different structure types is not different enough, and present a heuristic to merge models with the aim of minimizing the total size of the compressed database. This technique gives an additional improvement over the plain technique. The comparison against existing prototypes shows that our method is a competitive choice for compressed text databases.
KeywordsText Compression Compression Model Semistructured Documents Text Databases
Unable to display preview. Download preview PDF.
- J. Cheney. Compressing XML with multiplexed hierarchical PPM models. In Proc. Data Compression Conference (DCC 2001), pages 163–, 2001.Google Scholar
- D. Harman. Overviewof the Third Text REtrieval Conference. In Proc. Third Text REtrieval Conference (TREC-3), pages 1–19, 1995. NIST Special Publication 500-207.Google Scholar
- H. S. Heaps. Information Retrieval — Computational and Theoretical Aspects. Academic Press, 1978.Google Scholar
- D. A. Huffman. A method for the construction of minimum-redundancy codes. Proc. Inst. Radio Engineers, 40(9):1098–1101, 1952.Google Scholar
- H. Liefke and D. Suciu. XMill: an efficient compressor for XML data. In Proc. ACM SIGMOD 2000, pages 153–164, 2000.Google Scholar
- Ian H. Witten, Timothy C. Bell, John G. Cleary. Text Compression. Prentice Hall, Englewood Cliffs, N. J., 1990.Google Scholar
- P. Tolani and J. R. Haritsa. XGRIND: A query-friendly XML compressor. In ICDE, 2002. http://citeseer.nj.nec.com/503319.html.
- I. H. Witten, A. Moffat, and T. C. Bell. Managing Gigabytes. Morgan Kaufmann Publishers, Inc., second edition, 1999.Google Scholar
- N. Ziviani, E. Moura, G. Navarro, and R. Baeza-Yates. Compression: A key for next-generation text retrieval systems. IEEE Computer, 33(11):37–44, November 2000.Google Scholar