Move-to-Front, Distance Coding, and Inversion Frequencies Revisited
Move-to-Front, Distance Coding and Inversion Frequencies are three somewhat related techniques used to process the output of the Burrows-Wheeler Transform. In this paper we analyze these techniques from the point of view of how effective they are in the task of compressing low-entropy strings, that is, strings which have many regularities and are therefore highly compressible. This is a non-trivial task since many compressors have non-constant overheads that become non-negligible when the input string is highly compressible.
Because of the properties of the Burrows-Wheeler transform, being locally optimal ensures an algorithm compresses low-entropy strings effectively. Informally, local optimality implies that an algorithm is able to effectively compress an arbitrary partition of the input string. We show that in their original formulation neither Move-to-Front, nor Distance Coding, nor Inversion Frequencies is locally optimal. Then, we describe simple variants of the above algorithms which are locally optimal. To achieve local optimality with Move-to-Front it suffices to combine it with Run Length Encoding. To achieve local optimality with Distance Coding and Inversion Frequencies we use a novel “escape and re-enter” strategy. Since we build on previous results, our analyses are simple and shed new light on the inner workings of the three techniques considered in this paper.
KeywordsLocal Optimality Compression Algorithm Input String Arithmetic Code Alphabet Size
Unable to display preview. Download preview PDF.
- 1.Arnavut, Z., Magliveras, S.: Block sorting and compression. In: Procs of IEEE Data Compression Conference (DCC), pp. 181–190 (1997)Google Scholar
- 3.Binder, E.: Distance coder, Usenet group (2000) comp.compression Google Scholar
- 4.Burrows, M., Wheeler, D.: A block-sorting lossless data compression algorithm. Technical Report 124, Digital Equipment Corporation (1994)Google Scholar
- 10.Foschini, L., Grossi, R., Gupta, A., Vitter, J.: Fast compression with a static model in high-order entropy. In: Procs of IEEE Data Compression Conference (DCC), pp. 62–71 (2004)Google Scholar
- 15.Navarro, G., Mäkinen, V.: Compressed full-text indexes. ACM Computing Surveys (to Appear)Google Scholar
- 16.Ryabko, B.Y.: Data compression by means of a ’book stack’. Prob.Inf.Transm, 16(4) (1980)Google Scholar