Compressing Genomic Sequence Fragments Using SlimGene
With the advent of next generation sequencing technologies, the cost of sequencing whole genomes is poised to go below $1000 per human individual in a few years. As more and more genomes are sequenced, analysis methods are undergoing rapid development, making it tempting to store sequencing data for long periods of time so that the data can be re-analyzed with the latest techniques. The challenging open research problems, huge influx of data, and rapidly improving analysis techniques have created the need to store and transfer very large volumes of data.
Compression can be achieved at many levels, including trace level (compressing image data), sequence level (compressing a genomic sequence), and fragment-level (compressing a set of short, redundant fragment reads, along with quality-values on the base-calls). We focus on fragment-level compression, which is the pressing need today.
Our paper makes two contributions, implemented in a tool, SlimGene. First, we introduce a set of domain specific loss-less compression schemes that achieve over 40× compression of fragments, outperforming bzip2 by over 6×. Including quality values, we show a 5× compression using less running time than bzip2. Second, given the discrepancy between the compression factor obtained with and without quality values, we initiate the study of using ‘lossy’ quality values. Specifically, we show that a lossy quality value quantization results in 14× compression but has minimal impact on downstream applications like SNP calling that use the quality values. Discrepancies between SNP calls made between the lossy and lossless versions of the data are limited to low coverage areas where even the SNP calls made by the lossless version are marginal.
KeywordsCompression Scheme Personal Health Record Lossy Compression Generation Sequencing Technology Illumina Genome Analyzer
Unable to display preview. Download preview PDF.
- 1.Bellamy, J.C.: Digital Telephony, vol. 3rd. Wiley, Chichester (2000)Google Scholar
- 3.The CASAVA software toolkit, http://www.illumina.com/pages.ilmn?ID=314
- 6.Dublin, M.: So Long, Data Depression (2009), http://www.genomeweb.com/informatics/so-long-data-depression
- 8.Helicos Biosciences, http://www.helicosbio.com/
- 10.The Illumina Genome Analyzer, http://www.illumina.com/sequencing/
- 15.Pacific BioSciences, http://www.pacificbiosciences.com/index.php
- 16.Roche 454 Sequencing, http://www.454.com/
- 17.The SAM/BAM format, http://samtools.sourceforge.net/SAM1.pdf
- 18.Sharp, A.J., Cheng, Z., Eichler, E.E.: Structural variation of the human genome. Annu. Rev. Genomics Hum. Genet. (June 2006)Google Scholar
- 19.Steven, E.A., Mccanne, S., Vetterli, E.: A Layered Dct Coder For Internet Video. In: Proceedings of the IEEE International Conference on Image Processing, pp. 13–16 (1996)Google Scholar
- 20.Ziv, J., Lempel, A.: Compression of Individual Sequences Via Variable-Rate Coding. IEEE Transactions on Information Theory (1978)Google Scholar