Compresso: Efficient Compression of Segmentation Data for Connectomics

  • Brian Matejek
  • Daniel Haehn
  • Fritz Lekschas
  • Michael Mitzenmacher
  • Hanspeter Pfister
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10433)

Abstract

Recent advances in segmentation methods for connectomics and biomedical imaging produce very large datasets with labels that assign object classes to image pixels. The resulting label volumes are bigger than the raw image data and need compression for efficient storage and transfer. General-purpose compression methods are less effective because the label data consists of large low-frequency regions with structured boundaries unlike natural image data. We present Compresso, a new compression scheme for label data that outperforms existing approaches by using a sliding window to exploit redundancy across border regions in 2D and 3D. We compare our method to existing compression schemes and provide a detailed evaluation on eleven biomedical and image segmentation datasets. Our method provides a factor of 600–2200x compression for label volumes, with running times suitable for practice.

Keywords

Compression Encoding Segmentation Connectomics 

References

  1. 1.
    Aimar, L., Merritt, L., Petit, E., et al.: x264-a free h264/avc encoder (2005)Google Scholar
  2. 2.
    Collet, T.: Smaller and faster data compression with zstandard (2016). https://code.facebook.com/posts/1658392934479273/smaller-and-faster-data-compression-with-zstandard/. Accessed 23 Oct 2016
  3. 3.
    Deutsch, P., Gailly, J.L.: Zlib compressed data format specification version 3.3. Technical report (1996)Google Scholar
  4. 4.
    Everingham, M., Van Gool, L., Williams, C.K.I., et al.: The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html
  5. 5.
    Fredman, M., Saks, M.: The cell probe complexity of dynamic data structures. In: Proceedings of the Twenty-first Annual ACM Symposium on Theory of Computing, pp. 345–354. ACM (1989)Google Scholar
  6. 6.
    Google: Brotli compression format (2016). https://github.com/google/brotli. Accessed 11 Oct 2016
  7. 7.
  8. 8.
    Haehn, D., Knowles-Barley, S., Roberts, M., et al.: Design and evaluation of interactive proofreading tools for connectomics. IEEE Trans. Vis. Comput. Graph. 20(12), 2466–2475 (2014). Proceedings of IEEE SciVis 2014Google Scholar
  9. 9.
    He, L., Chao, Y., Suzuki, K., Wu, K.: Fast connected-component labeling. Pattern Recogn. 42(9), 1977–1987 (2009)CrossRefMATHGoogle Scholar
  10. 10.
    Kasthuri, N., Hayworth, K.J., Berger, D.R., et al.: Saturated reconstruction of a volume of neocortex. Cell 162(3), 648–661 (2015)CrossRefGoogle Scholar
  11. 11.
    Lehmann, M.: Liblzf (2016). http://oldhome.schmorp.de/marc/liblzf.html. Accessed 13 Oct 2016
  12. 12.
    Liu, T., Jones, C., Seyedhosseini, M., Tasdizen, T.: A modular hierarchical approach to 3d electron microscopy image segmentation. J. Neurosci. Meth. 226, 88–102 (2014)CrossRefGoogle Scholar
  13. 13.
    Martin, D., Fowlkes, C., Tal, D., et al.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of 8th International Conference on CV, vol. 2, pp. 416–423, July 2001Google Scholar
  14. 14.
    Nunez-Iglesias, J., Kennedy, R., Plaza, S.M., et al.: Graph-based active learning of agglomeration (GALA): a python library to segment 2d and 3d neuroimages. Front. Neuroinform. 8, 34 (2014)CrossRefGoogle Scholar
  15. 15.
    Oberhumer, M.: LZO real-time data compression library. User manual for LZO version 0.28 (2005). http://www.infosys.tuwien.ac.at/Staff/lux/marco/lzo.html. Accessed Feb 1997
  16. 16.
    Pavlov, I.: LZMA SDK (software development kit) (2007)Google Scholar
  17. 17.
    Roelofs, G., Koman, R.: PNG: The Definitive Guide. O’Reilly, Inc., Sebastopol (1999)Google Scholar
  18. 18.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). doi:10.1007/978-3-319-24574-4_28 CrossRefGoogle Scholar
  19. 19.
    Seward, J.: bzip. 2 (1998)Google Scholar
  20. 20.
    Skodras, A., Christopoulos, C., Ebrahimi, T.: The JPEG 2000 still image compression standard. IEEE Sig. Process. Mag. 18(5), 36–58 (2001)CrossRefMATHGoogle Scholar
  21. 21.
    Suissa-Peleg, A., Haehn, D., Knowles-Barley, S., et al.: Automatic neural reconstruction from petavoxel of electron microscopy data. Microsc. Microanal. 22(S3), 536–537 (2016)Google Scholar
  22. 22.
    Vandevenne, A.: Zopfli compression algorithm (2016). https://github.com/google/zopfli. Accessed 11 Oct 2016
  23. 23.
    Welch, T.A.: A technique for high-performance data compression. Computer 17(6), 8–19 (1984)CrossRefGoogle Scholar
  24. 24.
    Ziv, J., Lempel, A.: Compression of individual sequences via variable-rate coding. IEEE Trans. Inf. Theor. 24(5), 530–536 (1978)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Brian Matejek
    • 1
  • Daniel Haehn
    • 1
  • Fritz Lekschas
    • 1
  • Michael Mitzenmacher
    • 1
  • Hanspeter Pfister
    • 1
  1. 1.Harvard UniversityCambridgeUSA

Personalised recommendations