Skip to main content
Log in

Tailored data compression using stream partitioning and prediction: application to Gaia

  • Original article
  • Published:
Experimental Astronomy Aims and scope Submit manuscript


Gaia is the most ambitious space astrometry mission currently envisaged and it will be a technological challenge in all its aspects. Here we describe a proposal for the data compression system of Gaia, specifically designed for this mission but based on concepts that can be applied to other missions and systems as well. Realistic simulations have been performed with our Telemetry CODEC software, which performs a stream partitioning and pre-compression to the science data. In this way, standard compressors such as bzip2 or szip boost their performance and decrease their processing requirements when applied to such pre-processed data. These simulations have shown that a lossless compression factor of 3 can be achieved, whereas standard compression systems were unable to reach a factor of 2.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others


  1. Perryman, M.A.C., de Boer, K.S., Gilmore, G., Hoeg, E., Lattanzi, M.G., Lindegren, L., Luri, X., Mignard, F., Pace, O., de Zeeuw, P.T.: A&A, 369, 339 (2001)

    Article  ADS  Google Scholar 

  2. Portell, J., Luri, X., García-Berro, E.: IEEE Transactions on Aerospace and Electronic Systems 42, 2 (2006)

    Article  Google Scholar 

  3. Høg, E., de Bruijne, J.: Gaia sampling scheme (EHs4), Gaia internal document (GAIA-CA-TN-NBI-EH-176-2) (2006)

  4. Lammers, U.: Gaia telemetry rate simulations: A results update, Gaia internal document (GAIA-CH-TN-ESAC-UL-014-2) (2006)

  5. Portell, J., García-Berro, E., Luri, X.: Flux data codification: Proposals of simple codification schemes, Gaia internal document (GAIA-BCN-001) (2001)

  6. Gibson, J.D.: Adaptive prediction in speech differential encoding systems. Proc. IEEE 68 (1980)

  7. Langdon, G.G., Mealy, B.: On prediction error coding methods for lossless image compression. IEEE, in Conference Record of the Thirty-First Asilomar Conference on Signals. Syst. Comput. 2, 1442 (1997)

    Google Scholar 

  8. van Assche, S., de Rycke, D., Philips, W., Lemahieu, I.: Exploiting interframe redundancies in the lossless compression of 3D medical images. IEEE, Data Compression Conference, Proc. 575 (2000)

  9. Salomon, D.: Data Compression, the complete reference, Springer (2004)

  10. Masana, E., Luri, X., Anglada-Escudé, G., Llimona, P.: In: Proceedings of the Symposium “The Three-Dimensional Universe with Gaia”, ESA-SP-576 (2004)

  11. Babusiaux, C.: In: Proceedings of the Symposium “The Three-Dimensional Universe with Gaia”, ESA-SP-576 (2004)

  12. Masana, E., Fabricius, C., Luri, X., Torra, J., Figueras, F., Jordi, C., García-Berro, E., Portell, J.: GDAAS Telemetry Model, Gaia internal document (UB-GDAAS2-TN-003) (2004)

  13. Pace, O.: Clarification on the Probability Frame Loss (PFL) in data transmission to ground, Gaia internal document (SAG-OP-001) (1998)

  14. Geijo, E.M.: Modelo de canal y sistema de control de errores para SIXE (in Spanish), Master Thesis, UPC (2002)

  15. Portell, J., García-Berro, E., Luri, X., Castro, R., Prudhomme, M.: In: Highlights of Spanish Astrophysics IV, Springer Verlag (2007)

Download references


This work has been partially supported by the Spanish MEC grant AYA05-08013-C03-01, by the European Union FEDER funds and by the AGAUR.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Jordi Portell.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Portell, J., García-Berro, E., Luri, X. et al. Tailored data compression using stream partitioning and prediction: application to Gaia. Exp Astron 21, 125–149 (2006).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: