Experimental Astronomy

, Volume 21, Issue 3, pp 125–149 | Cite as

Tailored data compression using stream partitioning and prediction: application to Gaia

  • Jordi Portell
  • Enrique García-Berro
  • Xavier Luri
  • Alberto G. Villafranca
Original article


Gaia is the most ambitious space astrometry mission currently envisaged and it will be a technological challenge in all its aspects. Here we describe a proposal for the data compression system of Gaia, specifically designed for this mission but based on concepts that can be applied to other missions and systems as well. Realistic simulations have been performed with our Telemetry CODEC software, which performs a stream partitioning and pre-compression to the science data. In this way, standard compressors such as bzip2 or szip boost their performance and decrease their processing requirements when applied to such pre-processed data. These simulations have shown that a lossless compression factor of 3 can be achieved, whereas standard compression systems were unable to reach a factor of 2.


Instrumentation: miscellaneous Methods: data analysis Techniques: miscellaneous Astrometry 


07.87.+v 95.40.+s 95.55.Pe 95.10.Jk 95.80.+p 07.05.Hd 



This work has been partially supported by the Spanish MEC grant AYA05-08013-C03-01, by the European Union FEDER funds and by the AGAUR.


  1. 1.
    Perryman, M.A.C., de Boer, K.S., Gilmore, G., Hoeg, E., Lattanzi, M.G., Lindegren, L., Luri, X., Mignard, F., Pace, O., de Zeeuw, P.T.: A&A, 369, 339 (2001)CrossRefADSGoogle Scholar
  2. 2.
    Portell, J., Luri, X., García-Berro, E.: IEEE Transactions on Aerospace and Electronic Systems 42, 2 (2006)CrossRefGoogle Scholar
  3. 3.
    Høg, E., de Bruijne, J.: Gaia sampling scheme (EHs4), Gaia internal document (GAIA-CA-TN-NBI-EH-176-2) (2006)Google Scholar
  4. 4.
    Lammers, U.: Gaia telemetry rate simulations: A results update, Gaia internal document (GAIA-CH-TN-ESAC-UL-014-2) (2006)Google Scholar
  5. 5.
    Portell, J., García-Berro, E., Luri, X.: Flux data codification: Proposals of simple codification schemes, Gaia internal document (GAIA-BCN-001) (2001)Google Scholar
  6. 6.
    Gibson, J.D.: Adaptive prediction in speech differential encoding systems. Proc. IEEE 68 (1980)Google Scholar
  7. 7.
    Langdon, G.G., Mealy, B.: On prediction error coding methods for lossless image compression. IEEE, in Conference Record of the Thirty-First Asilomar Conference on Signals. Syst. Comput. 2, 1442 (1997)Google Scholar
  8. 8.
    van Assche, S., de Rycke, D., Philips, W., Lemahieu, I.: Exploiting interframe redundancies in the lossless compression of 3D medical images. IEEE, Data Compression Conference, Proc. 575 (2000)Google Scholar
  9. 9.
    Salomon, D.: Data Compression, the complete reference, Springer (2004)Google Scholar
  10. 10.
    Masana, E., Luri, X., Anglada-Escudé, G., Llimona, P.: In: Proceedings of the Symposium “The Three-Dimensional Universe with Gaia”, ESA-SP-576 (2004)Google Scholar
  11. 11.
    Babusiaux, C.: In: Proceedings of the Symposium “The Three-Dimensional Universe with Gaia”, ESA-SP-576 (2004)Google Scholar
  12. 12.
    Masana, E., Fabricius, C., Luri, X., Torra, J., Figueras, F., Jordi, C., García-Berro, E., Portell, J.: GDAAS Telemetry Model, Gaia internal document (UB-GDAAS2-TN-003) (2004)Google Scholar
  13. 13.
    Pace, O.: Clarification on the Probability Frame Loss (PFL) in data transmission to ground, Gaia internal document (SAG-OP-001) (1998)Google Scholar
  14. 14.
    Geijo, E.M.: Modelo de canal y sistema de control de errores para SIXE (in Spanish), Master Thesis, UPC (2002)Google Scholar
  15. 15.
    Portell, J., García-Berro, E., Luri, X., Castro, R., Prudhomme, M.: In: Highlights of Spanish Astrophysics IV, Springer Verlag (2007)Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2007

Authors and Affiliations

  • Jordi Portell
    • 1
    • 3
  • Enrique García-Berro
    • 2
    • 3
  • Xavier Luri
    • 1
    • 3
  • Alberto G. Villafranca
    • 2
    • 3
  1. 1.Departament d’Astronomia i Meteorologia, Facultat de FísicaUniversitat de BarcelonaBarcelonaSpain
  2. 2.Departament de Física Aplicada, Escola Politècnica Superior de CastelldefelsUniversitat Politècnica de CatalunyaCastelldefelsSpain
  3. 3.Institut d’Estudis Espacials de Catalunya (IEEC)BarcelonaSpain

Personalised recommendations