Experimental Astronomy

, Volume 36, Issue 1–2, pp 59–76 | Cite as

An AIPS-based, distributed processing method for large radio interferometric datasets

  • Stephen BourkeEmail author
  • Huib Jan van Langevelde
  • Karl Torstensson
  • Aaron Golden
Original Article


The data output rates of modern radio interferometric telescopes make the traditional data reduction process impractical in many cases. We report on the implementation of a lightweight infrastructure, named AIPSLite, that enables the deployment of AIPS interferometric processing routines on distributed systems in an autonomous and fault tolerant manner. We discuss how this approach was used to search for sources of 6.7 GHz methanol maser emission in the Cep A region with the European VLBI Network (EVN). The field was searched out to a radius of 1.25 min−1 at milli-arcsecond spatial resolution and 1024 frequency channels with 0.088 km s−1 velocity resolution. The imaged data was on the order of 30 TB. Processing was performed on 128 processors of the Irish Centre for High End Computing (ICHEC) linux cluster with a run time of 42 h, and a total of 212 CPU days.


Interferometry data processing Distributed processing AIPS ParselTongue AIPSLite 



George Heald is thanked for his useful comments on the manuscript. Salvador Curiel is thanked for his continuum image of Cep A. S.B. acknowledges support by Enterprise Ireland, Science Foundation Ireland, and the Higher Education Authority. K.T. acknowledges support by the EU Framework 6 Marie Curie Early Stage Training programme under contract number MEST-CT-2005-19669 “ESTRELA”. This effort is supported by the European Community Framework Programme 7, Advanced Radio Astronomy in Europe, grant agreement no.: 227290. ParselTongue was developed in the context of the ALBUS project, which has benefited from research funding from the European Community’s sixth Framework Programme under RadioNet R113CT 2003 5058187. The authors wish to acknowledge the SFI/HEA Irish Centre for High-End Computing (ICHEC) for the provision of computational facilities and support.


  1. 1.
    Bartkiewicz, A., Szymczak, M., Cohen, R.J., Richards, A.M.S.: Magnetic field in Cepheus A as deduced from OH maser polarimetric observations. Mon. Not. RAS 361, 623–632 (2005). doi: 10.1111/j.1365-2966.2005.09203.x ADSCrossRefGoogle Scholar
  2. 2.
    Beswick, R., Garrington, S.T., Gonzalez-Solarez, E., Harrison, P.A., Kettenis, M., Muxlow, T.W.B., Richards, A.M.S., van Langevelde, H.J., Walton, N.A., Winstanley, N.: On-demand radio imaging and access to calibrated data for all astronomers. In: Proceedings of the 8th European VLBI Network Symposium (2006)Google Scholar
  3. 3.
    Campbell, B.: Field of view calculations for the EVN correlator at JIVE. DOI: (2006)
  4. 4.
    Clark, B.G.: An efficient implementation of the algorithm ‘CLEAN’. Astron. Astrophys. 89, 377 (1980)ADSGoogle Scholar
  5. 5.
    Cornwell, T.J.: Ska and Evla computing costs for wide field imaging. Exp. Astron. 17, 329–343 (2004). doi: 10.1007/s10686-005-7449-z ADSCrossRefGoogle Scholar
  6. 6.
    Cornwell, T.J., Golap, K., Bhatnagar, S.: W Projection: a new algorithm for wide field imaging with radio synthesis arrays. In: Shopbell, P., Britton, M., Ebert, R. (eds.) Astronomical Data Analysis Software and Systems XIV. Astronomical Society of the Pacific Conference Series, vol. 347, p. 86 (2005)Google Scholar
  7. 7.
    Deller, A.T., Tingay, S.J., , Bailes, M., West, C.: DiFX: a software correlator for very long baseline interferometry using multiprocessor computing environments. Publ. ASP 119, 318–336 (2007). doi: 10.1086/513572 ADSGoogle Scholar
  8. 8.
    Deller, A.T., Tingay, S.J., Brisken, W.: Precision southern hemisphere pulsar VLBI astrometry: techniques and results for PSR J1559-4438. Astrophys. J. 690, 198–209 (2009). doi: 10.1088/0004-637X/690/1/198 ADSCrossRefGoogle Scholar
  9. 9.
    Guzman, J.C., Humphreys, B.: The Australian SKA Pathfinder (ASKAP) software architecture. In: Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series. Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, vol. 7740 (2010). doi: 10.1117/12.856962
  10. 10.
    Kettenis, M., van Langevelde, H.J., Reynolds, C., Cotton, B.: ParselTongue: AIPS Talking Python. In: Gabriel, C., Arviset, C., Ponz, D., Enrique, S. (eds.) Astronomical Data Analysis Software and Systems XV. Astronomical Society of the Pacific Conference Series, vol. 351, p. 497 (2006)Google Scholar
  11. 11.
    Kogan, L., Greisen, E.: Faceted imaging in AIPS. In: AIPS Memo 113. AIPS Memo Series, vol. 113 (2009)Google Scholar
  12. 12.
    Minier, V., Conway, J.E., Booth, R.S.: VLBI observations of 6.7 and 12.2 GHz methanol masers toward high mass star-forming regions. II. Tracing massive protostars. Astron. Astrophys. 369, 278–290 (2001). doi: 10.1051/0004-6361:20010124 ADSCrossRefGoogle Scholar
  13. 13.
    Perley, R.A.: Wide Field Imaging II: imaging with non-coplanar arrays. In: Perley, R.A., Schwab, F.R., Bridle, A.H. (eds.) Synthesis Imaging in Radio Astronomy. Astronomical Society of the Pacific Conference Series, vol. 6, p. 259 (1989)Google Scholar
  14. 14.
    Roy, J., Gupta, Y., Pen, U.-L., Peterson, J.B., Kudale, S., Kodilkar, J.: A real-time software backend for the GMRT. Exp. Astron. 28, 25–60 (2010). doi: 10.1007/s10686-010-9187-0 ADSCrossRefGoogle Scholar
  15. 15.
    Schilizzi, R.T., Aldrich, W., Anderson, B., Bos, A., Campbell, R.M., Canaris, J., Cappallo, R., Casse, J.L., Cattani, A., Goodman, J., van Langevelde, H.J., Maccafferri, A., Millenaar, R., Noble, R.G., Olnon, F., Parsley, S.M., Phillips, C., Pogrebenko, S.V., Smythe, D., Szomoru, A., Verkouter, H., Whitney, A.R.: The EVN-MarkIV VLBI data processor. Exp. Astron. 12, 49–67 (2001)ADSCrossRefGoogle Scholar
  16. 16.
    Schwab, F.R.: Relaxing the isoplanatism assumption in self-calibration; applications to low-frequency radio interferometry. Astron. J. 89, 1076–1081 (1984). doi: 10.1086/113605 ADSCrossRefGoogle Scholar
  17. 17.
    Strom, R.: What is the primary beam response of an interferometer with unequal elements? In: Bachiller, R., Colomer, F., Desmurs, J.-F., de Vicente, P. (eds.) European VLBI Network on New Developments in VLBI Science and Technology, pp. 273–274 (2004)Google Scholar
  18. 18.
    Taylor, G.B., Carilli, C.L., Perley, R.A. (eds.): Synthesis Imaging in Radio Astronomy II. Astronomical Society of the Pacific Conference Series, vol. 180 (1999)Google Scholar
  19. 19.
    Thompson, A.R., Moran, J.M., Swenson, G.W.: Interferometry and Synthesis in Radio Astronomy. Wiley (1986)Google Scholar
  20. 20.
    Torstensson, K.J.E., van Langevelde, H.J., Vlemmings, W.H.T., Bourke, S.: Dynamics of the 6.7 and 12.2 GHz methanol masers around Cepheus A HW2. Astron. Astrophys. 526, A38 (2011). doi: 10.1051/0004-6361/201015583 ADSCrossRefGoogle Scholar
  21. 21.
    Vlemmings, W.H.T., Diamond, P.J., van Langevelde, H.J., Torrelles, J.M.: The magnetic field in the star-forming region Cepheus A. From H2O maser polarization observations. Astron. Astrophys. 448, 597–611 (2006). doi: 10.1051/0004-6361:20054275 ADSCrossRefGoogle Scholar
  22. 22.
    Walker, R.C.: The multiple-point fringe-rate method of mapping spectral-line VLBI sources with application to H2O masers in W3-IRS5 and W3/OH/. Astrophys. J. 86, 1323–1331 (1981). doi: 10.1086/113013 ADSGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2013

Authors and Affiliations

  • Stephen Bourke
    • 1
    • 2
    Email author
  • Huib Jan van Langevelde
    • 1
  • Karl Torstensson
    • 1
    • 3
  • Aaron Golden
    • 2
  1. 1.Joint Institute for VLBI in EuropeDwingelooThe Netherlands
  2. 2.Centre for AstronomyNational University of IrelandGalwayIreland
  3. 3.Leiden ObservatoryLeiden UniversityLeidenThe Netherlands

Personalised recommendations