The LOFAR Central Processing Facility Architecture
Reconfiguration is a key feature characteristic of the LOFAR telescope. Software platforms are utilised to program out the required data transformations in the generation of scientific end-products. Reconfigurable resources nowadays often replace the hard-wired processing systems from the past. This paper describes how this paradigm is implemented in a purely general-purpose telescope back-end. Experiences from high performance computing, stream processing and software engineering have been combined, leading to a state-of-the-art processing platform. The processing platform offers a total processing power of 35 TFlops, which is used to process a sustained input data-stream of 320 Gbps. The architecture of this platform is optimised for streaming data processing and offers appropriate processing resources for each step in the data processing chains. Typical data processing chains include Fourier transformations and correlation tasks along with controlling tasks such as fringe rotation correction. These tasks are defined in a high level programming language and mapped onto the available resources at run time. A scheduling system is used to control a collection of concurrently executing observations, providing each associated application with the appropriate resources to meet its timing constraint and give the integrated system the correct on-line and off-line look and feel.
Keywordscluster correlator high performance computing HPC parallel processing pipeline streaming data supercomputer
Unable to display preview. Download preview PDF.
- Adiga, N.R., et al.: 2002, An Overview of the BlueGene/L Supercomputer, Proceedings of SC2002, http://sc-2002.org/paperpdfs/pap.pap207.pdf.Google Scholar
- Clos, C.: 1953, Bell Syst. Tech. J. 32, 406.Google Scholar
- de Vos, C.M., Schaaf, K.v.d. and Bregman, J.D.: 2001, Cluster Computers and Grid Processing in the First Radio-Telescope of a New Generation, Proceedings of IEEE CCGrid.Google Scholar
- Hellwagner, H. and Reinefeld, A.: 1999, Sci: Scalable Coherent Interface: Architecture and Software for High-Performance Compute Clusters, Springer-Verlag, ISBN 3540666966.Google Scholar
- Leiserson, C.E.: 1985, October, Fat-Trees: IEEE Trans. Comput. 34, 892.Google Scholar
- Silva, L.M.E. and Buyya, R.: 1999, Parallel Programing Models and Paradigms, chapter 1 of High Performance Cluster Computing, Vol. 2, R. Buyya, Prentice Hall PTR 1999, ISBN 0130137855.Google Scholar
- Snir, M.: 1998, MPI: The Complete Reference, MIT press, ISBN 0262692163.Google Scholar
- Schaaf, K.v.d., Bregman, J.D. and de Vos, C.M.: 2003a, Hybrid Cluster Computing Hardware and Software in the LOFAR Radio Telescope, Proceedings of PDPTA, CSREA Press.Google Scholar
- Schaaf, K.v.d., Bregman, J.D. and de Vos, C.M.: 2003b, Hybrid Cluster Computing Hardware and Software in the LOFARRadio Telescope, Proceedings of IEEE PDPTA.Google Scholar