Evaluation of the Suitability of Intel Xeon Phi Clusters for the Simulation of Ultrasound Wave Propagation Using Pseudospectral Methods

  • Filip VaverkaEmail author
  • Bradley E. Treeby
  • Jiri Jaros
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11538)


The ability to perform large-scale ultrasound simulations using Fourier pseudospectral methods has generated significant interest in medical ultrasonics, including for treatment planning in therapeutic ultrasound and image reconstruction in photoacoustic tomography. However, the routine execution of such simulations is computationally very challenging. Nowadays, the trend in parallel computing is towards the use of accelerated clusters where computationally intensive parts are offloaded from processors to accelerators. During last five years, Intel has released two generations of Xeon Phi accelerators. The goal of this paper is to investigate the performance on both architectures with respect to current processors, and evaluate the suitability of accelerated clusters for the distributed simulation of ultrasound propagation using Fourier-based methods. The paper reveals that the former version of Xeon Phis, the Knight’s Corner architecture, suffers from several flaws that reduce the performance far below the Haswell processors. On the other hand, the second generation called Knight’s Landing shows very promising performance comparable with current processors.


Ultrasound simulations Pseudospectral methods k-Wave toolbox Intel Xeon Phi KNC KNL MPI OpenMP Performance evaluation Scaling 



This work was supported by The Ministry of Education, Youth and Sports from the National Programme of Sustainability (NPU II) project IT4Innovations excellence in science - LQ1602 and by the IT4Innovations infrastructure which is supported from the Large Infrastructures for Research, Experimental Development and Innovations project IT4Innovations National Supercomputing Center - LM2015070. This project has received funding from the European Union’s Horizon 2020 research and innovation programme H2020 ICT 2016–2017 under grant agreement No 732411 and is an initiative of the Photonics Public Private Partnership. This work was also supported by the Engineering and Physical Sciences Research Council, UK, grant numbers EP/L020262/1 and EP/P008860/1.


  1. 1.
    Andrews, L.C.: Special Functions of Mathematics for Engineers. SPIE Pub. (1997)Google Scholar
  2. 2.
    Beard, P.: Biomedical photoacoustic imaging. Interface Focus 1(4), 602–631 (2011). Scholar
  3. 3.
    Boyd, J.P.: A comparison of numerical algorithms for fourier extension of the first, second, and third kinds. J. Comput. Phys. 178(1), 118–160 (2002). Scholar
  4. 4.
    Boyd, J.P.: Asymptotic fourier coefficients for a C\(\infty \) bell (Smoothed-“Top-Hat”) & the fourier extension problem. J. Sci. Comput. 29(1), 1–24 (2006). Scholar
  5. 5.
    Dubinsky, T.J., Cuevas, C., Dighe, M.K., Kolokythas, O., Joo, H.H.: High-intensity focused ultrasound: current potential and oncologic applications. Am. J. Roentgenol. 190(1), 191–199 (2008)CrossRefGoogle Scholar
  6. 6.
    Frigo, M., Johnson, S.G.: The design and implementation of FFTW3. Proc. IEEE 93(2), 216–231 (2005)CrossRefGoogle Scholar
  7. 7.
    Gholami, A., Hill, J., Malhotra, D., Biros, G.: AccFFT: a library for distributed-memory FFT on CPU and GPU architectures, May 2016.
  8. 8.
    Gu, J., Jing, Y.: Modeling of wave propagation for medical ultrasound: a review. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 62(11), 1979–1992 (2015). Scholar
  9. 9.
    Intel Corporation: Math Kernel Library 11.3 Developer Reference. Intel Corporation (2015)Google Scholar
  10. 10.
    Israeli, M., Vozovoi, L., Averbuch, A.: Spectral multidomain technique with local Fourier basis. J. Sci. Comput. 8(2), 135–149 (1993)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Jaros, J., Rendell, A.P., Treeby, B.E.: Full-wave nonlinear ultrasound simulation on distributed clusters with applications in high-intensity focused ultrasound. J. High Perform. Comput. Appl. 30(2), 137–155 (2016)CrossRefGoogle Scholar
  12. 12.
    Jaros, J., Vaverka, F., Treeby, B.E.: Spectral domain decomposition using local fourier basis: application to ultrasound simulation on a cluster of GPUs. Supercomput. Frontiers Innov. 3(3), 40–55 (2016)Google Scholar
  13. 13.
    Jeffers, J., Reinders, J.: Intel Xeon Phi Coprocessor High Performance Programming, vol. 1. Elsevier Inc., Waltham (2013)Google Scholar
  14. 14.
    Meairs, S., Alonso, A.: Ultrasound, microbubbles and the blood-brain barrier. Progress Biophys. Mol. Biol. 93(1–3), 354–362 (2007)CrossRefGoogle Scholar
  15. 15.
    Nandapalan, N., Jaros, J., Treeby, E.B., AlistairRendell, P.: Implementation of 3D FFTs across multiple GPUs in shared memory environments. In: Proceedings of the Thirteenth International Conference on Parallel and Distributed Computing, Applications and Technologies, pp. 167–172 (2012).
  16. 16.
    Pekurovsky, D.: P3DFFT: a framework for parallel computations of Fourier transforms in three dimensions (2012). Scholar
  17. 17.
    Pinton, G.F., Dahl, J., Rosenzweig, S., Trahey, G.E.: A heterogeneous nonlinear attenuating full-wave model of ultrasound. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 56(3), 474–488 (2009)CrossRefGoogle Scholar
  18. 18.
    Pippig, M.: PFFT: an extension of FFTW to massively parallel architectures. SIAM J. Sci. Comput. 35(3), 213–236 (2013)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Sorensen, H., Jones, D., Heideman, M., Burrus, C.: Real-valued fast Fourier transform algorithms. IEEE Trans. Acoust. Speech Signal Process. 35(6), 849–863 (1987). Scholar
  20. 20.
    Tabei, M., Mast, T.D., Waag, R.C.: A k-space method for coupled first-order acoustic propagation equations. J. Acoust. Soc. Am. 111(1 Pt 1), 53–63 (2002). Scholar
  21. 21.
    Treeby, B.E., Jaros, J., Rendell, A.P., Cox, B.T.: Modeling nonlinear ultrasound propagation in heterogeneous media with power law absorption using a k-space pseudospectral method. J. Acoust. Soc. Am. 131(6), 4324–4336 (2012). Scholar
  22. 22.
    Treeby, B.E., Vaverka, F., Jaros, J.: Performance and accuracy analysis of nonlinear k-wave simulations using local domain decomposition with an 8-GPU server. Proc. Meet. Acoust. 34(1), 022002 (2018)Google Scholar
  23. 23.
    Tufail, Y., Yoshihiro, A., Pati, S., Li, M.M., Tyler, W.J.: Ultrasonic neuromodulation by brain stimulation with transcranial ultrasound. Nat. Protoc. 6(9), 1453–1470 (2011). Scholar
  24. 24.
    Wang, E., et al.: High-Performance Computing on the Intel® Xeon Phi™. Springer, Cham (2014). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Faculty of Information Technology, Centre of Excellence IT4InnovationsBrno University of TechnologyBrnoCzech Republic
  2. 2.Medical Physics and Biomedical Engineering, Biomedical Ultrasound GroupUniversity College LondonLondonUK

Personalised recommendations