Deep Generative Models to Simulate 2D Patient-Specific Ultrasound Images in Real Time

Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1248)


We present a computational method for real-time, patient-specific simulation of 2D ultrasound (US) images. The method uses a large number of tracked ultrasound images to learn a function that maps position and orientation of the transducer to ultrasound images. This is a first step towards realistic patient-specific simulations that will enable improved training and retrospective examination of complex cases. Our models can simulate a 2D image in under 4 ms (well within real-time constraints), and produce simulated images that preserve the content (anatomical structures and artefacts) of real ultrasound images.


Ultrasound Simulation Deep learning 


  1. 1.
    Blum, T., Rieger, A., Navab, N., Friess, H., Martignoni, M.: A review of computer-based simulators for ultrasound training. Simul. Healthc. 8(2), 98–108 (2013)CrossRefGoogle Scholar
  2. 2.
    Ehricke, H.: SONOSim3D: a multimedia system for sonography simulation and education with an extensible case database. Eur, J, Ultrasound. 7, 225–230 (1998)CrossRefGoogle Scholar
  3. 3.
    Arkhurst, W.: Ein interaktiver Atlas für die Sonographie und Anatomie des Säuglingsgehirns. University Hamburg, Hamburg, Germany (2005). [PhD thesis]Google Scholar
  4. 4.
    Sclaverano, S., Chevreaua, G., Vadcardc, L., Mozerb, P., BiopSym, T.J.: A simulator for enhanced learning of ultrasound-guided prostate biopsy. Stud. Health Technol. Inform. 142, 301–306 (2009)Google Scholar
  5. 5.
    Aiger, D., Cohen-Or, D.: Real-time ultrasound imaging simulation. In: Real-Time Imaging, vol. 4, pp. 263–274 (1998)Google Scholar
  6. 6.
    Jensen, J.: Field: a program for simulating ultrasound systems. In: Nordicbaltic Conf Biomed Imaging, pp. 351–353 (1996)Google Scholar
  7. 7.
    Shams, R., Hartley, R., Navab, N.: Real-time simulation of medical ultrasound from CT images. MICCAI 11, 734–741 (2008)Google Scholar
  8. 8.
    Abkai, C., Becherer, N., Hesser, J., Männer, R.: Real-time simulator for intravascular ultrasound (IVUS). In: SPIE Med Imaging, vol. 6513, pp. 1–10 (2007)Google Scholar
  9. 9.
    Sun, B., McKenzie, F.: Medical student evaluation using virtual pathology echocardiography (VPE) for augmented standardized patients. Stud. Health Technol. Inform. 132, 508–510 (2008)Google Scholar
  10. 10.
    Berlage, T., Fox, T., Grunst, G., Quast, K.: Supporting ultrasound diagnosis using an animated 3D model of the heart. In: ICMCS, pp. 34–39 (1996)Google Scholar
  11. 11.
    Bürger, B., Abkai, C., Hesser, J.: Simulation of dynamic ultrasound based on CT models for medical education. Stud. Health Technol. Inform. 132, 56–61 (2008)Google Scholar
  12. 12.
    Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11((3371–3408) (2010)Google Scholar
  13. 13.
    Rifai, S., Vincent, P., Muller, X., Glorot, X., Bengio, Y.: Contractive auto-encoders: Explicit invariance during feature extraction. In: ICML, pp. 833–840 (2011)Google Scholar
  14. 14.
    Odena, A., Dumoulin, V., Olah, C.: Deconvolution and checkerboard artifacts. In: Distill (2016)Google Scholar
  15. 15.
    Horé, A., Ziou, D.: Image quality metrics: PSNR vs. SSIM. In: ICPR, pp. 2366–2369 (2010)Google Scholar
  16. 16.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: ICML, pp. 807–814 (2010)Google Scholar
  17. 17.
    Hu, Y., et al.: Freehand ultrasound image simulation with spatially-conditioned generative adversarial networks. In: MICCAI-RAMBO, pp. 105–115 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.School of Biomedical Engineering and Imaging SciencesKing’s College LondonLondonUK
  2. 2.Epsom and St Helier University NHS TrustEpsomUK

Personalised recommendations