Advertisement

Ultrasound simulation with deformable and patient-specific scatterer maps

  • 117 Accesses

Abstract

Purpose

Ray-tracing-based simulations model ultrasound (US) interactions with a custom geometric anatomical model, where US texture can be emulated via real-time point-spread function convolutions of a tissue scatterer representation. Such scatterer representations for realistic appearance are difficult to parameterize or model manually and do not respond to volumetric deformations such as those caused with tissue compression by the probe. Herein we utilize brightness mode (B-mode) estimated scatterer maps for ray tracing and propose to enhance the realism of ray-tracing-based simulations by incorporating dynamic speckle patterns that change compliant with tissue deformation.

Methods

In this work, we realistically simulate US texture deformations in the scatterer domain via back-projection of ray segments into a nominal state before sampling during simulation runtime. We estimate scatterer maps from background in vivo images using a pretrained generative adversarial network.

Results

We demonstrated our proposed scatterer estimation and runtime background fusion method on simulated transvaginal US scans of detailed surface-based foetal models. We show the viability of modelling deformations in the scatterer domain at interactive frame rates of 28 frames per second. A quantitative and a qualitative evaluations indicated improved realism in comparison to the state of the art.

Conclusions

Transferring a background image in a scatterer representation enables us to capture anatomical content in a physical space, in which deformations can be incorporated physically consistently before convolving with a US point-spread function during simulation runtime. This then uses the same imaging model on both the background and the hand-crafted models leading to a consistent and seamless compounding of contents in the scatterer space.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 99

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Notes

  1. 1.

    Provided as supplementary material.

References

  1. 1.

    Maul H, Scharf A, Baier P, Wüstemann M, Günter H, Gebauer G, Sohn C (2004) Ultrasound simulators: experience with the SonoTrainer and comparative review of other training systems. Ultrasound Obstet Gynecol 24(5):581–585

  2. 2.

    Ehricke H (1998) SONOSim3D: a multimedia system for sonography simulation and education with an extensible case database. Eur J Ultrasound 7(3):225–300

  3. 3.

    Arkhurst W, Pommert A, Richter E, Frederking H, Kim S-I, Schubert R, Höhne KH (2001) A virtual reality training system for pediatric sonography. Proc Int Congr Ser 1230:483–487

  4. 4.

    Tahmasebi AM, Abolmaesumi P, Hashtrudi-Zaad K (2007) A haptic-based ultrasound training/examination system (HUTES). In: Procedings of IEEE international conference on robotics and automation (ICRA), pp 3130–3131

  5. 5.

    Sclaverano S, Chevreau G, Vadcard L, Mozer P, Troccaz J (2009) BiopSym: a simulator for enhanced learning of ultrasound-guided prostate biopsy. Stud Health Technol Inform 142:301–306

  6. 6.

    Goksel O, Salcudean SE (2009) B-mode ultrasound image simulation in deformable 3-D medium. IEEE Trans Med Imaging 28(11):1657–1669

  7. 7.

    Reichl T, Passenger J, Acosta O, Salvado O (2009) Ultrasound goes GPU: real-time simulation using CUDA. In: Proceedings of SPIE medical imaging, p 726116

  8. 8.

    Gao H, Choi HF, Claus P, Boonen S, Jaecques S, Van Lenthe GH, Van der Perre G, Lauriks W, D’Hooge J (2009) A fast convolution-based methodology to simulate 2-D/3-D cardiac ultrasound images. IEEE Trans Ultrason Ferroelectr Freq Control 56(2):404–409

  9. 9.

    Bürger B, Bettinghausen S, Radle M, Hesser J (2013) Real-time GPU-based ultrasound simulation using deformable mesh models. IEEE Trans Med Imaging 32(3):609–618

  10. 10.

    Mattausch O, Goksel O (2016) Monte-Carlo ray tracing for realistic ultrasound training simulation. In: Proceedings of the eurographics workshop visual computing biomedicine (EG VCBM), pp 173–181

  11. 11.

    Mattausch O, Makhinya M, Goksel O (2018) Realistic ultrasound simulation of complex surface models using interactive Monte-Carlo path tracing. Comput Graph Forum 37(1):202–213

  12. 12.

    Tanner C, Starkov R, Bajka M, Goksel O (2018) Framework for fusion of data- and model-based approaches for ultrasound simulation. In: Proceedings of MICCAI, pp 332–339

  13. 13.

    Flach B, Makhinya M, Goksel O (2016) PURE: panoramic ultrasound reconstruction by seamless stitching of volumes. In: Proceedings of MICCAI workshop simulation and synthesis in medical imaging (SASHIMI), pp 75–84

  14. 14.

    Mattausch O, Goksel O (2018) Image-based reconstruction of tissue scatterers using beam steering for ultrasound simulation. IEEE Trans Med Imag 37(3):767–780

  15. 15.

    Al Bahou A, Tanner C, Goksel O (2019) SCATGAN for reconstruction of ultrasound scatterers using generative adversarial networks. In: Proceedings of IEEE international symposium Biomedical Imaging (ISBI), accepted (also arXiv:1902.00469)

  16. 16.

    Starkov R, Tanner C, Bajka M, Goksel O (2019) Ultrasound simulation with animated anatomical models and on-the-fly fusion with real images via path-tracing. Comput Graph 82:44–52

  17. 17.

    Kajiya JT (1986) The rendering equation. ACM SIGGRAPH Comput Graph 20(4):143–150

  18. 18.

    Mattausch O, Ren E, Bajka M, Vanhoey K, Goksel O (2017) Comparison of texture synthesis methods for content generation in ultrasound simulation for training. In: Proceedings of SPIE Med Imaging, p 1013523

  19. 19.

    Bamber JC, Dickinson RJ (1980) Ultrasonic B-scanning: a computer simulation. Phys Med Biol 25(3):463

  20. 20.

    Jensen J (2004) Simulation of advanced ultrasound systems using field II. Proc IEEE Int Symp Biomed Imaging 1:636–639

  21. 21.

    Mattausch O, Goksel O (2015) Scatterer reconstruction and parametrization of homogeneous tissue for ultrasound image simulation. In: Proceedings of IEEE engineering medicine and biology conference (EMBC), pp 6350–6353

  22. 22.

    Müller M, Stam J, James D, Thürey N (2008) Real time physics: class notes. In: Proceedings of ACM SIGGRAPH classes, pp 88:1–88:90

  23. 23.

    Petrinec K (2013) Patient-specific interactive ultrasound image simulation with soft-tissue deformation. Ph.D. thesis, University of California

  24. 24.

    Zikic D, Wein W, Khamene A, Clevert D-A, Navab N (2006) Fast deformable registration of 3D-ultrasound data using a variational approach. In: Proceedings of MICCAI, pp 915–923

  25. 25.

    Virga S, Göbl R, Baust M, Navab N, Hennersperger C (2018) Use the force: deformation correction in robotic 3D ultrasound. Int J Comput Assist Radiol Surg 13(5):619–627

  26. 26.

    Flach B, Makhinya M, Goksel O (2016) Model-based compensation of tissue deformation during data acquisition for interpolative ultrasound simulation. In: Proceedings of IEEE international symposium biomedical imaging (ISBI)

  27. 27.

    Selmi S-Y, Promayon E, Sarrazin J, Troccaz J (2014) 3D interactive ultrasound image deformation for realistic prostate biopsy simulation. In: Proceedings of biomedical simulation, pp 122–130

  28. 28.

    Bro-Nielsen M (1998) Finite element modeling in surgery simulation. Proc IEEE 86(3):490–503

  29. 29.

    Clark JH (1976) Hierarchical geometric models for visible surface algorithms. Commun ACM 19(10):547–554

  30. 30.

    Stich M, Friedrich H, Dietrich A (2009) Spatial splits in bounding volume hierarchies. In: Proceedings of high-perform graph (HPG), pp 7–13

  31. 31.

    Wald I, Boulos S, Shirley P (2007) Ray tracing deformable scenes using dynamic bounding volume hierarchies. ACM Trans Graph. https://doi.org/10.1145/1189762.1206075

  32. 32.

    Karras T, Aila T (2013) Fast parallel construction of high-quality bounding volume hierarchies. In: Proceedings of high-perform graph (HPG), pp 89–99

  33. 33.

    Lext J, Akenine-Möller T (2001) Towards rapid reconstruction for animated ray tracing. In: Proceedings of eurograph short present, pp 311–318

  34. 34.

    Loughna P, Chitty L, Evans T, Chudleigh T (2009) Fetal size and dating: charts recommended for clinical obstetric practice. Ultrasound 17(3):160–166

  35. 35.

    Rubin SM, Whitted T (1980) A 3-dimensional representation for fast rendering of complex scenes. ACM SIGGRAPH Comput Graph 14(3):110–116

  36. 36.

    Faure F, Duriez C, Delingette H, Allard J, Gilles B, Marchesseau S, Talbot H, Courtecuisse H, Bousquet G, Peterlik I, Cotin S (2012) Sofa: a multi-model framework for interactive physical simulation. In: Soft tissue biomechanical modeling for computer assisted surgery. Springer, Berlin, pp 283–321

Download references

Acknowledgements

This work was funded by Innosuisse MoCaFrame grant and the Swiss National Science Foundation (Grant No. 179116). We thank Fabien Péan for the 3D deformation meshes.

Author information

Correspondence to Orcun Goksel.

Ethics declarations

Conflict of interest

The authors Rastislav Starkov, Lin Zhang, Michael Bajka, Christine Tanner, and Orcun Goksel declare no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the provincial ethics committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 26413 KB)

Supplementary material 1 (mp4 26413 KB)

Appendix A: Shape function

Appendix A: Shape function

Consider a FEM mesh consisting of tetrahedral elements, which are commonly used in tessellations. The position \({\mathbf {x}}\) inside a tetrahedral element can be represented in terms of the barycentric coordinates \(r_{k}\) of \({\mathbf {x}}\) with respect to the four element corner positions \({\mathbf {c}}_{k}\) that correspond to the deformed state of the mesh, i.e. \({\mathbf {x}} = \sum _{k=1}^{4} r_{k} {\mathbf {c}}_{k}\). This can be written as [6]:

$$\begin{aligned} \begin{bmatrix} x \\ y \\ z\\ 1 \end{bmatrix} = \begin{bmatrix} c_{11}&c_{12}&c_{13}&c_{14} \\ c_{21}&c_{22}&c_{23}&c_{24} \\ c_{31}&c_{32}&c_{33}&c_{34} \\ 1&1&1&1 \end{bmatrix} \begin{bmatrix} r_{1} \\ r_{2} \\ r_{3}\\ r_{4} \end{bmatrix} \quad \text{ i.e. } \quad {\mathbf {x}} = {\mathbf {C}}{\mathbf {r}}. \end{aligned}$$
(1)

Similarly, the position \({\mathbf {x}}^{0}\) corresponding to the same barycentric coordinates \(r_{k}\) in the nominal mesh can be found as \({\mathbf {x}}^{0}={\mathbf {C}}^{0}{\mathbf {r}}\), where \({\mathbf {C}}^{0}\) contains the nominal corner position. The transformation given by \({\mathbf {x}}^{0}=f^{-1}({\mathbf {x}})\) can be expressed as \({\mathbf {x}}^{0}={\mathbf {C}}^{0}{\mathbf {C}}^{-1}{\mathbf {x}}\), where \({\mathbf {C}}^{0}{\mathbf {C}}^{-1}\) is the inverse transformation \(f^{-1}\) at \({\mathbf {x}}\).

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Starkov, R., Zhang, L., Bajka, M. et al. Ultrasound simulation with deformable and patient-specific scatterer maps. Int J CARS 14, 1589–1599 (2019) doi:10.1007/s11548-019-02054-5

Download citation

Keywords

  • Ultrasound simulation
  • Medical training
  • Sonography
  • Virtual reality
  • Monte-Carlo
  • Ray tracing