Skip to main content

Advertisement

Log in

Five-dimensional ultrasound system for soft tissue visualization

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

A five-dimensional ultrasound (US) system is proposed as a real-time pipeline involving fusion of 3D B-mode data with the 3D ultrasound elastography (USE) data as well as visualization of these fused data and a real-time update capability over time for each consecutive scan. 3D B-mode data assist in visualizing the anatomy of the target organ, and 3D elastography data adds strain information.

Methods

We investigate the feasibility of such a system and show that an end-to-end real-time system, from acquisition to visualization, can be developed. We present a system that consists of (a) a real-time 3D elastography algorithm based on a normalized cross-correlation (NCC) computation on a GPU; (b) real-time 3D B-mode acquisition and network transfer; (c) scan conversion of 3D elastography and B-mode volumes (if acquired by 4D wobbler probe); and (d) visualization software that fuses, visualizes, and updates 3D B-mode and 3D elastography data in real time.

Results

We achieved a speed improvement of 4.45-fold for the threaded version of the NCC-based 3D USE versus the non-threaded version. The maximum speed was 79 volumes/s for 3D scan conversion. In a phantom, we validated the dimensions of a 2.2-cm-diameter sphere scan-converted to B-mode volume. Also, we validated the 5D US system visualization transfer function and detected 1- and 2-cm spherical objects (phantom lesion). Finally, we applied the system to a phantom consisting of three lesions to delineate the lesions from the surrounding background regions of the phantom.

Conclusion

A 5D US system is achievable with real-time performance. We can distinguish between hard and soft areas in a phantom using the transfer functions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Siegel R, Naishadham D, Jemal A (2013) Cancer statistics, 2013. CA Cancer J Clin 63(1):11–30

    Article  PubMed  Google Scholar 

  2. Ophir J, Céspedes I, Ponnekanti H, Yazdi Y, Li X (1991) Elastography: a quantitative method for imaging the elasticity of biological tissues. Ultrason Imaging 13(2):111–134

    Article  CAS  PubMed  Google Scholar 

  3. Boctor EM, Deshmukh N, Ayad MS, Clarke C, Dickie K, Choti MA, Burdette EC (2009) Three-dimensional heat-induced echo-strain imaging for monitoring high-intensity acoustic ablation. In: McAleavey SA, D’hooge J (eds) Medical imaging 2009: ultrasonic imaging and signal processing, Lake Buena Vista, FL, p 72650R. doi:10.1117/12.811287

  4. Falou O, Sadeghi-Naini A, Prematilake S, Sofroni E, Papanicolau N, Iradji S, Jahedmotlagh Z, Lemon-Wong S, Pignol J-P, Rakovitch E, Zubovits J, Spayne J, Dent R, Trudeau M, Boileau JF, Wright FC, Yaffe MJ, Czarnota GJ (2013) Evaluation of neoadjuvant chemotherapy response in women with locally advanced breast cancer using ultrasound elastography. Transl Oncol 6(1):17–24

    Article  PubMed Central  PubMed  Google Scholar 

  5. Billings S, Deshmukh N, Kang HJ, Taylor R, Boctor EM (2012) System for robot-assisted real-time laparoscopic ultrasound elastography. In: Holmes III DR, Wong HW (eds) Medical imaging 2012: image-guidedprocedures, robotic interventions, and modeling, San Diego, California, USA, p 83161W. doi:10.1117/12.911086

  6. Sen HT, Deshmukh N, Goldman R, Kazanzides P, Taylor RH, Boctor E, Simaan N (2012) Enabling technologies for natural orifice transluminal endoscopic surgery (NOTES) using robotically guided elasticity imaging. In: Holmes III DR, Wong KH (eds) Medical Imaging 2012: Image-Guided Procedures, Robotic Interventions, and Modeling, San Diego, California, USA, p 83161Y. doi:10.1117/12.912383

  7. Weismann C, Mayr C, Egger H, Auer A (2011) Breast sonography—2D, 3D, 4D ultrasound or elastography? Breast Care (Basel) 6(2):98–103

    Article  Google Scholar 

  8. Boctor EM, Choti M, Fictinger G, Taylor R, Prince JL (2011) Robotic 5-dimensional ultrasound. US Patent 7901357 B2

  9. Torres F, Fanti Z, Arambula Cosío F (2013) 3D freehand ultrasound for medical assistance in diagnosis and treatment of breast cancer: preliminary results. In: IX international seminar on medical information processing and analysis, p 89220K

  10. Sayed A, Layne G, Abraham J, Mukdadi O (2013) Nonlinear characterization of breast cancer using multi-compression 3D ultrasound elastography in vivo. Ultrasonics 53(5):979–991

    Article  PubMed Central  PubMed  Google Scholar 

  11. Jedrzejewski G, Ben-Skowronek I, Wozniak MM, Brodzisz A, Budzynska E, Wieczorek AP (2013) Testicular adrenal rest tumors in boys with congenital adrenal hyperplasia: 3D US and elastography—do we get more information for diagnosis and monitoring? J Pediatr Urol 9(6):1032–1037

    Article  PubMed  Google Scholar 

  12. Ying M, Zheng Y-P, Kot BC-W, Cheung JC-W, Cheng SC-H, Kwong DL-W (2013) Three-dimensional elastography for cervical lymph node volume measurements: a study to investigate feasibility, accuracy and reliability. Ultrasound Med Biol 39(3):396–406

    Article  PubMed  Google Scholar 

  13. Rivaz H, Fleming I, Assumpcao L, Fichtinger G, Hamper U, Choti M, Hager G, Boctor E (2008) Ablation monitoring with elastography: 2D in vivo and 3D ex vivo studies. In: Metaxas D, Axel L, Fichtinger G, Székely G (eds) Medical image computing and computer-assisted intervention (MICCAI 2008), vol 5242. Springer, Berlin, pp 458–466

  14. Foroughi P, Burgner J, Choti MA, Webster III RJ, Hager GD, Boctor EM (2012) Towards intra-operative monitoring of ablation using tracked 3D ultrasound elastography and internal palpation. In: Bosch JG, Doyley MM (eds) Medical imaging 2012: ultrasonic imaging, tomography, and therapy, San Diego, California, USA, p 83200T. doi:10.1117/12.913988

  15. Kanenishi K, Hanaoka U, Noguchi J, Marumo G, Hata T (2013) 4D ultrasound evaluation of fetal facial expressions during the latter stages of the second trimester. Int J Gynecol Obstet 121(3):257–260

    Article  Google Scholar 

  16. Wittek A, Karatolios K, Bihari P, Schmitz-Rixen T, Moosdorf R, Vogt S, Blase C (2013) In vivo determination of elastic properties of the human aorta based on 4D ultrasound data. J Mech Behav Biomed Mater 27:167–183

    Article  PubMed  Google Scholar 

  17. Hilde G, Staer-Jensen J, Siafarikas F, Gjestland K, Ellström Engh M, Bø K (2013) How well can pelvic floor muscles with major defects contract? A cross-sectional comparative study 6 weeks after delivery using transperineal 3D/4D ultrasound and manometer. BJOG 120(11):1423–1429

    Article  CAS  PubMed  Google Scholar 

  18. Braekken IH, Majida M, Engh ME, Bø K (2009) Test–retest reliability of pelvic floor muscle contraction measured by 4D ultrasound. Neurourol Urodyn 28(1):68–73

    Article  PubMed  Google Scholar 

  19. Yagel S, Cohen SM, Shapiro I, Valsky DV (2007) 3D and 4D ultrasound in fetal cardiac scanning: a new look at the fetal heart. Ultrasound Obstet Gynecol 29(1):81–95

    Article  CAS  PubMed  Google Scholar 

  20. Vijayan S, Klein S, Hofstad EF, Lindseth F, Ystgaard B, Lango T (2013) Validation of a non-rigid registration method for motion compensation in 4D ultrasound of the liver. In: IEEE 10th international symposium on biomedical imaging, pp 792–795

  21. Gritzmann N, Weismann CF, Datz L (2007) Diagnostic algorithm: how to make use of new 2D, 3D and 4D ultrasound technologies in breast imaging. Eur J Radiol 64(2):250–257

    Article  Google Scholar 

  22. Achiron R, Gindes L, Zalel Y, Lipitz S, Weisz B (2008) Three- and four-dimensional ultrasound: new methods for evaluating fetal thoracic anomalies. Ultrasound Obstet Gynecol 32(1):36–43

    Article  CAS  PubMed  Google Scholar 

  23. Mahdavi SS, Moradi M, Morris WJ, Goldenberg SL, Salcudean SE (2012) Fusion of ultrasound B-mode and vibro-elastography images for automatic 3D segmentation of the prostate. IEEE Trans Med Imaging 31(11):2073–2082

    Article  PubMed  Google Scholar 

  24. Moradi M, Abolmaesumi P, Mousavi P (2010) Tissue typing using ultrasound RF time series: experiments with animal tissue samples. Med Phys 37(8):4401

    Article  PubMed  Google Scholar 

  25. Uniyal N, Eskandari H, Abolmaesumi P, Sojoudi S, Gordon P, Warren L, Rohling RN, Salcudean SE, Moradi M (2015) Ultrasound RF time series for classification of breast lesions. IEEE Trans Med Imaging 34(2):652–661

    Article  PubMed  Google Scholar 

  26. Itoh A, Ueno E, Tohno E, Kamma H, Takahashi H, Shiina T, Yamakawa M, Matsumura T (2006) Breast disease: clinical application of US elastography for diagnosis. Radiology 239(2):341–350

    Article  PubMed  Google Scholar 

  27. Burnside ES, Hall TJ, Sommer AM, Hesley GK, Sisney GA, Svensson WE, Fine JP, Jiang J, Hangiandreou NJ (2007) Differentiating benign from malignant solid breast masses with US strain imaging. Radiology 245(2):401–410

    Article  PubMed  Google Scholar 

  28. Kang H-J, Deshmukh NP, Stolka P, Burdette EC, Boctor EM (2012) Ultrasound imaging software framework for real-time monitoring of acoustic ablation therapy. In: Bosch JG, Doyley MM (eds) Medical imaging 2012: ultrasonic imaging, tomography, and therapy, San Diego, California, USA, p 83201E.doi:10.1117/12.912362

  29. Stolka P, Kang H, Boctor E (2010) The MUSiiC Toolkit: modular real-time toolkit for advanced ultrasound research. http://hdl.handle.net/10380/3172

  30. Shin KG, Ramanathan P (1994) Real-time computing: a new discipline of computer science and engineering. Proc IEEE 82(1):6–24

    Article  Google Scholar 

  31. Heinecke A, Klemm M, Bungartz H-J (2012) From GPGPU to many-core: Nvidia Fermi and intel many integrated core architecture. Comput Sci Eng 14(2):78–83

    Article  Google Scholar 

  32. Deshmukh NP, Kang HJ, Billings SD, Taylor RH, Hager GD, et al (2014) Elastography using multi-stream GPU: an application to online tracked ultrasound elastography, in-vivo and the da Vinci surgicalsystem. PLoS One 9(12):e115881. doi: 10.1371/journal.pone.0115881

  33. Pospisil E, Rohling R, Azar R, Salcudean S (2010) 4-D \(\times \) 3-D ultrasound: real-time scan conversion, filtering, and display of displacement vectors with a motorized curvilinear transducer. IEEE Trans Ultrason Ferroelectr Freq Control 57(10):2271–2283

    Article  PubMed  Google Scholar 

  34. Boctor EM, Matinfar M, Ahmad O, Rivaz H, Choti M, Taylor RH (2009) Elasticity-based three dimensional ultrasound real-time volume rendering. In: Miga MI, Wong KH (eds) Medical imaging 2009: visualization, image-guided procedures, and modeling, Lake Buena Vista, FL, , p 72612V. doi:10.1117/12.815166

  35. Mann D, Caban JJ, Stolka PJ, Boctor EM, Yoo TS (2011) Multi-dimensional transfer functions for effective visualization of streaming ultrasound and elasticity images. In: Wong KH, Holmes III DR (eds) Medical Imaging 2011: Visualization, Image-Guided Procedures, and Modeling, Lake Buena Vista, FL, p 796439. doi:10.1117/12.878935

  36. Weis JA, Yankeelov TE, Munoz SA, Sastry RA, Barnes SL, Arlinghaus LR, Li X, Miga MI (2013) A consistent pre-clinical/clinical elastography approach for assessing tumor mechanical properties in therapeutic systems. In: Weaver JB, Molthen RC (eds) Medical imaging 2013: biomedical applications in molecular, structural, and functional imaging, Lake Buena Vista (Orlando Area), FL, USA, p 86721F. doi:10.1117/12.2007425

  37. Heid V, Evers H, Henn C, Glombitza G, Meinzer H-P (2000) 5D interactive real time Doppler ultrasound visualization of the heart. Med Imaging 2000:494–500

    Google Scholar 

  38. Takahashi H, Hasegawa H, Kanai H (2013) Improvement of automated identification of the heart wall in echocardiography by suppressing clutter component. Jpn J Appl Phys 52(7S):07HF17

    Article  Google Scholar 

  39. Deshmukh N, Rivaz H, Boctor E (2009) GPU-based elasticity imaging algorithms. In: Proceedings of the international conference in medical imaging computing and computer assisted intervention

Download references

Acknowledgments

NIH SBIR Grant (PI: Everett Burdette) CA134169, NIH Grant CA112852, and The Johns Hopkins University internal funds supported this work. The authors thank Dr. Pezhman Foroughi, Dr. Daniel Carnegie, Dr. Philipp Stolka, Dr. Terry S. Yoo, Dr. Ioana N. Fleming, Balázs P. Vágvölgyi, Dr. Hyun Jae Kang, Joyce Choi, Seth D. Billings, Tina Vashistha, Joseph Wang, and David Mann for their valuable inputs.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Nishikant P. Deshmukh.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Appendix: 3D scan conversion on GPU

Appendix: 3D scan conversion on GPU

Scan conversion helps to convert the data collected on the surface of 4D probe that depicts spherical sector, which is an array of images, to the appropriate location in space. This section details the equations used for scan conversion on GPU. We calculate \(\varphi \), the angle of field of view by

$$\begin{aligned} \varphi =N\times \delta \end{aligned}$$
(2)

where N is the number of slices per volume and \(\delta \) is the step angle of the motor. We then determine the dimension of the scan-converted volume by

$$\begin{aligned} y^{\prime }=\frac{x^{\prime }\times \left( {\tau +y-\tau } \times \cos \left( {\frac{\varphi }{2}} \right) \right) }{\left( {2\times (\tau +y)\times \tan \left( {\frac{\varphi }{2}} \right) } \right) } \end{aligned}$$
(3)

where \(( {x^{\prime },y^{\prime }})\) is the new (elevation, axial) size in pixels. In our case, \(x^{\prime }\) is user defined, \(\tau \) is the radius of the curvature in pixels and y is the number of pixels in the axial direction in the original volume. We then calculate the step size in the axial direction by

$$\begin{aligned} r_i =\tau +i+1 \end{aligned}$$
(4)

where \(r_i\) is the step size in the axial direction (pixels) from the origin of the probe for all \(i=0\)y. We then calculate the angle \(\theta _j\) from the center slice by

$$\begin{aligned} \theta _{j} =-\frac{\varphi }{2}+j\times \delta \end{aligned}$$
(5)

where \(\theta _{j}\) is negative or positive on the opposite sides of the center slice, \(j=0\)x, where x is the number of pixels in the elevational direction. Typically, \(x=N\) when the slice thickness is 1 pixel. We now calculate the forward index \(i^{\prime \prime }\) into the final volume along the axial direction by

$$\begin{aligned} i^{\prime \prime }=\frac{x^{\prime }\times \left( {\hbox {round}\left( r_i \times \cos \left( \theta _{j}\right) \right) -\tau \times \cos \left( {\frac{\varphi }{2}} \right) } \right) }{\hbox {round}\left( {2+(\tau +y)\times \tan \left( {\frac{\varphi }{2}} \right) } \right) }. \end{aligned}$$
(6)

We now calculate the forward index \(j^{\prime \prime }\) in the final volume along the elevation direction by

$$\begin{aligned} j^{\prime \prime }=\frac{y^{\prime }\times \left( {(\tau +y)\times \tan \left( {\frac{\varphi }{2}} \right) +\hbox {round}\left( r_i \times \sin \left( \theta _j\right) \right) } \right) }{\left( {\tau +y-\tau \times \cos \left( {\frac{\varphi }{2}} \right) } \right) }.\nonumber \\ \end{aligned}$$
(7)

The following equation gives final scan conversion after substituting values of \(y^{\prime }\), \(i^{\prime \prime }\), and \(j^{\prime \prime }\) from Eqs. (3), (6), and (7), respectively:

$$\begin{aligned}&\hbox {output}\left[ k\times x^{\prime }\times y^{\prime }+i^{\prime \prime }\times x^{\prime }+j^{\prime \prime }\right] \nonumber \\&\quad = \hbox {input}\left[ k\times x\times y+i\times x+j\right] \end{aligned}$$
(8)

where output is the output volume buffer which will contain the scan-converted volume, input is the input volume buffer which contains the US machine acquired volume, \(k=0\)zz is the number of pixels in the lateral direction.

GPU kernels execute Eqs. (3)–(8) that are highly parallel since index mappings are independent of each other. In the case of conflict, where multiple pixels from the different locations in the original buffer get mapped to the same pixel in the output buffer, we simply consider the maximum pixel value. There are holes created into the output slices, which are recovered using interpolation by a simple GPU-based averaging filter.

For this scan conversion to work and to keep the equations simple, we need to rotate the volume around an axis at the center of the volume and axis-align to elevation–axial–lateral directions. The rotation depicts swapping of elevation–axial and lateral–axial slices. The following equations give us an easy way to swap the values in the 3D buffers:

$$\begin{aligned}&\hbox {postswap}\left[ j\times y\times x+k\times x+i\right] \nonumber \\&\quad = \hbox {preswap}\left[ i\times y\times z+k\times z+(z-j-1)\right] \end{aligned}$$
(9)
$$\begin{aligned}&\hbox {preswap}\left[ j\times y\times x+k\times x+(x-i-1)\right] \nonumber \\&\quad = \hbox {postswap}\left[ i\times y\times z+k\times z+j)\right] \end{aligned}$$
(10)

where pre- and post-swap are the buffers before and after swapping of the pixels. A GPU kernel executes Eq. 9 just before the scan conversion and Eq. 10 after the scan conversion.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deshmukh, N.P., Caban, J.J., Taylor, R.H. et al. Five-dimensional ultrasound system for soft tissue visualization. Int J CARS 10, 1927–1939 (2015). https://doi.org/10.1007/s11548-015-1277-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-015-1277-z

Keywords

Navigation