Detection and grading of prostate cancer using temporal enhanced ultrasound: combining deep neural networks and tissue mimicking simulations

  • Shekoofeh AziziEmail author
  • Sharareh Bayat
  • Pingkun Yan
  • Amir Tahmasebi
  • Guy Nir
  • Jin Tae Kwak
  • Sheng Xu
  • Storey Wilson
  • Kenneth A. Iczkowski
  • M. Scott Lucia
  • Larry Goldenberg
  • Septimiu E. Salcudean
  • Peter A. Pinto
  • Bradford Wood
  • Purang AbolmaesumiEmail author
  • Parvin MousaviEmail author
Original Article



Temporal Enhanced Ultrasound (TeUS) has been proposed as a new paradigm for tissue characterization based on a sequence of ultrasound radio frequency (RF) data. We previously used TeUS to successfully address the problem of prostate cancer detection in the fusion biopsies.


In this paper, we use TeUS to address the problem of grading prostate cancer in a clinical study of 197 biopsy cores from 132 patients. Our method involves capturing high-level latent features of TeUS with a deep learning approach followed by distribution learning to cluster aggressive cancer in a biopsy core. In this hypothesis-generating study, we utilize deep learning based feature visualization as a means to obtain insight into the physical phenomenon governing the interaction of temporal ultrasound with tissue.


Based on the evidence derived from our feature visualization, and the structure of tissue from digital pathology, we build a simulation framework for studying the physical phenomenon underlying TeUS-based tissue characterization.


Results from simulation and feature visualization corroborated with the hypothesis that micro-vibrations of tissue microstructure, captured by low-frequency spectral features of TeUS, can be used for detection of prostate cancer.


Temporal enhanced ultrasound Deep learning Deep belief network Cancer grading Prostate cancer 



This work was supported in part by the Natural Sciences and Engineering Research Council of Canada (NSERC) and in part by the Canadian Institutes of Health Research (CIHR).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.


  1. 1.
    Azizi S, Imani F, Ghavidel S, Tahmasebi A, Wood B, Mousavi P, Abolmaesumi P (2016) Detection of prostate cancer using temporal sequences of ultrasound data: a large clinical feasibility study. Int J Comput Assist Radiol Surg. 11(6):947–956CrossRefPubMedGoogle Scholar
  2. 2.
    Azizi S, Imani F, Kwak JT, Tahmasebi A, Xu S, Yan P, Kruecker J, Turkbey B, Choyke P, Pinto P, Wood B, Mousavi P, Abolmaesumi P (2016) Classifying cancer grades using temporal ultrasound for transrectal prostate biopsy. International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 653–661Google Scholar
  3. 3.
    Azizi S, Imani F, Zhuang B, Tahmasebi A, Kwak JT, Xu S, Uniyal N, Turkbey B, Choyke P, Pinto P, Wood B, Mousavi P, Abolmaesumi P (2015) Ultrasound-based detection of prostate cancer using automatic feature selection with deep belief networks. International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 70–77Google Scholar
  4. 4.
    Bell AJ, Sejnowski TJ (1997) The independent components of natural scenes are edge filters. Vis Res 37(23):3327–3338CrossRefPubMedPubMedCentralGoogle Scholar
  5. 5.
    Bengio Y, Lamblin P, Popovici D, Larochelle H (2007) Greedy layer-wise training of deep networks. Adv Neural Inf Process Syst 19:153Google Scholar
  6. 6.
    Correas JM, Tissier AM, Khairoune A, Vassiliu V, Méjean A, Hélénon O, Memo R, Barr RG (2014) Prostate cancer: diagnostic performance of real-time shear-wave elastography. Radiology 275(1):280–289CrossRefPubMedGoogle Scholar
  7. 7.
    Daoud MI, Lacefield JC (2011) Three-dimensional computer simulation of high-frequency ultrasound imaging of healthy and cancerous murine liver tissues. In: SPIE Medical Imaging, pp. 79,680H–79,680H. International Society for Optics and PhotonicsGoogle Scholar
  8. 8.
    Daoud MI, Mousavi P, Imani F, Rohling R, Abolmaesumi P (2013) Tissue classification using ultrasound-induced variations in acoustic backscattering features. IEEE Trans Biomed Eng 60(2):310–320CrossRefPubMedGoogle Scholar
  9. 9.
    Epstein JI, Feng Z, Trock BJ, Pierorazio PM (2012) Upgrading and downgrading of prostate cancer from biopsy to radical prostatectomy: incidence and predictive factors using the modified Gleason grading system and factoring in tertiary grades. Eur Urol 61(5):1019–1024CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Fawcett T (2006) An introduction to ROC analysis. Pattern Recognit Lett 27(8):861–874CrossRefGoogle Scholar
  11. 11.
    Feleppa E, Porter C, Ketterling J, Dasgupta S, Ramachandran S, Sparks D (2007) Recent advances in ultrasonic tissue-type imaging of the prostate. Acoustical imaging. Springer, Netherlands, pp 331–339CrossRefGoogle Scholar
  12. 12.
    Hunt JW, Worthington AE, Kerr AT (1995) The subtleties of ultrasound images of an ensemble of cells: simulation from regular and more random distributions of scatterers. Ultrasound Med Biol 21(3):329–341CrossRefPubMedGoogle Scholar
  13. 13.
    Hunt JW, Worthington AE, Xuan A, Kolios MC, Czarnota GJ, Sherar MD (2002) A model based upon pseudo regular spacing of cells combined with the randomisation of the nuclei can explain the significant changes in high-frequency ultrasound signals during apoptosis. Ultrasound Med Biol 28(2):217–226CrossRefPubMedGoogle Scholar
  14. 14.
    Iczkowski KA, Torkko KC, Kotnis GR, Wilson RS, Huang W, Wheeler TM, Abeyta AM, La Rosa FG, Cook S, Werahera PN (2011) Digital quantification of five high-grade prostate cancer patterns, including the cribriform pattern, and their association with adverse outcome. Am J Clin Pathol 136(1):98–107CrossRefPubMedPubMedCentralGoogle Scholar
  15. 15.
    Imani F, Abolmaesumi P, Gibson E, Khojaste A, Gaed M, Moussa M, Gomez JA, Romagnoli C, Leveridge M, Chang S (2015) Computer-aided prostate cancer detection using ultrasound RF time series: in vivo feasibility study. IEEE Trans Med Imaging 34(11):2248–2257CrossRefPubMedGoogle Scholar
  16. 16.
    Imani F, Ramezani M, Nouranian S, Gibson E, Khojaste A, Gaed M, Moussa M, Gomez JA, Romagnoli C, Leveridge M (2015) Ultrasound-based characterization of prostate cancer using joint independent component analysis. IEEE Trans Biomed Eng 62(7):1796–1804CrossRefPubMedGoogle Scholar
  17. 17.
    Imani F, Zhuang B, Tahmasebi A, Kwak JT, Xu S, Agarwal H, Bharat S, Uniyal N, Turkbey IB, Choyke P, Pinto P (2015) Augmenting MRI–transrectal ultrasound-guided prostate biopsy with temporal ultrasound data: a clinical feasibility study. Int J Comput Assist Radiol Surg 10(6):727–735CrossRefPubMedGoogle Scholar
  18. 18.
    Jensen JA (2004) Simulation of advanced ultrasound systems using field II. In: IEEE international symposium on biomedical imaging: nano to macro, IEEE 2004 , pp. 636–639Google Scholar
  19. 19.
    Khojaste A, Imani F, Moradi M, Berman D, Siemens DR, Sauerberi EE, Boag AH, Abolmaesumi P, Mousavi P (2015) Characterization of aggressive prostate cancer using ultrasound RF time series. In: SPIE Medical Imaging, pp. 94,141A–94,141A. International society for optics and photonicsGoogle Scholar
  20. 20.
    Kuru TH, Roethke MC, Seidenader J, Simpfendörfer T, Boxler S, Alammar K, Rieker P, Popeneciu VI, Roth W, Pahernik S (2013) Critical evaluation of magnetic resonance imaging targeted, transrectal ultrasound guided transperineal fusion biopsy for detection of prostate cancer. J Urol 190(4):1380–1386CrossRefPubMedGoogle Scholar
  21. 21.
    Li S, Chen M, Wang W, Zhao W, Wang J, Zhao X, Zhou C (2011) A feasibility study of MR elastography in the diagnosis of prostate cancer at 3.0 T. Acta Radiol 52(3):354–358CrossRefPubMedGoogle Scholar
  22. 22.
    Llobet R, Pérez-Cortés JC, Toselli AH, Juan A (2007) Computer-aided detection of prostate cancer. Int J Med Inf 76(7):547–556CrossRefGoogle Scholar
  23. 23.
    Marks L, Young S, Natarajan S (2013) MRI–US fusion for guidance of targeted prostate biopsy. Curr Opin Urol 23(1):43CrossRefPubMedPubMedCentralGoogle Scholar
  24. 24.
    Moradi M, Abolmaesumi P, Mousavi P (2010) Tissue typing using ultrasound RF time series: experiments with animal tissue samples. Med Phys 37(8):4401–4413CrossRefPubMedGoogle Scholar
  25. 25.
    Moradi M, Abolmaesumi P, Siemens DR, Sauerbrei EE, Boag AH, Mousavi P (2009) Augmenting detection of prostate cancer in transrectal ultrasound images using SVM and RF time series. IEEE Trans Biomed Eng 56(9):2214–2224CrossRefPubMedGoogle Scholar
  26. 26.
    Moradi M, Mahdavi SS, Nir G, Jones EC, Goldenberg SL, Salcudean SE (2013) Ultrasound RF time series for tissue typing: first in vivo clinical results. In: SPIE Medical Imaging, pp. 86,701I–86,701I. International society for optics and photonicsGoogle Scholar
  27. 27.
    Nelson ED, Slotoroff CB, Gomella LG, Halpern EJ (2007) Targeted biopsy of the prostate: the impact of color doppler imaging and elastography on prostate cancer detection and Gleason score. Urology 70(6):1136–1140CrossRefPubMedGoogle Scholar
  28. 28.
    de Rooij M, Hamoen EH, Fütterer JJ, Barentsz JO, Rovers MM (2014) Accuracy of multiparametric MRI for prostate cancer detection: a meta-analysis. Am J Roentgenol 202(2):343–351CrossRefGoogle Scholar
  29. 29.
    Siddiqui MM, Rais-Bahrami S, Turkbey B, George AK, Rothwax J, Shakir N, Okoro C, Raskolnikov D, Parnes HL, Linehan WM (2015) Comparison of MR/ultrasound fusion–guided biopsy with ultrasound-guided biopsy for the diagnosis of prostate cancer. Jama 313(4):390–397CrossRefPubMedPubMedCentralGoogle Scholar
  30. 30.
    Singer EA, Kaushal A, Turkbey B, Couvillon A, Pinto PA, Parnes HL (2012) Active surveillance for prostate cancer: past, present and future. Curr Opin Oncol 24(3):243–250Google Scholar
  31. 31.
    Turkbey B, Mani H, Aras O, Ho J, Hoang A, Rastinehad AR, Agarwal H, Shah V, Bernardo M, Pang Y (2013) Prostate cancer: Can multiparametric MR imaging help identify patients who are candidates for active surveillance? Radiology 268(1):144–152CrossRefPubMedPubMedCentralGoogle Scholar
  32. 32.
    Xu L, Jordan MI (1996) On convergence properties of the EM algorithm for Gaussian mixtures. Neural Comput 8(1):129–151CrossRefGoogle Scholar
  33. 33.
    Xu S, Kruecker J, Turkbey B, Glossop N, Singh AK, Choyke P, Pinto P, Wood BJ (2008) Real-time MRI-TRUS fusion for guidance of targeted prostate biopsies. Comput Aided Surg 13(5):255–264CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© CARS 2017

Authors and Affiliations

  • Shekoofeh Azizi
    • 1
    Email author
  • Sharareh Bayat
    • 1
  • Pingkun Yan
    • 2
  • Amir Tahmasebi
    • 2
  • Guy Nir
    • 1
  • Jin Tae Kwak
    • 3
  • Sheng Xu
    • 6
  • Storey Wilson
    • 4
  • Kenneth A. Iczkowski
    • 4
  • M. Scott Lucia
    • 4
  • Larry Goldenberg
    • 5
  • Septimiu E. Salcudean
    • 1
  • Peter A. Pinto
    • 6
  • Bradford Wood
    • 6
  • Purang Abolmaesumi
    • 1
    Email author
  • Parvin Mousavi
    • 7
    Email author
  1. 1.The University of British ColumbiaVancouverCanada
  2. 2.Philips Research North AmericaCambridgeUSA
  3. 3.Sejong University, Gwangjin-GuSeoulSouth Korea
  4. 4.University of ColoradoDenverUSA
  5. 5.Vancouver Prostate CentreVancouverCanada
  6. 6.National Institutes of HealthBethesdaUSA
  7. 7.Queen’s UniversityKingstonCanada

Personalised recommendations