Detection of prostate cancer using temporal sequences of ultrasound data: a large clinical feasibility study

  • Shekoofeh AziziEmail author
  • Farhad Imani
  • Sahar Ghavidel
  • Amir Tahmasebi
  • Jin Tae Kwak
  • Sheng Xu
  • Baris Turkbey
  • Peter Choyke
  • Peter Pinto
  • Bradford Wood
  • Parvin Mousavi
  • Purang Abolmaesumi
Original Article



This paper presents the results of a large study involving fusion prostate biopsies to demonstrate that temporal ultrasound can be used to accurately classify tissue labels identified in multi-parametric magnetic resonance imaging (mp-MRI) as suspicious for cancer.


We use deep learning to analyze temporal ultrasound data obtained from 255 cancer foci identified in mp-MRI. Each target is sampled in axial and sagittal planes. A deep belief network is trained to automatically learn the high-level latent features of temporal ultrasound data. A support vector machine classifier is then applied to differentiate cancerous versus benign tissue, verified by histopathology. Data from 32 targets are used for the training, while the remaining 223 targets are used for testing.


Our results indicate that the distance between the biopsy target and the prostate boundary, and the agreement between axial and sagittal histopathology of each target impact the classification accuracy. In 84 test cores that are 5 mm or farther to the prostate boundary, and have consistent pathology outcomes in axial and sagittal biopsy planes, we achieve an area under the curve of 0.80. In contrast, all of these targets were labeled as moderately suspicious in mp-MR.


Using temporal ultrasound data in a fusion prostate biopsy study, we achieved a high classification accuracy specifically for moderately scored mp-MRI targets. These targets are clinically common and contribute to the high false-positive rates associated with mp-MRI for prostate cancer detection. Temporal ultrasound data combined with mp-MRI have the potential to reduce the number of unnecessary biopsies in fusion biopsy settings.


Temporal ultrasound data Deep learning Deep belief network Cancer diagnosis Prostate cancer 



This work was supported in part by the Natural Sciences and Engineering Research Council of Canada (NSERC) and the Canadian Institutes of Health Research (CIHR).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.


  1. 1.
    Azizi S, Imani F, Zhuang B, Tahmasebi A, Kwak JT, Xu S, Uniyal N, Turkbey B, Choyke P, Pinto P, Wood B (2015) Ultrasound-based detection of prostate cancer using automatic feature selection with deep belief networks. Med Image Comput Comput-Assist Interv—MICCAI 2015, pp 70–77. Springer (2015)Google Scholar
  2. 2.
    Bengio Y, Lamblin P, Popovici D, Larochelle H (2007) Greedy layer-wise training of deep networks. Adv Neural Inf Process Syst 19:153Google Scholar
  3. 3.
    Correas JM, Tissier AM, Khairoune A, Khoury G, Eiss D, Hélénon O (2013) Ultrasound elastography of the prostate: state of the art. Diagn Interv Imaging 94(5):551–560CrossRefPubMedGoogle Scholar
  4. 4.
    Daoud MI, Mousavi P, Imani F, Rohling R, Abolmaesumi P (2013) Tissue classification using ultrasound-induced variations in acoustic backscattering features. IEEE Trans Biomed Eng 60(2):310–320CrossRefPubMedGoogle Scholar
  5. 5.
    Epstein JI, Feng Z, Trock BJ, Pierorazio PM (2012) Upgrading and downgrading of prostate cancer from biopsy to radical prostatectomy: incidence and predictive factors using the modified Gleason grading system and factoring in tertiary grades. Eur Urol 61(5):1019–1024CrossRefPubMedPubMedCentralGoogle Scholar
  6. 6.
    Erhan D, Bengio Y, Courville A, Manzagol PA, Vincent P, Bengio S (2010) Why does unsupervised pre-training help deep learning? J Mach Learn Res 11:625–660Google Scholar
  7. 7.
    Fawcett T (2006) An introduction to roc analysis. Pattern Recognit Lett 27(8):861–874CrossRefGoogle Scholar
  8. 8.
    Feleppa, E., Porter, C., Ketterling, J., Dasgupta, S., Ramachandran, S., Sparks, D.: Recent advances in ultrasonic tissue-type imaging of the prostate. In: Acoustical imaging, pp 331–339. Springer (2007)Google Scholar
  9. 9.
    Goossen T, Wijkstra H (2003) Transrectal ultrasound imaging and prostate cancer. Archivio Italiano di Urologia Andrologia 75(1):68–74Google Scholar
  10. 10.
    Hinton G (2010) A practical guide to training restricted Boltzmann machines. Momentum 9(1):926Google Scholar
  11. 11.
    Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554CrossRefPubMedGoogle Scholar
  12. 12.
    Imani F, Abolmaesumi P, Gibson E, Khojaste A, Gaed M, Moussa M, Gomez JA, Romagnoli C, Siemens DR, Leviridge M, Chang S, Fenster A, Ward AD, Mousavi P (2013) Ultrasound-based characterization of prostate cancer: an in vivo clinical feasibility study. Med Image Comput Comput-Assist Interv–MICCAI 2013, pp 279–286, SpringerGoogle Scholar
  13. 13.
    Imani F, Abolmaesumi P, Gibson E, Khojaste A, Gaed M, Moussa M, Siemens DR, Fenster A, Ward A, Mousavi P (2015) Computer-aided prostate cancer detection using ultrasound RF time series: in vivo feasibility study. IEEE Trans Med Imaging 34(11):2248–2257CrossRefPubMedGoogle Scholar
  14. 14.
    Imani F, Ramezani M, Nouranian S, Gibson E, Khojaste A, Gaed M, Moussa M, Gomez J, Romagnoli C, Leveridge M, Chang S (2015) Ultrasound-based characterization of prostate cancer using joint independent component analysis. IEEE Trans Biomed Eng 62(7):1796–1804CrossRefPubMedGoogle Scholar
  15. 15.
    Imani, F., Zhuang, B., Tahmasebi, A., Kwak, J.T., Xu S, Agarwal H, Bharat S, Uniyal N, Turkbey I, Choyke P, Pinto P (2015) Augmenting MRI-transrectal ultrasound-guided prostate biopsy with temporal ultrasound data: a clinical feasibility study. Int J Comput Assist Radiol Surg, pp 1–9 (2015)Google Scholar
  16. 16.
    Kuru TH, Roethke MC, Seidenader J, Simpfendörfer T, Boxler S, Alammar K, Rieker P, Popeneciu V, Roth W, Pahernik S, Schlemmer H (2013) Critical evaluation of magnetic resonance imaging targeted, transrectal ultrasound guided transperineal fusion biopsy for detection of prostate cancer. J Urol 190(4):1380–1386CrossRefPubMedGoogle Scholar
  17. 17.
    Marks L, Young S, Natarajan S (2013) MRI-ultrasound fusion for guidance of targeted prostate biopsy. Curr Opin Urol 23(1):43CrossRefPubMedPubMedCentralGoogle Scholar
  18. 18.
    Miyagawa T, Tsutsumi M, Matsumura T, Kawazoe N, Ishikawa S, Shimokama T, Miyanaga N, Akaza H (2009) Real-time elastography for the diagnosis of prostate cancer: evaluation of elastographic moving images. Jpn J Clin Oncol 39(6):394–398CrossRefPubMedGoogle Scholar
  19. 19.
    Moradi M, Abolmaesumi P, Mousavi P (2010) Tissue typing using ultrasound RF time series: experiments with animal tissue samples. Med Phys 37(8):4401–4413CrossRefPubMedGoogle Scholar
  20. 20.
    Moradi, M., Mahdavi, S.S., Nir, G., Jones, E.C., Goldenberg, S.L., Salcudean, S.E.: Ultrasound RF time series for tissue typing: first in vivo clinical results. In: SPIE Medical Imaging, pp. 86,701I–86,701I. International Society for Optics and Photonics (2013)Google Scholar
  21. 21.
    Moradi M, Mahdavi SS, Nir G, Mohareri O, Koupparis A, Gagnon L, Fazli L, Casey R, Ischia J, Jones E, Goldenberg S (2014) Multiparametric 3D in vivo ultrasound vibroelastography imaging of prostate cancer: Preliminary results. Med Phys 41(7)Google Scholar
  22. 22.
    Moradi M, Mousavi P, Boag A, Sauerbrei EE, Siemens D, Abolmaesumi P (2009) Augmenting detection of prostate cancer in transrectal ultrasound images using SVM and RF time series. IEEE Trans Biomed Eng 56(9):2214–2224CrossRefPubMedGoogle Scholar
  23. 23.
    Nelson ED, Slotoroff CB, Gomella LG, Halpern EJ (2007) Targeted biopsy of the prostate: The impact of color Doppler imaging and elastography on prostate cancer detection and Gleason score. Urology 70(6):1136–1140CrossRefPubMedGoogle Scholar
  24. 24.
    Park S, Aglyamov SR, Emelianov SY (2007) Elasticity imaging using conventional and high-frame rate ultrasound imaging: Experimental study. IEEE Trans Ultrason Ferroelectr Freq Control 54(11):2246–2256CrossRefPubMedGoogle Scholar
  25. 25.
    Rapiti E, Schaffar R, Iselin C, Miralbell R, Pelte MF, Weber D, Zanetti R, Neyroud-Caspar I, Bouchardy C (2013) Importance and determinants of Gleason score undergrading on biopsy sample of prostate cancer in a population-based study. BMC Urol 13(1):19CrossRefPubMedPubMedCentralGoogle Scholar
  26. 26.
    de Rooij M, Hamoen EH, Fütterer JJ, Barentsz JO, Rovers MM (2014) Accuracy of multiparametric MRI for prostate cancer detection: a meta-analysis. Am J Roentgenol 202(2):343–351CrossRefGoogle Scholar
  27. 27.
    Schmitz G, Ermert H, Senge T (1999) Tissue-characterization of the prostate using RF ultrasonic signals. IEEE Trans Ultrason Ferroelectr Freq Control 46(1):126–138CrossRefPubMedGoogle Scholar
  28. 28.
    Tanaka M, Okutomi M (2014) A novel inference of a restricted Boltzmann machine. In: International conference on pattern recognition (ICPR), 2014 22nd, pp 1526–1531. IEEE (2014)Google Scholar
  29. 29.
    Turkbey B, Mani H, Aras O, Ho J, Hoang A, Rastinehad A, Agarwal H, Shah V, Bernardo M, Pang Y, Daar D (2013) Prostate cancer: Can mp-MR imaging help identify patients who are candidates for active surveillance? Radiology 268(1):144–152Google Scholar
  30. 30.
    Xie SW, Li HL, Du J, Xia JG, Guo YF, Xin M, Li FH (2013) Influence of serum PSA level, prostate volume, and PSA density on prostate cancer detection with contrast-enhanced sonography using contrast-tuned imaging technology. J Ultrasound Med 32(5):741–748CrossRefPubMedGoogle Scholar
  31. 31.
    Xu S, Kruecker J, Turkbey B, Glossop N, Singh AK, Choyke P, Pinto P, Wood B (2008) Real-time MRI-TRUS fusion for guidance of targeted prostate biopsies. Comput Aided Surg 13(5):255–264CrossRefPubMedPubMedCentralGoogle Scholar
  32. 32.
    Yosinski J, Lipson H (2012) Visually debugging restricted Boltzmann machine training with a 3D example. In: 29th international conference on machine learning, representation learning workshop 2012Google Scholar

Copyright information

© CARS 2016

Authors and Affiliations

  • Shekoofeh Azizi
    • 1
    Email author
  • Farhad Imani
    • 1
  • Sahar Ghavidel
    • 2
  • Amir Tahmasebi
    • 3
  • Jin Tae Kwak
    • 4
  • Sheng Xu
    • 4
  • Baris Turkbey
    • 4
  • Peter Choyke
    • 4
  • Peter Pinto
    • 4
  • Bradford Wood
    • 4
  • Parvin Mousavi
    • 2
  • Purang Abolmaesumi
    • 1
  1. 1.University of British ColumbiaVancouverCanada
  2. 2.Queen’s UniversityKingstonCanada
  3. 3.Philips Research North AmericaCambridgeUSA
  4. 4.National Institutes of HealthBethesdaUSA

Personalised recommendations