Transfer learning from RF to B-mode temporal enhanced ultrasound features for prostate cancer detection

  • Shekoofeh AziziEmail author
  • Parvin Mousavi
  • Pingkun Yan
  • Amir Tahmasebi
  • Jin Tae Kwak
  • Sheng Xu
  • Baris Turkbey
  • Peter Choyke
  • Peter Pinto
  • Bradford Wood
  • Purang Abolmaesumi
Original Article



We present a method for prostate cancer (PCa) detection using temporal enhanced ultrasound (TeUS) data obtained either from radiofrequency (RF) ultrasound signals or B-mode images.


For the first time, we demonstrate that by applying domain adaptation and transfer learning methods, a tissue classification model trained on TeUS RF data (source domain) can be deployed for classification using TeUS B-mode data alone (target domain), where both data are obtained on the same ultrasound scanner. This is a critical step for clinical translation of tissue classification techniques that primarily rely on accessing RF data, since this imaging modality is not readily available on all commercial scanners in clinics. Proof of concept is provided for in vivo characterization of PCa using TeUS B-mode data, where different nonlinear processing filters in the pipeline of the RF to B-mode conversion result in a distribution shift between the two domains.


Our in vivo study includes data obtained in MRI-guided targeted procedure for prostate biopsy. We achieve comparable area under the curve using TeUS RF and B-mode data for medium to large cancer tumor sizes in biopsy cores (>4 mm).


Our result suggests that the proposed adaptation technique is successful in reducing the divergence between TeUS RF and B-mode data.


Temporal enhanced ultrasound Radiofrequency signal B-mode Deep learning Deep belief network Transfer learning Cancer diagnosis Prostate cancer 



This work was supported in part by the Natural Sciences and Engineering Research Council of Canada (NSERC) and in part by the Canadian Institutes of Health Research (CIHR).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.


  1. 1.
    Azizi S, Imani F, Ghavidel S, Tahmasebi A, Wood B, Mousavi P, Abolmaesumi P (2016) Detection of prostate cancer using temporal sequences of ultrasound data: a large clinical feasibility study. Int J Comput Assist Radiol Surg 11:1–10CrossRefGoogle Scholar
  2. 2.
    Azizi S, Imani F, Kwak JT, Tahmasebi A, Xu S, Yan P, Kruecker J, Turkbey B, Choyke P, Pinto P, Wood B, Mousavi P, Abolmaesumi P (2016) Classifying cancer grades using temporal ultrasound for transrectal prostate biopsy. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 653–661Google Scholar
  3. 3.
    Azizi S, Imani F, Zhuang B, Tahmasebi A, Kwak JT, Xu S, Uniyal N, Turkbey B, Choyke P, Pinto P, Wood B, Mousavi P, Abolmaesumi P (2015) Ultrasound-based detection of prostate cancer using automatic feature selection with deep belief networks. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 70–77Google Scholar
  4. 4.
    Bengio Y (2012) Deep learning of representations for unsupervised and transfer learning. Unsuperv Transf Learn Chall Mach Learn 7:19Google Scholar
  5. 5.
    Conjeti S, Katouzian A, Roy AG, Peter L, Sheet D, Carlier S, Laine A, Navab N (2016) Supervised domain adaptation of decision forests: transfer of models trained in vitro for in vivo intravascular ultrasound tissue characterization. Med Image Anal 32:1–17CrossRefPubMedGoogle Scholar
  6. 6.
    Daoud MI, Mousavi P, Imani F, Rohling R, Abolmaesumi P (2013) Tissue classification using ultrasound-induced variations in acoustic backscattering features. IEEE Trans Biomed Eng 60(2):310–320CrossRefPubMedGoogle Scholar
  7. 7.
    Epstein JI, Feng Z, Trock BJ, Pierorazio PM (2012) Upgrading and downgrading of prostate cancer from biopsy to radical prostatectomy: incidence and predictive factors using the modified gleason grading system and factoring in tertiary grades. Eur Urol 61(5):1019–1024CrossRefPubMedPubMedCentralGoogle Scholar
  8. 8.
    Feleppa E, Porter C, Ketterling J, Dasgupta S, Ramachandran S, Sparks D (2007) Recent advances in ultrasonic tissue-type imaging of the prostate. In: Acoustical imaging. Springer, Netherlands, pp 331–339Google Scholar
  9. 9.
    Fernando B, Habrard A, Sebban M, Tuytelaars T (2013) Unsupervised visual domain adaptation using subspace alignment. In: Proceedings of the IEEE international conference on computer vision, pp 2960–2967Google Scholar
  10. 10.
    Glorot X, Bordes A, Bengio Y (2011) Domain adaptation for large-scale sentiment classification: a deep learning approach. In: Proceedings of the 28th international conference on machine learning (ICML-11), pp 513–520Google Scholar
  11. 11.
    Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: Computer vision and pattern recognition (CVPR), 2012 IEEE conference on, IEEE, pp 2066–2073Google Scholar
  12. 12.
    Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554CrossRefPubMedGoogle Scholar
  13. 13.
    Imani F, Abolmaesumi P, Gibson E, Khojaste A, Gaed M, Moussa M, Gomez JA, Romagnoli C, Leveridge M, Chang S (2015) Computer-aided prostate cancer detection using ultrasound rf time series: in vivo feasibility study. IEEE Trans Med Imaging 34(11):2248–2257CrossRefPubMedGoogle Scholar
  14. 14.
    Imani F, Ramezani M, Nouranian S, Gibson E, Khojaste A, Gaed M, Moussa M, Gomez JA, Romagnoli C, Leveridge M (2015) Ultrasound-based characterization of prostate cancer using joint independent component analysis. IEEE Trans Biomed Eng 62(7):1796–1804CrossRefPubMedGoogle Scholar
  15. 15.
    Imani F, Zhuang B, Tahmasebi A, Kwak JT, Xu S, Agarwal H, Bharat S, Uniyal N, Turkbey IB, Choyke P, Pinto P (2015) Augmenting mri-transrectal ultrasound-guided prostate biopsy with temporal ultrasound data: a clinical feasibility study. Int J Comput Assist Radiol Surg 10(6):727–735CrossRefPubMedGoogle Scholar
  16. 16.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105Google Scholar
  17. 17.
    Mirzaalian H, Ning L, Savadjiev P, Pasternak O, Bouix S, Michailovich O, Grant G, Marx C, Morey R, Flashman L (2016) Inter-site and inter-scanner diffusion MRI data harmonization. NeuroImage 135:311–323CrossRefPubMedGoogle Scholar
  18. 18.
    Moradi M, Abolmaesumi P, Mousavi P (2010) Tissue typing using ultrasound RF time series: experiments with animal tissue samples. Med Phys 37(8):4401–4413CrossRefPubMedGoogle Scholar
  19. 19.
    Moradi M, Abolmaesumi P, Siemens DR, Sauerbrei EE, Boag AH, Mousavi P (2009) Augmenting detection of prostate cancer in transrectal ultrasound images using svm and RF time series. IEEE Trans Biomed Eng 56(9):2214–2224CrossRefPubMedGoogle Scholar
  20. 20.
    Moradi M, Mahdavi SS, Nir G, Jones EC, Goldenberg SL, Salcudean SE (2013) Ultrasound RF time series for tissue typing: first in vivo clinical results. In: SPIE medical imaging. International society for optics and photonics, pp 86,701I–86,701IGoogle Scholar
  21. 21.
    Moradi M, Mousavi P, Abolmaesumi P (2007) Computer-aided diagnosis of prostate cancer with emphasis on ultrasound-based approaches: a review. Ultrasound Medicine Biol 33(7):1010–1028CrossRefGoogle Scholar
  22. 22.
    Oelze ML, O’Brien WD, Blue JP, Zachary JF (2004) Differentiation and characterization of rat mammary fibroadenomas and 4t1 mouse carcinomas using quantitative ultrasound imaging. IEEE Trans Med Imaging 23(6):764–771CrossRefPubMedGoogle Scholar
  23. 23.
    Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359CrossRefGoogle Scholar
  24. 24.
    Razavian A, Azizpour H, Sullivan J, Carlsson S (2014) CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 806–813Google Scholar
  25. 25.
    Seabra J, Sanches JM (2012) RF ultrasound estimation from b-mode images. In: Ultrasound imaging. Springer, US, pp 3–24Google Scholar
  26. 26.
    Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 99:1–1Google Scholar
  27. 27.
    Tanaka M, Okutomi M (2014) A novel inference of a restricted Boltzmann machine. In: International conference on pattern recognition (ICPR), 2014 22nd, IEEE, pp 1526–1531Google Scholar
  28. 28.
    van Engelen A, van Dijk AC, Truijman MT, van’t Klooster R, van Opbroek A, van der Lugt A, Niessen WJ, Kooi ME, de Bruijne M (2015) Multi-center MRI carotid plaque component segmentation using feature normalization and transfer learning. IEEE Trans Med Imaging 34(6):1294–1305CrossRefPubMedGoogle Scholar
  29. 29.
    Van Opbroek A, Ikram MA, Vernooij MW, De Bruijne M (2015) Transfer learning improves supervised image segmentation across imaging protocols. IEEE Trans Med Imaging 34(5):1018–1030CrossRefPubMedGoogle Scholar
  30. 30.
    Zhuang F, Cheng X, Luo P, Pan SJ, He Q (2015) Supervised representation learning: transfer learning with deep autoencoders. In: Int. Joint Conf. Artif. IntellGoogle Scholar

Copyright information

© CARS 2017

Authors and Affiliations

  • Shekoofeh Azizi
    • 1
    Email author
  • Parvin Mousavi
    • 2
  • Pingkun Yan
    • 3
  • Amir Tahmasebi
    • 3
  • Jin Tae Kwak
    • 4
  • Sheng Xu
    • 5
  • Baris Turkbey
    • 5
  • Peter Choyke
    • 5
  • Peter Pinto
    • 5
  • Bradford Wood
    • 5
  • Purang Abolmaesumi
    • 1
  1. 1.The University of British ColumbiaVancouverCanada
  2. 2.Queen’s UniversityKingstonCanada
  3. 3.Philips Research North AmericaCambridgeUSA
  4. 4.Sejong UniversityGwangjin-GuKorea
  5. 5.National Institutes of HealthBethesdaUSA

Personalised recommendations