Skip to main content
SpringerLink
Log in

A Comparative Study of Modern Machine Learning Approaches for Focal Lesion Detection and Classification in Medical Images: BoVW, CNN and MTANN

  • Chapter
  • First Online:
Artificial Intelligence in Decision Support Systems for Diagnosis in Medical Imaging

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 140))

Abstract

Two dominant classes of modern approaches for the detection and classification of focal lesions are a bag of visual words and end-to-end learning machines. In this study, we reviewed and compared these approaches for lung nodule detection, colorectal polyp detection, and lung nodule classification in CT images. Specifically, we considered massive-training artificial neural networks (MTANNs) and convolutional neural networks (CNNs) as representatives of end-to-end learning machines, and Fisher vectors as a representative of the bag of visual words. We first compared CNNs with Fisher vectors in nodule detection, nodule classification, and polyp detection, concluding that the best performing CNN model achieved comparable performance to that of Fisher vectors. We also analyzed the performance of CNNs with varying depths for the 3 studied applications. Our experiments showed that the CNN architectures with 3 or 4 convolutional layers were more effective than shallower architectures, but we did not observe a further performance gain by using deeper architectures. We then compared CNNs with MTANNs, concluding that MTANNs outperformed CNNs for nodule detection and classification particularly given limited training data. Specifically, for nodule detection, the MTANNs generated 0.08 false positives per section at 100% sensitivity, which was significantly (p < 0.05) lower than the best performing CNN model with 0.67 false positives per section at the same level of sensitivity. We showed that the best performing CNN model achieved comparable performance to that of Fisher vectors in the 3 studied applications, and that MTANNs outperformed CNNs in nodule detection and classification, especially given limited training data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Suzuki, K., Armato III, S.G., Li, F., Sone, S., Doi, K.: Massive training artificial neural network (mtann) for reduction of false positives in computerized detection of lung nodules in low-dose computed tomography. Med. Phys. 30(7), 1602–1617 (2003)

    Article  Google Scholar 

  2. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  3. Perronnin, F., Sánchez, J., Mensink, T.: Improving the fisher kernel for large-scale image classification. In: Computer Vision–ECCV 2010, pp. 143–156. Springer (2010)

    Google Scholar 

  4. Suzuki, K., Horiba, I., Sugie, N.: Efficient approximation of neural filters for removing quantum noise from images. IEEE Trans. Signal Process. 50(7), 1787–1799 (2002)

    Article  Google Scholar 

  5. Suzuki, K., Horiba, I., Sugie, N., Nanki, M.: Neural filter with selection of input features and its application to image quality improvement of medical image sequences. IEICE Trans. Inf. Syst. 85(10), 1710–1718 (2002)

    Google Scholar 

  6. Suzuki, K., Li, F., Sone, S., et al.: Computer-aided diagnostic scheme for distinction between benign and malignant nodules in thoracic low-dose ct by use of massive training artificial neural network. IEEE Trans. Med. Imag. 24(9), 1138–1150 (2005)

    Article  Google Scholar 

  7. Suzuki, K., Yoshida, H., Näppi, J., Dachman, A.H.: Massive-training artificial neural network (mtann) for reduction of false positives in computer-aided detection of polyps: suppression of rectal tubes. Med. Phys. 33(10), 3814–3824 (2006)

    Article  Google Scholar 

  8. Suzuki, K., Yoshida, H., Näppi, J., Armato III, S.G., Dachman, A.H.: Mixture of expert 3d massive-training anns for reduction of multiple types of false positives in cad for detection of polyps in ct colonography. Med. Phys. 35(2), 694–703 (2008)

    Article  Google Scholar 

  9. Suzuki, K., Rockey, D.C., Dachman, A.H.: Ct colonography: advanced computer-aided detection scheme utilizing mtanns for detection of missed polyps in a multicenter clinical trial. Med. Phys. 37(1), 12–21 (2010)

    Article  Google Scholar 

  10. Suzuki, K., Zhang, J., Xu, J.: Massive-training artificial neural network coupled with laplacian-eigenfunction-based dimensionality reduction for computer-aided detection of polyps in ct colonography. IEEE Trans. Med. Imag. 29(11), 1907–1917 (2010)

    Article  Google Scholar 

  11. Xu, J.-W., Suzuki, K.: Massive-training support vector regression and gaussian process for false-positive reduction in computer-aided detection of polyps in ct colonography. Med. Phys. 38(4), 1888–1902 (2011)

    Article  Google Scholar 

  12. Fukushima, K.: Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern. 36(4), 193–202 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  13. Fukushima, K.: Neocognitron capable of incremental learning. Neural Netw. 17(1), 37–46 (2004)

    Article  MATH  Google Scholar 

  14. Deutsch, S.: A simplified version of kunihiko fukushima’s neocognitron. Biol. Cybern. 42(1), 17–21 (1981)

    Article  Google Scholar 

  15. LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)

    Article  Google Scholar 

  16. Le Cun, B.B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Handwritten digit recognition with a back-propagation network. In: Advances in Neural Information Processing Systems. Citeseer (1990)

    Google Scholar 

  17. Tajbakhsh, N., Gotway, M.B., Liang, J.: Computer-aided pulmonary embolism detection using a novel vessel-aligned multi-planar image representation and convolutional neural networks. In: Medical Image Computing and Computer-Assisted Intervention MICCAI 2015 (2015)

    Google Scholar 

  18. Shen, W., Zhou, M., Yang, F., Yang, C., Tian, J.: Multi-scale convolutional neural networks for lung nodule classification. In: Ourselin, S., Alexander, D.C., Westin, C.-F., Cardoso, M.J. (eds.), Information Processing in Medical Imaging. Lecture Notes in Computer Science, vol. 9123, pp. 588–599. Springer International Publishing (2015)

    Google Scholar 

  19. Roth, H., Lu, L., Seff, A., Cherry, K., Hoffman, J., Wang, S., Liu, J., Turkbey, E., Summers, R.: A new 2.5d representation for lymph node detection using random sets of deep convolutional neural network observations. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.), Medical Image Computing and Computer-Assisted Intervention MICCAI 2014. Lecture Notes in Computer Science, vol. 8673, pp. 520–527. Springer International Publishing (2014)

    Google Scholar 

  20. Tajbakhsh, N., Gurudu, S.R., Liang, J.: A comprehensive computer-aided polyp detection system for colonoscopy videos. In: Information Processing in Medical Imaging, pp. 327–338. Springer (2015)

    Google Scholar 

  21. Zhang, W., Li, R., Deng, H., Wang, L., Lin, W., Ji, S., Shen, D.: Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. NeuroImage 108, 214–224 (2015)

    Article  Google Scholar 

  22. Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.-M., Larochelle, H.: Brain tumor segmentation with deep neural networks (2015). arXiv:1505.03540

  23. Tajbakhsh, N., Shin, J.Y., Gurudu, S.R., Hurst, R.T., Kendall, C.B., Gotway, M.B., Liang, J.: Convolutional neural networks for medical image analysis: Fine tuning or full training? IEEE Trans. Med. Imag. 35, 1299–1312 (2016)

    Article  Google Scholar 

  24. Tajbakhsh, N., Gurudu, S.R., Liang, J.: Automatic polyp detection in colonoscopy videos using an ensemble of convolutional neural networks. In: 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), pp. 79–83. IEEE (2015)

    Google Scholar 

  25. Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)

    Google Scholar 

  26. Fei-Fei, L., Perona, P.: A bayesian hierarchical model for learning natural scene categories. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 2, pp. 524–531. IEEE (2005)

    Google Scholar 

  27. Song, Y., Cai, W., Zhang, F., Huang, H., Zhou, Y., Feng, D.D.: Bone texture characterization with fisher encoding of local descriptors. In: 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), pp. 5–8. IEEE (2015)

    Google Scholar 

  28. Kwitt, R., Hegenbart, S., Rasiwasia, N., Vécsei, A., Uhl, A.: Do we need annotation experts? a case study in celiac disease classification. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2014, pp. 454–461. Springer (2014)

    Google Scholar 

  29. Twinanda, A.P., De Mathelin, M., Padoy, N.: Fisher kernel based task boundary retrieval in laparoscopic database with single video query. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2014, pp. 409–416. Springer (2014)

    Google Scholar 

  30. Manivannan, S., Wang, R., Trucco, E.: Inter-cluster features for medical image classification. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2014, pp. 345–352. Springer (2014)

    Google Scholar 

  31. Lowe, D.G.: Object recognition from local scale-invariant features. In: The Proceedings of the Seventh IEEE International Conference on Computer vision, 1999, vol. 2, pp. 1150–1157. IEEE (1999)

    Google Scholar 

  32. Sone, S., Takashima, S., Li, F., Yang, Z., Honda, T., Maruyama, Y., Hasegawa, M., Yamanda, T., Kubo, K., Hanamura, K., et al.: Mass screening for lung cancer with mobile spiral computed tomography scanner. The Lancet 351(9111), 1242–1245 (1998)

    Article  Google Scholar 

  33. Li, F., Sone, S., Abe, H., MacMahon, H., Armato, S.G., Doi, K.: Lung cancers missed at low-dose helical ct screening in a general population: comparison of clinical, histopathologic, and imaging findings 1. Radiology 225(3), 673–683 (2002)

    Article  Google Scholar 

  34. Näppi, J., Yoshida, H.: Feature-guided analysis for reduction of false positives in cad of polyps for computed tomographic colonography. Med. Phys. 30(7), 1592–1601 (2003)

    Article  Google Scholar 

  35. Yoshida, H., Näppi, J.: Three-dimensional computer-aided diagnosis scheme for detection of colonic polyps. IEEE Trans. Med. Imag. 20(12), 1261–1274 (2001)

    Article  Google Scholar 

  36. Egan, J.P., Greenberg, G.Z., Schulman, A.I.: Operating characteristics, signal detectability, and the method of free response. J. Acoust. Soc. Am. 33(8), 993–1007 (1961)

    Article  Google Scholar 

  37. Chakraborty, D.P., Berbaum, K.S.: Observer studies involving detection and localization: modeling, analysis, and validation. Med. Phys. 31(8), 2313–2330 (2004)

    Article  Google Scholar 

  38. Zhai, X., Chakraborty, D.: RJafroc: Analysis of Data Acquired Using the Receiver Operating Characteristic Paradigm and Its Extensions (2015). R package version 0.1.1

    Google Scholar 

  39. Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., Mller, M.: Proc: an open-source package for r and s+ to analyze and compare roc curves. BMC Bioinform. 12(77) (2011)

    Google Scholar 

  40. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding (2014). arXiv:1408.5093

  41. Vedaldi, A., Fulkerson, B.: VLFeat: an open and portable library of computer vision algorithms (2008). http://www.vlfeat.org/

  42. Edwards, D.C., Kupinski, M.A., Metz, C.E., Nishikawa, R.M.: Maximum likelihood fitting of froc curves under an initial-detection-and-candidate-analysis model. Med. Phys. 29(12), 2861–2870 (2002)

    Article  Google Scholar 

  43. van Ginneken, B., Setio, A.A., Jacobs, C., Ciompi, F.: Off-the-shelf convolutional neural network features for pulmonary nodule detection in computed tomography scans. In: 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), pp. 286–289, April 2015

    Google Scholar 

  44. Simonyan, K., Vedaldi, A., Zisserman, A.: Deep fisher networks for large-scale image classification. In: Advances in Neural Information Processing Systems, pp. 163–171 (2013)

    Google Scholar 

  45. Sydorov, V., Sakurada, M., Lampert, C.: Deep fisher kernels-end to end learning of the fisher kernel gmm parameters. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1402–1409 (2014)

    Google Scholar 

  46. Perronnin, F., Larlus, D.: Fisher vectors meet neural networks: A hybrid classification architecture. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3743–3752 (2015)

    Google Scholar 

  47. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Computer Vision–ECCV 2014, pp. 818–833. Springer (2014)

    Google Scholar 

  48. Eigen, D., Rolfe, J., Fergus, R., LeCun, Y.: Understanding deep architectures using a recursive convolutional network (213). arXiv:1312.1847

  49. Li, Q., Li, F., Shiraishi, J., Katsuragawa, S., Sone, S., Doi, K.: Investigation of new psychophysical measures for evaluation of similar images on thoracic computed tomography for distinction between benign and malignant nodules. Med. Phys. 30(10), 2584–2593 (2003)

    Article  Google Scholar 

  50. Margeta, J., Criminisi, A., Cabrera Lozoya, R., Lee, D.C., Ayache, N.: Fine-tuned convolutional neural nets for cardiac mri acquisition plane recognition. Comput. Methods Biomech. Biomed. Eng. Imag. Vis., 1–11 (2015)

    Google Scholar 

  51. Bar, Y., Diamant, I., Wolf, L., Greenspan, H.: Deep learning with non-medical training used for chest pathology identification, vol. 9414, pp. 94140V–94140V–7 (2015)

    Google Scholar 

  52. Kobetski, M., Sullivan, J.: Improved boosting performance by explicit handling of ambiguous positive examples. In: Pattern Recognition Applications and Methods. Advances in Intelligent Systems and Computing, vol. 318, pp. 17–37 (2015)

    Google Scholar 

  53. Ba, J., Caruana, R.: Do deep nets really need to be deep? In: Advances in Neural Information Processing Systems, pp. 2654–2662 (2014)

    Google Scholar 

  54. Suzuki, K., Doi, K.: How can a massive training artificial neural network (mtann) be trained with a small number of cases in the distinction between nodules and vessels in thoracic ct? 1. Acad. Radiol. 12(10), 1333–1341 (2005)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Medical Imaging Research Center & Department of Electrical and Computer Engineering, Illinois Institute of Technology, 3440 S. Dearborn St., Chicago, IL, 60616, USA

    Nima Tajbakhsh & Kenji Suzuki

Authors
  1. Nima Tajbakhsh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kenji Suzuki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nima Tajbakhsh .

Editor information

Editors and Affiliations

  1. Medical Imaging Research Center and Department of Electrical and Computer Engineering, Illinois Institute of Technology, Chicago, Illinois, USA

    Kenji Suzuki

  2. Key Laboratory of Machine Perception (Ministry of Education), School of Electronics Engineering and Computer Science, Peking University, Beijing, China

    Yisong Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Tajbakhsh, N., Suzuki, K. (2018). A Comparative Study of Modern Machine Learning Approaches for Focal Lesion Detection and Classification in Medical Images: BoVW, CNN and MTANN. In: Suzuki, K., Chen, Y. (eds) Artificial Intelligence in Decision Support Systems for Diagnosis in Medical Imaging. Intelligent Systems Reference Library, vol 140. Springer, Cham. https://doi.org/10.1007/978-3-319-68843-5_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-68843-5_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-68842-8

  • Online ISBN: 978-3-319-68843-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation