Food and Environmental Virology

, Volume 10, Issue 2, pp 201–208 | Cite as

Virus Particle Detection by Convolutional Neural Network in Transmission Electron Microscopy Images

  • Eisuke Ito
  • Takaaki Sato
  • Daisuke Sano
  • Etsuko Utagawa
  • Tsuyoshi KatoEmail author
Original Paper


A new computational method for the detection of virus particles in transmission electron microscopy (TEM) images is presented. Our approach is to use a convolutional neural network that transforms a TEM image to a probabilistic map that indicates where virus particles exist in the image. Our proposed approach automatically and simultaneously learns both discriminative features and classifier for virus particle detection by machine learning, in contrast to existing methods that are based on handcrafted features that yield many false positives and require several postprocessing steps. The detection performance of the proposed method was assessed against a dataset of TEM images containing feline calicivirus particles and compared with several existing detection methods, and the state-of-the-art performance of the developed method for detecting virus was demonstrated. Since our method is based on supervised learning that requires both the input images and their corresponding annotations, it is basically used for detection of already-known viruses. However, the method is highly flexible, and the convolutional networks can adapt themselves to any virus particles by learning automatically from an annotated dataset.


Feline calicivirus Virus detection Transmission electron microscopy Image processing Convolutional neural network Machine learning 



This work was supported by the Japan Society for the Promotion of Science through Grants-in-Aid for Scientific Research (A) (17H01299) and (C) (15K00591).

Supplementary material

12560_2018_9335_MOESM1_ESM.pdf (282 kb)
Supplementary material 1 (PDF 281 kb)


  1. Bosch, A., Guix, S., Sano, D., & Pinto, R. M. (2008). New tools for the study and direct surveillance of viral pathogens in water. Current Opinion in Biotechnology, 19, 295–310.CrossRefPubMedGoogle Scholar
  2. Bradski, G. (2000). The OpenCV library. Dr. Dobb’s Journal of Software Tools, 120, 122–125.Google Scholar
  3. Ciresan, D., Giusti, A., Gambardella, L. M., & Schmidhuber, J. (2012). Deep neural networks segment neuronal membranes in electron microscopy images. In Proceedings of the Advances in Neural Information Processing Systems (pp. 2843–2851).Google Scholar
  4. Ciresan, D. C., Giusti, A., Gambardella, L. M., & Schmidhuber, J. (2013). Mitosis detection in breast cancer histology images with deep neural networks. In International Conference on Medical Image Computing and Computer-assisted Intervention (pp. 411–418). Berlin, Heidelberg: Springer.Google Scholar
  5. Doane, F. W. (1980). Virus morphology as an aid for rapid diagnosis. The Yale Journal of Biology and Medicine, 53(1), 19–25.PubMedPubMedCentralGoogle Scholar
  6. Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., et al. (2017). Brain tumor segmentation with deep neural networks. Medical Image Analysis, 35, 18–31.CrossRefPubMedGoogle Scholar
  7. Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012). Improving neural networks by preventing co-adaptation of feature detectors. arXiv Preprint arXiv:1207.0580.
  8. Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv Preprint arXiv:1502.03167.
  9. Ishii, S., Kitamura, G., Segawa, T., Kobayashi, A., Miura, T., Sano, D., et al. (2014). Microfluidic quantitative PCR for simultaneous quantification of multiple viruses in environmental water samples. Applied and Environmental Microbiology, 80(24), 7505–7511.CrossRefPubMedPubMedCentralGoogle Scholar
  10. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Proceedings of the Advances in Neural Information Processing Systems (pp. 1097–1105).Google Scholar
  11. Kylberg, G., Uppström, M., Hedlund, K. O., Borgefors, G., & Sintorn, I. M. (2012). Segmentation of virus particle candidates in transmission electron microscopy images. Journal of Microscopy, 245(2), 140–147.CrossRefPubMedGoogle Scholar
  12. Kylberg, G., Uppström, M., & Sintorn, I. M. (2011). Virus texture analysis using local binary patterns and radial density profiles. In Proceedings of the Iberoamerican Congress on Pattern Recognition (pp. 573–580). Berlin, Heidelberg: Springer.Google Scholar
  13. LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.CrossRefGoogle Scholar
  14. Martin, I. M. B., Marinescu, D. C., Lynch, R. E., & Baker, T. S. (1997). Identification of spherical virus particles in digitized images of entire electron micrographs. Journal of Structural Biology, 120(2), 146–157.CrossRefGoogle Scholar
  15. Matuszewski, B. J. & Shark, L. K. (2001). Hierarchical iterative Bayesian approach to automatic recognition of biological viruses in electron microscope images. In Proceedings of the 2001 International Conference on Image Processing (Vol. 2, pp. 347–350). IEEE.Google Scholar
  16. Matuszewski, B., Shark, L. K., & Hall, G. (1997). A novel segmentation and classification method for identification of viruses in electron microscope images. In Proceedings of the Sixth International Conference on Image Processing and Its Applications (Vol. 2, pp. 819–823). IET.Google Scholar
  17. Nicholson, W. V., & Glaeser, R. M. (2001). Review: Automatic particle detection in electron microscopy. Journal of Structural Biology, 133(2–3), 90–101.CrossRefPubMedGoogle Scholar
  18. Ong, H., & Chandran, V. (2005). Identification of gastroenteric viruses by electron microscopy using higher order spectral features. Journal of Clinical Virology, 34(3), 195–206.CrossRefPubMedGoogle Scholar
  19. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.Google Scholar
  20. Proença, Md C M S, Nunes, J. F. M., & de Matos, A. P. A. (2013). Automatic virus particle selection—the entropy approach. IEEE Transactions on Image Processing, 22(5), 1996–2003.CrossRefGoogle Scholar
  21. Roingeard, P. (2008). Viral detection by electron microscopy: Past, present and future. Biology of the Cell, 100(8), 491–501.CrossRefPubMedGoogle Scholar
  22. Ryner, M., Strömberg, J. O., Söderberg-Nauclér, C., & Homman-Loudiyi, M. (2006). Identification and classification of human cytomegalovirus capsids in textured electron micrographs using deformed template matching. Virology Journal, 3(1), 57.CrossRefPubMedPubMedCentralGoogle Scholar
  23. Schramlová, J., Arientova, S., & Hulinska, D. (2010). The role of electron microscopy in the rapid diagnosis of viral infections—review. Folia Microbiologica, 55(1), 88–101.CrossRefPubMedGoogle Scholar
  24. Sintorn, I. M., Homman-Loudiyi, M., Söderberg-Nauclér, C., & Borgefors, G. (2004). A refined circular template matching method for classification of human cytomegalovirus capsids in TEM images. Computer Methods and Programs in Biomedicine, 76(2), 95–102.CrossRefPubMedGoogle Scholar
  25. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1), 1929–1958.Google Scholar
  26. Thi Nguyen, H.-T., Nakagomi, T., Sano, D., Sherchand, J. B., Pandey, B. D., Cunliffe, N. A., et al. (2015). Molecular epidemiology of noroviruses detected in Nepalese children with acute diarrhea between 2005 and 2011: Increase and predominance of minor genotype GII.13. Infection, Genetics and Evolution, 30, 27–36.CrossRefGoogle Scholar
  27. van der Walt, S., Schönberger, J. L., Nunez-Iglesias, J., Boulogne, F., Warner, J. D., Yager, N., et al. (2014). scikit-image: Image processing in Python. PeerJ, 2, e453.CrossRefPubMedPubMedCentralGoogle Scholar
  28. Wan, L., Zeiler, M., Zhang, S., Cun, Y. L., & Fergus, R. (2013). Regularization of neural networks using DropConnect. In Proceedings of the 30th International Conference on Machine Learning (ICML-13) (pp. 1058–1066).Google Scholar
  29. Wu, H., & Gu, X. (2015). Towards dropout training for convolutional neural networks. Neural Networks, 71, 1–10.CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Eisuke Ito
    • 1
  • Takaaki Sato
    • 1
  • Daisuke Sano
    • 2
  • Etsuko Utagawa
    • 3
  • Tsuyoshi Kato
    • 1
    • 4
    • 5
    Email author
  1. 1.Division of Electronics and Informatics, Faculty of Science and TechnologyGunma UniversityKiryuJapan
  2. 2.Department of Civil and Environmental Engineering, Graduate School of EngineeringTohoku UniversitySendaiJapan
  3. 3.Laboratory of Viral Infection I, Graduate School of Infection Control Sciences, Kitasato Institute for Life SciencesKitasato UniversityTokyoJapan
  4. 4.Center for Research on Adoption of NextGen Transportation Systems (CRANTS)Gunma UniversityKiryuJapan
  5. 5.Integrated Institute for Regulatory ScienceWaseda UniversityTokyoJapan

Personalised recommendations