Skip to main content
SpringerLink
Log in
Book cover

International Conference on Image Analysis and Processing

ICIAP 2022: Image Analysis and Processing. ICIAP 2022 Workshops pp 533–544Cite as

Enabling Efficient Training of Convolutional Neural Networks for Histopathology Images

Part of the Lecture Notes in Computer Science book series (LNCS,volume 13373)

Abstract

Convolutional Neural Networks (CNNs) have gained lots of attention in various digital imaging applications. They have proven to produce incredible results, especially on big data, that require high processing demands. With the increasing size of datasets, especially in computational pathology, CNN processing takes even longer and uses higher computational resources. Considerable research has been conducted to improve the efficiency of CNN, such as quantization. This paper aims to apply efficient training and inference of ResNet using quantization on histopathology images, the Patch Camelyon (PCam) dataset. An analysis for efficient approaches to classify histopathology images is presented. First, the original RGB-colored images are evaluated. Then, compression methods such as channel reduction and sparsity are applied. When comparing sparsity on grayscale with RGB modes, classification accuracy is relatively the same, but the total number of MACs is less in sparsity on grayscale by 77% than RGB. A higher classification result was achieved by grayscale mode, which requires much fewer MACs than the original RGB mode. Our method’s low energy and processing make this project suitable for inference on wearable healthcare low powered devices and mobile hospitals in rural areas or developing countries. This also assists pathologists by presenting a preliminary diagnosis.

Keywords

  • Deep learning
  • Quantization
  • Computational Pathology

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Araújo, T., et al.: Classification of breast cancer histology images using convolutional neural networks. PLoS ONE 12(6), e0177544 (2017). https://doi.org/10.1371/journal.pone.0177544

    CrossRef  Google Scholar 

  2. Balzer, W., et al.: Weight quantization in Boltzmann machines. Neural Netw. 4(3), 405–409 (1991). https://doi.org/10.1016/0893-6080(91)90077-I

    CrossRef  Google Scholar 

  3. Beevi, K.S., et al.: Automatic mitosis detection in breast histopathology images using convolutional neural network based deep transfer learning. Biocybern. Biomed. Eng. 39(1), 214–223 (2019). https://doi.org/10.1016/j.bbe.2018.10.007

    CrossRef  Google Scholar 

  4. Bejnordi, B.E., et al.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318(22), 2199–2210 (2017). https://doi.org/10.1001/jama.2017.14585

    CrossRef  Google Scholar 

  5. Chakradhar, S., et al.: A dynamically configurable coprocessor for convolutional neural networks. In: Proceedings of the 37th Annual International Symposium on Computer Architecture, pp. 247–257 (2010). https://doi.org/10.1145/1815961.1815993

  6. Chen, T., et al.: DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning. ACM SIGARCH Comput. Archit. News 42(1), 269–284 (2014). https://doi.org/10.1145/2654822.2541967

    CrossRef  Google Scholar 

  7. Courbariaux, M., et al.: BinaryConnect: training deep neural networks with binary weights during propagations. In: Advances in Neural Information Processing Systems, pp. 3123–3131 (2015). 10.48550/arXiv. 1511.00363

    Google Scholar 

  8. Courbariaux, M., et al.: Binarized neural networks: training deep neural networks with weights and activations constrained to +1 or \(-\)1. arXiv preprint arXiv:1602.02830 (2016). https://doi.org/10.48550/arXiv.1602.02830

  9. Cruz-Roa, A., et al.: High-throughput adaptive sampling for whole-slide histopathology image analysis (HASHI) via convolutional neural networks: application to invasive breast cancer detection. PloS One 13(5) (2018). https://doi.org/10.1371/journal.pone.0196828

  10. Du, Z., et al.: ShiDianNao: shifting vision processing closer to the sensor. In: Proceedings of the 42nd Annual International Symposium on Computer Architecture, pp. 92–104 (2015). https://doi.org/10.1145/2749469.2750389

  11. Farabet, C., et al.: CNP: an FPGA-based processor for convolutional networks. In: 2009 Intl Conference on Field Programmable Logic and Applications, pp. 32–37. IEEE (2009). https://doi.org/10.1109/FPL.2009.5272559

  12. Fiesler, E., et al.: Weight discretization paradigm for optical neural networks. In: Optical Interconnections and Networks, vol. 1281, pp. 164–173. Intl Society for Optics and Photonics (1990). https://doi.org/10.1117/12.20700

  13. Graves, A., et al.: Speech recognition with deep recurrent neural networks. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 6645–6649. IEEE (2013). https://doi.org/10.1109/ICASSP.2013.6638947

  14. Gurcan, M.N., et al.: Histopathological image analysis: a review. IEEE Rev. Biomed. Eng. 2, 147–171 (2009). https://doi.org/10.1109/RBME.2009.2034865

    CrossRef  Google Scholar 

  15. He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90

  16. Horowitz, M.: 1.1 computing’s energy problem (and what we can do about it). In: 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC), pp. 10–14. IEEE (2014). https://doi.org/10.1109/ISSCC.2014.6757323

  17. Horowitz, M.: Energy table for 45nm process. In: Stanford VLSI wiki (2014)

    Google Scholar 

  18. Kim, M., Kim, J., Lee, D., Son, J., Lee, W.: A study on autoplay model using DNN in turn-based RPG. In: Bhatia, S.K., Tiwari, S., Ruidan, S., Trivedi, M.C., Mishra, K.K. (eds.) Advances in Computer, Communication and Computational Sciences. AISC, vol. 1158, pp. 399–407. Springer, Singapore (2021). https://doi.org/10.1007/978-981-15-4409-5_36

    CrossRef  Google Scholar 

  19. Komura, D., Ishikawa, S.: Machine learning methods for histopathological image analysis. Comput. Struct. Biotechnol. J. 16, 34–42 (2018). https://doi.org/10.1016/j.csbj.2018.01.001

    CrossRef  Google Scholar 

  20. Korbar, B., et al.: Deep learning for classification of colorectal polyps on whole-slide images. J. Pathol. Inform. 8 (2017). https://doi.org/10.4103/jpi.jpi_34_17

  21. Krizhevsky, A., et al.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems 25, pp. 1097–1105 (2012). https://doi.org/10.1145/3065386

  22. Landolt, S., et al.: A taxonomy for deep learning in natural language processing. In: Hawaii International Conference on System Sciences (2021). https://doi.org/10.24251/HICSS.2021.129

  23. Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017). https://doi.org/10.1016/j.media.2017.07.005

    CrossRef  Google Scholar 

  24. Mejbri, S., et al.: Deep analysis of CNN settings for new cancer whole-slide histological images segmentation: the case of small training sets. In: 6th International Conference on BioImaging (BIOIMAGING 2019), pp. 120–128 (2019). https://doi.org/10.5220/0007406601200128

  25. Mohamed, M., et al.: A data and compute efficient design for limited-resources deep learning. arXiv preprint arXiv:2004.09691 (2020)

  26. Parashar, A., et al.: Automated guided autonomous car using deep learning and computer vision. In: Autonomous Driving and Advanced Driver-Assistance Systems (ADAS): Apps, Development, Legal Issues, and Testing, p. 219 (2021). https://doi.org/10.1201/9781003048381-10

  27. Qadeer, W., et al.: Convolution engine: balancing efficiency & flexibility in specialized computing. In: Proceedings of the 40th Annual International Symposium on Computer Architecture, pp. 24–35 (2013). https://doi.org/10.1145/2485922.2485925

  28. Qiu, J., et al.: Going deeper with embedded FPGA platform for convolutional neural network. In: Proceedings of the 2016 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, pp. 26–35 (2016). https://doi.org/10.1145/2847263.2847265

  29. Roohi, A., et al.: ApGAN: approximate GAN for robust low energy learning from imprecise components. IEEE Trans. Comput. 69(3), 349–360 (2019). https://doi.org/10.1109/TC.2019.2949042

    CrossRef  MATH  Google Scholar 

  30. Roohi, A., et al.: Processing-in-memory acceleration of convolutional neural networks for energy-efficiency, and power-intermittency resilience. In: 20th International Symposium on Quality Electronic Design (ISQED), pp. 8–13. IEEE (2019). https://doi.org/10.1109/ISQED.2019.8697572

  31. Roohi, A., et al.: RNSim: efficient deep neural network accelerator using residue number systems. In: 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD), pp. 1–9. IEEE (2021). https://doi.org/10.1109/ICCAD51958.2021.9643531

  32. Sharma, H., et al.: Bit fusion: bit-level dynamically composable architecture for accelerating deep neural network. In: 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA), pp. 764–775. IEEE (2018). https://doi.org/10.1109/ISCA.2018.00069

  33. Shen, D., et al.: Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19, 221–248 (2017). https://doi.org/10.1146/annurev-bioeng-071516-044442

    CrossRef  Google Scholar 

  34. Srinidhi, C.L., et al.: Deep neural network models for computational histopathology: a survey. Med. Image Anal. 101813 (2020). https://doi.org/10.1016/j.media.2020.101813

  35. Vanhoucke, V., Senior, A., Mao, M.Z.: Improving the speed of neural networks on CPUs. In: Deep Learning and Unsupervised Feature Learning Workshop, NIPS 2011 (2011)

    Google Scholar 

  36. Vasiljević, J., et al.: Towards histopathological stain invariance by unsupervised domain augmentation using generative adversarial networks. Neurocomputing 460, 277–291 (2021). https://doi.org/10.1016/j.neucom.2021.07.005

    CrossRef  Google Scholar 

  37. Veeling, B.S., Linmans, J., Winkens, J., Cohen, T., Welling, M.: Rotation equivariant CNNs for digital pathology. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 210–218. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_24

    CrossRef  Google Scholar 

  38. Veta, M., et al.: Breast cancer histopathology image analysis: a review. IEEE Trans. Biomed. Eng. 61(5), 1400–1411 (2014). https://doi.org/10.1109/TBME.2014.2303852

    CrossRef  Google Scholar 

  39. Wang, D., et al.: Deep learning for identifying metastatic breast cancer. arXiv preprint arXiv:1606.05718 (2016)

  40. Wei, J.W., et al.: Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks. Sci. Rep. 9(1) (2019). https://doi.org/10.1038/s41598-019-40041-7

  41. Worrall, D., Welling, M.: Deep scale-spaces: equivariance over scale. In: Advances in Neural Information Processing Systems 32 (2019). https://doi.org/10.5555/3454287.3454949

  42. Zhou, S., et al.: DoReFa-Net: training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv:1606.06160 (2016)

Download references

Author information

Authors and Affiliations

  1. University of Nebraska-Lincoln, Lincoln, NE, 68503, USA

    Mohammed H. Alali, Arman Roohi & Jitender S. Deogun

  2. Prince Sattam Bin Abdulaziz University, Al-Kharj, Saudi Arabia

    Mohammed H. Alali

Authors
  1. Mohammed H. Alali
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arman Roohi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jitender S. Deogun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammed H. Alali .

Editor information

Editors and Affiliations

  1. National Research Council, Lecce, Italy

    Pier Luigi Mazzeo

  2. Università Politecnica delle Marche, Ancona, Italy

    Emanuele Frontoni

  3. Boston University, Boston, MA, USA

    Prof. Stan Sclaroff

  4. National Research Council, Lecce, Italy

    Cosimo Distante

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Alali, M.H., Roohi, A., Deogun, J.S. (2022). Enabling Efficient Training of Convolutional Neural Networks for Histopathology Images. In: Mazzeo, P.L., Frontoni, E., Sclaroff, S., Distante, C. (eds) Image Analysis and Processing. ICIAP 2022 Workshops. ICIAP 2022. Lecture Notes in Computer Science, vol 13373. Springer, Cham. https://doi.org/10.1007/978-3-031-13321-3_47

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-13321-3_47

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-13320-6

  • Online ISBN: 978-3-031-13321-3

  • eBook Packages: Computer ScienceComputer Science (R0)

search

Navigation