Skip to main content
SpringerLink
Log in

Shape-aware fine-grained classification of erythroid cells

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Fine-grained classification and counting of bone marrow erythroid cells are vital for evaluating the health status and formulating therapeutic schedules for leukemia or hematopathy. Due to the subtle visual differences between different types of erythroid cells, it is challenging to apply existing image-based deep learning models for fine-grained erythroid cell classification. Moreover, there is no large open-source datasets on erythroid cells to support the model training. In this paper, we introduce BMEC (Bone Morrow Erythroid Cells), the first large fine-grained image dataset of erythroid cells, to facilitate more deep learning research on erythroid cells. BMEC contains 5,666 images of individual erythroid cells, each of which is extracted from the bone marrow erythroid cell smears and professionally annotated to one of the four types of erythroid cells. To distinguish the erythroid cells, one key indicator is the cell shape which is closely related to the cell growth and maturation. Therefore, we design a novel shape-aware image classification network for fine-grained erythroid cell classification. The shape feature is extracted from the shape mask image and aggregated to the raw image feature with a shape attention module. With the shape-attended image feature, our network achieved superior classification performance (81.12% top-1 accuracy) on the BMEC dataset comparing to the baseline methods. Ablation studies also demonstrate the effectiveness of incorporating the shape information for the fine-grained cell classification. To further verify the generalizability of our method, we tested our network on two additional public white blood cells (WBC) datasets and the results show our shape-aware method can generally outperform recent state-of-the-art works on classifying the WBC. The code and BMEC dataset can be found on https://github.com/wangye8899/BMEC.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4

Similar content being viewed by others

Data Availability

The code and dataset have been published on github (https://github.com/wangye8899/BMEC).

Code Availability

https://github.com/wangye8899/BMEC

References

  1. Chitra P, Jebarani M, Kavipriya P, Srilatha K, Sumathi M, Lakshmi S (2019) Detection of aml in blood microscopic images using local binary pattern and supervised classifier. Res J Pharm Technol 12(4):1717–1720

    Article  Google Scholar 

  2. Alomari YM, Sheikh Abdullah SNH, Zaharatul Azma R, Omar K (2014) Automatic detection and quantification of wbcs and rbcs using iterative structured circle detection algorithm. Comput Math. Meth Med

  3. Lippeveld M, Knill C, Ladlow E, Fuller A, Michaelis LJ, Saeys Y, Filby A, Peralta D (2020) Classification of human white blood cells using machine learning for stain-free imaging flow cytometry. Cytometry A 97(3):308–319

    Article  Google Scholar 

  4. Petrović N , Moyà-alcover G, Jaume-i-capó A, González-Hidalgo M (2020) Sickle-cell disease diagnosis support selecting the most appropriate machine learning method : towards a general and interpretable approach for cell morphology analysis from microscopy images. Comput Biol Med 126:104027

    Article  Google Scholar 

  5. Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B (2021) Swin transformer : hierarchical vision transformer using shifted windows. arXiv:2103.14030

  6. Xu M, Zhang Z, Hu H, Wang J, Wang L, Wei F, Bai X, Liu Z (2021) End-to-end semi-supervised object detection with soft teacher. Proc ICCV

  7. Strudel R, Garcia R, Laptev I, Schmid C (2021) Segmenter: transformer for semantic segmentation

  8. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  9. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE CVPR pp 770–778

  10. Huang G, Liu Z, Van Der Maaten L , Weinberger KQ (2017) Densely connected convolutional networks. In: IEEE CVPR pp 4700–4708

  11. Zhang H, Wu C, Zhang Z, Zhu Y, Zhang Z, Lin H, Sun Y, He T, Mueller J, Manmatha R, Li M, Smola A J (2020) Resnest : split-attention networks. arxiv: Comput Vis Pattern Recognit

  12. Rezatofighi SH, Soltanian-Zadeh H (2011) Automatic recognition of five types of white blood cells in peripheral blood. Comput Med Imaging Graph 35(4):333–343

    Article  Google Scholar 

  13. https://github.com/Shenggan/BCCDDataset

  14. Kouzehkanan ZM, et al., Saghari S, Tavakoli E, Rostami P, Abaszadeh M, Mirzadeh F, Satlsar ES, Gheidishahran M, Gorgi F, Mohammadi S (2021) Raabin-wbc: a large free access dataset of white blood cells from normal peripheral blood. bioRxiv

  15. Alam MM, Islam MT (2019) Machine learning approach of automatic identification and counting of blood cells. Healthc Technol Lett 6(4):103–108

    Article  Google Scholar 

  16. Gonzalez-Hidalgo M, Guerrero-Pena F, Herold-garcía S, Jaume-i-capó A, Marrero-Fernández PD (2014) Red blood cell cluster separation from digital images for use in sickle cell disease. IEEE J Biomed Health Inform 19(4):1514–1525

    Article  Google Scholar 

  17. Labati RD, Piuri V, Scotti F (2011) All-idb : the acute lymphoblastic leukemia image database for image processing. In: 2011 18th IEEE international conference on image processing, IEEE, pp 2045–2048

  18. Zheng X, Wang Y, Wang G, Liu J (2018) Fast and robust segmentation of white blood cell images by self-supervised learning. Micron 107:55–71

    Article  Google Scholar 

  19. Naruenatthanaset K, Chalidabhongse TH, Palasuwan D, Anantrasirichai N, Palasuwan A (2020) Red blood cell segmentation with overlapping cell separation and classification on imbalanced dataset. arXiv:2012.01321

  20. Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  MATH  Google Scholar 

  21. Friedman JH (2002) Stochastic gradient boosting. Comput Stat Data Anal 38(4):367–378

    Article  MathSciNet  MATH  Google Scholar 

  22. Tavakoli E, Ghaffari A, Kouzehkanan ZM, Hosseini R (2021) New segmentation and feature extraction algorithm for classification of white blood cells in peripheral smear images. bioRxiv

  23. Liu C, Huang L, Wei Z, Zhang W (2021) Subtler mixed attention network on fine-grained image classification. Appl Intell 51(11):7903–7916

    Article  Google Scholar 

  24. Wang L, He K, Feng X, Ma X (2022) Multilayer feature fusion with parallel convolutional block for fine-grained image classification. Appl Intell 52(3):2872–2883

    Article  Google Scholar 

  25. Sun C, Ai Y, Wang S, Zhang W (2021) Mask-guided ssd for small-object detection. Appl Intell 51(6):3311–3322

    Article  Google Scholar 

  26. Zhou Y, Wang Y, Wu J, Hassan M, Pang W, Lv L, Wang L, Cui H (2022) Erythroidcounter : an automatic pipeline for erythroid cell detection, identification and counting based on deep learning. Multimed Tools Appl, 1–16

  27. Toğaçar M, Ergen B, Cömert Z (2020) Classification of white blood cells using deep features obtained from convolutional neural network models based on the combination of feature selection methods. Appl Soft Comput 97:106810

    Article  Google Scholar 

  28. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf. Process Syst 25:1097–1105

    Google Scholar 

  29. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D , Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: IEEE CVPR, pp 1–9

  30. Sahlol AT, Kollmannsberger P, Ewees AA (2020) Efficient classification of white blood cell leukemia with improved swarm optimization of deep features. Sci Rep 10(1):1–11

    Article  Google Scholar 

  31. Mirjalili S, Gandomi AH, Mirjalili SZ, Saremi S, Faris H, Mirjalili SM (2017) Salp swarm algorithm : a bio-inspired optimizer for engineering design problems. Adv Eng Softw 114:163–191

    Article  Google Scholar 

  32. Alzubaidi L, Fadhel MA, Al-Shamma O, Zhang J, Duan Y (2020) Deep learning models for classification of red blood cells in microscopy images to aid in sickle cell anemia diagnosis. Electronics 9(3):427

    Article  Google Scholar 

  33. Pasupa K, Vatathanavaro S, Tungjitnob S (2020) Convolutional neural networks based focal loss for class imbalance problem : a case study of canine red blood cells morphology classification. J Ambient Intell Humanized Comput, 1–17

  34. Lin T-Y, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988

  35. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: ECCV

  36. Tavakoli S, Ghaffari A, Kouzehkanan ZM (2021) Generalizability in white blood cells’ classification problem. bioRxiv

  37. Gregory TR (2001) The bigger the c-value, the larger the cell : genome size and red blood cell size in vertebrates. Blood Cells Mol Dis 27(5):830–843

    Article  Google Scholar 

  38. Otsu N (1979) A threshold selection method from gray level histograms. IEEE Trans Syst Man Cybern 9:62–66

    Article  Google Scholar 

  39. Hu J, Shen L, Albanie S, Sun G, Wu E (2018) Squeeze-and-excitation networks. In: Computer vision and pattern recognition

  40. Wightman R (2019) Pytorch Image Models GitHub. https://doi.org/10.5281/zenodo.4414861

  41. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in pytorch. In: NIPS-W

  42. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers : surpassing human-level performance on imagenet classification. In: International conference on computer vision

  43. Loshchilov I, Hutter F (2016) Sgdr: stochastic gradient descent with warm restarts. arXiv:1608.03983

  44. Jung C, Abuhamad M, Alikhanov J, Mohaisen A, Han K, Nyang D (2019) W-net: a cnn-based architecture for white blood cells image classification. arXiv:1910.01091

  45. Baydilli YY, Atila Ü (2020) Classification of white blood cells using capsule networks. Comput Med Imaging Graph 80:101699

    Article  Google Scholar 

  46. Harshanand B, Sangaiah AK (2020) Comprehensive analysis of deep learning methodology in classification of leukocytes and enhancement using swish activation units. Mobile networks and applications 25 (6):2302–2320

    Article  Google Scholar 

  47. Khan MA, Qasim M, Lodhi HMJ, Nazir M, Javed K, Rubab S, Din A, Habib U (2021) Automated design for recognition of blood cells diseases from hematopathology using classical features selection and elm. Microsc Res Tech 84(2):202–216

    Article  Google Scholar 

Download references

Funding

This research is supported by the National Natural Science Foundation of China (Grants No. 61772227, 61972174, 61972175, 62202199), Science and Technology Development Foundation of Jilin Province (No. 20180201045GX, 20200201300JC, 20200401083GX, 20200201163JC), the Jilin Development and Reform Commission Fund (No. 2020C020-2).

Author information

Author notes

    Authors and Affiliations

    1. School of Computer Science and Technology, Jilin University, Changchun, 130012, Jilin Province, China

      Ye Wang, Xiaoqing Ma, Yubin Xiao, Xuan Wu & You Zhou

    2. School of Artificial Intelligence, Jilin University, Changchun, 130012, Jilin Province, China

      Rui Ma

    3. Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, Changchun, 130012, Jilin Province, China

      Rui Ma

    4. Second Hospital of Jilin University, Changchun, 130041, Jilin Province, China

      Honghua Cui

    Authors
    1. Ye Wang
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Rui Ma
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Xiaoqing Ma
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Honghua Cui
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. Yubin Xiao
      View author publications

      You can also search for this author in PubMed Google Scholar

    6. Xuan Wu
      View author publications

      You can also search for this author in PubMed Google Scholar

    7. You Zhou
      View author publications

      You can also search for this author in PubMed Google Scholar

    Contributions

    Conceptualization, Y.W. and R.M.; methodology, Y.W., R.M. and Y.Z.; validation, X.M., Y.X. and X.W.; formal analysis, Y.W. and R.M.; investigation, H.C.; resources, Y.Z.; data curation, H.C. and X.M.; writing—original draft preparation, Y.W.; writing—review and editing, R.M.; visualization, Y.W. and R.M.; supervision, Y.Z.; project administration, Y.Z.; All authors have read and agreed to the published version of the manuscript.

    Corresponding author

    Correspondence to You Zhou.

    Ethics declarations

    Conflict of Interests

    The authors declare no conflict of interest.

    Additional information

    Publisher’s note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Ye Wang and Rui Ma contributed equally to this work.

    Rights and permissions

    Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Wang, Y., Ma, R., Ma, X. et al. Shape-aware fine-grained classification of erythroid cells. Appl Intell 53, 19115–19127 (2023). https://doi.org/10.1007/s10489-023-04465-z

    Download citation

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s10489-023-04465-z

    Keywords

    Search

    Navigation