Skip to main content

Advertisement

SpringerLink
Log in

Identification of diatom taxonomy by a combination of region-based full convolutional network, online hard example mining, and shape priors of diatoms

  • Original Article
  • Published:
International Journal of Legal Medicine Aims and scope Submit manuscript

Abstract

Diatom test is one of the commonly used diagnostic methods for drowning in forensic pathology, which provides supportive evidence for drowning. However, in forensic practice, it is time-consuming and laborious for forensic experts to classify and count diatoms, whereas artificial intelligence (AI) is superior to human experts in processing data and carrying out classification tasks. Some AI techniques have focused on searching diatoms and classifying diatoms. But, they either could not classify diatoms correctly or were time-consuming. Conventional detection deep network has been used to overcome these problems but failed to detect the occluded diatoms and the diatoms similar to the background heavily, which could lead to false positives or false negatives. In order to figure out the problems above, an improved region-based full convolutional network (R-FCN) with online hard example mining and the shape prior of diatoms was proposed. The online hard example mining (OHEM) was coupled with the R-FCN to boost the capacity of detecting the occluded diatoms and the diatoms similar to the background heavily and the priors of the shape of the common diatoms were explored and introduced to the anchor generation strategy of the region proposal network in the R-FCN to locate the diatoms precisely. The results showed that the proposed approach significantly outperforms several state-of-the-art methods and could detect the diatom precisely without missing the occluded diatoms and the diatoms similar to the background heavily. From the study, we could conclude that (1) the proposed model can locate the position and identify the genera of common diatoms more accurately; (2) this method can reduce the false positives or false negatives in forensic practice; and (3) it is a time-saving method and can be introduced.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Piette MHA, Letter EAD (2006) Drowning: still a difficult autopsy diagnosis. Forensic Sci Int 163(1–2):1–9

    Article  Google Scholar 

  2. Zhang P, Kang X, Zhang S et al (2020) The length and width of diatoms in drowning cases as the evidence of diatoms penetrating the alveoli-capillary barrier. Int J Legal Med 134(3):1037–1042

    Article  Google Scholar 

  3. Lunetta P, Penttil A, Hällfors G (1998) Scanning and transmission electron microscopical evidence of the capacity of diatoms to penetrate the alveolo-capillary barrier in drowning. Int J Legal Med 111(5):229–237

    Article  CAS  Google Scholar 

  4. Lunetta P, Modell JH (2005) Macroscopical, microscopical, and laboratory findings in drowning victims. In: Tsokos M (ed) Forensic pathology reviews, vol 3. Humana Press, Totowa

    Chapter  Google Scholar 

  5. Rajvinder S, Rajinder S, Thakar MK (2006) Extraction methods of diatoms-a review. IIJFMT, 4(2)

  6. Ming M, Meng X, Wang E (2007) Evaluation of four digestive methods for extracting diatoms. Forensic Sci Int 170(1):29–34

    Article  Google Scholar 

  7. Zhao J, Liu C, He S et al (2013) Microwave digestion-vacuum filtration-automated scanning electron microscopy as a sensitive method for forensic diatom test. Int J Legal Med 127(2):459–463

    Article  Google Scholar 

  8. Eiji K, Sonoda N, Shinkawa N et al (2019) A new enzymatic method for extracting diatoms from organs of suspected drowning cases using papain: optimal digestion and first practical application. Forensic Sci Int 297:204–216

    Article  Google Scholar 

  9. Culverhouse P, Simpson R, Ellis R et al (1996) Automatic classification of field-collected dinoflagellates by artificial neural network. Mar Ecol Prog Ser 139(1–3):281–287

    Article  Google Scholar 

  10. Zhou Y, Zhang J, Huang J et al (2019) Digital whole-slide image analysis for automated diatom test in forensic cases of drowning using a convolutional neural network algorithm. Forensic Sci Int 302:109922

    Article  Google Scholar 

  11. Anibal P, Gloria B, Oscar D et al (2017) Automated diatom classification (part B): a deep learning approach. Appl Sci 7(5):460

    Article  Google Scholar 

  12. Krause LMK, Koc J, Rosenhahn B et al (2020) A fully convolutional neural network for detection and counting of diatoms on coatings after short-term field exposure. Environ Sci Technol 54(16):10022–10030

    Article  CAS  Google Scholar 

  13. Yu W, Xue Y, Knoops R et al (2020) Automated diatom searching in the digital scanning electron microscopy images of drowning cases using the deep neural networks. Int J Legal Med 135(2):497–508

    Article  Google Scholar 

  14. Guiry MD (2012) How many species of algae are there? J Phycol 48(5):1057–1063

    Article  Google Scholar 

  15. Mann DG, Vanormelingen P (2013) An inordinate fondness? The number, distributions, and origins of diatom species. J Eukaryot Microbiol 60:414–420

    Article  Google Scholar 

  16. Zhao J, Wang Y, Zhang Y et al (2014) Types of diatoms in China’s three major rivers and the possible application for an automatic forensic diatom test. Aust J Forensic Sci 47(3):268–274

    Article  Google Scholar 

  17. Eiji K, Shuji K, Masahiro S et al (2011) Numbers, sizes, and types of diatoms around estuaries for a diatom test. Am J Forensic Med Pathol 32(3):269–274

    Article  Google Scholar 

  18. Gloria B, Oscar D, Anibal P et al (2017) Automated diatom classification (part A): handcrafted feature approaches. Appl Sci 7(8):753

    Article  Google Scholar 

  19. Du Buf H (2002) Automatic diatom identification. World Scientific, Singapore

    Book  Google Scholar 

  20. Dimitrovski I, Kocev D, Loskovska S et al (2012) Hierarchical classification of diatom images using ensembles of predictive clustering trees. Eco Inform 7(1):19–29

    Article  Google Scholar 

  21. Deng J, Wang X, Zhao J et al (2019) Cyclotella recognition of high-resolution electron microscopy with complex background. Comput Eng Des 40:167–172

    Google Scholar 

  22. Liu W, Anguelov D, Erhan D et al (2016) Ssd: single shot multibox detector. European conference on computer vision, pp 21–37

  23. Redmon J, Divvala S, Girshick R et al (2016) You only look once: unified, real-time object detection. Proceedings of 29th IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, USA

  24. Lin T, Goyal P, Girshick R (2017) Focal loss for dense object detection. Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, USA

  25. Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, USA

  26. Zhang S, Wen L, Bian X et al (2018) Single-shot refinement neural network for object detection. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

  27. Ren S, He K, Girshick R et al (2015) Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  28. He K, Gkioxari G, Dollár P et al (2017) Mask r-cnn. Proceedings of the IEEE international conference on computer vision, pp 2961–2969

  29. Girshick R, Donahue J, Darrell T et al (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of 27th IEEE Conference on Computer Vision and Pattern Recognition, Columbus, Ohio, USA

  30. Girshick R (2015) Fast R-CNN. Proceedings of the2015IEEE International Conference on Computer Vision, Santiago, Chile

  31. Dai J, Li Y, He K et al R-fcn: object detection via region-based fully convolutional networks Advances. Neural Information Processing System

  32. Lin TY, Dollár P, Girshick R et al (2017) Lin T et al (2017) Feature pyramid networks for object detection. Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, USA

  33. Si J, Lin J, Jiang F et al (2019) Hand-raising gesture detection in real classrooms using improved r-fcn. Neurocomputing 359:69–76

    Article  Google Scholar 

  34. Shrivastava A, Gupta A, Girshick R (2016) Training region-based object detectors with online hard example mining, pp 761–769

  35. Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. Proceedings of 29th IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Nevada, USA

  36. Lecun Y, Boser B, Denker J et al (2014) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551

    Article  Google Scholar 

  37. Everingham M, Van Gool L, Williams CK et al (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88(2):303–338

    Article  Google Scholar 

  38. Lin TY, Maire M, Belongie S et al (2014) Microsoft coco: common objects in context, pp 740–755

  39. Jia Y, Shelhamer E, J, Donahue J et al (2014) Caffe: convolutional architecture for fast feature embedding, pp 675–678

  40. Ludes B, Coste M, North N et al (1999) Diatom analysis in victim’s tissues as an indicator of the site of drowning. Int J Legal Med 112(3):163–166

    Article  CAS  Google Scholar 

Download references

Funding

This study was financially supported by the National Nature Science Foundation of China (61202267), Guangzhou Municipal Science and Technology Project (2019030001; 2019030012); Zhejiang Basic Public Welfare Research Plan Projects (LGG19F030009), and Grant-in Aids for Scientific Research from the Ministry of Public Security of the People’s Republic of China (2020GABJC38).

Author information

Authors and Affiliations

  1. School of Computers, Guangdong University of Technology, Guangzhou, 510006, China

    Jiehang Deng, Wenquan Guo, Youwei Zhao, Runhao Lai & Guosheng Gu

  2. Kunming Medical University, Chunrong Road West 1168, Chenggong District, Kunming, China

    Jingjian Liu

  3. College of Electrical and Information Engineering, Quzhou University, Quzhou, 324000, China

    Yalong Zhang

  4. Faculty of Forensic Medicine, Zhongshan School of Medicine, Sun Yat-Sen University & Guangdong Province Translational Forensic Medicine Engineering Technology Research Center, Zhongshan 2nd Road 74, Yuexiu District, Guangzhou, People’s Republic of China

    Qi Li, Chao Liu & Jian Zhao

  5. Guangzhou Forensic Science Institute & Key Laboratory of Forensic Pathology, Ministry of Public Security, Baiyun Avenue 1708, Baiyun District, Guangzhou, People’s Republic of China

    Chao Liu & Jian Zhao

Authors
  1. Jiehang Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenquan Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Youwei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jingjian Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Runhao Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guosheng Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yalong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Qi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Chao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Jian Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Guosheng Gu, Chao Liu or Jian Zhao.

Ethics declarations

Ethical approval

Not applicable.

Informed consent

Not applicable.

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deng, J., Guo, W., Zhao, Y. et al. Identification of diatom taxonomy by a combination of region-based full convolutional network, online hard example mining, and shape priors of diatoms. Int J Legal Med 135, 2519–2530 (2021). https://doi.org/10.1007/s00414-021-02664-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00414-021-02664-2

Keywords

Search

Navigation