Abstract
Artificial intelligence and nanotechnology are two areas of science that have changed the world and made life easier during this last decade. Both fields are undergoing significant knowledge expansion, and both bear the promise of a better future for humankind. This research study used convolutional neural networks to classify images of nanostructured materials of different chemical components, obtained through transmission electron microscopy (TEM). A total of 685 ground truth images from a reduced collection of nanostructured TEM images were analyzed. They were classified into three groups: silicate, silica, and coating, each type belonging to chemical compounds of yttrium silicate, silicon oxide nanoparticles, and silicon oxide nanoparticles as a thin layer (coating), respectively. The classification, location, and segmentation of chemical compounds were conducted using Mask R-CNN (Region-Convolution Neural Network) with ResNet101 as the backbone for convolutional neural networks and trained with the collection of images created. The results showed accuracy scores from 85 to 99% for the three classes. The trained model was also able to classify overlapping and agglomerated clusters of these three compounds.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bayda S, Adeel M, Tuccinardi T, Cordani M, Rizzolio F (2020) The history of nanoscience and nanotechnology: from chemical-physical applications to nanomedicine. Molecules 25(1):112
Cervantes-Vásquez D, Contreras OE, Hirata GA (2013) Quantum efficiency of silica-coated rare-earth doped yttrium silicate. J Lumin 143:226–232
Pitts WH, McCulloch WS (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5:115–133
Lecun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444
Fisker R, Carstensen JM, Hansen MF, Bodker F, Morup S (2000) Estimation of nanoparticle size distributions by image analysis. Nanoparticle Res 2:267–277
Zafari S, Eerola T, Sampo J, Kälviäinen H, Haario H (2017) Segmentation of partially overlapping convex objects using branch and bound algorithm. Lecture Notes Comput Sci 10118:76–90
Phoulady HA, Goldgof D, Hall LO, Mouton PR (2017) A framework for nucleus and overlapping cytoplasm segmentation in cervical cytology extended depth of field and volume images. Comput Med Imag Gr 59:38–49
Saleh HM, Saad NH, Ashidi NMI (2019) Overlapping chromosome segmentation using u-net: convolutional networks with test time augmentation. Proced Comput Sci 159:524–533
Bohm A, Tatarchenko M, Falk T (2019) “Isoov2 dl - semantic instance segmentation of touching and overlapping objects,” in 2019 IEEE 16th International symposium on biomedical imaging (ISBI 2019), pp 343–347
Kowal M, Zejmo M, Skobel M, Korbicz J, Monczak R (2020) Cell nuclei segmentation in cytological images using convolutional neural network and seeded watershed algorithm. J Digital Imag 33:231–242
Fukushima K (1980) Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybern 36:193–202
He K, Gkioxari G, Dollár P, Girshick R (2020) Mask R-CNN. IEEE Trans Pattern Anal Mach Intell 42(2):386–397
Yu J, Li Y, Gao C, Gao H, Xia G, Yu ZL, Li Y (2020) Exemplar-based recursive instance segmentation with application to plant image analysis. IEEE Trans Image Process 29:389–404
Romera-Paredes B, Torr PHS (2016) Recurrent instance segmentation. Recurrent instance segmentation. Springer International Publishing, Heidelberg, pp 312–329
Schumann O, Hahn M, Dickmann J, Wöhler C (2018) “Supervised clustering for radar applications : on the way to radar instance segmentation,” in IEEE MTT-S International conference on microwaves for intelligent mobility (ICMIM), (Munich), pp 1–4, IEEE
Loussaief S, Abdelkrim A (2018) Convolutional neural network hyper-parameters optimization based on genetic algorithms. Int J Adv Comput Sci Appl 9(10):252–266
Girshick R (2015) “Fast R-CNN,” in Proceedings of the IEEE international conference on computer vision, pp 1440–1448
Ren S, He K, Girshick R, Sun J (2016) Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149
He K, Gkioxari G, Dollár P, Girshick R (2017) “Mask R-CNN,” in 2017 IEEE international conference on computer vision (ICCV), pp 2980–2988
Flores D-L, Gutierrez E, Cervantes D, Chacon M, Hirata G (2017) White-light emission from y2sio5:ce3+, tb3+ and sr2si5n8:eu2+ phosphor blends: a predictive model. Micro Nano Lett 12(7):500–504
Raj S, Kumar S, Raj S (2015) An improved histogram equalization technique for image contrast enhancement. In: International conference on communication and computing, pp 1–6
Pizer SM, Johnston RE, Ericksen JP, Yankaskas BC, Muller KE (1990) Contrast-limited adaptive histogram equalization: speed and effectiveness. In: Proceedings of the first conference on visualization in biomedical computing, pp 337–345. https://doi.org/10.1109/VBC.1990.109340
Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6(1):1–48
Giryes R, Sapiro G, Bronstein AM (2016) Deep neural networks with random gaussian weights: A universal classification strategy? IEEE Trans Signal Process 64(13):3444–3457
Acknowledgements
We thank Fernanda García and Karla Juárez for their exhaustive work creating the ground truth images. Carlos Cabrera thanks CICESE and CONACYT for the scholarship grant.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Cabrera, C., Cervantes, D., Muñoz, F. et al. Classifying nanostructured and heterogeneous materials from transmission electron microscopy images using convolutional neural networks. Neural Comput & Applic 34, 11035–11047 (2022). https://doi.org/10.1007/s00521-022-07029-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-022-07029-3