Skip to main content
SpringerLink
Log in

AtomGAN: unsupervised deep learning for fast and accurate defect detection of 2D materials at the atomic scale

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

The extraction of atomic-level material features from electron microscope images is crucial for studying structure-property relationships and discovering new materials. However, traditional electron microscope analyses rely on time-consuming and complex human operations; thus, they are only applicable to images with a small number of atoms. In addition, the analysis results vary due to observers’ individual deviations. Although efforts to introduce automated methods have been performed previously, many of these methods lack sufficient labeled data or require various conditions in the detection process that can only be applied to the target material. Thus, in this study, we developed AtomGAN, which is a robust, unsupervised learning method, that segments defects in classical 2D material systems and the heterostructures of MoS2/WS2 automatically. To solve the data scarcity problem, the proposed model is trained on unpaired simulated data that contain point and line defects for MoS2/WS2. The proposed AtomGAN was evaluated on both simulated and real electron microscope images. The results demonstrate that the segmented point defects and line defects are presented perfectly in the resulting figures, with a measurement precision of 96.9%. In addition, the cycled structure of AtomGAN can quickly generate a large number of simulated electron microscope images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Han R, Feng S, Sun D M, et al. Properties and photodetector applications of two-dimensional black arsenic phosphorus and black phosphorus. Sci China Inf Sci, 2021, 64: 140402

    Article  Google Scholar 

  2. Wu S, Wang X, Jiang W, et al. Interface engineering of ferroelectric-gated MoS2 phototransistor. Sci China Inf Sci, 2021, 64: 140407

    Article  Google Scholar 

  3. Yu X, Ng S, Putri L K, et al. Point-defect engineering: leveraging imperfections in graphitic carbon nitride (g-C3N4) photocatalysts toward artificial photosynthesis (small 48/2021). Small, 2021, 17: 2170252

    Article  Google Scholar 

  4. Fu L, Li H, Wang L, et al. Defect passivation strategies in perovskites for an enhanced photovoltaic performance. Energy Environ Sci, 2020, 13: 4017–4056

    Article  Google Scholar 

  5. Sangwan V K, Jariwala D, Kim I S, et al. Gate-tunable memristive phenomena mediated by grain boundaries in single-layer MoS2. Nat Nanotech, 2015, 10: 403–406

    Article  Google Scholar 

  6. Sangwan V K, Lee H S, Bergeron H, et al. Multi-terminal memtransistors from polycrystalline monolayer molybdenum disulfide. Nature, 2018, 554: 500–504

    Article  Google Scholar 

  7. An J, Sun T, Wang B, et al. Efficient graphene in-plane homogeneous p−n−p junction based infrared photodetectors with low dark current. Sci China Inf Sci, 2021, 64: 140403

    Article  Google Scholar 

  8. Wu R, Zhu R Z, Zhao S H, et al. Filling the gap: thermal properties and device applications of graphene. Sci China Inf Sci, 2021, 64: 140401

    Article  Google Scholar 

  9. Yang Y, Zhou J, Zhu F, et al. Determining the three-dimensional atomic structure of an amorphous solid. Nature, 2021, 592: 60–64

    Article  Google Scholar 

  10. Maksov A, Dyck O, Wang K, et al. Deep learning analysis of defect and phase evolution during electron beam-induced transformations in WS2. npj Comput Mater, 2019, 5: 1

    Article  Google Scholar 

  11. Ziatdinov M, Dyck O, Maksov A, et al. Deep learning of atomically resolved scanning transmission electron microscopy images: chemical identification and tracking local transformations. ACS Nano, 2017, 11: 12742–12752

    Article  Google Scholar 

  12. Guo Y, Kalinin S V, Cai H, et al. Defect detection in atomic-resolution images via unsupervised learning with translational invariance. npj Comput Mater, 2021, 7: 180

    Article  Google Scholar 

  13. Cho P, Wood A, Mahalingam K, et al. Defect detection in atomic resolution transmission electron microscopy images using machine learning. Mathematics, 2021, 9: 1209

    Article  Google Scholar 

  14. Choi Y, Choi M, Kim M, et al. StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the 31st IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, 2018. 8789–8797

  15. Jain S, Seth G, Paruthi A, et al. Synthetic data augmentation for surface defect detection and classification using deep learning. J Intell Manuf, 2020, 33: 1007–1020

    Article  Google Scholar 

  16. Ren X, Lin W, Yang X, et al. Data augmentation in defect detection of sanitary ceramics in small and non-i.i.d datasets. IEEE Trans Neural Netw Learn Syst, 2022,: 1–10

  17. Lian J, Jia W, Zareapoor M, et al. Deep-learning-based small surface defect detection via an exaggerated local variation-based generative adversarial network. IEEE Trans Ind Inf, 2020, 16: 1343–1351

    Article  Google Scholar 

  18. Cannizzaro D, Varrella A G, Paradiso S, et al. In-situ defect detection of metal additive manufacturing: an integrated framework. IEEE Trans Emerg Top Comput, 2022, 10: 74–86

    Article  Google Scholar 

  19. Tang T W, Kuo W H, Lan J H, et al. Anomaly detection neural network with dual auto-encoders GAN and its industrial inspection applications. Sensors, 2020, 20: 3336

    Article  Google Scholar 

  20. Madsen J, Susi T. The abTEM code: transmission electron microscopy from first principles. Open Res Europe, 2021, 1: 24

    Article  Google Scholar 

  21. Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the 16th IEEE International Conference on Computer Vision (ICCV), Venice, 2017. 2242–2251

  22. Zhu J Y, Zhang R, Pathak D, et al. Toward multimodal image-to-image translation. In: Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS), Long Beach, 2017

  23. Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets. In: Proceedings of the 28th Conference on Neural Information Processing Systems (NIPS), Montreal, 2014. 2672–2680

  24. Senel H G, Peters R A, Dawant B. Topological median filters. IEEE Trans Image Process, 2002, 11: 89–104

    Article  MathSciNet  Google Scholar 

  25. Huh W, Lee D, Lee C. Memristors based on 2D materials as an artificial synapse for neuromorphic electronics. Adv Mater, 2020, 32: 2002092

    Article  Google Scholar 

  26. Lv Z, Xing X, Huang S, et al. Self-assembling crystalline peptide microrod for neuromorphic function implementation. Matter, 2021, 4: 1702–1719

    Article  Google Scholar 

  27. He K, Zhang X, Ren S, et al. Identity mappings in deep residual networks. In: Proceedings of European Conference on Computer Vision, 2016

  28. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016. 770–778

  29. Gong Y, Lin J, Wang X, et al. Vertical and in-plane heterostructures from WS2/MoS2 monolayers. Nat Mater, 2014, 13: 1135–1142

    Article  Google Scholar 

  30. Bhola R, Krishna N H, Ramesh K N, et al. Detection of the power lines in UAV remote sensed images using spectral-spatial methods. J Environ Manage, 2018, 206: 1233–1242

    Article  Google Scholar 

  31. Chaouch H, Charfeddine S, Aoun S B, et al. Multiscale monitoring using machine learning methods: new methodology and an industrial application to a photovoltaic system. Mathematics, 2022, 10: 890

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by National Key R&D Program of China (Grant No. 2022YFB2404303), National Natural Science Foundation of China (Grant Nos. 52107224, 52077096), State Grid Corporation of China (Grant No. 520626210064), and China Postdoctoral Science Foundation (Grant No. 2019M662612).

Author information

Authors and Affiliations

  1. State Key Laboratory of Advanced Electromagnetic Engineering and Technology, School of Electrical and Electronic Engineering, Huazhong University of Science and Technology, Wuhan, 430074, China

    Danpeng Cheng, Shun Tang & Yuan-Cheng Cao

  2. School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China

    Wuxin Sha

  3. CITIC Dicastal Co., Ltd., Qinhuangdao, 066011, China

    Zuo Xu, Shide Li, Zhigao Yin & Yuling Lang

Authors
  1. Danpeng Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wuxin Sha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zuo Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shide Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhigao Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yuling Lang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Shun Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yuan-Cheng Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuan-Cheng Cao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cheng, D., Sha, W., Xu, Z. et al. AtomGAN: unsupervised deep learning for fast and accurate defect detection of 2D materials at the atomic scale. Sci. China Inf. Sci. 66, 160410 (2023). https://doi.org/10.1007/s11432-022-3757-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-022-3757-x

Keywords

Search

Navigation