Skip to main content
SpringerLink
Log in

Lab on a beam—Big data and artificial intelligence in scanning transmission electron microscopy

  • Technical Feature
  • Published:
MRS Bulletin Aims and scope Submit manuscript

Abstract

Atomically resolved imaging of materials enabled by the advent of aberration-corrected scanning transmission electron microscopy (STEM) has become a mainstay of modern materials science. However, much of the wealth of quantitative information contained in the fine details of atomic structure or spectra remains largely unexplored. In this article, we discuss new opportunities enabled by physics-informed big data and machine learning technologies to extract physical information from static and dynamic STEM images, ranging from statistical thermodynamics of alloys to kinetics of solid-state reactions at a single defect level. The synergy of deep-learning image analytics and real-time feedback further allows harnessing beam-induced atomic and bond dynamics to enable direct atom-by-atom fabrication. Examples of direct atomic motion over mesoscopic distances, engineered doping at selected lattice sites, and assembly of multiatomic structures are reviewed. These advances position the scanning transmission electron microscope to transition from a mere imaging tool toward a complete nanoscale laboratory for exploring electronic, phonon, and quantum phenomena in atomically engineered structures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9

Similar content being viewed by others

References

  1. J.D. Martin, Solid State Insurrection: How the Science of Substance Made American Physics Matter (University of Pittsburgh Press, Pittsburgh, 2018).

    Book  Google Scholar 

  2. C.C.M. Mody, Instrumental Community (MIT Press, Cambridge, MA, 2011).

    Book  Google Scholar 

  3. O.L. Krivanek, N. Dellby, A.J. Spence, R.A. Camps, L.M. Brown, “Aberration Correction in the STEM,” in Electron Microscopy and Analysis 1997, J.M. Rodenburg, Ed. (IOP Publishing, Bristol, 1997), p. 35.

    Google Scholar 

  4. M. Haider, S. Uhlemann, E. Schwan, H. Rose, B. Kabius, K. Urban, Nature 392, 768 (1998).

    Article  CAS  Google Scholar 

  5. O. Scherzer, Optik 2, 114 (1947).

    CAS  Google Scholar 

  6. N. Dellby, O.L. Krivanek, P.D. Nellist, P.E. Batson, A.R. Lupini, J. Electron Microsc. 50, 177 (2001).

    CAS  Google Scholar 

  7. S.J. Pennycook, P.D. Nellist, Eds., Scanning Transmission Electron Microscopy: Imaging and Analysis (Springer, New York, 2011).

  8. M. Varela, S.D. Findlay, A.R. Lupini, H.M. Christen, A.Y. Borisevich, N. Dellby, O.L. Krivanek, P.D. Nellist, M.P. Oxley, L.J. Allen, S.J. Pennycook, Phys. Rev. Lett. 92, 095502 (2004).

    Article  CAS  Google Scholar 

  9. J.C. Idrobo, A.R. Lupini, T.L. Feng, R.R. Unocic, F.S. Walden, D.S. Gardiner, T.C. Lovejoy, N. Dellby, S.T. Pantelides, O.L. Krivanek, Phys. Rev. Lett. 120, 095901 (2018).

    Article  CAS  Google Scholar 

  10. H. Larocque, F. Bouchard, V. Grillo, A. Sit, S. Frabboni, R.E. Dunin-Borkowski, M.J. Padgett, R.W. Boyd, E. Karimi, Phys. Rev. Lett. 117, 154801 (2016).

    Article  Google Scholar 

  11. Y. Jiang, Z. Chen, Y.M. Hang, P. Deb, H. Gao, S.E. Xie, P. Purohit, M.W. Tate, J. Park, S.M. Gruner, V. Elser, D.A. Muller, Nature 559, 343 (2018).

    Article  CAS  Google Scholar 

  12. O. Dyck, S. Kim, E. Jimenez-Izal, A.N. Alexandrova, S.V. Kalinin, S. Jesse, Small 14, e1801771 (2018).

    Article  Google Scholar 

  13. T. Susi, J.C. Meyer, J. Kotakoski, Ultramicroscopy 180, 163 (2017).

    Article  CAS  Google Scholar 

  14. D.M. Eigler, E.K. Schweizer, Nature 344, 524 (1990).

    CAS  Google Scholar 

  15. S.J. Pennycook, S.V. Kalinin, Nature 515, 487 (2014).

    Article  CAS  Google Scholar 

  16. M. Ziatdinov, O. Dyck, S. Jesse, S.V. Kalinin, “Atomic Mechanisms for the Si Atom Dynamics in Graphene: Chemical Transformations at the Edge and in the Bulk,” arXiv preprint arXiv:09322 (2019).

  17. S. Jesse, Q. He, A.R. Lupini, D.N. Leonard, M.P. Oxley, O. Ovchinnikov, R.R. Unocic, A. Tselev, M. Fuentes-Cabrera, B.G. Sumpter, S.J. Pennycook, S.V. Kalinin, A.Y. Borisevich, Small 11, 5895 (2015).

    Article  CAS  Google Scholar 

  18. J. Feng, A.V. Kvit, C. Zhang, J. Hoffman, A. Bhattacharya, D. Morgan, P.M. Voyles, “Imaging of Single La Vacancies in LaMnO3,” preprint, submitted arXiv:06308 (2017).

  19. M. Ziatdinov, A. Maksov, S.V. Kalinin, npj Comp. Mater. 3, 31 (2017).

    Article  Google Scholar 

  20. Pycroscopy: Scientific Analysis of Nanoscale Materials Imaging Data, https://github.com/pycroscopy/AICrystallographer.

  21. K. Simonyan, A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” preprint, submitted arXiv:1409.1556 (2014).

  22. https://colab.research.google.com/github/pycroscopy/AICrystallographer/blob/master/Tutorials/Dogs_vs_atoms.ipynb

  23. M. Ziatdinov, O. Dyck, A. Maksov, X. Li, X. Sang, K. Xiao, R.R. Unocic, R. Vasudevan, S. Jesse, S.V. Kalinin, ACS Nano 11, 12742 (2017).

    Article  CAS  Google Scholar 

  24. A. Maksov, O. Dyck, K. Wang, K. Xiao, D.B. Geohegan, B.G. Sumpter, R.K. Vasudevan, S. Jesse, S.V. Kalinin, M. Ziatdinov, npj Comp. Mater. 5, 12 (2019).

    Article  Google Scholar 

  25. J. Madsen, P. Liu, J. Kling, J.B. Wagner, T.W. Hansen, O. Winther, J. Schiøtz, Adv. Theory Simul. 1, 1800037 (2018).

    Article  Google Scholar 

  26. M. Rashidi, R.A. Wolkow, ACS Nano 12, 5185 (2018).

    Article  CAS  Google Scholar 

  27. Nion Swift, www.nion.com/swift.

  28. O.L. Krivanek, M.F. Chisholm, V. Nicolosi, T.J. Pennycook, G.J. Corbin, N. Dellby, M.F. Murfitt, C.S. Own, Z.S. Szilagyi, M.P. Oxley, S.T. Pantelides, S.J. Pennycook, Nature 464, 571 (2010).

    Article  CAS  Google Scholar 

  29. A. Govind Rajan, K.S. Silmore, J. Swett, A.W. Robertson, J.H. Warner, D. Blankschtein, M.S. Strano, Nat. Mater. 18, 129 (2019).

    Article  CAS  Google Scholar 

  30. M. Ziatdinov, O. Dyck, B.G. Sumpter, S. Jesse, R.K. Vasudevan, S.V. Kalinin, “Building and Exploring Libraries of Atomic Defects in Graphene: Scanning Transmission Electron and Scanning Tunneling Microscopy Study,” preprint, submitted arXiv:04256 (2018).

  31. Si-Vacancy Complexes in Graphene, https://doi.org/10.25920/0xv3-8459.

  32. H. Takagi, T. Takayama, G. Jackeli, G. Khaliullin, S.E. Nagler, Nat. Rev. Phys. 1, 264 (2019).

    Article  Google Scholar 

  33. A.Y. Borisevich, A.N. Morozovska, Y.M. Kim, D. Leonard, M.P. Oxley, M.D. Biegalski, E.A. Eliseev, S.V. Kalinin, Phys. Rev. Lett. 109, 065702 (2012).

    Article  CAS  Google Scholar 

  34. Q. Li, C.T. Nelson, S.L. Hsu, A.R. Damodaran, L.L. Li, A.K. Yadav, M. McCarter, L.W. Martin, R. Ramesh, S.V. Kalinin, Nat. Commun. 8, 1468 (2017).

    Article  CAS  Google Scholar 

  35. A.V. Ievlev, S. Jesse, T.J. Cochell, R.R. Unocic, V.A. Protopopescu, S.V. Kalinin, ACS Nano 9, 11784 (2015).

    Article  CAS  Google Scholar 

  36. J.P. Sethna, Statistical Mechanics: Entropy, Order Parameters and Complexity, 1st ed. (Oxford University Press, Oxford, UK, 2006).

    Google Scholar 

  37. L. Vlcek, A. Maksov, M.H. Pan, R.K. Vasudevan, S.V. Kalinin, ACS Nano 11, 10313 (2017).

    Article  CAS  Google Scholar 

  38. L. Vlcek, R.K. Vasudevan, S. Jesse, S.V. Kalinin, J. Chem. Theory Comput. 13, 5179 (2017).

    Article  CAS  Google Scholar 

  39. L. Vlcek, M. Ziatdinov, A. Maksov, A. Tselev, A.P. Baddorf, S.V. Kalinin, R.K. Vasudevan, ACS Nano 13, 718 (2019).

    Article  CAS  Google Scholar 

  40. A. Mardt, L. Pasquali, H. Wu, F. Noe, Nat. Commun. 9, 5 (2018).

    Article  Google Scholar 

  41. S.V. Kalinin, A. Borisevich, S. Jesse, Nature 539, 485 (2016).

    Article  CAS  Google Scholar 

  42. S. Jesse, B.M. Hudak, E. Zarkadoula, J. Song, A. Maksov, M. Fuentes-Cabrera, P. Ganesh, I. Kravchenko, P.C. Snijders, A.R. Lupini, A.Y. Borisevich, S.V. Kalinin, Nanotechnology 29, 255303 (2018).

    Article  Google Scholar 

  43. Richard Feynman’s blackboard at time of his death (1988) Caltech Photo Archives, ID Number 1.10-29.

Download references

Author information

Authors and Affiliations

  1. The Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, USA

    Sergei V. Kalinin, Andrew R. Lupini, Ondrej Dyck, Stephen Jesse, Maxim Ziatdinov & Rama K. Vasudevan

Authors
  1. Sergei V. Kalinin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrew R. Lupini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ondrej Dyck
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stephen Jesse
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Maxim Ziatdinov
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Rama K. Vasudevan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sergei V. Kalinin.

Additional information

This article is based on the Symposium X (Frontiers of Materials Research) presentation given at the 2018 MRS Fall Meeting in Boston, Mass.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kalinin, S.V., Lupini, A.R., Dyck, O. et al. Lab on a beam—Big data and artificial intelligence in scanning transmission electron microscopy. MRS Bulletin 44, 565–575 (2019). https://doi.org/10.1557/mrs.2019.159

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1557/mrs.2019.159

Search

Navigation