, Volume 68, Issue 11, pp 2963–2972 | Cite as

IDEAL: Images Across Domains, Experiments, Algorithms and Learning

  • Daniela M. UshizimaEmail author
  • Hrishikesh A. Bale
  • E. Wes Bethel
  • Peter Ercius
  • Brett A. Helms
  • Harinarayan Krishnan
  • Lea T. Grinberg
  • Maciej Haranczyk
  • Alastair A. Macdowell
  • Katarzyna Odziomek
  • Dilworth Y. Parkinson
  • Talita Perciano
  • Robert O. Ritchie
  • Chao Yang


Research across science domains is increasingly reliant on image-centric data. Software tools are in high demand to uncover relevant, but hidden, information in digital images, such as those coming from faster next generation high-throughput imaging platforms. The challenge is to analyze the data torrent generated by the advanced instruments efficiently, and provide insights such as measurements for decision-making. In this paper, we overview work performed by an interdisciplinary team of computational and materials scientists, aimed at designing software applications and coordinating research efforts connecting (1) emerging algorithms for dealing with large and complex datasets; (2) data analysis methods with emphasis in pattern recognition and machine learning; and (3) advances in evolving computer architectures. Engineering tools around these efforts accelerate the analyses of image-based recordings, improve reusability and reproducibility, scale scientific procedures by reducing time between experiments, increase efficiency, and open opportunities for more users of the imaging facilities. This paper describes our algorithms and software tools, showing results across image scales, demonstrating how our framework plays a role in improving image understanding for quality control of existent materials and discovery of new compounds.


Mean Square Error Scanning Transmission Electron Microscopy Convolutional Neural Network Ceramic Matrix Composite CBIR System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported by the Director, Office of Science, Advanced Scientific Computing Research and Basic Energy Sciences, of the US Department of Energy. Both the Early Career Research project and the Center for Applied Mathematics for Energy Related Applications (CAMERA) are under Contract No. DE-AC02- 05CH11231. This research used resources of the National Energy Research Scientific Computing Center, which is supported by the Office of Science of the US Department of Energy under Contract No. DE-AC02- 05CH11231. We would like to thank Przemyslaw Oberbek, Materials Design Division, Faculty of Materials Science and Engineering, Warsaw University of Technology, Warsaw, Poland for preparing the SEM images. Also, the authors thank P. Nico and A. Wills for sharing samples of microCT of geological materials, and STEM of PMO, respectively. Additional thanks to B. Loring for supporting visualization schemes, and M. Alegro for participating on the development of multimodal registration methods.


  1. 1.
    D. Martin, C. Fowlkes, D. Tal, and J. Malik, in Proceedings of IEEE International Conference on Computer Vision (2001), p. 416Google Scholar
  2. 2.
    B. Hariharan, P. Arbeláez, R. Girshick, and J. Malik, in Proceedings of IEEE ECCV– European Conference on Computer Vision (2014) p. 297Google Scholar
  3. 3.
    K. Singha, S. Maity, M. Singha, and S. Pal, Front. Sci. 11 (2012)Google Scholar
  4. 4.
    D. Ushizima, A. Bianchi, C. deBianchi, and W. Bethel, in ImageJ User and Developer Conference (2012)Google Scholar
  5. 5.
    L. Martin, A. Tuysuzoglu, W.C. Karl, and P. Ishwar, IEEE Trans. Image Process. 24(11), 4069 (2015)MathSciNetCrossRefGoogle Scholar
  6. 6.
    E.J. Tuegel, A.R. Ingraffea, T.G. Eason, and S. Spottswood, Int. J. Aerosp. Eng. 1, 5 (2011)Google Scholar
  7. 7.
    D. Ushizima, T. Perciano, H. Krishnan, B. Loring, H. Bale, D. Parkinson, and J. Sethian, in Proceedings of IEEE International Conference on Big Data (2014)Google Scholar
  8. 8.
    A. Krizhevsky, I. Sutskever, and G.E. Hinton, Adv. Neural Inf. Process. Syst. 2, 1106 (2012)Google Scholar
  9. 9.
    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A.C. Berg, and L. Fei-Fei, Int. J. Comput. Vis. 115(3), 211 (2015)MathSciNetCrossRefGoogle Scholar
  10. 10.
  11. 11.
    Argonne National Laboratory, TomoPy. Accessed 7 July 2016
  12. 12.
    T. Bicer, D. Gürsoy, R. Kettimuthu, F. De Carlo, and I. Foster, J. Synchrotron Radiat. 23, 4 (2016)CrossRefGoogle Scholar
  13. 13.
    Argonne National Laboratory, Advanced photon source: an office of science national user facility. Accessed 7 July 2016
  14. 14.
    B.N. Cox, H.A. Bale, M. Blacklock, M.N.T. Fast, V. Rajan, R. Rinaldi, R.O. Ritchie, M. Rossol, J. Shaw, Q.D. Yang, F. Zok, and D.B. Marshall, Annu. Rev. Mater. Res. 44, 479 (2014)CrossRefGoogle Scholar
  15. 15.
    General Electric, Ceramic matrix composites improve engine efficiency. 2016. Accessed 7 July 2016
  16. 16.
    H.A. Bale, A. Haboub, A.A. Macdowell, J.R. Nasiatka, D.Y. Parkinson, B.N. Cox, and D.B. Marshall, Nat. Mater. 12, 40 (2012)CrossRefGoogle Scholar
  17. 17.
    R. Nock and F. Nielsen, IEEE Trans. Pattern Anal. Mach. Intell. 26, 1452 (2004)CrossRefGoogle Scholar
  18. 18.
    Fiji, Imagej. Accessed 7 July 2016
  19. 19.
    R.C. Gonzalez and R.E. Woods, Digital Image Processing, 3rd edn. (Prentice-Hall, 2006), p. 861Google Scholar
  20. 20.
    R.O. Duda, P. Hart, and D.G. Stork, Pattern Classification (Wiley, New York, 2001)zbMATHGoogle Scholar
  21. 21.
    A. Gajewicz, N. Schaeublin, B. Rasulev, E. Maurer, S. Hussain, T. Puzyn, and J. Leszczynski, Nanotoxicology 9, 313 (2014)CrossRefGoogle Scholar
  22. 22.
    A. Mikolajczyk, A. Gajewicz, B. Rasulev, N. Schaeublin, E. Maurer-Gardner, S. Hussain, J. Leszczynski, and T. Puzyn, Chem. Mater. 27, 2400 (2015)CrossRefGoogle Scholar
  23. 23.
    E. Burello and A.P. Worth, Wiley Interdiscip. Rev. Nanomed. Nanobiotechnol. 3(3), 298 (2011)CrossRefGoogle Scholar
  24. 24.
    K. Odziomek, D. Ushizima, M. Haranczyk, and T. Puzyn, in Proceedings of American Chemical Society (2014)Google Scholar
  25. 25.
    A.W. Wills, D.J. Michalak, P. Ercius, E.R. Rosenberg, T. Perciano, D. Ushizima, R. Runser, and B.A. Helms, Adv. Funct. Mater., 1 (2015)Google Scholar
  26. 26.
    D.M. Ushizima-Sabino, L. da Fontoura Costa, E.G., Rizzatti, and M.A. Zago, Real Time Imaging 10(4), 205 (2004)CrossRefGoogle Scholar
  27. 27.
    Y. Zhong, and A.K. Jain, Pattern Recogn. 33, 671 (2000)CrossRefGoogle Scholar
  28. 28.
    M. Knezevic, A. Levinson, R. Harris, R.K. Mishra, R.D. Doherty, and S.R. Kalidindi, Acta Mater. 58(19), 6230 (2010)CrossRefGoogle Scholar
  29. 29.
    T. Pietzsch, S. Saalfeld, S. Preibisch, and P. Tomancak, Nat. Methods 12, 6 (2015)CrossRefGoogle Scholar
  30. 30.
    D. Ushizima, T. Perciano, and D. Parkinson, in Proceedings of IEEE International Conference on Big Data (2014)Google Scholar
  31. 31.
    T. Pietzsch, in BigDataViewer. Accessed 7 July 2016
  32. 32.
    I. Paula Jr., F. Medeiros, F. Bezerra, and D. Ushizima, J. Math. Imaging Vis. 45, 251 (2013)MathSciNetCrossRefGoogle Scholar
  33. 33.
    F. Araujo, R. Silva, and D.M. Ushizima, in Proceedings of PyData San Francisco (2016)Google Scholar
  34. 34.
    J. Sethian, Level Set Methods and Fast Marching Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Materials Science (Cambridge University Press, Cambridge, 1999)zbMATHGoogle Scholar
  35. 35.
    T. Perciano, D. Ushizima, E.W. Bethel, Y.D. Mizhahi, and J.A. Sethian, in Proc.eedings of IEEE ICIP Conference (2016)Google Scholar
  36. 36.
    D. Ushizima, A. Bianchi, and C. Carneiro, in Proceedings of IEEE ISBI Symposium (2014)Google Scholar
  37. 37.
    Y.D. Mizrahi, M. Denil, and N. de Freitas, Proc. ICML 32, 1 (2014)Google Scholar
  38. 38.
    S.Z. Li, Markov Random Field Modeling in Image Analysis (Springer, London, 2009)zbMATHGoogle Scholar
  39. 39.
    G.C. Leite, D.M. Ushizima, F.N.S. Medeiros, and G.G. de Lima, Sensors 10(6), 5994 (2010)CrossRefGoogle Scholar
  40. 40.
    D.M. Ushizima, D. Morozov, G.H. Weber, A.G. Bianchi, J.A. Sethian, and E.W. Bethel, IEEE. Trans, Vis. Comput. Gr. 18(12), 2041 (2012)CrossRefGoogle Scholar
  41. 41.
    M. Alegro, E. Amaro-Jr, B. Loring, H. Heinsen, E. Alho, L. Zollei, D. Ushizima, and L.T. Grinberg, in Proceedings of IEEE CVPR Conference (2016)Google Scholar
  42. 42.
    T. Peterka, R. Ross, A. Gyulassy, V. Pascucci, W. Kendall, H.W. Shen, T.Y. Lee, and A. Chaudhuri, in Proceedings of IEEE LDAV Symposium (2011) p. 105Google Scholar

Copyright information

© The Minerals, Metals & Materials Society (outside the U.S.) 2016

Authors and Affiliations

  • Daniela M. Ushizima
    • 1
    • 7
    Email author
  • Hrishikesh A. Bale
    • 2
  • E. Wes Bethel
    • 1
  • Peter Ercius
    • 3
  • Brett A. Helms
    • 3
  • Harinarayan Krishnan
    • 1
  • Lea T. Grinberg
    • 4
  • Maciej Haranczyk
    • 1
  • Alastair A. Macdowell
    • 5
  • Katarzyna Odziomek
    • 6
  • Dilworth Y. Parkinson
    • 5
  • Talita Perciano
    • 1
  • Robert O. Ritchie
    • 2
    • 7
  • Chao Yang
    • 1
  1. 1.Computational Research DivisionLawrence Berkeley National LaboratoryBerkeleyUSA
  2. 2.Materials Sciences DivisionLawrence Berkeley National LaboratoryBerkeleyUSA
  3. 3.Molecular FoundryLawrence Berkeley National LaboratoryBerkeleyUSA
  4. 4.Memory and Aging CenterUniversity of California San FranciscoSan FranciscoUSA
  5. 5.Advanced Light Source DivisionLawrence Berkeley National LaboratoryBerkeleyUSA
  6. 6.Laboratory of Environmental Chemometrics, Faculty of ChemistryUniversity of GdanskGdańskPoland
  7. 7.University of California BerkeleyBerkeleyUSA

Personalised recommendations