Abstract
Artificial intelligence (AI) is transforming research through analysis of massive datasets and accelerating simulations by factors of up to a billion. Such acceleration eclipses the speedups that were made possible though improvements in CPU process and design and other kinds of algorithmic advances. It sets the stage for a new era of discovery in which previously intractable challenges will become surmountable, with applications in fields such as discovering the causes of cancer and rare diseases, developing effective, affordable drugs, improving food sustainability, developing detailed understanding of environmental factors to support protection of biodiversity, and developing alternative energy sources as a step toward reversing climate change. To succeed, the research community requires a high-performance computational ecosystem that seamlessly and efficiently brings together scalable AI, general-purpose computing, and large-scale data management. The authors, at the Pittsburgh Supercomputing Center (PSC), launched a second-generation computational ecosystem to enable AI-enabled research, bringing together carefully designed systems and groundbreaking technologies to provide at no cost a uniquely capable platform to the research community. It consists of two major systems: Neocortex and Bridges-2. Neocortex embodies a revolutionary processor architecture to vastly shorten the time required for deep learning training, foster greater integration of artificial deep learning with scientific workflows, and accelerate graph analytics. Bridges-2 integrates additional scalable AI, high-performance computing (HPC), and high-performance parallel file systems for simulation, data pre- and post-processing, visualization, and Big Data as a Service. Neocortex and Bridges-2 are integrated to form a tightly coupled and highly flexible ecosystem for AI- and data-driven research.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brown, T.B., et al.: Language models are few-shot learners (2020)
Buitrago, P.A., Nystrom, N.A.: Open compass: accelerating the adoption of AI in open research. In: Proceedings of the Practice and Experience in Advanced Research Computing on Rise of the Machines (Learning), PEARC 2019. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3332186.3332253
Buitrago, P.A., Nystrom, N.A.: Strengthening the adoption of AI in research and cyberinfrastructure. In: Pascucci, V., et al. (eds.) Report from the NSF Workshop on Smart Cyberinfrastructure 2020, Alexandria, Virginia (2020)
Buitrago, P.A., Nystrom, N.A., Gupta, R., Saltz, J.: Delivering scalable deep learning to research with bridges-AI. In: Crespo-Mariño, J.L., Meneses-Rojas, E. (eds.) CARLA 2019. CCIS, vol. 1087, pp. 200–214. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-41005-6_14
Cerebras Systems: Cerebras wafer scale engine: an introduction (2019)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding (2018)
Gale, T., Elsen, E., Hooker, S.: The state of sparsity in deep neural networks (2019)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)
Kasim, M.F., et al.: Up to two billion times acceleration of scientific simulations with deep neural architecture search (2020)
Khan, A., Huerta, E.A., Wang, S., Gruendl, R., Jennings, E., Zheng, H.: Deep learning at scale for the construction of galaxy catalogs in the Dark Energy Survey. Phys. Lett. B 795, 248–258 (2019). https://doi.org/10.1016/j.physletb.2019.06.009
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105. Curran Associates, Inc. (2012)
Lie, S.: Wafer scale deep learning. In: Hot Chips 31 (2019)
Nystrom, N.A., Buitrago, P.A., Blood, P.D.: Bridges: converging HPC, AI, and big data for enabling discovery. In: Vetter, J.S. (ed.) Contemporary High Performance Computing: From Petascale toward Exascale. Contemporary High Performance Computing, vol. 3. CRC Press, Boca Raton (2019)
Nystrom, N.A., Levine, M.J., Roskies, R.Z., Scott, J.R.: Bridges: a uniquely flexible HPC resource for new communities and data analytics. In: Proceedings of the 2015 XSEDE Conference: Scientific Advancements Enabled by Enhanced Cyberinfrastructure, XSEDE 2015. Association for Computing Machinery, New York (2015). https://doi.org/10.1145/2792745.2792775
Nystrom, N.A., Welling, J., Blood, P.D., Goh, E.L.G.: BlackLight: coherent shared memory for enabling science. In: Vetter, J.S. (ed.) Contemporary High Performance Computing: From Petascale toward Exascale. Contemporary High Performance Computing. Taylor & Francis Group, Boca Raton (2013)
Shoeybi, M., et al.: Megatron-LM: training multi-billion parameter language models using model parallelism (2019)
Smith, J.S., et al.: Approaching coupled cluster accuracy with a general-purpose neural network potential through transfer learning. Nat. Commun. 10(1), 2903 (2019). https://doi.org/10.1038/s41467-019-10827-4
Vassilieva, N., Buitrago, P.A., Nystrom, N.A., Sanielevici, S.E.: Technical overview of the Cerebras CS-1, the AI compute engine for Neocortex (webinar) (2020). https://www.cmu.edu/psc/aibd/neocortex/technical-overview-webinar.html
Acknowledgments
Thanks to Natalia Vassilieva for collaboration on the Cerebras CS-1. The Bridges system, including Bridges-AI, is supported by NSF award number 1445606. The Bridges-2 system is supported by NSF award number 1928147. The Neocortex system is supported by NSF award number 2005597. The Open Compass project is supported by NSF award number 1833317.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Buitrago, P.A., Nystrom, N.A. (2021). Neocortex and Bridges-2: A High Performance AI+HPC Ecosystem for Science, Discovery, and Societal Good. In: Nesmachnow, S., Castro, H., Tchernykh, A. (eds) High Performance Computing. CARLA 2020. Communications in Computer and Information Science, vol 1327. Springer, Cham. https://doi.org/10.1007/978-3-030-68035-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-68035-0_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68034-3
Online ISBN: 978-3-030-68035-0
eBook Packages: Computer ScienceComputer Science (R0)