Skip to main content

Advertisement

SpringerLink
Log in

The requirements for performing artificial-intelligence-related research and model development

  • Artificial intelligence in pediatric radiology
  • Published:
Pediatric Radiology Aims and scope Submit manuscript

Abstract

Artificial intelligence research in health care has undergone tremendous growth in the last several years thanks to the explosion of digital health care data and systems that can leverage large amounts of data to learn patterns that can be applied to clinical tasks. In addition, given broad acceleration in machine learning across industries like transportation, media and commerce, there has been a significant growth in demand for machine-learning practitioners such as engineers and data scientists, who have skill sets that can be applied to health care use cases but who simultaneously lack important health care domain expertise. The purpose of this paper is to discuss the requirements of building an artificial-intelligence research enterprise including the research team, technical software/hardware, and procurement and curation of health care data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Topol EJ (2019) High-performance medicine: the convergence of human and artificial intelligence. Nat Med 25:44–56

    Article  CAS  Google Scholar 

  2. Massachusetts Institute of Technology (n.d.) Artificial intelligence in healthcare. MIT Management Executive Education. https://mit-online.getsmarter.com/presentations/lp/mit-ai-in-healthcare-online-short-course/. Accessed 21 Apr 2022

  3. Stanford University (2022) AI in healthcare specialization. Coursera. https://www.coursera.org/specializations/ai-healthcare. Accessed 21 Apr 2022

  4. University of Glasgow (2022) Informed clinical decision making using deep learning specialization. Coursera. https://www.coursera.org/specializations/clin-decision-deep-learning. Accessed 23 Jun 2022

  5. Radiological Society of North America (2022) The RSNA imaging AI certificate program. RSNA. https://www.rsna.org/ai-certificate. Accessed 21 Apr 2022

  6. Schuur F, Rezazade Mehrizi MH, Ranschaert E (2021) Training opportunities of artificial intelligence (AI) in radiology: a systematic review. Eur Radiol 31:6021–6029

    Article  Google Scholar 

  7. Society for Imaging Informatics in Medicine (2022) National imaging informatics course — radiology. SIIM. https://siim.org/page/niic. Accessed 21 Apr 2022

  8. Willemink MJ, Koszek WA, Hardell C et al (2020) Preparing medical imaging data for machine learning. Radiology 295:4–15

    Article  Google Scholar 

  9. Recht MP, Zbontar J, Sodickson DK et al (2020) Using deep learning to accelerate knee MRI at 3 T: results of an interchangeability study. AJR Am J Roentgenol 215:1421–1429

    Article  Google Scholar 

  10. Chaudhari AS, Mittra E, Davidzon GA et al (2021) Low-count whole-body PET with deep learning in a multicenter and externally validated study. NPJ Digit Med 4:1–11

    Google Scholar 

  11. Irvin J, Rajpurkar P, Ko M et al (2019) CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 590–597

  12. Davenport TH, Patil DJ (2012) Data scientist. Harv Bus Rev 90:70–76

    PubMed  Google Scholar 

  13. Dai W, Berleant D (2019) Benchmarking contemporary deep learning hardware and frameworks: a survey of qualitative metrics. In: 2019 IEEE First International Conference on Cognitive Machine Intelligence (CogMI), pp 148–155

  14. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444

    Article  CAS  Google Scholar 

  15. Jouppi NP, Young C, Patil N et al (2017) In-datacenter performance analysis of a tensor processing unit. In: Proceedings of the 44th Annual International Symposium on Computer Architecture, ACM, New York, pp 1–12

  16. Lemley J, Bazrafkan S, Corcoran P (2017) Deep learning for consumer devices and services: pushing the limits for machine learning, artificial intelligence, and computer vision. IEEE Consum Electron Mag 6:48–56

    Article  Google Scholar 

  17. Van Rossum G, Drake FL (2009) Python 3 reference manual. CreateSpace Independent Publishing Platform, North Charleston

  18. Abadi M, Barham P, Chen J et al (2016) TensorFlow: a system for large-scale machine learning. In: 12th USENIX Symposium on operating Systems Design and Implementation (OSDI 16), pp 265–283

  19. Paszke A, Gross S, Massa F et al (2019) PyTorch: an imperative style, high-performance deep learning library. In: Advances in neural information processing systems 32. Curran Associates Inc., pp 8024–8035

  20. Chirodea MC, Novac OC, Novac CM et al (2021) Comparison of Tensorflow and PyTorch in convolutional neural network–based applications. In: 2021 13th International Conference on Electronics, Computers and Artificial Intelligence (ECAI), pp 1–6

  21. Chollet F (2015) Keras. https://keras.io. Accessed 26 Jul 2022

  22. Falcon W (2019) PyTorch Lightning. https://www.pytorchlightning.ai. Accessed 21 Apr 2022

  23. Pedregosa F, Varoquaux G, Gramfort A et al (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830

    Google Scholar 

  24. Esteva A, Robicquet A, Ramsundar B et al (2019) A guide to deep learning in healthcare. Nat Med 25:24–29

    Article  CAS  Google Scholar 

  25. McKinney SM, Sieniek M, Godbole V et al (2020) International evaluation of an AI system for breast cancer screening. Nature 577:89–94

    Article  CAS  Google Scholar 

  26. Hannun AY, Rajpurkar P, Haghpanahi M et al (2019) Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nat Med 25:65–69

    Article  CAS  Google Scholar 

  27. Wang F, Casalino LP, Khullar D (2019) Deep learning in medicine — promise, progress, and challenges. JAMA Intern Med 179:293–294

    Article  Google Scholar 

  28. Voss SD, Reaman GH, Kaste SC, Slovis TL (2009) The ALARA concept in pediatric oncology. Pediatr Radiol 39:1142–1146

    Article  Google Scholar 

  29. Candemir S, Nguyen XV, Folio LR, Prevedello LM (2021) Training strategies for radiology deep learning models in data — limited scenarios. Radiol Artif Intell 3:e210014

    Article  Google Scholar 

  30. Azizi S, Mustafa B, Ryan F et al (2021) Big self-supervised models advance medical image classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 3478–3488

  31. Kotia J, Kotwal A, Bharti R, Mangrulkar R (2021) Few shot learning for medical imaging. In: Das SK, Das SP, Dey N, Hassanien A-E (eds) Machine learning algorithms for industrial applications. Springer International Publishing, Cham, pp 107–132

    Chapter  Google Scholar 

  32. Medela A, Picon A, Saratxaga CL et al (2019) Few shot learning in histopathological images: reducing the need of labeled data on biological datasets. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp 1860–1864

  33. Rieke N, Hancox J, Li W et al (2020) The future of digital health with federated learning. NPJ Digit Med 3:119

    Article  Google Scholar 

  34. Huang S-C, Pareek A, Seyyedi S et al (2020) Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines. NPJ Digit Med 3:136

    Article  Google Scholar 

  35. Bluemke DA, Moy L, Bredella MA et al (2020) Assessing radiology research on artificial intelligence: a brief guide for authors, reviewers, and readers — from the radiology editorial board. Radiology 294:487–489

    Article  Google Scholar 

  36. Eelbode T, Sinonquel P, Maes F, Bisschops R (2021) Pitfalls in training and validation of deep learning systems. Best Pract Res Clin Gastroenterol 52–53:101712

    Article  Google Scholar 

  37. Hestness J, Ardalani N, Diamos G (2019) Beyond human-level accuracy: computational challenges in deep learning. In: Proceedings of the 24th Symposium on Principles and Practice of Parallel Programming. ACM, New York, pp 1–14

  38. Agarwal V, Podchiyska T, Banda JM et al (2016) Learning statistical models of phenotypes using noisy labeled training data. J Am Med Inform Assoc 23:1166–1173

    Article  Google Scholar 

  39. Karimi D, Dou H, Warfield SK, Gholipour A (2020) Deep learning with noisy labels: exploring techniques and remedies in medical image analysis. Med Image Anal 65:101759

    Article  Google Scholar 

  40. Rolnick D, Veit A, Belongie S, Shavit N (2017) Deep learning is robust to massive label noise. arXiv:1705.10694

  41. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2261–2269

  42. Tan M, Le Q (2019) EfficientNet: rethinking model scaling for convolutional neural networks. Proc Mach Learn Res 97:6105–6114

    Google Scholar 

  43. Bischl B, Binder M, Lang M et al (2021) Hyperparameter optimization: foundations, algorithms, best practices and open challenges. arXiv:210705847

  44. Alibrahim H, Ludwig SA (2021) Hyperparameter optimization: comparing genetic algorithm against grid search and Bayesian optimization. In: 2021 IEEE Congress on Evolutionary Computation (CEC), pp 1551–1559

  45. Akiba T, Sano S, Yanase T et al (2019) Optuna: a next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, New York, pp 2623–2631

  46. Singh R, Bharti V, Purohit V et al (2021) MetaMed: few-shot medical image classification using gradient-based meta-learning. Pattern Recogn 120:108111

    Article  Google Scholar 

  47. Chen T, Kornblith S, Norouzi M, Hinton G (2020) A simple framework for contrastive learning of visual representations. Proc Mach Learn Res 119:1597–1607

    Google Scholar 

  48. Sowrirajan H, Yang J, Ng AY, Rajpurkar P (2021) MoCo pretraining improves representation and transferability of chest X-ray models. Proc Mach Learn Res 143:728–744

    Google Scholar 

  49. Couture HD, Williams LA, Geradts J et al (2018) Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype. NPJ Breast Cancer 4:30

    Article  Google Scholar 

  50. Chaudhari AS, Sandino CM, Cole EK et al (2021) Prospective deployment of deep learning in MRI: a framework for important considerations, challenges, and recommendations for best practices. J Magn Reson Imaging 54:357–371

    Article  Google Scholar 

  51. Abràmoff MD, Lavin PT, Birch M et al (2018) Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med 1:39

    Article  Google Scholar 

  52. Wu E, Wu K, Daneshjou R et al (2021) How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nat Med 27:582–584

    Article  CAS  Google Scholar 

  53. Benjamens S, Dhunnoo P, Meskó B (2020) The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database. NPJ Digit Med 3:118

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Stanford AIMI Center, Stanford University, 1701 Page Mill Road, Palo Alto, CA, 94304, USA

    Anuj Pareek & Matthew P. Lungren

  2. Department of Medical Imaging, Ann & Robert H. Lurie Children’s Hospital of Chicago, Chicago, IL, USA

    Safwan S. Halabi

Authors
  1. Anuj Pareek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthew P. Lungren
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Safwan S. Halabi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anuj Pareek.

Ethics declarations

Conflicts of interest

Anuj Pareek is an associate at Cerebriu. Safwan S. Halabi is an adviser for Change Healthcare. Matthew P. Lungren has nothing to disclose.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pareek, A., Lungren, M.P. & Halabi, S.S. The requirements for performing artificial-intelligence-related research and model development. Pediatr Radiol 52, 2094–2100 (2022). https://doi.org/10.1007/s00247-022-05483-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00247-022-05483-8

Keywords

Search

Navigation