Skip to main content

Artificial intelligence and radiomics in nuclear medicine: potentials and challenges

A Publisher Correction to this article was published on 28 November 2019

This article has been updated

Abstract

Artificial intelligence involves a wide range of smart techniques that are applicable to medical services including nuclear medicine. Recent advances in computer power, availability of accumulated digital archives containing large amount of patient images, and records bring new opportunities for the implementation of artificial techniques in nuclear medicine. As a subset of artificial intelligence, machine learning is an emerging tool that possibly perform many clinical tasks. Nuclear medicine community needs to adapt to this fast approaching smart era, to exploit the opportunities and tackle the problems associated with artificial intelligence tools. It is aimed in this editorial to outline the potentials and challenges of artificial intelligence applications in nuclear medicine.

Background

Artificial intelligence (AI) fueled by increased volume of digital archives containing “big data” (actual patient images and records), powerful computer technology (thanks to advances in the microelectronics, computer vision, and pattern recognition), and cutting-edge learning algorithms has just become a reality in healthcare. AI has been a topic of interest since the 1950s, but its application in healthcare has been progressing from theory to practice in the last decade. There are several different definitions of AI, but in its simplest definition, the capability of a computer program to think, learn, react, adapt to solve problems, and perform reasoning like a human. Extracting clinically meaningful quantitative features from medical images is called radiomics. It can give valuable diagnostic and predictive details of tumors. Radiomic workup can be experimented in all medical disciplines for different biological phenomena but most of the radiomics research efforts have been reported in oncology [1, 2]. Radiomic work even in a single image requires time-consuming calculations and statistical analysis, and its potential has therefore not been exploited fully. AI techniques using radiomics would potentially excel as a clinically valuable tool for detailed decision support in precision medicine. If radiomic features can be correlated with genomic data through AI techniques, it would provide more precise radiogenomic results.

Computer-aided lesion detection (CAD) is a technology that has been used for a long time in some imaging technologies. AI should not be confused with CAD, which was designed to perform a standard, specific task using predefined criteria with no ability to learn from the data it has been dealing with. Theoretically, AI is able to perform several tasks in a “smart” fashion mimicking human cognition. There are several techniques of AI [3, 4]. Machine learning (ML) is a subset of AI in which the computer (“the machine”) has the ability to learn from the data that it has been exposed to without prior programming. It can work either in a supervised or unsupervised way.

In imaging practice, the ML program can be “trained” by annotated cases (including real patient images, reports and records with known outcomes) and learns to predict the diagnosis of new cases (supervised learning) [5,6,7,8]. A supervised learning algorithm can learn and modify itself to solve the problem by reasoning through predefined criteria with little or no human intervention. It requires large amount of labeled (structured) data to “train” the program so that the ML program can successfully manage by using data science techniques, the variations commonly seen in human studies. Supervised machine learning is potentially applicable to most of the routine patient images by training the computer with the digital patient archive created in the last 2–3 decades (thanks to DICOM and PACS).

The AI program can also learn itself from a set of unlabeled (unstructured) data without known outcome and create its own model of intelligence to solve the problem without prior training (unsupervised learning). In unsupervised learning, a set of unlabeled data without predefined criteria is enough for the computer to recognize the pattern and the problem.

Artificial neural networks (ANNs) are one of the examples of supervised machine learning (although there are some ANN algorithms that do not involve supervised learning). It is not uncommon to see that deep learning, deep neural learning, and deep neural network are interchangeably used in ANN texts despite the fact that there is nuance in meaning in computer science. ANNs can use a multilayer network similar to neural network in human brain to solve the problems. The most commonly used multilayer neural network in medical AI tools is convolutional neural network [9].

Despite great achievements in its theoretical and experimental applications, AI is yet at a preliminary but fast growing phase in practice [5]. Significant investment has been made in AI research and development efforts by giant companies, and therefore, rapid expansion of its applications in medicine is expected.

Currently, one of the most popular area of AI applications is the healthcare with the ultimate aims of increasing the quality of care, decreasing the cost, optimizing the workflow, achieving more efficient individualized care and precision medicine, and decreasing the need for human workforce in this most costly service sector in the world with an ever ageing population. There are many AI algorithms claiming to serve almost in each medical discipline [10,11,12,13,14,15,16,17,18], but initial efforts were heavily directed to diagnostic radiology which has many monotonous repetitive tasks in daily practice including diagnostic assessment of screening mammography and chest X-rays in which AI could easily separate “normal” from “abnormal.” With the newer algorithms, AI is experimented in solving more complex problems and sophisticatedly assessing more detailed diagnostic studies including computed tomography and magnetic resonance imaging. There is now a growing body of literature detailing the use of AI in several qualitative and quantitative radiological tasks [5,6,7,8]. The ultimate impact of AI on radiology profession as a whole in near and far future is therefore a hot topic in radiology community [19]. There are two opposing views in medical community about the impact of AI technologies on medical profession (Fig. 1). In the pessimistic view, AI is seen as a threat to our professional future and a rival to medical professionals while in the optimistic view; AI is expected to support medical professionals in their daily tasks.

Fig. 1
figure1

Pessimistic (a) and optimistic (b) views about the impact of AI on medical professionals (original cartoons by Vacip Örger and Turgut Özalp)

Surprisingly, despite a large portion of its daily routine is devoted to high-tech diagnostic imaging, there have been fewer efforts of AI in nuclear medicine. Advanced AI efforts are progressing towards performing the complex tasks which have currently been done by “human” nuclear medicine physicians. AI techniques have been experimented in brain studies and oncological imaging [20,21,22]. Also, involvement of AI in diagnostic radiology will eventually influence Nuclear Medicine practice and research, at least through AI-assisted CT and MR applications in SPECT/CT, PET/CT, and PET/MR hybrid imaging [5,6,7,8]. Nuclear medicine professionals should therefore have to learn how to deal with and make the most of AI tools and also how to manage their impact on our practice. This special issue of EJNMMI is rightly and timely devoted to AI and radiomics in nuclear medicine.

AI and radiomics are the titles of new age smart technologies that are the products of long-lasting efforts and cumulative knowledge. It is therefore impossible to cover the entire pool of knowledge and experience in AI in this single issue of EJNMMI. The basics and emerging applications will be described in this issue. For the readers interested in more technical details, there are several more specific texts in computer science literature.

Potentials

It is now time for nuclear medicine community to discover the opportunities and tackle the challenges that will be created by AI. Data mining is one of the potential uses of AI to organize and classify the huge amount of patient records (“big data”) stored in our digital archive. The data obtained by image mining using AI techniques can potentially be used for statistical analysis which could be helpful for both routine practice and research. By natural language processing, AI program can even be trained to convert the narrative patient reports and records stored in digital archive to a more machine friendly format that could be further processed statistically and analyzed to train itself or make it usable for any other research or clinical purposes.

Before the arrival of patients in the nuclear medicine department, AI tools can be coded to manage patient appointments, organize daily schedule, distinguish patients with special needs and priority, determine the amount of radioactivity for each patient, and optimize the daily and weekly program for imaging devices and several other pre-arrival tasks needed for a seamless and cost-effective daily routine.

Triage of requests is another potential role for AI by screening the examination requests and distinguishing the high probability of disease from low probability based on the clinical information in the request form and other information available in institutional archive. This might be of use in making the daily imaging schedule by giving priority to patients with high probability of disease.

Once the patient arrives in the department, AI is technically able to determine the scanning time and prepare the imaging device for each patient, and is potentially able to serve the nuclear medicine physicians by performing pre-reporting workup of raw data obtained from the patients. With the introduction of AI-assisted nuclear medicine systems, qualitative and quantitative image processing and lesion size and volume measurements can be made, radiomic features can be extracted, and SUV calculations and time-dependent changes in the radionuclide uptake in the lesions are theoretically quantified by machine learning programs even more quickly and accurately. A critical role for AI techniques at this stage would be the quality assurance by reviewing the raw data, detecting the motion artifacts and noise and correcting them by applying relevant correction tools without human supervision and intervention.

Also, once the patients are scanned, the AI tools would be potentially coded to screen the scans and decide if additional scans in different positions/regions or at a different time point is needed or in SPECT imaging if and in which area of body the SPECT/CT sections are needed. This feature would reduce the patient waiting time and physician work load.

More importantly, several radiomic features of a tumor or other biological phenomena which are currently not used for routine diagnostic assessment could be analyzed in detail with precision by ML algorithms. New radiomic features that cannot be noticed by human eyes and current qualitative and quantitative tools might be discovered using micro-measurements and highly advanced calculation abilities of AI tools. These radiomic features can be used by reporting physicians or algorithm itself for making a more structured diagnosis. This function of ML may potentially open the way to a better individualized patient care and precision medicine.

As an aid to reporting, advanced machine learning techniques have the potential to withdraw the relevant previous reports, tests, and scans of the same patient from the digital archive and align them with the current scan and even compare key findings and detect disease progress or regression for follow-up and monitoring purposes. Currently, these tedious pre-reporting tasks are performed by the nuclear medicine physicians spending a significant amount of time. ML would be of further use in listing the most recently published papers and guidelines that are relevant to the key findings/disease detected on the current scan aiding to prepare a most up-to-date customized patient report. This time-consuming focused literature search made by reporting physician and its inclusion in the patient report as an addendum is already a part of routine clinical reporting for selected examinations in some academic departments.

Dosimetry is another potential area of work for AI tools by performing precise segmentation of organs and lesions and calculations of absorbed dose using data obtained from three-dimensional imaging. This would be potentially useful for both diagnostic and therapeutic radionuclide procedures. Also, optimization of imaging equipment technology, criteria, and parameters through AI tools integrated to gamma camera and PET systems can possibly bring the opportunity of reducing administered activity and imaging time.

The strongest claim to adapt AI in healthcare is indeed the machine reporting of routine diagnostic imaging tests and making a clinically meaningful diagnosis as efficient as a human physician [5,6,7,8, 19]. Most AI papers in medical literature therefore focus on head-to-head comparison of the performance of machine algorithms with human physicians. Although repetitive screening tests (particularly two-dimensional scans such as mammography and chest X-rays) and possibly CT can easily be assessed and reported by machine learning, probably in a shorter time, with less inter-observer variability and greater accuracy than “human” radiologists, more effort and time are required for AI tools to assess more complex diagnostic scans including MR, PET, and SPECT. As the most repetitive and universal standard tests in nuclear medicine practice, renal scan, whole body bone scan, and myocardial perfusion imaging would possibly be the first examinations that could be successfully assessed and reported by ML techniques. AI programs can easily take over the reporting tasks for bone mineral densitometry studies which are already performed and reported in highly standardized universal protocols.

Recent studies reported the use of machine learning techniques in the prediction of diagnosis on CT, MR, SPECT, and PET in a limited number of cases; these results are yet to be validated in a larger data set obtained in clinical setting [20,21,22,23]. While machine reporting sounds to be a threat to imaging professionals (replacement of physicians by AI tools), there are several technical, organizational, financial, ethical, and administrative issues to be resolved before it is implemented in practice.

For follow-up purposes, AI tools have the technical potential to detect (even subtle) changes in the current and previous scans and compare them. This ability would allow us to intervene at an earlier stage of progress or recurrence. Also, this ability can be exploited for monitoring the efficacy of radionuclide or non-radionuclide therapy enabling the physician to timely modify the treatment protocol saving costs and preventing unnecessary side effects.

Once they are validated, ML programs has the potential to make the evidence-based diagnostic reporting a standard in practice and to take over some exhaustive work load by performing repetitive, time-consuming, and monotonous tasks. This may even result in improved patient care and increased life quality of physicians in whom “burnout syndrome” due to heavy work load has been getting a serious problem. For example, voice interaction with AI-aided imaging devices and reporting computers through speech recognition technology which is currently used for reporting only is another potential area of AI applications to save physician’s time that can be used more efficiently for patient interaction.

Challenges

While AI promises several potentials for future clinical applications, there are also several technical, administrative, financial, and legal challenges that should be taken into consideration in the implementation and commercialization of AI tools in nuclear medicine. The first challenge is the need for huge amount of machine consumable data (millions of digital images, records, and reports) to train the AI programs. It is not always possible to have this amount of archived data in a single institution, city, or region. The first solution would be to initiate collaboration to gather sufficient amount of data in a city, region, or country. Even this effort has its own inherent problems: first of all, it may be difficult to have in different institutions, cities, and regions the digital data stored in a standard machine-consumable format. Second problem with this effort could be the legal aspects of protected health information (PHI) as almost all digital archives contain metadata (for example patient identifiers) for each image. The PHI must be removed in a tedious time-consuming effort from the images before they are fed to the AI machine. Third issue would be the copyright of the training data in such a multilateral collaboration. Another problem associated with a comprehensive collaboration to have sufficient amount of data needed to train the algorithm covering different regions or countries is that initial experience showed the same AI algorithm trained with data of one region might give misleading results, possibly due to racial and demographic features [24]. So, it does not seem to be possible to simply buy an AI program trained with data from different regions or countries and implement it in your region or country. Technical advances and further refinements in future may allow the AI algorithms to be trained by a smaller amount of data or even simulated data (images), eliminating the need for multilateral collaboration.

The second challenge for AI application in nuclear medicine is the need for enormous amount of expert labor to label the patient images, records, and reports to make it ready for the machine to consume because even small mistakes in labeling the images and feeding the machine for training can cause systematic error in the final output. The time-consuming task of annotation should therefore be done by expert nuclear medicine professionals for the current AI techniques at the moment.

A time-consuming challenge includes the tedious work of anonymization of digital data. The images, records, and reports in digital archive should be anonymized for deidentification before feeding to the AI program. Recent initial efforts in progress promise hope for machine deidentification through AI tools.

An imminent challenge is associated with the content of nuclear medicine education. The practicing nuclear medicine professionals should be educated for AI literacy in order to use and manage the algorithm effectively. This education must be further extended to residency training and medical school curriculums that should be supported with relevant mathematics, algebra, statistics, and computing knowledge.

Another challenge is about the need for collaboration of computer scientists and nuclear medicine professionals. In addition to the requirement of knowledge in mathematics, algebra, statistics, and computing, implementation of medical AI algorithms requires branch specific knowledge. Computer scientists who are equipped with mathematics, algebra, statistics and computing knowledge are not expected to have the essential medical background to be able to define medical criteria for the algorithm, to train the machine with correct diagnostic findings or therapeutic decisions and to validate the whole program in a clinical setting. Active, time-consuming involvement of nuclear medicine professionals in collaboration with computer scientists (both whom have different working culture) is required at several stages of creating and implementing an AI tool.

The medicolegal aspect of AI-aided diagnosis and recommendations is a significant challenge. Who will be held responsible for the mistakes of AI program (“AI malpractice”) is not yet widely studied. This issue requires close collaboration of nuclear medicine professionals, government agencies, legal professionals, and insurance companies. The reimbursement of AI-aided services will add new dimensions to the discussion on the pattern of payment to the physicians: the fee-for-service model may be replaced by value-based salaried system.

As a diagnostic aid, licensing of AI programs as a “medical device” will be mandatory for real-life clinical applications before commercialization. This requirement will bring further challenges of validation standards and approval criteria and new quality control parameters.

Finally, the cyber security of AI programs containing sensitive health information brings another challenge to the efforts of implementation in clinical practice. Integration of AI programs in current imaging technologies (PET and SPECT systems), digital archives, and patient management systems requires connection to the Internet, creating significant cyber security risks and making patient data vulnerable to hacking.

One question still remains unanswered: if machine learning tools learn forever from the data they are exposed to, could they become one day a “superintelligent” machine (i.e., more intelligent than human, surpassing human’s intelligence) or should we stop them at some point of development?

For a smooth transition to new smart era, new guidelines within EANM, SNM, and national societies are needed for the involvement of AI tools in our practice. Also, current guidelines might need to be updated to address the AI-associated issues. This special issue of EJNMMI may serve to stimulate interest in AI concepts and techniques in nuclear medicine community and also the efforts to establish a task force or committee within EANM specifically working on the impact of AI applications in our practice and the policies required for a seamless implementation of AI tools in the field of nuclear medicine.

Change history

  • 28 November 2019

    The publisher regret to inform the readers that the original version of this article contained errors in the text and published inadvertently.

  • 28 November 2019

    The publisher regret to inform the readers that the original version of this article contained errors in the text and published inadvertently.

References

  1. 1.

    Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data. Radiology. 2016;278:563–77.

    Article  Google Scholar 

  2. 2.

    Lam A, Bui K, Hernandez Rangel E, Nguyentat M, Fernando D, Nelson K, et al. Radiogenomics and IR. J Vasc Interv Radiol. 2018;29:706–13.

    Article  Google Scholar 

  3. 3.

    Jordan MI, Mitchell TM. Machine learning: trends, perspectives, and prospects. Science. 2015;349:255–60.

    CAS  Article  Google Scholar 

  4. 4.

    LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436–44.

    CAS  Article  Google Scholar 

  5. 5.

    Martín Noguerol T, Paulano-Godino F, Martín-Valdivia MT, Menias CO, Luna A. Strengths, weaknesses, opportunities, and threats analysis of artificial intelligence and machine learning applications in radiology. J Am Coll Radiol. 2019;16:1239–47.

    Article  Google Scholar 

  6. 6.

    Tang A, Tam R, Cadrin-Chênevert A, Guest W, Chong J, Barfett J, et al. Canadian Association of Radiologists (CAR) Artificial Intelligence Working Group. Canadian Association of Radiologists White Paper on Artificial Intelligence in Radiology. Can Assoc Radiol J. 2018;69:120–35.

    Article  Google Scholar 

  7. 7.

    Thrall JH, Li X, Li Q, Cruz C, Do S, Dreyer K, et al. Artificial intelligence and machine learning in radiology: opportunities, challenges, pitfalls, and criteria for success. J Am Coll Radiol. 2018;15:504–8.

    Article  Google Scholar 

  8. 8.

    Choy G, Khalilzadeh O, Michalski M, Do S, Samir AE, Pianykh OS, et al. Current applications and future impact of machine learning in radiology. Radiology. 2018;288:318–28.

    Article  Google Scholar 

  9. 9.

    Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging. 2016;35:1285–98.

    Article  Google Scholar 

  10. 10.

    Obermeyer Z, Emanuel EJ. Predicting the future—big data, machine learning, and clinical medicine. N Engl J Med. 2016;375:1216–9.

    Article  Google Scholar 

  11. 11.

    Emin EI, Emin E, Papalois A, Willmott F, Clarke S, Sideris M. Artificial intelligence in obstetrics and gynaecology: is this the way forward? In Vivo. 2019;33:1547–51.

    Article  Google Scholar 

  12. 12.

    Patel V, Khan MN, Shrivastava A, Sadiq K, Ali SA, Moore SR, et al. Artificial intelligence applied to gastrointestinal diagnostics: a review. J Pediatr Gastroenterol Nutr. 2019. https://doi.org/10.1097/MPG.0000000000002507.

  13. 13.

    Niel O, Bastard P. Artificial Intelligence in nephrology: core concepts, clinical applications, and perspectives. Am J Kidney Dis. 2019. https://doi.org/10.1053/j.ajkd.2019.05.020.

    Article  Google Scholar 

  14. 14.

    Tran WT, Jerzak K, Lu FI, Klein J, Tabbarah S, Lagree A, et al. Personalized breast cancer treatments using artificial intelligence in radiomics and pathomics. J Med Imaging Radiat Sci. 2019. https://doi.org/10.1016/j.jmir.2019.07.010.

  15. 15.

    Jo T, Nho K, Saykin AJ. Deep learning in Alzheimer’s disease: diagnostic classification and prognostic prediction using neuroimaging data. Front Aging Neurosci. 2019;11:220. https://doi.org/10.3389/fnagi.2019.00220.

    Article  PubMed  PubMed Central  Google Scholar 

  16. 16.

    Rashidi HH, Tran NK, Betts EV, Howell LP, Green R. Artificial intelligence and machine learning in pathology: the present landscape of supervised methods. Acad Pathol. 2019;6. https://doi.org/10.1177/2374289519873088.

    Article  Google Scholar 

  17. 17.

    Johnson KM, Johnson HE, Zhao Y, Dowe DA, Staib LH. Scoring of coronary artery disease characteristics on coronary CT angiograms by using machine learning. Radiology. 2019;292:354–62.

    Article  Google Scholar 

  18. 18.

    Hamet P, Tremblay J. Artificial intelligence in medicine. Metabolism. 2017;69S:S36–40.

    Article  Google Scholar 

  19. 19.

    Chockley K, Emanuel E. The end of radiology? Three threats to the future practice of radiology. J Am Coll Radiol. 2016;13:1415–20.

    Article  Google Scholar 

  20. 20.

    Ingrisch M, Schöppe F, Paprottka K, Fabritius M, Strobl FF, De Toni EN, et al. Prediction of 90Y radioembolization outcome from pretherapeutic factors with random survival forests. J Nucl Med. 2018;59:769–73.

    CAS  Article  Google Scholar 

  21. 21.

    Mehta R, Cai K, Kumar N, Knuttinen MG, Anderson TM, Lu H, et al. A lesion-based response prediction model using pretherapy PET/CT image features for Y90 radioembolization to hepatic malignancies. Technol Cancer Res Treat. 2017;16:620–9.

    CAS  Article  Google Scholar 

  22. 22.

    Van Weehaeghe D, Ceccarini J, Delva A, Robberecht W, Van Damme P, Van Laere K. Prospective validation of 18F-FDG brain PET discriminant analysis methods in the diagnosis of amyotrophic lateral sclerosis. J Nucl Med. 2016;57:1238–43.

    Article  Google Scholar 

  23. 23.

    Uribe CF, Mathotaarachchi S, Gaudet V, Smith KC, Rosa-Neto P, Benard F, et al. Machine learning in nuclear medicine: part 1—introduction. J Nucl Med. 2019;60:451–8.

    Article  Google Scholar 

  24. 24.

    Pickup LC, Gleeson F, Talwar A, Kadir T. Lung nodule risk stratification using CNNs: can we generalize from screening training data? In: Society for imaging informatics in medicine conference on machine intelligence in medical imaging. September 26-27. Baltimore: Johns Hopkins; 2017.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Cumali Aktolun.

Ethics declarations

Ethics approval and consent to participate

Not Applicable.

Conflict of interest

The author has no relation, connection, or affiliation of any kind to any artificial intelligence company or similar commercial entity and does not hold any patent or copyright directly or indirectly related to artificial intelligence business.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original version of this article was revised because it contained errors in the text and was published inadvertently.

This article is part of the Topical Collection on Advanced Image Analyses (Radiomics and Artificial Intelligence)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Aktolun, C. Artificial intelligence and radiomics in nuclear medicine: potentials and challenges. Eur J Nucl Med Mol Imaging 46, 2731–2736 (2019). https://doi.org/10.1007/s00259-019-04593-0

Download citation

Keywords

  • Artificial intelligence
  • Radiomics
  • Machine learning
  • Deep learning
  • Artificial neural networks
  • Supervised learning
  • Unsupervised learning