Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

ETL Framework for Real-Time Business Intelligence over Medical Imaging Repositories

Abstract

In the last decades, the amount of medical imaging studies and associated metadata has been rapidly increasing. Despite being mostly used for supporting medical diagnosis and treatment, many recent initiatives claim the use of medical imaging studies in clinical research scenarios but also to improve the business practices of medical institutions. However, the continuous production of medical imaging studies coupled with the tremendous amount of associated data, makes the real-time analysis of medical imaging repositories difficult using conventional tools and methodologies. Those archives contain not only the image data itself but also a wide range of valuable metadata describing all the stakeholders involved in the examination. The exploration of such technologies will increase the efficiency and quality of medical practice. In major centers, it represents a big data scenario where Business Intelligence (BI) and Data Analytics (DA) are rare and implemented through data warehousing approaches. This article proposes an Extract, Transform, Load (ETL) framework for medical imaging repositories able to feed, in real-time, a developed BI (Business Intelligence) application. The solution was designed to provide the necessary environment for leading research on top of live institutional repositories without requesting the creation of a data warehouse. It features an extensible dashboard with customizable charts and reports, with an intuitive web-based interface that empowers the usage of novel data mining techniques, namely, a variety of data cleansing tools, filters, and clustering functions. Therefore, the user is not required to master the programming skills commonly needed for data analysts and scientists, such as Python and R.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Notes

  1. 1.

    www.djangoproject.com

  2. 2.

    www.elastic.co/products/kibana

  3. 3.

    www.grafana.com

  4. 4.

    The same PatientID was given for records with different PatientName, PatientBirthDate, PatientSex tuples

References

  1. 1.

    Hamilton B: Big data is the future of healthcare. Teaneck: Cognizant, 2012

  2. 2.

    Godinho TM, Viana-Ferreira C, Bastiao Silva LA, Costa C: A routing mechanism for cloud outsourcing of medical imaging repositories. IEEE J Biomed Health Inform 20(1):367–375, 2016. https://doi.org/10.1109/JBHI.2014.2361633

  3. 3.

    Santos M, Bastiao L, Costa C, Silva A, Rocha N: “Clinical Data Mining in Small Hospital PACS: Contributions for Radiology Department Improvement,” in Information Systems and Technologies for Enhancing Health and Social Care. Hershey: IGI Global, 2013, pp. 236–251

  4. 4.

    Mildenberger P, Eichelberg M, Martin E: Introduction to the DICOM standard. Eur Radiol 12(4):920–927, 1, 2002. https://doi.org/10.1007/s003300101100

  5. 5.

    Digital Imaging and Communications in Medicine (DICOM) Part 3: Information object definitions, NEMA, Standard, 2017.

  6. 6.

    Digital Imaging and Communications in Medicine (DICOM) Part 7: Message Exchange, NEMA, Standard, 2017.

  7. 7.

    Pianykh OS: Digital Imaging and Communications in Medicine (DICOM): a practical introduction and survival guide, Vol. 26. Berlin: Springer Science & Business Media, 2009, 424 pp

  8. 8.

    Wang S, Pavlicek W, Roberts CC, Langer SG, Zhang M, Hu M, Morin RL, Schueler BA, Wellnitz CV, Wu T: An automated DICOM database capable of arbitrary data mining (including radiation dose indicators) for quality monitoring. J Digit Imaging 24(2):223–233, 2011. https://doi.org/10.1007/s10278-010-9329-y

  9. 9.

    Hu M, Pavlicek W, Liu PT, Zhang M, Langer SG, Wang S, Place V, Miranda R, Wu TT: Informatics in radiology: efficiency metrics for imaging device productivity. RadioGraphics 31(2):603–616, 2011. https://doi.org/10.1148/rg.312105714

  10. 10.

    Ondategui-Parra S, Erturk SM, Ros PR: Survey of the use of quality indicators in academic radiology departments. Am J Roentgenol 187(5):W451–W455, 1, 2006. https://doi.org/10.2214/AJR.05.1064

  11. 11.

    Raghupathi W, Raghupathi V: Big data analytics in healthcare: promise and potential. Health Inf Sci Syst 2(1):3, 1, 2014. https://doi.org/10.1186/2047-2501-2-3

  12. 12.

    Nagy PG, Warnock MJ, Daly M, Toland C, Meenan CD et al.: Informatics in radiology: automated Web-based graphical dashboard for radiology operational business intelligence. RadioGraphics 29(7):1897–1906, 1, 2009. https://doi.org/10.1148/rg.297095701

  13. 13.

    Andreu-Perez J, Poon CCY, Merrifield RD, Wong STC, Yang G-Z: Big data for health. IEEE J Biomed Health Inform 19(4):1193–1208, 2015. https://doi.org/10.1109/JBHI.2015.2450362

  14. 14.

    Viceconti M, Hunter P, Hose R: Big data, big knowledge: big data for personalized healthcare. IEEE J Biomed Health Inform 19(4):1209–1215, 2015. https://doi.org/10.1109/JBHI.2015.2406883

  15. 15.

    Langer SG: Challenges for data storage in medical imaging research. J Digit Imaging 24(2):203–207, 2011. https://doi.org/10.1007/s10278-010-9311-8

  16. 16.

    Watson HJ, Wixom BH: The current state of business intelligence. Computer 40(9):96–99, 2007. https://doi.org/10.1109/MC.2007.331

  17. 17.

    Chen H, Chiang RHL, Storey VC: Business intelligence and analytics: from big data to big impact. MIS Q 36(4):1165–1188, 2012

  18. 18.

    Langer SG: A flexible database architecture for mining DICOM objects: the DICOM data warehouse. J Digit Imaging 25(2):206–212, 2012. https://doi.org/10.1007/s10278-011-9434-6

  19. 19.

    J. T. L. Wang, M. J. Zaki, H. T. T. Toivonen, and D. Shasha, Introduction to Data Mining in Bioinformatics. In: Data Mining in Bioinformatics, Springer, London, 2005, pp. 3–8. https://doi.org/10.1007/1-84628-059-11.

  20. 20.

    Valente F, Silva LAB, Godinho TM, Costa C: Anatomy of an extensible open source PACS. J Digit Imaging, 2015. https://doi.org/10.1007/s10278-015-9834-0

  21. 21.

    Costa C, Freitas F, Pereira M, Silva A, Oliveira JL: Indexing and retrieving DICOM data in disperse and unstructured archives. Int J Comput Assist Radiol Surg 4(1):71–77, 1, 2009. https://doi.org/10.1007/s11548-008-0269-7

  22. 22.

    Bastiao L, Santos M, Costa C, Silva A, Rocha N: Dicoogle statistics: Analyzing efficiency and service quality of digital imaging laboratories. Heildelberg: Springer, 2013

  23. 23.

    Santos M, Bastiao L, Costa C, Silva A, Rocha N: Dicom and clinical data mining in a small hospital pacs: A pilot study. In: International Conference on ENTERprise Information Systems. Berlin: Springer, 2011, pp. 254–263

Download references

Funding

Tiago Marques Godinho is funded by Fundação para a Ciência e Tecnologia (FCT) under grant agreement SFRH/BD/104647/2014. Rui Lebre received support by the Integrated Programme of SR&TD “SOCA” (Ref.CENTRO-01-0145-FEDER-000010), co-funded by Centro 2020 program, Portugal 2020, European Union, through the European Regional Development Fund. This work has received support from the ERDF European Regional Development Fund through the Operational Programme for Competitiveness and Internationalization, COMPETE 2020 Programme, and by National Funds through the FCT, Fundação para a Ciência e a Tecnologia within the project PTDC/EEI-ESS/6815/2014.

Author information

Correspondence to Rui Lebre.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Godinho, T.M., Lebre, R., Almeida, J.R. et al. ETL Framework for Real-Time Business Intelligence over Medical Imaging Repositories. J Digit Imaging 32, 870–879 (2019). https://doi.org/10.1007/s10278-019-00184-5

Download citation

Keywords

  • PACS
  • Business Intelligence
  • DICOM
  • Data Analytics
  • Cloud
  • Big data