Skip to main content

Advertisement

SpringerLink
Log in

Rapid Quality Assessment of Nonrigid Image Registration Based on Supervised Learning

  • Original Paper
  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

When preprocedural images are overlaid on intraprocedural images, interventional procedures benefit in that more structures are revealed in intraprocedural imaging. However, image artifacts, respiratory motion, and challenging scenarios could limit the accuracy of multimodality image registration necessary before image overlay. Ensuring the accuracy of registration during interventional procedures is therefore critically important. The goal of this study was to develop a novel framework that has the ability to assess the quality (i.e., accuracy) of nonrigid multimodality image registration accurately in near real time. We constructed a solution using registration quality metrics that can be computed rapidly and combined to form a single binary assessment of image registration quality as either successful or poor. Based on expert-generated quality metrics as ground truth, we used a supervised learning method to train and test this system on existing clinical data. Using the trained quality classifier, the proposed framework identified successful image registration cases with an accuracy of 81.5%. The current implementation produced the classification result in 5.5 s, fast enough for typical interventional radiology procedures. Using supervised learning, we have shown that the described framework could enable a clinician to obtain confirmation or caution of registration results during clinical procedures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

Available upon request.

Code Availability

Available upon request.

References

  1. B. Furlow, Radiologic technology 90(6), 581CT (2019)

  2. G. Antoch, H. Kuehl, F.M. Vogt, J.F. Debatin, J. Stattaus, Journal of vascular and interventional radiology: JVIR 13(11), 1155 (2002)

  3. M. Sato, Y. Watanabe, K. Tokui, K. Kawachi, S. Sugata, J. Ikezoe, The American journal of gastroenterology 95(8), 2102 (2000). https://doi.org/10.1111/j.1572-0241.2000.02275.x

  4. S. Goshima, M. Kanematsu, H. Kondo, R. Yokoyama, T. Miyoshi, H. Nishibori, H. Kato, H. Hoshi, M. Onozuka, N. Moriyama, AJR. American journal of roentgenology 187(1), W25 (2006). https://doi.org/10.2214/AJR.04.1878

  5. D.M. Paushter, R.K. Zeman, M.L. Scheibler, P.L. Choyke, M.H. Jaffe, L.R. Clark, AJR. American journal of roentgenology 152(2), 267 (1989). https://doi.org/10.2214/ajr.152.2.267

  6. M. Montorsi, R. Santambrogio, P. Bianchi, M. Donadon, E. Moroni, A. Spinelli, M. Costa, J Gastrointest Surg 9(1), 62 (2005). DOI S1091-255X(04)00452-4[pii]10.1016/j.gassur.2004.10.003

  7. M.S.S. Chen, J.Q.Q. Li, Y. Zheng, R.P.P. Guo, H.H.H. Liang, Y.Q.Q. Zhang, X.J.J. Lin, W.Y. Lau, Ann Surg 243(3), 321 (2006). https://doi.org/10.1097/01.sla.0000201480.65519.b8

  8. M. Abu-Hilal, J.N. Primrose, A. Casaril, M.J. McPhail, N.W. Pearce, N. Nicoli, J Gastrointest Surg 12(9), 1521 (2008). https://doi.org/10.1007/s11605-008-0553-4

  9. H.H. Liang, M.S. Chen, Z.W. Peng, Y.J. Zhang, Y.Q. Zhang, J.Q. Li, W.Y. Lau, Ann Surg Oncol 15(12), 3484 (2008). https://doi.org/10.1245/s10434-008-0076-y

  10. Z.W. Peng, Y.J. Zhang, M.S. Chen, X.J. Lin, H.H. Liang, M. Shi, Eur J Surg Oncol 36(11), 1054 (2010). DOI S0748-7983(10)00475-0[pii]10.1016/j.ejso.2010.08.133

  11. Q. Yang, H. Qi, R. Zhang, C. Wan, Z. Song, L. Zhang, W. Fan, Journal of Vascular and Interventional Radiology 28(4), 481 (2017). https://doi.org/10.1016/j.jvir.2016.11.042

  12. V.S. Sotirchos, L.M. Petrovic, M. Gönen, D.S. Klimstra, R.K.G. Do, E.N. Petre, A.R. Garcia, A. Barlas, J.P. Erinjeri, K.T. Brown, A.M. Covey, W. Alago, L.A. Brody, R.P. DeMatteo, N.E. Kemeny, S.B. Solomon, K.O. Manova-Todorova, C.T. Sofocleous, Radiology 280(3), 949 (2016). https://doi.org/10.1148/radiol.2016151005.  http://www.ncbi.nlm.nih.gov/pmc/articles/PMC5006720/. 27010254[pmid]

  13. Y.S. Kim, W.J. Lee, H. Rhim, H.K. Lim, D. Choi, J.Y. Lee, AJR Am J Roentgenol 195(3), 758 (2010). https://doi.org/10.2214/AJR.09.2954

  14. G. Laimer, P. Schullian, N. Jaschke, D. Putzer, G. Eberle, A. Alzaga, B. Odisio, R. Bale, European radiology 30(5), 2463 (2020)

  15. D. Wei, S. Ahmad, J. Huo, P. Huang, P.T. Yap, Z. Xue, J. Sun, W. Li, D. Shen, Q. Wang, Medical image analysis 65, 101763 (2020). https://doi.org/10.1016/j.media.2020.101763

  16. H. Elhawary, S. Oguro, K. Tuncali, P.R. Morrison, S. Tatli, P.B. Shyn, S.G. Silverman, N. Hata, Acad Radiol 17(11), 1334 (2010). https://doi.org/10.1016/j.acra.2010.06.004

  17. T. Rohlfing, C.R. Maurer Jr., W.G. O’Dell, J. Zhong, C.R. Maurer, W.G.O. Dell, Med Phys 31(3), 427 (2004)

  18. J. Wu, S.S. Samant, Medical physics 34(6Part1), 2099 (2007)

  19. J. Wu, M.J. Murphy, Medical physics 37(11), 5756 (2010)

  20. S.E. Muenzing, B. van Ginneken, K. Murphy, J.P. Pluim, Medical image analysis 16(8), 1521 (2012)

  21. R. Shams, Y. Xiao, F. H´ebert, M. Abramowitz, R. Brooks, H. Rivaz, IEEE transactions on medical imaging 37(2), 428 (2018)

  22. V. Walimbe, R. Shekhar, Medical Image Analysis 10(6), 899 (2006). https://doi.org/10.1016/j.media.2006.09.002

  23. C. Studholme, D.L.G. Hill, D.J. Hawkes, Pattern Recognition 32(1), 71 (1999). https://doi.org/10.1016/S0031-3203(98)00091-0

  24. A. Bharatha, M. Hirose, N. Hata, S.K.Warfield, M. Ferrant, K.H. Zou, E. Suarez-Santana, J. Ruiz-Alzola, A. D’Amico, R.A. Cormack, R. Kikinis, F.A. Jolesz, C.M. Tempany, Med Phys 28(12), 2551 (2001)

  25. K.H. Zou, S.K. Warfield, A. Bharatha, C.M. Tempany, M.R. Kaus, S.J. Haker, W.M. Wells 3rd, F.A. Jolesz, R. Kikinis, Acad Radiol 11(2), 178 (2004)

  26. J.P. Pluim, J.A. Maintz, M.A. Viergever, IEEE transactions on medical imaging 22(8), 986 (2003)

  27. T. Gass, G. Székely, O. Goksel, Journal of Medical Imaging 2(1), 014005 (2015)

  28. R.D. Datteri, Y. Liu, P.F. D’Haese, B.M. Dawant, IEEE transactions on medical imaging 34(1), 86 (2015)

  29. M.P. Heinrich, I.J. Simpson, B.W. Papież, M. Brady, J.A. Schnabel, Medical image analysis 27, 57 (2016)

  30. H. Sokooti, G. Saygili, B. Glocker, B.P. Lelieveldt, M. Staring, in International Conference on Medical Image Computing and Computer-Assisted Intervention (Springer, 2016), pp. 107–115

  31. M. Schlachter, T. Fechter, M. Jurisic, T. Schimek-Jasch, O. Oehlke, S. Adebahr,W. Birkfellner, U. Nestle, K. Bühler, IEEE transactions on medical imaging 35(10), 2319 (2016)

  32. J. Kybic, IEEE Transactions on Image Processing 19(1), 64 (2010)

  33. J. Tokuda, W. Plishker, M. Torabi, O.I. Olubiyi, G. Zaki, S. Tatli, S.G. Silverman, R. Shekher, N. Hata, Academic Radiology 22(6), 722 (2015). https://doi.org/10.1016/j.acra.2015.01.007. http://linkinghub.elsevier.com/retrieve/pii/S1076633215000434

  34. L. Breiman, Mach. Learn. 45(1), 5 (2001). https://doi.org/10.1023/A:1010933404324

  35. T. Yoo, M. Ackerman, W. Lorensen, W. Schroeder, V. Chalana, S. Aylward, D. Metaxas, R. Whitaker, in Studies in Health Technology and Informatics, vol. 85 (2002), vol. 85, pp. 586–592. https://doi.org/10.3233/978-1-60750-929-5-586

  36. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, E. Duchesnay, Journal of Machine Learning Research 12, 2825 (2011)

Download references

Funding

No funding was received with this work.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Maryland, College Park, MD, USA

    Eung-Joo Lee & Shuvra S. Bhattacharyya

  2. Institute for Advanced Computer Studies, University of Maryland, College Park, MD, USA

    William Plishker & Raj Shekhar

  3. IGI Technologies, Silver Spring, MD, USA

    Nobuhiko Hata, Paul B. Shyn & Stuart G. Silverman

  4. Department of Radiology, Brigham and Women’s Hospital and Harvard Medical School, Boston, MA, USA

    Shuvra S. Bhattacharyya

  5. Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Hospital, Washington, DC, USA

    Raj Shekhar

Authors
  1. Eung-Joo Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. William Plishker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nobuhiko Hata
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Paul B. Shyn
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stuart G. Silverman
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shuvra S. Bhattacharyya
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Raj Shekhar
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

In this work, we report on a novel image registration quality assessment framework designed to integrate into existing interventional radiology workflows and deliver image fusion results that are quantifiably accurate relative to expert-validated solutions. This framework was constructed based on image registration accuracy metrics that can be computed in near real time and combined to form the assessment of multimodality registration quality using supervised learning. This work is significant because it adds a critical quality control step in clinical implementation of multimodality image registration and fusion. By establishing a quality threshold enforced by our framework, the fusion of MR and CT images will be more reliable in our specific implementation and enable faster and more accurate procedures. The manuscript is entirely original and has not been copyrighted, published, or accepted for publication elsewhere.

Corresponding author

Correspondence to Eung-Joo Lee.

Ethics declarations

Ethics Approval

The study was institutional review board-approved (Protocol 2002-P-001166/24).

Consent to Participate

Informed consent to participate in the study was obtained from participants.

Consent for Publication

The patient, or parent, guardian or next of kin (in case of deceased patients) provided written informed consent for the publication of any associated data and accompanying images.

Conflicts of Interest/Competing Interests

William Plishker and Raj Shekhar are founders of IGI Technologies, a medical technology startup company. Other authors have nothing to disclose.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, EJ., Plishker, W., Hata, N. et al. Rapid Quality Assessment of Nonrigid Image Registration Based on Supervised Learning. J Digit Imaging 34, 1376–1386 (2021). https://doi.org/10.1007/s10278-021-00523-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-021-00523-5

Keywords

Search

Navigation