A 2D/3D Non-rigid Registration Method for Lung Images Based on a Non-linear Correlation Between Displacement Vectors and Similarity Measures

Abstract

Purpose

This paper presents a novel 2D/3D non-rigid registration method for lung lesions tracking in image-guided diagnoses and treatments.

Methods

Preoperative 3D lung CT volumes were obtained at a series of respiratory phases and important anatomical points were extracted. A CT volume was selected as reference volume and others were considered floating volumes. Displacement vectors of anatomical points were calculated using coherent point drift (CPD) and diffeomorphic-demon methods. For each CT volume, 2D digitally reconstructed radiographs (DRR) were generated by ray projection to simulate intraoperative 2D X-ray, and grayscale-based similarity measures of DRR between reference and floating volumes were calculated. A pulmonary respiration model was constructed using cubic polynomial, which represents the non-linear correlation between displacement vectors and similarity measures. During operation, the pulmonary respiration model used X-ray to obtain displacement vectors of important anatomical points; deformation fields were calculated from the displacement vectors through B-spline transformations. Finally, simulated lung CT volumes corresponding to intraoperative X-ray were generated by applying deformation fields to reference volume. The proposed method was compared to a state-of-the-art 2D/3D non-rigid registration method named respiratory-phase matching.

Results

Experimental results on two datasets respectively from a lung cancer patient and an ex vivo pig lung, showed mean registration accuracy of the proposed method was 1.57 ± 0.92 mm and 2.65 ± 1.35 mm; an improvement of 23% and 9% over the matching method. Moreover, mean lung structure overlaps were 0.91 and 0.88, comparable to matching method.

Conclusion

The proposed method has potential in aiding lung intervention diagnoses and real-time lesion tracking.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Data Availability

The datasets used or analysed during the current study are available from the corresponding author on reasonable request.

Code Availability

The code required to reproduce these findings cannot be shared at this time as the code also forms part of an ongoing study.

References

  1. 1.

    Siegel, R. L., Miller, K. D., & Jemal, A. (2019). Cancer statistics, 2019. CA: A Cancer Journal for Clinicians, 69(1), 7–34.

    Google Scholar 

  2. 2.

    Cleary, K., & Peters, T. M. (2010). Image-guided interventions: Technology review and clinical applications. Annual Review of Biomedical Engineering, 12, 119–142.

    Article  Google Scholar 

  3. 3.

    Matsuoka, T., Sonobe, M., & Date, H. (2015). Intraoperative fine-needle aspiration biopsy (FNA) for lung cancer: Diagnostic value and risk of pleural dissemination. Surgery Today, 45(6), 695–699.

    Article  Google Scholar 

  4. 4.

    Geneser, S. E., Hinkle, J., Kirby, R. M., Wang, B., Salter, B., & Joshi, S. (2011). Quantifying variability in radiation dose due to respiratory-induced tumor motion. Medical Image Analysis, 15(4), 640–649.

    Article  Google Scholar 

  5. 5.

    Foote, M. D., Zimmerman, B., Sawant, A., & Joshi, S. (2019). Real-time 2D-3D deformable registration with deep learning and application to lung radiotherapy targeting. In International conference information processing, 2019 (pp. 265–276).

  6. 6.

    Chen, F., Wu, D., & Liao, H. (2016). Registration of CT and ultrasound images of the spine with neural network and orientation code mutual information. In International Conference on Medical Imaging and Augmented Reality, 2016 (pp. 292–301). Springer.

  7. 7.

    Markelj, P., Tomaževič, D., Likar, B., & Pernuš, F. (2012). A review of 3D/2D registration methods for image-guided interventions. Medical Image Analysis, 16(3), 642–661.

    Article  Google Scholar 

  8. 8.

    Rueckert, D., & Aljabar, P. (2010). Nonrigid registration of medical images: Theory, methods, and applications [applications corner]. IEEE Signal Processing Magazine, 27(4), 113–119.

    Article  Google Scholar 

  9. 9.

    Gendrin, C., Furtado, H., Weber, C., Bloch, C., Figl, M., Pawiro, S. A., et al. (2012). Monitoring tumor motion by real time 2D/3D registration during radiotherapy. Radiotherapy and Oncology, 102(2), 274–280.

    Article  Google Scholar 

  10. 10.

    Zvonarev, P., Farrell, T., Hunter, R., Wierzbicki, M., Hayward, J., & Sur, R. (2013). 2D/3D registration algorithm for lung brachytherapy. Medical Physics, 40(2), 021913.

    Article  Google Scholar 

  11. 11.

    Su, P., Yang, J., Lu, K., Yu, N., Wong, S. T., & Xue, Z. (2013). A fast CT and CT-fluoroscopy registration algorithm with respiratory motion compensation for image-guided lung intervention. IEEE Transactions on Biomedical Engineering, 60(7), 2034–2041.

    Article  Google Scholar 

  12. 12.

    Su, P., Xue, Z., Lu, K., Yang, J., & Wong, S. T. (2012). Fast CT-CT fluoroscopy registration with respiratory motion compensation for image-guided lung intervention. In D. R. Holmes III & K. H. Wong (Eds.), Medical imaging 2012: Image-guided procedures, robotic interventions, and modeling, 2012 (Vol. 8316, pp. 83161J). International Society for Optics and Photonics.

  13. 13.

    Khamene, A., Fluck, O., & Aharon, S. (2012). Deformable 2D-3D registration. Google Patents.

  14. 14.

    Liu, Y., Cheng, E., Wibowo, H., & Rai, L. (2016). A GPU-based method in recovering the full 3D deformation field using multiple 2D fluoroscopic views in lung navigation. In 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), 2016 (pp. 294–297). IEEE.

  15. 15.

    Chou, C.-R., Frederick, B., Mageras, G., Chang, S., & Pizer, S. (2013). 2D/3D image registration using regression learning. Computer Vision and Image Understanding, 117(9), 1095–1106.

    Article  Google Scholar 

  16. 16.

    Weon, C., Kim, M., Park, C. M., & Ra, J. B. (2017). Real-time respiratory phase matching between 2D fluoroscopic images and 3D CT images for precise percutaneous lung biopsy. Medical Physics, 44(11), 5824–5834.

    Article  Google Scholar 

  17. 17.

    Yin, Y., Hoffman, E. A., Ding, K., Reinhardt, J. M., & Lin, C.-L. (2010). A cubic B-spline-based hybrid registration of lung CT images for a dynamic airway geometric model with large deformation. Physics in Medicine & Biology, 56(1), 203.

    Article  Google Scholar 

  18. 18.

    Xia, W., Jin, Q., Ni, C., Wang, Y., & Gao, X. (2018). Thorax x-ray and CT interventional dataset for nonrigid 2D/3D image registration evaluation. Medical Physics, 45(11), 5343–5351.

    Article  Google Scholar 

  19. 19.

    Özkan, H., Osman, O., Şahin, S., Atasoy, M., Barutca, H., Boz, A., et al. (2011). Lung Segmentation algorithm for CAD system in CTA images. International Journal of Biomedical and Biological Engineering, 5(5), 190–193.

    Google Scholar 

  20. 20.

    Frangi, A. F., Niessen, W. J., Vincken, K. L., & Viergever, M. A. (1998). Multiscale vessel enhancement filtering. In International conference on medical image computing and computer-assisted intervention, 1998 (pp. 130–137). Springer.

  21. 21.

    Myronenko, A., & Song, X. (2010). Point set registration: Coherent point drift. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(12), 2262–2275.

    Article  Google Scholar 

  22. 22.

    Byrd, R. H., Lu, P., Nocedal, J., & Zhu, C. (1995). A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing, 16(5), 1190–1208.

    MathSciNet  Article  Google Scholar 

  23. 23.

    Xia, W., & Gao, X. (2014). A fast deformable registration method for 4D lung CT in hybrid framework. International Journal of Computer Assisted Radiology and Surgery, 9(4), 523–533.

    Article  Google Scholar 

  24. 24.

    Mattes, D., Haynor, D. R., Vesselle, H., Lewellen, T. K., & Eubank, W. (2003). PET-CT image registration in the chest using free-form deformations. IEEE Transactions on Medical Imaging, 22(1), 120–128.

    Article  Google Scholar 

  25. 25.

    Gorbunova, V., Durrleman, S., Lo, P., Pennec, X., & De Bruijne, M. (2010). Lung CT registration combining intensity, curves and surfaces. In 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 2010 (pp. 340–343). IEEE.

  26. 26.

    Mu, Z. (2016). A fast DRR generation scheme for 3D-2D image registration based on the block projection method. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2016 (pp. 169–177).

  27. 27.

    Spoerk, J., Bergmann, H., Wanschitz, F., Dong, S., & Birkfellner, W. (2007). Fast DRR splat rendering using common consumer graphics hardware. Medical Physics, 34(11), 4302–4308.

    Article  Google Scholar 

  28. 28.

    Foote, M. D., Zimmerman, B. E., Sawant, A., & Joshi, S. C. (2019). Real-time 2d-3d deformable registration with deep learning and application to lung radiotherapy targeting. In International Conference on Information Processing in Medical Imaging, 2019 (pp. 265–276). Springer.

  29. 29.

    Murphy, K., van Ginneken, B., Klein, S., Staring, M., de Hoop, B. J., Viergever, M. A., et al. (2011). Semi-automatic construction of reference standards for evaluation of image registration. Medical Image Analysis, 15(1), 71–84.

    Article  Google Scholar 

  30. 30.

    Crum, W. R., Camara, O., Rueckert, D., Bhatia, K. K., Jenkinson, M., & Hill, D. L. (2015). Generalised overlap measures for assessment of pairwise and groupwise image registration and segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 2005 (pp. 99–106). Springer.

  31. 31.

    Avants, B. B., Tustison, N. J., Song, G., Cook, P. A., Klein, A., & Gee, J. C. (2011). A reproducible evaluation of ANTs similarity metric performance in brain image registration. NeuroImage, 54(3), 2033–2044.

    Article  Google Scholar 

  32. 32.

    Rohlfing, T. (2012). Image similarity and tissue overlaps as surrogates for image registration accuracy: Widely used but unreliable. IEEE Transactions on Medical Imaging, 31(2), 153–163.

    Article  Google Scholar 

  33. 33.

    Vlachopoulos, G., Korfiatis, P., Skiadopoulos, S., Kazantzi, A., Kalogeropoulou, C., Pratikakis, I., et al. (2015). Selecting registration schemes in case of interstitial lung disease follow-up in CT. Medical Physics, 42(8), 4511–4525.

    Article  Google Scholar 

Download references

Acknowledgements

We thank Science and Technology Plan Projects of Tianjin [Grant Number 19YDYGHZ00030] and Science and Technology Plan Projects of Jiangsu [Grant Number BE2019665] for supporting this work, and we thank MedEditing LLC for linguistic assistance during the preparation of this manuscript.

Funding

This work was supported by Science and Technology Plan Projects of Tianjin [Grant Number 19YDYGHZ00030]; Science and Technology Plan Projects of Jiangsu [Grant Number BE2019665].

Author information

Affiliations

Authors

Contributions

Conceptualization: JZ, WX, QJ; Methodology: JZ, WX, QJ; Writing-original draft preparation: JZ; Writing-review and editing: WX, XG; Supervision: XG.

Corresponding author

Correspondence to Xin Gao.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The study was approved by the Bioethics Committee of the Wuxi No. 2 People’s Hospital (No. 2018B05).

Consent to Participate

Informed consent was obtained from all individual participants included in the study.

Consent for Publication

Patients signed informed consent regarding publishing their data and photographs.

Appendix

Appendix

The principal notations used in this paper were described in Table 3.

Table 3 Description of principal notations

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhang, J., Xia, W., Jin, Q. et al. A 2D/3D Non-rigid Registration Method for Lung Images Based on a Non-linear Correlation Between Displacement Vectors and Similarity Measures. J. Med. Biol. Eng. (2021). https://doi.org/10.1007/s40846-021-00609-z

Download citation

Keywords

  • 2D/3D non-rigid registration
  • Cubic polynomial
  • Similarity measure
  • B-spline transformation
  • Lung image