This paper presents a novel 2D/3D non-rigid registration method for lung lesions tracking in image-guided diagnoses and treatments.
Preoperative 3D lung CT volumes were obtained at a series of respiratory phases and important anatomical points were extracted. A CT volume was selected as reference volume and others were considered floating volumes. Displacement vectors of anatomical points were calculated using coherent point drift (CPD) and diffeomorphic-demon methods. For each CT volume, 2D digitally reconstructed radiographs (DRR) were generated by ray projection to simulate intraoperative 2D X-ray, and grayscale-based similarity measures of DRR between reference and floating volumes were calculated. A pulmonary respiration model was constructed using cubic polynomial, which represents the non-linear correlation between displacement vectors and similarity measures. During operation, the pulmonary respiration model used X-ray to obtain displacement vectors of important anatomical points; deformation fields were calculated from the displacement vectors through B-spline transformations. Finally, simulated lung CT volumes corresponding to intraoperative X-ray were generated by applying deformation fields to reference volume. The proposed method was compared to a state-of-the-art 2D/3D non-rigid registration method named respiratory-phase matching.
Experimental results on two datasets respectively from a lung cancer patient and an ex vivo pig lung, showed mean registration accuracy of the proposed method was 1.57 ± 0.92 mm and 2.65 ± 1.35 mm; an improvement of 23% and 9% over the matching method. Moreover, mean lung structure overlaps were 0.91 and 0.88, comparable to matching method.
The proposed method has potential in aiding lung intervention diagnoses and real-time lesion tracking.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
The datasets used or analysed during the current study are available from the corresponding author on reasonable request.
The code required to reproduce these findings cannot be shared at this time as the code also forms part of an ongoing study.
Siegel, R. L., Miller, K. D., & Jemal, A. (2019). Cancer statistics, 2019. CA: A Cancer Journal for Clinicians, 69(1), 7–34.
Cleary, K., & Peters, T. M. (2010). Image-guided interventions: Technology review and clinical applications. Annual Review of Biomedical Engineering, 12, 119–142.
Matsuoka, T., Sonobe, M., & Date, H. (2015). Intraoperative fine-needle aspiration biopsy (FNA) for lung cancer: Diagnostic value and risk of pleural dissemination. Surgery Today, 45(6), 695–699.
Geneser, S. E., Hinkle, J., Kirby, R. M., Wang, B., Salter, B., & Joshi, S. (2011). Quantifying variability in radiation dose due to respiratory-induced tumor motion. Medical Image Analysis, 15(4), 640–649.
Foote, M. D., Zimmerman, B., Sawant, A., & Joshi, S. (2019). Real-time 2D-3D deformable registration with deep learning and application to lung radiotherapy targeting. In International conference information processing, 2019 (pp. 265–276).
Chen, F., Wu, D., & Liao, H. (2016). Registration of CT and ultrasound images of the spine with neural network and orientation code mutual information. In International Conference on Medical Imaging and Augmented Reality, 2016 (pp. 292–301). Springer.
Markelj, P., Tomaževič, D., Likar, B., & Pernuš, F. (2012). A review of 3D/2D registration methods for image-guided interventions. Medical Image Analysis, 16(3), 642–661.
Rueckert, D., & Aljabar, P. (2010). Nonrigid registration of medical images: Theory, methods, and applications [applications corner]. IEEE Signal Processing Magazine, 27(4), 113–119.
Gendrin, C., Furtado, H., Weber, C., Bloch, C., Figl, M., Pawiro, S. A., et al. (2012). Monitoring tumor motion by real time 2D/3D registration during radiotherapy. Radiotherapy and Oncology, 102(2), 274–280.
Zvonarev, P., Farrell, T., Hunter, R., Wierzbicki, M., Hayward, J., & Sur, R. (2013). 2D/3D registration algorithm for lung brachytherapy. Medical Physics, 40(2), 021913.
Su, P., Yang, J., Lu, K., Yu, N., Wong, S. T., & Xue, Z. (2013). A fast CT and CT-fluoroscopy registration algorithm with respiratory motion compensation for image-guided lung intervention. IEEE Transactions on Biomedical Engineering, 60(7), 2034–2041.
Su, P., Xue, Z., Lu, K., Yang, J., & Wong, S. T. (2012). Fast CT-CT fluoroscopy registration with respiratory motion compensation for image-guided lung intervention. In D. R. Holmes III & K. H. Wong (Eds.), Medical imaging 2012: Image-guided procedures, robotic interventions, and modeling, 2012 (Vol. 8316, pp. 83161J). International Society for Optics and Photonics.
Khamene, A., Fluck, O., & Aharon, S. (2012). Deformable 2D-3D registration. Google Patents.
Liu, Y., Cheng, E., Wibowo, H., & Rai, L. (2016). A GPU-based method in recovering the full 3D deformation field using multiple 2D fluoroscopic views in lung navigation. In 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), 2016 (pp. 294–297). IEEE.
Chou, C.-R., Frederick, B., Mageras, G., Chang, S., & Pizer, S. (2013). 2D/3D image registration using regression learning. Computer Vision and Image Understanding, 117(9), 1095–1106.
Weon, C., Kim, M., Park, C. M., & Ra, J. B. (2017). Real-time respiratory phase matching between 2D fluoroscopic images and 3D CT images for precise percutaneous lung biopsy. Medical Physics, 44(11), 5824–5834.
Yin, Y., Hoffman, E. A., Ding, K., Reinhardt, J. M., & Lin, C.-L. (2010). A cubic B-spline-based hybrid registration of lung CT images for a dynamic airway geometric model with large deformation. Physics in Medicine & Biology, 56(1), 203.
Xia, W., Jin, Q., Ni, C., Wang, Y., & Gao, X. (2018). Thorax x-ray and CT interventional dataset for nonrigid 2D/3D image registration evaluation. Medical Physics, 45(11), 5343–5351.
Özkan, H., Osman, O., Şahin, S., Atasoy, M., Barutca, H., Boz, A., et al. (2011). Lung Segmentation algorithm for CAD system in CTA images. International Journal of Biomedical and Biological Engineering, 5(5), 190–193.
Frangi, A. F., Niessen, W. J., Vincken, K. L., & Viergever, M. A. (1998). Multiscale vessel enhancement filtering. In International conference on medical image computing and computer-assisted intervention, 1998 (pp. 130–137). Springer.
Myronenko, A., & Song, X. (2010). Point set registration: Coherent point drift. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(12), 2262–2275.
Byrd, R. H., Lu, P., Nocedal, J., & Zhu, C. (1995). A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing, 16(5), 1190–1208.
Xia, W., & Gao, X. (2014). A fast deformable registration method for 4D lung CT in hybrid framework. International Journal of Computer Assisted Radiology and Surgery, 9(4), 523–533.
Mattes, D., Haynor, D. R., Vesselle, H., Lewellen, T. K., & Eubank, W. (2003). PET-CT image registration in the chest using free-form deformations. IEEE Transactions on Medical Imaging, 22(1), 120–128.
Gorbunova, V., Durrleman, S., Lo, P., Pennec, X., & De Bruijne, M. (2010). Lung CT registration combining intensity, curves and surfaces. In 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 2010 (pp. 340–343). IEEE.
Mu, Z. (2016). A fast DRR generation scheme for 3D-2D image registration based on the block projection method. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2016 (pp. 169–177).
Spoerk, J., Bergmann, H., Wanschitz, F., Dong, S., & Birkfellner, W. (2007). Fast DRR splat rendering using common consumer graphics hardware. Medical Physics, 34(11), 4302–4308.
Foote, M. D., Zimmerman, B. E., Sawant, A., & Joshi, S. C. (2019). Real-time 2d-3d deformable registration with deep learning and application to lung radiotherapy targeting. In International Conference on Information Processing in Medical Imaging, 2019 (pp. 265–276). Springer.
Murphy, K., van Ginneken, B., Klein, S., Staring, M., de Hoop, B. J., Viergever, M. A., et al. (2011). Semi-automatic construction of reference standards for evaluation of image registration. Medical Image Analysis, 15(1), 71–84.
Crum, W. R., Camara, O., Rueckert, D., Bhatia, K. K., Jenkinson, M., & Hill, D. L. (2015). Generalised overlap measures for assessment of pairwise and groupwise image registration and segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 2005 (pp. 99–106). Springer.
Avants, B. B., Tustison, N. J., Song, G., Cook, P. A., Klein, A., & Gee, J. C. (2011). A reproducible evaluation of ANTs similarity metric performance in brain image registration. NeuroImage, 54(3), 2033–2044.
Rohlfing, T. (2012). Image similarity and tissue overlaps as surrogates for image registration accuracy: Widely used but unreliable. IEEE Transactions on Medical Imaging, 31(2), 153–163.
Vlachopoulos, G., Korfiatis, P., Skiadopoulos, S., Kazantzi, A., Kalogeropoulou, C., Pratikakis, I., et al. (2015). Selecting registration schemes in case of interstitial lung disease follow-up in CT. Medical Physics, 42(8), 4511–4525.
We thank Science and Technology Plan Projects of Tianjin [Grant Number 19YDYGHZ00030] and Science and Technology Plan Projects of Jiangsu [Grant Number BE2019665] for supporting this work, and we thank MedEditing LLC for linguistic assistance during the preparation of this manuscript.
This work was supported by Science and Technology Plan Projects of Tianjin [Grant Number 19YDYGHZ00030]; Science and Technology Plan Projects of Jiangsu [Grant Number BE2019665].
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The study was approved by the Bioethics Committee of the Wuxi No. 2 People’s Hospital (No. 2018B05).
Consent to Participate
Informed consent was obtained from all individual participants included in the study.
Consent for Publication
Patients signed informed consent regarding publishing their data and photographs.
The principal notations used in this paper were described in Table 3.
About this article
Cite this article
Zhang, J., Xia, W., Jin, Q. et al. A 2D/3D Non-rigid Registration Method for Lung Images Based on a Non-linear Correlation Between Displacement Vectors and Similarity Measures. J. Med. Biol. Eng. (2021). https://doi.org/10.1007/s40846-021-00609-z
- 2D/3D non-rigid registration
- Cubic polynomial
- Similarity measure
- B-spline transformation
- Lung image