Abstract
Convolutional neural networks (CNNs) are well suited for image restoration tasks, like super resolution, deblurring, and denoising, in which the information required for restoring each pixel is mostly concentrated in a small neighborhood around it in the degraded image. However, they are less natural for highly non-local reconstruction problems, such as computed tomography (CT). To date, this incompatibility has been partially circumvented by using CNNs with very large receptive fields. Here, we propose an alternative approach, which relies on the rearrangement of the CT projection measurements along the CNN’s 3rd (channels’) dimension. This leads to a more local inverse problem, which is suitable for CNNs. We demonstrate our approach on sparse-view and limited-view CT, and show that it significantly improves reconstruction accuracy for any given network model. This allows achieving the same level of accuracy with significantly smaller models, and thus induces shorter training and inference times.
Y. Bahat—Part of the work was done while the author was affiliated with the Technion.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Andersen, A., Kak, A.: Simultaneous algebraic reconstruction technique (sart): a superior implementation of the art algorithm. Ultrasonic Imaging 6(1), 81–94 (1984)
Cai, J.F., Jia, X., Gao, H., Jiang, S.B., Shen, Z., Zhao, H.: Cine cone beam ct reconstruction using low-rank matrix factorization: algorithm and a proof-of-princple study (2012)
Chen, H., et al.: Low-dose CT with a residual encoder-decoder convolutional neural network. IEEE Trans. Med. Imaging 36(12), 2524–2535 (2017)
Chen, Y.: Artifact suppressed dictionary learning for low-dose CT image processing. IEEE Trans. Med. Imaging 33(12), 2271–2292 (2014)
Dreike, P., Boyd, D.P.: Convolution reconstruction of fan beam projections. Comput. Graph. Image Process. 5(4), 459–469 (1976)
Feldkamp, L.A., Davis, L.C., Kress, J.W.: Practical cone-beam algorithm. J. Opt. Soc. Am. A-optics Image Sci. Vision 1, 612–619 (1984)
Gordon, R., Bender, R., Herman, G.T.: Algebraic reconstruction techniques (art) for three-dimensional electron microscopy and x-ray photography. J. Theor. Biol. 29(3), 471–481 (1970)
He, J., Wang, Y., Ma, J.: Radon inversion via deep learning. IEEE Trans. Med. Imaging 39(6), 2076–2087 (2020)
Jin, K.H., McCann, M.T., Froustey, E., Unser, M.: Deep convolutional neural network for inverse problems in imaging. IEEE Trans. Image Process. 26(9), 4509–4522 (2017)
Kang, E., Min, J., Ye, J.C.: A deep convolutional neural network using directional wavelets for low-dose x-ray CT reconstruction. Med. Phys. 44(10), e360–e375 (2017)
Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: Fast and accurate image super-resolution with deep laplacian pyramid networks. IEEE Trans. Pattern Anal. Mach. Intell. 41(11), 2599–2613 (2018)
Laine, S., Lehtinen, J., Aila, T.: Self-supervised deep image denoising. CoRR abs/1901.10277 (2019)
Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. CoRR abs/1609.04802 (2016)
Lee, H., Lee, J., Kim, H., Cho, B., Cho, S.: Deep-neural-network-based sinogram synthesis for sparse-view CT image reconstruction. IEEE Trans. Radiat. Plasma Med. Sci. 3(2), 109–119 (2019)
Li, S., Cao, Q., Chen, Y., Hu, Y., Luo, L., Toumoulin, C.: Dictionary learning based sinogram inpainting for ct sparse reconstruction. Optik - Int. J. Light Electron Optics 125, 2862–2867 (2014)
Mao, X., Shen, C., Yang, Y.: Image denoising using very deep fully convolutional encoder-decoder networks with symmetric skip connections. CoRR abs/1603.09056 (2016)
Radon, J.: On the determination of functions from their integral values along certain manifolds. IEEE Trans. Med. Imaging 5(4), 170–176 (1986)
Ramachandran, G.N., Lakshminarayanan, A.V.: Three-dimensional reconstruction from radiographs and electron micrographs: application of convolutions instead of fourier transforms. Proc. Natl. Acad. Sci. USA 68(9), 2236–40 (1971)
Ren, D., Zuo, W., Hu, Q., Zhu, P., Meng, D.: Progressive image deraining networks: a better and simpler baseline. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Sidky, E.Y., Pan, X.: Image reconstruction in circular cone-beam computed tomography by constrained, total-variation minimization. Phys. Med. Biol. 53, 4777 (2008)
Tao, X., Gao, H., Wang, Y., Shen, X., Wang, J., Jia, J.: Scale-recurrent network for deep image deblurring. CoRR abs/1802.01770 (2018)
Wang, W., et al.: An end-to-end deep network for reconstructing CT images directly from sparse sinograms. IEEE Trans. Comput. Imaging 6, 1548–1560 (2020)
Wecksung, G.W., Kruger, R.P., Morris, R.A.: Fan-to parallel-beam conversion in cat by rubber sheet transformation. Appl. Digital Image Process. III(0207), 76–83 (1979)
Xie, Q., et al.: Robust low-dose CT sinogram preprocessing via exploiting noise-generating mechanism. IEEE Trans. Med. Imaging 36(12), 2487–2498 (2017)
Xu, L., Ren, J.S., Liu, C., Jia, J.: Deep convolutional neural network for image deconvolution. Adv. Neural Inf. Process. Syst. 27, 1790–1798 (2014)
Xu, Q., Yu, H., Mou, X., Zhang, L., Hsieh, J., Wang, G.: Low-dose x-ray ct reconstruction via dictionary learning. IEEE Trans. Med. Imaging 31, 1682–1697 (2012)
Yan, K., Wang, X., Lu, L., Summers, R.M.: Deeplesion: automated deep mining, categorization and detection of significant radiology image findings using large-scale clinical lesion annotations. CoRR abs/1710.01766 (2017)
Yang, Q., et al.: Low-dose CT image denoising using a generative adversarial network with wasserstein distance and perceptual loss. IEEE Trans. Med. Imaging 37(6), 1348–1357 (2018)
Yu, W., Wang, C., Nie, X., Huang, M., Wu, L.: Image reconstruction for few-view computed tomography based on l0 sparse regularization. Procedia Comput. Sci. 107, 808–813 (2017)
Zhang, H., Sindagi, V., Patel, V.M.: Joint transmission map estimation and dehazing using deep networks. IEEE Trans. Circ. Syst. Video Technol. 30(7), 1975–1986 (2019)
Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)
Zhang, Y., Zhang, W.H., Chen, H., Yang, M., Li, T.Y., Zhou, J.L.: Few-view image reconstruction combining total variation and a high-order norm. Int. J. Imaging Syst. Technol. 23, 249–255 (2013)
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. CoRR abs/1807.02758 (2018)
Zhu, Y., Zhao, M., Zhao, Y., Li, H., Zhang, P.: Noise reduction with low dose CT data based on a modified ROF model. Opt. Express 20(16), 17987–18004 (2012)
Acknowledgement
This research was partially supported by the Ollendorff Miverva Center at the Viterbi Faculty of Electrical and Computer Engineering, Technion. Yuval Bahat is funded by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 945422.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Hamoud, B., Bahat, Y., Michaeli, T. (2023). Beyond Local Processing: Adapting CNNs for CT Reconstruction. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13803. Springer, Cham. https://doi.org/10.1007/978-3-031-25066-8_29
Download citation
DOI: https://doi.org/10.1007/978-3-031-25066-8_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25065-1
Online ISBN: 978-3-031-25066-8
eBook Packages: Computer ScienceComputer Science (R0)