Abstract
We propose a method to jointly super-resolve an anisotropic image volume along with its corresponding voxel labels without external training data. Our method is inspired by internally trained super-resolution, or self-super-resolution (SSR) techniques that target anisotropic, low-resolution (LR) magnetic resonance (MR) images. While resulting images from such methods are quite useful, their corresponding LR labels—derived from either automatic algorithms or human raters—are no longer in correspondence with the super-resolved volume. To address this, we develop an SSR deep network that takes both an anisotropic LR MR image and its corresponding LR labels as input and produces both a super-resolved MR image and its super-resolved labels as output. We evaluated our method with 50 \(T_1\)-weighted brain MR images \(4\times \) down-sampled with 10 automatically generated labels. In comparison to other methods, our method had superior Dice across all labels and competitive metrics on the MR image. Our approach is the first reported method for SSR of paired anisotropic image and label volumes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Source code is publicly available at https://gitlab.com/iacl/label_smore.
References
American College of Radiology: American College of Radiology magnetic resonance imaging accreditation program: Phantom test guidance for use of the large MRI phantom for the ACR MRI accreditation program, p. 16 (2018)
Chaudhari, A.S., et al.: Super-resolution musculoskeletal MRI using deep learning. Magn. Reson. Med. 80(5), 2139–2154 (2018)
Chen, Y., Xie, Y., Zhou, Z., Shi, F., Christodoulou, A.G., Li, D.: Brain MRI super resolution using 3D deep densely connected neural networks. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 739–742 (2018)
Chen, Y., Liu, S., Wang, X.: Learning continuous image representation with local implicit image function. arXiv preprint arXiv:2012.09161 (2020)
Delannoy, Q., et al.: SegSRGAN: super-resolution and segmentation using generative adversarial networks - application to neonatal brain MRI. Computers in Biology and Medicine 120, 103755 (2020)
Delbracio, M., Sapiro, G.: Removing camera shake via weighted Fourier burst accumulation. IEEE Trans. Image Process. 24(11), 3293–3307 (2015)
Du, J., et al.: Super-resolution reconstruction of single anisotropic 3D MR images using residual convolutional neural network. Neurocomputing 392, 209–220 (2020)
He, Y., et al.: Structured layer surface segmentation for retina OCT using fully convolutional regression networks. Med. Image Anal. 68, 101856 (2021)
Huo, Y., et al.: 3D whole brain segmentation using spatially localized atlas network tiles. Neuroimage 194, 105–119 (2019)
Jog, A., Carass, A., Prince, J.L.: Self super-resolution for magnetic resonance images. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9902, pp. 553–560. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46726-9_64
LaMontagne, P.J., et al.: OASIS-3: longitudinal neuroimaging, clinical, and cognitive dataset for normal aging and Alzheimer disease. medRxiv (2019)
Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4681–4690 (2017)
Lim, B., Son, S., Kim, H., Nah, S., Mu Lee, K.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, July 2017
Liu, D., Liu, J., Liu, Y., Tao, R., Prince, J.L., Carass, A.: Label super resolution for 3D magnetic resonance images using deformable U-net. In: Medical Imaging 2021: Image Processing, vol. 11596, pp. 597–602. International Society for Optics and Photonics, SPIE (2021)
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Conference on Learning Representations (2019)
Masutani, E.M., Bahrami, N., Hsiao, A.: Deep learning single-frame and multiframe super-resolution for cardiac MRI. Radiology 295(3), 552–561 (2020)
Pham, C., Ducournau, A., Fablet, R., Rousseau, F.: Brain MRI super-resolution using deep 3D convolutional networks. In: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), pp. 197–200 (2017)
Pham, D.L., Xu, C., Prince, J.L.: Current methods in medical image segmentation. Ann. Rev. Biomed. Eng. 2(1), 315–337 (2000)
Smith, L.N.: A disciplined approach to neural network hyper-parameters: part 1-learning rate, batch size, momentum, and weight decay. arXiv preprint arXiv:1803.09820 (2018)
Wang, S.L., et al.: Thalamus segmentation using convolutional neural networks. In: Medical Imaging 2021: Image Processing, vol. 11596. International Society for Optics and Photonics, SPIE (2021)
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), September 2018
Zhao, C., Dewey, B.E., Pham, D.L., Calabresi, P.A., Reich, D.S., Prince, J.L.: SMORE: a self-supervised anti-aliasing and super-resolution algorithm for MRI using deep learning. IEEE Trans. Med. Imaging 40, 805–817 (2020)
Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: UNet++: a nested U-net architecture for medical image segmentation. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 3–11. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_1
Acknowledgements
This material is supported by the National Science Foundation Graduate Research Fellowship under Grant No. DGE-1746891. This work was partially supported by National Multiple Sclerosis Society grant RG-1907-34570, and by the Center for Neuroscience and Regenerative Medicine in the Department of Defense. Theoretical development partially supported by DoD/CDMRP grant MS190131.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Remedios, S.W., Han, S., Dewey, B.E., Pham, D.L., Prince, J.L., Carass, A. (2021). Joint Image and Label Self-super-Resolution. In: Svoboda, D., Burgos, N., Wolterink, J.M., Zhao, C. (eds) Simulation and Synthesis in Medical Imaging. SASHIMI 2021. Lecture Notes in Computer Science(), vol 12965. Springer, Cham. https://doi.org/10.1007/978-3-030-87592-3_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-87592-3_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87591-6
Online ISBN: 978-3-030-87592-3
eBook Packages: Computer ScienceComputer Science (R0)
