Skip to main content
SpringerLink
Log in

TransEM: Residual Swin-Transformer Based Regularized PET Image Reconstruction

  • Conference paper
  • First Online:
Medical Image Computing and Computer Assisted Intervention – MICCAI 2022 (MICCAI 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13434))

Abstract

Positron emission tomography (PET) image reconstruction is an ill-posed inverse problem and suffers from high level of noise due to limited counts received. Recently deep neural networks especially convolutional neural networks (CNN) have been successfully applied to PET image reconstruction. However, the local characteristics of the convolution operator potentially limit the image quality obtained by current CNN-based PET image reconstruction methods. In this paper, we propose a residual swin-transformer based regularizer (RSTR) to incorporate regularization into the iterative reconstruction framework. Specifically, a convolution layer is firstly adopted to extract shallow features, then the deep feature extraction is accomplished by the swin-transformer layer. At last, both deep and shallow features are fused with a residual operation and another convolution layer. Validations on the realistic 3D brain simulated low-count data show that our proposed method outperforms the state-of-the-art methods in both qualitative and quantitative measures.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 44.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Gunn, R., Slifstein, M., Searle, G., Price, J.: Quantitative imaging of protein targets in the human brain with PET. Phys. Med. Biol. 60, 363–411 (2015)

    Article  Google Scholar 

  2. Brooks, R.A.: Statistical limitations in x-ray reconstructive tomography. Med. Phys. 3(4), 237–240 (1976)

    Google Scholar 

  3. Shepp, L., Vardi, Y.: Maximum likelihood reconstruction for emission tomography. IEEE Trans. Med. Imaging 1, 113–122 (1982)

    Article  Google Scholar 

  4. Xie, N., et al.: Penalized-likelihood PET image reconstruction using 3D structural convolutional sparse coding. IEEE Trans. Biomed. Eng. 69, 4–14 (2022)

    Article  Google Scholar 

  5. Chen, S., Liu, H., Shi, P., Chen, Y.: Sparse representation and dictionary learning penalized image reconstruction for positron emission tomography. Phys. Med. Biol. 60, 807–823 (2015)

    Article  Google Scholar 

  6. Chen, S., Liu, H., Hu, Z., Zhang, H., Shi, P., Chen, Y.: Simultaneous reconstruction and segmentation of dynamic PET via low-rank and sparse matrix decomposition. IEEE Trans. Biomed. Eng. 62, 1784–1795 (2015)

    Article  Google Scholar 

  7. Wang, G., Qi, J.: PET image reconstruction using Kernel method. IEEE Trans. Med. Imaging 34, 61–71 (2014)

    Article  Google Scholar 

  8. Feruglio, P., Vinegoni, C., Gros, J., Sbarbati, A., Weissleder, R.: Block matching 3D random noise filtering for absorption optical projection tomography. Phys. Med. Biol. 55, 5401 (2010)

    Article  Google Scholar 

  9. Dutta, J., Leahy, R., Li, Q.: Non-local means denoising of dynamic PET images. PLoS ONE 8, e81390 (2013)

    Article  Google Scholar 

  10. Reader, A., Corda, G., Mehranian, A., Costa-Luis, C., Ellis, S., Schnabel,: J. Deep learning for PET image reconstruction. IEEE Trans. Radiat. Plasma Med. Sci. 5, 1–25 (2020)

    Google Scholar 

  11. Wang, B., Liu, H.: FBP-Net for direct reconstruction of dynamic PET images. Phys. Med. Biol. 65, 235008 (2020)

    Article  Google Scholar 

  12. Cui, J., et al.: PET image denoising using unsupervised deep learning. Eur. J. Nucl. Med. Mol. Imaging 46, 2780–2789 (2019)

    Google Scholar 

  13. Gong, K., et al.: Iterative PET image reconstruction using convolutional neural network representation. IEEE Trans. Med. Imaging 38, 675–685 (2018)

    Article  Google Scholar 

  14. Mehranian, A., Reader, A.: Model-based deep learning PET image reconstruction using forward-backward splitting expectation-maximization. IEEE Trans. Radiat. Plasma Med. Sci. 5, 54–64 (2020)

    Article  Google Scholar 

  15. Lim, H., Chun, I., Dewaraja, Y., Fessler, J.: Improved low-count quantitative PET reconstruction with an iterative neural network. IEEE Trans. Med. Imaging 39, 3512–3522 (2020)

    Article  Google Scholar 

  16. Dosovitskiy, A., et al..: An image is worth 16x16 words: transformers for image recognition at scale. ArXiv Preprint arXiv:2010.11929 (2020)

  17. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems 30 (2017)

    Google Scholar 

  18. Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022 (2021)

    Google Scholar 

  19. Combettes, P.L., Pesquet, J.C.: Proximal splitting methods in signal processing. In: Bauschke, H., Burachik, R., Combettes, P., Elser, V., Luke, D., Wolkowicz, H. (eds.) Fixed-Point Algorithms for Inverse Problems in Science and Engineering. Springer Optimization and its Applications, vol 49. Springer, New York (2011). https://doi.org/10.1007/978-1-4419-9569-8_10

  20. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  21. Wang, G., Qi, J.: Penalized likelihood PET image reconstruction using patch-based edge-preserving regularization. IEEE Trans. Med. Imaging 31, 2194–2204 (2012)

    Article  Google Scholar 

  22. Lange, K., Hunter, D., Yang, I.: Optimization transfer using surrogate objective functions. J. Comput. Graph. Stat. 9, 1–20 (2000)

    MathSciNet  Google Scholar 

  23. Hendrycks, D., Gimpel, K.: Gaussian error linear units (GELUs). ArXiv Preprint arXiv:1606.08415 (2016)

  24. Kingma, D., Ba, J.: Adam: a method for stochastic optimization. ArXiv Preprint arXiv:1412.6980 (2014)

  25. Hudson, H., Larkin, R.: Accelerated image reconstruction using ordered subsets of projection data. IEEE Trans. Med. Imaging 13, 601–609 (1994)

    Article  Google Scholar 

  26. De Pierro, A.: A modified expectation maximization algorithm for penalized likelihood estimation in emission tomography. IEEE Trans. Med. Imaging 14, 132–137 (1995)

    Article  Google Scholar 

  27. Häggström, I., Schmidtlein, C., Campanella, G., Fuchs, T.: DeepPET: a deep encoder-decoder network for directly solving the PET image reconstruction inverse problem. Med. Image Anal. 54, 253–262 (2019)

    Article  Google Scholar 

  28. Cocosco, C., Kollokian, V., Kwan, R., Pike, G., Evans, A.: BrainWeb: online interface to a 3D MRI simulated brain database. Neuroimage 5, 425 (1997)

    Google Scholar 

  29. Siddon, R.: Fast calculation of the exact radiological path for a three-dimensional CT array. Med. Phys. 12, 252–255 (1985)

    Article  Google Scholar 

  30. Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the Talent Program of Zhejiang Province (2021R51004) and by the National Natural Science Foundation of China (U1809204).

Author information

Authors and Affiliations

  1. State Key Laboratory of Modern Optical Instrumentation, Department of Optical Engineering, Zhejiang University, Hangzhou, 310027, China

    Rui Hu & Huafeng Liu

  2. Jiaxing Key Laboratory of Photonic Sensing & Intelligent Imaging, Jiaxing, 314000, China

    Huafeng Liu

  3. Intelligent Optics & Photonics Research Center, Jiaxing Research Institute, Zhejiang University, Jiaxing, 314000, China

    Huafeng Liu

Authors
  1. Rui Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huafeng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huafeng Liu .

Editor information

Editors and Affiliations

  1. Rochester Institute of Technology, Rochester, NY, USA

    Linwei Wang

  2. Chinese University of Hong Kong, Hong Kong, Hong Kong

    Qi Dou

  3. University of Virginia, Charlottesville, VA, USA

    P. Thomas Fletcher

  4. National Center for Tumor Diseases (NCT/UCC), Dresden, Germany

    Stefanie Speidel

  5. Case Western Reserve University, Cleveland, OH, USA

    Shuo Li

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hu, R., Liu, H. (2022). TransEM: Residual Swin-Transformer Based Regularized PET Image Reconstruction. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13434. Springer, Cham. https://doi.org/10.1007/978-3-031-16440-8_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-16440-8_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-16439-2

  • Online ISBN: 978-3-031-16440-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation