Skip to main content
SpringerLink
Log in

SMC-SRGAN-Lightning super-resolution algorithm based on optical micro-scanning thermal microscope image

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Due to the low spatial resolution of the existing optical micro-scanning thermal microscope imaging system, the acquired micro-scanning infrared images have inferior image quality and low contrast. Deep learning methods, represented by SRGAN, have shown promising results in super-resolution. However, this method still has artifacts, blurriness, low spatial resolution, and slow reconstruction speed. Therefore, we propose the SMC-SRGAN-Lightning super-resolution algorithm based on optical micro-scanning thermal microscope images in this study. Firstly, we enhance the model’s attention to features and improve the details and clarity of the reconstructed images. Removing the BN layer in residual blocks, replacing the ReLU with SMU, and introducing the CBAM to construct the SMC module. Secondly, we incorporate the attention mechanism SEnet into the Bottleneck structure of MobileNetV2. Reducing the channels in the first 1 × 1 convolution layer to 1/4 and creating the SE-MobileNetV2 module. It can enhance the model’s focus on essential features, computational efficiency, and accuracy. Finally, to validate the effectiveness of our method, we compare it with four other super-resolution algorithms on public datasets and images obtained from the optical micro-scanning thermal microscope imaging system. Experimental results indicate that our method improves image clarity, preserving details, and textures. Comprehensively considering super-resolved image quality and time costs, our method is superior to other methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data availability

The dataset is available in the [FLIR dataset] repository, [https://www.flir.com/oem/adas/adas-dataset-form/]. An additional portion of the dataset analyzed during the current study is available from the corresponding authors upon reasonable request.

References

  1. Staffa, E., Bernard, V., Kubicek, L., et al.: Infrared thermography as option for evaluating the treatment effect of percutaneous transluminal angioplasty by patients with peripheral arterial disease. Vascular 25(1), 42–49 (2017)

    Article  Google Scholar 

  2. Jingjing, Y., Zhimao, W.U., Erzhen, M.U., et al.: Research and applications of infrared thermal microscope imaging system. Infrared Technol. 39(11), 973–978 (2017)

    Google Scholar 

  3. Jing, D., Yonggang, S., Bin, W., et al.: Micro-fourier-transform infrared imaging for chemical analysis of fibrous microplastics in coastal seawater samples. Environ. Chem. 37(06), 1440–1443 (2018)

    Google Scholar 

  4. Yan, L., Yuwei, Z., Xiaodong, X., et al.: High temporal resolution transient thermoreflectance microscope. J. Astronaut. Metrol. Meas. 42(5), 73–78 (2022)

    Google Scholar 

  5. Huang, Y.F., Shih, L.P., Tsai, C.H., et al.: Describing video scenarios using deep learning techniques. Int. J. Intell. Syst. 36(6), 2465–2490 (2021)

    Article  Google Scholar 

  6. Li, Q., Yang, Y., Guo, Y., et al.: Performance evaluation of deep learning classification network for image features. IEEE Access 9, 9318–9333 (2021)

    Article  Google Scholar 

  7. Wickramasinghe, C.S., Marino, D.L., Manic, M.: ResNet autoencoders for unsupervised feature learning from high-dimensional data: deep models resistant to performance degradation. IEEE Access 9, 40511–40520 (2021)

    Article  Google Scholar 

  8. Ledig, C., Theis, L., Huszár, F., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 105–114 (2017)

  9. Dong, C., Loy, C.C., He, K., et al.: Learning a deep convolutional network for image super-resolution. In: Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, Proceedings, Part IV 13, pp. 184–199. Springer (2014)

  10. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks, pp. 1646–1654. IEEE (2016)

  11. Xintao, W., Ke, Y., Shixiang, W., et al.: Esrgan: enhanced super-resolution generative adversarial networks. In: Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pp. 63–79 (2019)

  12. Zhang, W., Liu, Y., Dong, C., et al.: RankSRGAN: generative adversarial networks with ranker for image super-resolution. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3096–3105 (2019)

  13. Ning, K., Zhang, Z., Han, K., Han, SiYu., Zhang, X.: Multi-frame super-resolution algorithm based on a WGAN. IEEE Access 9, 85839–85851 (2021)

    Article  Google Scholar 

  14. Meijing, G., Weiqi, J., Yinan, C., et al.: Thermal microscope imaging system for semiconductor device and IC invalidation analysis. In: International Symposium on Photoelectronic Detection and Imaging 2007: Photoelectronic Imaging and Detection. SPIE 6621, 62117–62117 (2008)

    Google Scholar 

  15. Meijing, G., Xia, W., Weiqi, J., et al.: Digital thermal microscope for biomedical application. In: 2007 IEEE/ICME International Conference on Complex Medical Engineering, pp. 1847–1850. IEEE (2007)

  16. Meijing, G., Wang Jingyuan, Xu., Wei, G.C.: High-resolution over-sampling reconstruction algorithm for a microscanning thermal microscope imaging system. Infrared Phys. Technol. 76, 661–666 (2016)

    Article  Google Scholar 

  17. Gao, M., Xu, J., Tan, A., et al.: Error correction based on micro-scanning preprocessing for an optical micro-scanning thermal microscope imaging system. Infrared Phys. Technol. 83, 252–258 (2017)

    Article  Google Scholar 

  18. Bozhi, Z., Meijing, G., Pan, C., et al.: Infrared and visible image fusion based on contrast enhancement guided filter and infrared feature decomposition. Infrared Phys. Technol. 127, 104404 (2022)

    Article  Google Scholar 

  19. Tan, A., Liao, H., Zhang, B., et al.: Infrared image enhancement algorithm based on detail enhancement guided image filtering. Vis. Comput. 39, 1–12 (2022)

    Google Scholar 

  20. Gao, M., Jin, W., Wang, X., et al.: Zero calibration for the designed micro scanning thermal microscope imaging system. Acta Opt. Sinica 29(8), 2175–2179 (2009)

    Article  Google Scholar 

  21. Gao, M., Tan, A., Zhang, B., et al.: Design and realization of high resolution optical micro-scanning thermal microscope imaging system. Infrared Phys. Technol. 95, 46–52 (2018)

    Article  Google Scholar 

  22. Bee, L., Sanghyun, S., Heewon, K., et al.: Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 1132–1140 (2017)

  23. Koushik, B., Sandeep, K., Shilpak, B., et al.: Smooth maximum unit: smooth activation function for deep networks using smoothing maximum technique. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022), pp. 784–793 (2022)

  24. Shao, B.: Research on infrared super resolution imaging and small target classification technology. In: Shanghai: University of Chinese Academy of Sciences (Shanghai Institute of Technical Physics, Chinese Academy of Sciences) PhD Thesis (2019)

  25. Yaohong, C., Kang Jin, U., Gaopeng, Z., et al.: Real-time infrared image detail enhancement based on fast guided image filter and plateau equalization. Appl. Opt. 59(21), 6407–6416 (2020)

    Article  Google Scholar 

  26. Ferzan, K., Yusuf, C., Zafer, C.: Infrared image enhancement model based on gravitational force and lateral inhibition networks. Infrared Phys. Technol. 100, 15–27 (2019)

    Article  Google Scholar 

  27. Ying, G., Lun, L., Peng, W.: Real time interpolation algorithm based on Lanczos kernel. J. Commun. 38(6), 142–147 (2017)

    Google Scholar 

  28. Passarella, L.S., Mahajan, S., Pal, A., et al.: Reconstructing high resolution ESM data through a novel fast super resolution convolutional neural network (FSRCNN). Geophys. Res. Lett. 49(4), 2021097571 (2022)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by National Nature Science Foundation of China (61971373) and Hebei Natural Science Foundation (F2023105001).

Author information

Authors and Affiliations

  1. College of Information and Electronic, Beijing Institute of Technology, Beijing, 100081, China

    Meijing Gao & Yunjia Xie

  2. The Key Laboratory for Special Fiber and Fiber Sensor of Hebei Province, School of Information Science and Engineering, Yanshan University, Qinhuangdao, 066004, China

    Yang Bai, Shiyu Li & Zhilong Li

  3. Tangshan Research Institute of Beijing Institute of Technology, Beijing, China

    Meijing Gao

  4. School of Aerospace Engineering, Beijing Institute of Technology, Beijing, 100081, China

    Bozhi Zhang

Authors
  1. Meijing Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yunjia Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bozhi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shiyu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhilong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meijing Gao.

Ethics declarations

Conflict of interest

The authors declared that they have no conflicts of interest to this work. We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, M., Bai, Y., Xie, Y. et al. SMC-SRGAN-Lightning super-resolution algorithm based on optical micro-scanning thermal microscope image. Vis Comput (2024). https://doi.org/10.1007/s00371-023-03247-5

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00371-023-03247-5

Keywords

Search

Navigation