Skip to main content
SpringerLink
Log in

Compressed video quality enhancement algorithm based on 3D-CNNs

  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

By exploring the current block-based lossy video coding process and compressed videos, this paper finds two unique characteristics namely quality fluctuation and pixel deficiency. And we use 3D convolutional neural network (3D-CNN) to make full use of the limited temporal and spatial information in compressed video and build compressed video quality enhancement network (CVQENet) to improve the compressed video quality. The experimental results show that compared with the videos encoded by High Efficiency Video Coding (HEVC/H.265), the mean value of the Peak Signal-to-Noise Ratio (PSNR) of enhanced videos has been improved by 0.4652 dB under Low Delay (LD) configuration with Quantization Parameter (QP) is set to 37.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Sullivan, G. J., Ohm, J. R., Han, W. J., & Wiegand, T. (2013). Overview of the high efficiency video coding (HEVC) standard. IEEE Transactions on Circuits & Systems for Video Technology, 22(12), 1649–1668.

    Article  Google Scholar 

  2. Liu, M., Liu, C., Chen, Y., Yan, Z., & Zhao, N. (2022). Radio frequency fingerprint collaborative intelligent blind identification for green radios. IEEE Transactions on Green Communications and Networking, 7, 1–10.

    Google Scholar 

  3. Liu, M., Zhang, H., Liu, Z., & Zhao, N. (2022). Attacking spectrum sensing with adversarial deep learning in cognitive radio-enabled internet of things. IEEE Transactions on Reliability. https://doi.org/10.1109/TR.2022.3179491

    Article  Google Scholar 

  4. Lim, B., Son, S., Kim, H., Nah, S., & Mu Lee, K. (2017). Enhanced deep residual networks for single image super-resolution (2017). In IEEE Conference on Computer Vision and Pattern Recognition Workshops, Hawaii, USA (pp. 136–144).

  5. Zheng, B., Chen, Y., Tian, X., Zhou, F., & Liu, X. (2018). Implicit dual-domain convolutional network for robust color image compression artifact reduction. IEEE Transactions on Circuits and Systems for Video Technology, 2018(99), 1–1.

    Google Scholar 

  6. Dong, C., Deng, Y., Loy, C. C., & Tang, X. (2015). Compression artifacts reduction by a deep convolutional network. In 2015 IEEE International Conference on Computer Vision, Santiago, Chile (pp. 576–584).

  7. Tai, Y., Yang, J., Liu, X., & Xu, C. (2017). MemNet: A persistent memory network for image restoration. In 2017 IEEE International Conference on Computer Vision, Venice, Italy (pp. 4539–4547).

  8. Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017). Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 26(7), 3142–3155.

    Article  MathSciNet  MATH  Google Scholar 

  9. Wang, T., Chen, M., & Chao, H. (2017). A novel deep learning-based method of improving coding efficiency from the decoder-end for HEVC. In 2017 Data Compression Conference, Snowbird, United States (pp. 410–419).

  10. Yang, R., Xu, M., Liu, T., Wang, Z., & Guan, Z. (2017). Enhancing quality for HEVC compressed videos. IEEE Transactions on Circuits and Systems for Video Technology. https://doi.org/10.1109/TCSVT.2018.2867568

    Article  Google Scholar 

  11. Yang, R., Sun, X., Xu, M., & Zeng, W. (2019). Quality-gated convolutional LSTM for enhancing compressed video. In 2019 IEEE International Conference on Multimedia and Expo, Shanghai, China (pp. 532–537).

  12. Yang, R., Mentzer, F., Gool, L. V., & Timofte, R. (2020). Learning for video compression with hierarchical quality and recurrent enhancement. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA (pp. 6628–6637).

  13. Feng, L., Zhang, X., Wang, S., Wang, Y., & Ma, S. (2019). Coding prior based high efficiency restoration for compressed video. In 2019 IEEE International Conference on Image Processing, Taiwan, China (pp. 769–774).

  14. Yang, R., Xu, M., Wang, Z., & Li, T. (2018). Multi-frame quality enhancement for compressed video. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA (pp. 6664–6673).

  15. Wang, T., Xiao, W., Chen, M., & Chao, H. (2018). The multi-scale deep decoder for the standard HEVC bitstreams. In Data Compression Conference, Snowbird, United States (pp. 197–206).

  16. He, X., Hu, Q., Zhang, X., Zhang, C., Lin, W., & Han, X. (2018). Enhancing HEVC compressed videos with a partition-masked convolutional neural network. In 2018 IEEE International Conference on Image Processing, Athens, Greece (pp. 216–220).

  17. Ma, L., Tian, Y., & Huang, T. (2018). Residual-based video restoration for HEVC intra coding. In 2018 IEEE Fourth International Conference on Multimedia Big Data, Xi’an, China (pp. 1–7).

  18. Kang, J., Kim, S., & Lee K. M. (2017). Multi-modal/multi-scale convolutional neural network based in-loop filter design for next generation video codec. In 2017 IEEE International Conference on Image Processing, Beijing, China (pp. 26–30).

  19. Song, X., Yao, J., Zhou, L., Wang, L., Wu, X., Xie, D., & Pu, S. (2018). A practical convolutional neural network as loop filter for intra frame. In 2018 IEEE International Conference on Image Processing, Madrid, Spain (pp. 1133–1137).

  20. Meng, X., Deng, X., Zhu, S., Liu, S., Wang, C., Chen, C., & Zeng, B. (2020). Mganet: A robust model for quality enhancement of compressed video. In 2020 Data Compression Conference, Snowbird, United States.

  21. Qiu, Z., Yao, T., Mei, T. (2017). Learning spatio-temporal representation with pseudo-3D residual networks. In 2017 IEEE International Conference on Computer Vision, Venice, Italy (pp. 5533–5541).

  22. Guan, Z., Xing, Q., Xu, M., Yang, R., Liu, T., & Wang, Z. (2021). MFQE 2.0: A new approach for multi-frame quality enhancement on compressed video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43, 949–963.

    Article  Google Scholar 

Download references

Acknowledgements

This research is supported by The Beijing Natural Science Foundation (Grant No. 4212001) and by Key R &D and Transformation Program of Qinghai Province (Grant No. 2022-QY-205).

Author information

Authors and Affiliations

  1. College of Physics and Electronic Information Engineering, Qinghai Minzu University, Xining, 810000, Qinghai, China

    Shanji Chen & Lingfei Zhang

  2. The Information Department, Beijing University of Technology, Beijing, 100124, China

    Pengyu Liu, Yue Zhang, Sirong Wang & Jing Yuan

  3. Advanced Information Network Beijing Laboratory, Beijing, 100124, China

    Pengyu Liu, Yue Zhang, Sirong Wang & Jing Yuan

  4. Computational Intelligence and Intelligent Systems Beijing Key Laboratory, Beijing, 100124, China

    Pengyu Liu, Yue Zhang, Sirong Wang & Jing Yuan

Authors
  1. Shanji Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pengyu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yue Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lingfei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sirong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jing Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pengyu Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, S., Liu, P., Zhang, Y. et al. Compressed video quality enhancement algorithm based on 3D-CNNs. Wireless Netw (2023). https://doi.org/10.1007/s11276-023-03392-8

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11276-023-03392-8

Keywords

Search

Navigation