Skip to main content
SpringerLink
Log in

Hardware architecture optimization for high-frequency zeroing and LFNST in H.266/VVC based on FPGA

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

To reduce the hardware implementation resource consumption of the two-dimensional transform component in H.266 VVC, a unified hardware structure is proposed that supports full-size Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), and full-size Low-Frequency Non-Separable Transform (LFNST). This paper presents an area-efficient hardware architecture for two-dimensional transforms based on a general Regular Multiplier (RM) and a high-throughput hardware design for LFNST in the context of H.266/VVC. The first approach utilizes the high-frequency zeroing characteristics of VVC and the symmetric properties of the DCT-II matrix, allowing the RM-based architecture to use only 256 general multipliers in a fully pipelined structure with a parallelism of 16. The second approach optimizes the transpose operation of the input matrix for LFNST in a parallelism of 16 architecture, aiming to save storage and logic resources.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

No datasets were generated or analyzed during the current study.

References

  1. Bross, B., Chen, J., Liu, S., et al: Versatile video coding editorial refinements on draft 10 [eb/ol] ((2020-11-24)[2022-11-15]). https://jvet-experts.org/doc_end_user/documents/20_Teleconference/wg11/JVET-T2001-v2.zip

  2. Bross, B., Wang, Y.K., Ye, Y., Liu, S., Chen, J., Sullivan, G.J., Ohm, J.R.: Overview of the versatile video coding (VVC) standard and its applications. IEEE Trans. Circuits Syst. Video Technol. 31(10), 3736–3764 (2021). https://doi.org/10.1109/TCSVT.2021.3101953

    Article  Google Scholar 

  3. Sullivan, G.J., Ohm, J.R., Han, W.J., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. (2012). https://doi.org/10.1109/tcsvt.2012.2221191

    Article  Google Scholar 

  4. Bossen, F., Suhring, K., Wieckowski, A., Liu, S.: VVC complexity and software implementation analysis. IEEE Trans. Circuits Syst. Video Technol. 31(10), 3765–3778 (2021). https://doi.org/10.1109/tcsvt.2021.3072204

    Article  Google Scholar 

  5. Zhao, X., Kim, S.H., Zhao, Y., Egilmez, H.E., Koo, M., Liu, S., Lainema, J., Karczewicz, M.: Transform coding in the VVC standard. IEEE Trans. Circuits Syst. Video Technol. (2021). https://doi.org/10.1109/tcsvt.2021.3087706

    Article  Google Scholar 

  6. Yan, S.: High efficiency video coding. Chin. J. Comput. 36, 2340–2355 (2013)

    Google Scholar 

  7. Bross, B., Andersson, K., Blaser, M., Drugeon, V., Kim, S.H., Lainema, J., Li, J., Liu, S., Ohm, J.R., Sullivan, G.J., Yu, R.: General video coding technology in responses to the joint call for proposals on video compression with capability beyond HEVC. IEEE Trans. Circuits Syst. Video Technol. 30(5), 1226–1240 (2020). https://doi.org/10.1109/tcsvt.2019.2949619

    Article  Google Scholar 

  8. Silveira, B., Neto, L., Palomino, D., Diniz, C., Correa, G.: Multiple transform selection hardware design for 4k@60fps real-time versatile video coding. In: 2022 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–5 (2022). https://doi.org/10.1109/ISCAS48785.2022.9937964

  9. Silveira, B., Palomino, D., Diniz, C., Correa, G.: A hardware design for the multi-transform module of the versatile video coding standard. In: 2023 36th SBC/SBMicro/IEEE/ACM Symposium on Integrated Circuits and Systems Design (SBCCI), pp. 1–6 (2023). https://doi.org/10.1109/SBCCI60457.2023.10261970

  10. Garrido, M.J., Pescador, F., Chavaríras, M., Lobo, P.J., Sanz, C.: A high performance FPGA-based architecture for the future video coding adaptive multiple core transform. IEEE Trans. Consum. Electron. 64(1), 53–60 (2018). https://doi.org/10.1109/TCE.2018.2812459

    Article  Google Scholar 

  11. Garrido, M.J., Pescador, F., Chavarrías, M., Lobo, P.J., Sanz, C.: A 2-D multiple transform processor for the versatile video coding standard. IEEE Trans. Consum. Electron. 65(3), 274–283 (2019). https://doi.org/10.1109/TCE.2019.2913327

    Article  Google Scholar 

  12. Garrido, M.J., Pescador, F., Chavarrías, M., Lobo, P.J., Sanz, C., Paz, P.: An FPGA-based architecture for the versatile video coding multiple transform selection core. IEEE Access 8, 81887–81903 (2020). https://doi.org/10.1109/ACCESS.2020.2991299

    Article  Google Scholar 

  13. Fan, Y., Zeng, Y., Sun, H., Katto, J., Zeng, X.: A pipelined 2D transform architecture supporting mixed block sizes for the VVC standard. IEEE Trans. Circuits Syst. Video Technol. 30(9), 3289–3295 (2020). https://doi.org/10.1109/TCSVT.2019.2934752

    Article  Google Scholar 

  14. Dempster, A., Macleod, M.: Use of minimum-adder multiplier blocks in fir digital filters. IEEE Trans. Circuits Syst. II Analog Digit. Signal Process. 42(9), 569–577 (1995). https://doi.org/10.1109/82.466647

    Article  Google Scholar 

  15. Goebel, J., Agostini, L., Zatt, B., Porto, M.: Low-frequency non-separable transform hardware system design for the VVC encoder. In: 2022 35th SBC/SBMicro/IEEE/ACM Symposium on Integrated Circuits and Systems Design (SBCCI), pp. 1–6 (2022). https://doi.org/10.1109/SBCCI55532.2022.9893228

  16. Koo, M., Salehifar, M., Lim, J., Kim, SH.: Low frequency non-separable transform (LFNST). In: 2019 Picture Coding Symposium (PCS), pp. 1–5 (2019). https://doi.org/10.1109/PCS48520.2019.8954507

  17. Mert, AC., Kalali, E., Hamzaoglu, I.: An FPGA implementation of future video coding 2D transform. In: 2017 IEEE 7th International Conference on Consumer Electronics-Berlin (ICCE-Berlin), pp. 31–36 (2017). https://doi.org/10.1109/ICCE-Berlin.2017.8210582

  18. Hao, Z., Sun, H., Xiang, G., Zhang, P., Zeng, X., Fan, Y.: A reconfigurable multiple transform selection architecture for VVC. IEEE Trans. Very Large Scale Integr. Syst. 31(5), 658–669 (2023). https://doi.org/10.1109/TVLSI.2023.3245291

    Article  Google Scholar 

  19. Goebel, J., Costa, V., Agostini, L., Zatt, B., Porto, M.: A high-throughput design for the H.266/VVC low-frequency non-separable transform. In: 2022 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1798–1802 (2022). https://doi.org/10.1109/ISCAS48785.2022.9937597

Download references

Acknowledgements

This work was supported in part by the National Key R &D Program of China under Grant 2022C01068, and in part by National College Students’ innovation and entrepreneurship training program under Grant 202310336045.

Author information

Authors and Affiliations

  1. The School of Electronics and Information, Hangzhou Dianzi University, 310018, Hangzhou, China

    Junxiang Zhang, Qinghua Sheng, Rui Pan, Jiawei Wang, Kuan Qin & Xiaoyan Niu

  2. The School of Electronic Engineering, Dianzi University Information Engineering College, Hangzhou, 311305, China

    Xiaofang Huang

Authors
  1. Junxiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qinghua Sheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rui Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiawei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kuan Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiaofang Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xiaoyan Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Zhang wrote the main manuscript text. Sheng, Pan and Niu guided the overall direction of the work. Zhang, Huang, Qin and Wang prepared Figs. 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10. All authors reviewed the manuscript.

Corresponding author

Correspondence to Junxiang Zhang.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, J., Sheng, Q., Pan, R. et al. Hardware architecture optimization for high-frequency zeroing and LFNST in H.266/VVC based on FPGA. J Real-Time Image Proc 21, 90 (2024). https://doi.org/10.1007/s11554-024-01470-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-024-01470-4

Keywords

Search

Navigation