Skip to main content
SpringerLink
Log in

Real-time H264/AVC encoder based on enhanced frame level parallelism for smart multicore DSP camera

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

The latest generation of multicore digital signal processors (DSP), their high computing power, low consumption, and integrated peripherals will allow them to be embedded in the next generation of smart camera. Such DSPs allow designers to evolve the vision landscape and simplify the developer’s tasks to run more complex image and video processing applications without the need to burden a separate personal computer. This paper explains the exploitation of the computing power of a multicore DSP TMS320C6472 to implement a real-time H264/AVC video encoder. This work can be considered as a milestone for the implementation of the new High Efficiency Video Coding standard (HEVC-H265). In fact, to improve the encoding speed, enhanced Frame Level Parallelism (FLP) approach is presented and implemented. A real-time fully functional video demo is given, taken into account video capture and bitstream storage. Experimental results show how we efficiently exploit the potentials and the features of the multicore platform without inducing video quality degradation in terms of PSNR or bitrate increase. The enhanced FLP using five DSP cores achieves a speedup factor of more than four times in average compared to a mono-core implementation for Common Intermediate Format (CIF 352 × 288), Standard Definition (SD 720 × 480), and High Definition (HD 1280 × 720) resolutions. This optimized implementation allows us to meet the real-time compliant by reaching an encoding speed of 99 f/s (frame/second) and 30 f/s for CIF and SD resolutions respectively, and saves up to 77 % of encoding time for HD resolution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Smart camera. http://en.wikipedia.org/wiki/Smart_camera

  2. Shi, Y., Lichman, S.: Smart cameras: a review. http://www.nicta.com.au/__data/assets/pdf_file/0004/16456/smart_camera_review.pdf

  3. Megapixel OEM smart camera module with removable storage. http://www.mosaicengineering.com/docs/OEM-SCM.2010-01-26.pdf

  4. Smart camera manufacturers and products links. http://www.smartcamera.it/links.htm#Top

  5. Freescale machine vision and camera. http://www.freescale.com/webapp/sps/site/application.jsp?code=APLMAVISMCA

  6. TMS320C66x high-performance multicore DSPs for video surveillance. http://www.ti.com/lit/ml/sprt643/sprt643.pdf

  7. Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG, draft ITU-T recommendation and final draft international standard of joint video specification (ITU-T Rec. H.264 ISO/IEC 14496-10 AVC), JVT-G050, (2003)

  8. Sullivan, G.J., Ohm, J.-R., Han, W.-J., Wiegand, T.: Overview of the high efficiency video coding (HEVC) Stand. IEEE Trans Circuit Syst Video Technology 22(12), 1649–1668 (2012)

    Article  Google Scholar 

  9. Xiao, Z., Le, S., Baas, B.: A fine-grained parallel implementation of a H.264/AVC encoder on a 167-processor computational platform, signals, systems and computers (ASILOMAR), 2011 conference record of the forty fifth asilomar conference on pp. 2067, 2071 (2011)

  10. Sun, S., Wang, D., Chen, S., Perrott, R., Chapman, B., Subhlok, J., Mello, R., Yang, L (eds) A highly efficient parallel algorithm for H.264 encoder based on macro-block region partition, high performance computing and communications. Springer, Berlin, vol. 4782, pp. 577–585 (2007)

  11. H264/AVC software joint model JM: http://iphome.hhi.de/suehring/tml/download/old_jm/

  12. Zhao, Z., Liang, P.: A highly efficient parallel algorithm for H.264 video encoder, acoustics, speech and signal processing, 2006. ICASSP 2006 In: Proceedings of 2006 IEEE international conference on, vol. 5, pp. V, V (2006)

  13. Chen, Y.-K., Tian, X., Ge, S., Girkar, M.: Towards efficient multi-level threading of H.264 encoder on Intel hyper-threading architectures, parallel and distributed processing symposium, 2004. In: Proceedings of 18th international. p. 63 (2004)

  14. Sankaraiah, S., Lam, H.S., Eswaran, C., Abdullah, J.: GOP level parallelism on H.264 video encoder for multicore architecture international conference on circuits, system and simulation(ICCSS 2011), IPCSIT vol. 7, pp. 127–132, May 2011, IACSIT Press, Singapore, ISSN:2010-460X, ISBN : 978-981-08-8638-7, Bangkok, (2011)

  15. Sankaraiah, S., Shuan, L.H., Eswaran, C., Abdullah, J.: Performance optimization of video coding process on multi-core platform using gop level parallelism. International journal of parallel programming, ISSN:1573–7640. doi:10.1007/s10766-013-0267-4 (2013)

  16. Rodriguez, A., Gonzalez, A., Malumbres, M.P., Hierarchical parallelization of an H.264/AVC video encoder, parallel computing in electrical engineering, 2006. PAR ELEC 2006. International symposium on, p. 363 (2006)

  17. Chen, S., Chen, S., Gu, H., Chen, H., Yin, Y., Chen, X., Sun, S., Liu, S., Wang, Y.: Mapping of H.264/AVC encoder on a hierarchical chip multicore DSP platform, high performance computing and communications (HPCC), 2010 12th IEEE international conference on, pp. 465–470, (2010)

  18. Yang, M.-J., Tham, J.-Y., Rahardja, S., Wu, D.-J., Real-time H.264 encoder implementation on a low-power digital signal processor, multimedia and expo, 2009. ICME 2009. IEEE international conference on, pp. 1150–1153, (2009)

  19. Zrida, H.K., Jemai, A., Ammari, A.C., Abid, M.: High level H.264/AVC video encoder parallelization for multiprocessor implementation, design, automation and test in Europe conference and exhibition, 2009. DATE ‘09, pp. 940–945 (2009)

  20. Rodrigues, A., Roma, N., Sousa, L.: Open platform for designing parallel H.264/AVC video encoders on multi-core systems. In: Proceedings of the 20th international workshop on network and operating systems support for digital audio and video (NOSSDAV ‘10). ACM, New York, 81–86. p. 264 (2010). http://doi.acm.org/10.1145/1806565.1806586

  21. Lehtoranta, O., Hämäläinen, T., Lappalainen, V., Mustonen, J.: Parallel implementation of video encoder on quad DSP system. Microprocess Microsyst 26(1), 1–15 (2002)

    Article  Google Scholar 

  22. de Medeiros, B.A.: Video coding on multicore graphics processors (GPUs), dissertation submitted to obtain the master degree in information systems and computer engineering, High Institute of Techniques, Technical University of Lisboa (2012)

  23. Ji, F., Li, X.-Y., Yang, C.-L.: An algorithm based on AVS encoding on FPGA multi-core pipeline, computational and information sciences (ICCIS), 2013 Fifth international conference on, pp. 1521–1524 (2013)

  24. Peng, X., Xu, J., Zhou, Y., Wu, F.: Highly parallel line-based image coding for many cores. Image Process IEEE Trans 21(1), 196–206 (2012)

    Article  MathSciNet  Google Scholar 

  25. Su, H., Wen, M., Wu, N., Ren, J., Zhang, C.: Efficient parallel video processing techniques on GPU: from framework to implementation. Sci World J 2014(716020), 19 (2014)

    Google Scholar 

  26. Adeyemi-Ejeye, A.O., Walker, S.: 4kUHD H264 wireless live video streaming using CUDA. J Elect Comput Eng 2014(183716), 12 (2014)

    Google Scholar 

  27. Elhamzi, W., Dubois, J., Miteran, J., Atri, M., Heyrman, B., Ginhac, D. (2013) Efficient smart-camera accelerator: a configurable motion estimator dedicated to video codec. J Syst Archit Part A, 59(10): 870–877, ISSN 1383-7621

  28. Jo, S., Jo, S.H., Song, Y.H.: Exploring parallelization techniques based on OpenMP in H.264/AVC encoder for embedded multi-core processor. J Syst Archit 58(9), 339–353 (2012)

    Article  Google Scholar 

  29. TMS320C6472 datasheet. http://www.ti.com/lit/ds/sprs612g/sprs612g.pdf

  30. Multicore DSP vs GPUs. http://www.sagivtech.com/contentManagment/uploadedFiles/fileGallery/Multi_core_DSPs_vs_GPUs_TI_for_distribution.pdf

  31. TMS32C6472 low power consuption. http://www.ti.com/lit/wp/spry130/spry130.pdf

  32. TMS320C6472 Power consumption summary. http://www.ti.com/lit/an/sprab76a/sprab76a.pdf

  33. power spreadsheet for power consumption estimation. http://www.ti.com/general/docs/lit/getliterature.tsp?baseLiteratureNumber=sprab76&fileType=zip&track=no

  34. Werda, I., Dammak, T., Grandpierre, T., Ayed, M., Masmoudi, N.: Real-time H.264/AVC baseline decoder implementation on TMS320C6416. J Real Time Image Process, Springer. vol. 7. pp. 215–232 (2012)

  35. Bahri, N., Werda, I., Grandpierre, T., Ayed, M.B, Masmoudi, N., Akil, M.: Optimizations for real-time implementation of H264/AVC video encoder on DSP processor. International review on computers and software, 8(9) (2013)

  36. Xiph.org video test media. https://media.xiph.org/video/derf/

  37. Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG organizations. http://www.itu.int/en/ITU-T/studygroups/com16/video/Pages/jvt.aspx

  38. Open source computer vision library. http://opencv.org/

  39. TI network developer’s kit (NDK) v2.21 user’s guide. http://www.ti.com/lit/ug/spru523h/spru523h.pdf

  40. TMS320C6472 chip support library API reference guide. http://software-dl.ti.com/sdoemb/sdoemb_public_sw/csl/CSL_C6472/latest/index_FDS.html

  41. Throughput application report for the TMS320C6472 DSP. http://www.ti.com/lit/an/spraay0a/spraay0a.pdf

Download references

Acknowledgments

This work is fruit of cooperation between Sfax National School of Engineers and ESIEE Engineering PARIS. It is supported by the French ministries of Foreign Affairs and Tunisian ministry for Higher Education and Scientific Research in the context of Hubert Curien Partnership (PHC UTIQUE) under the CMCU project number 12G1108.

Author information

Authors and Affiliations

  1. LETI Laboratory, National School of Engineers, University of Sfax, Sfax, Tunisia

    Nejmeddine Bahri, Nidhameddine Belhadj, Mohamed Ali Ben Ayed & Nouri Masmoudi

  2. LIGM Laboratory, ESIEE Engineering, University Paris-EST, Créteil, France

    Thierry Grandpierre & Mohamed Akil

Authors
  1. Nejmeddine Bahri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nidhameddine Belhadj
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thierry Grandpierre
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohamed Ali Ben Ayed
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nouri Masmoudi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mohamed Akil
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nejmeddine Bahri.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bahri, N., Belhadj, N., Grandpierre, T. et al. Real-time H264/AVC encoder based on enhanced frame level parallelism for smart multicore DSP camera. J Real-Time Image Proc 12, 791–812 (2016). https://doi.org/10.1007/s11554-014-0470-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-014-0470-6

Keywords

Search

Navigation