Abstract
Using the multiple reference frames compensation in the H264 coder improves the coding efficiency for sequences which contain uncovered backgrounds, repetitive motions and highly textured areas. Unfortunately this technique requires excessive memory and computation resources. In this article, we proposed and implemented a technique based on Markov Random Fields Algorithm relying on robust moving pixel segmentation. By the introduction of this technique, we were able to decrease the number of reference frames from five to three while keeping similar video coding performances. The coding time decreased by 35% and the sequence quality was preserved. After the validation of our idea, we evaluated the processing time of the Markov algorithm on architectures intended for embedded multimedia applications. Both DSP and FPGA implementations were explored. We were able to process 50 frames(128 × 128)/s on the EP1S10 FPGA paltform and 35 frames(128 × 128)/s on the ADSP BF533.
Similar content being viewed by others
References
Wiegand, T., Sullivan, G., Reichel, J., Schwarz, H., Wien, M. (eds.): Joint draft 11 of SVC amendment, JVT-X201, Geneva, Switzerland (2007)
Kamaci, N., Altunbasak, Y.: Performance comparison of the emerging H.264 video coding standard with the existing standards. In: IEEE Intenational Conference on Multimedia and Expo, Baltimore, MD, July (2003)
Wiegand, T., Sullivan, J., Bjøntegaard, G., Luthra, A.: Overview of the H.264/AVC Video Coding Standard. IEEE Trans. Circuits Syst. Video Technol. 13(7), 560–576 (2003)
Wiegand, T., Zhang, X., Girod, B.: Block-based hybrid video coding using motion-compensated long-term memory prediction. In: Proceedings of the Picture Coding Symposium, pp. 153–158. Berlin, Germany, September (1997)
Wiegand, T., Zhang, X., Girod, B.: Long-Term Memory Motion-Compensated Prediction. IEEE Trans. Circuits Syst. Video Technol. 9(1), 70–84 (1999)
ITU-T, Recommendation H.263 (Video Coding for Low Bitrate Communication) Annex U (1999)
Huang, Y.W., et al.: Analysis and complexity reduction of multiple reference frames motion estimation in h.264/avc. IEEE Trans. Circuits Syst. Video Technol. 16(4), 507–522 (2006)
Liu, Z., Li, L., Song, Y., Ikenaga, T.: Goto, VLSI oriented fast multiple reference frame motion estimation algorithm for H.264/AVC. In: IEEE International Conference on Multimedia and Expo, 2–5 July 2007, pp. 1902–1905 (2007)
Su, Y., Sun, M.-T.: Fast multiple reference frame motion estimation for H.264/AVC. IEEE Trans. Circuits Syst. Video Technol. 16(3), 447–452 (2006)
Chen, M.J., et.al.: Efficient multi-frame motion estimation algorithms for mpeg-4 avc/jvt/h.264. In: Proceedings of the 2004 International Symposium on Circuits and Systems, vol. 3, pp. 737–740, May (2004)
Wu, P., Xiao, C.-B.: An adaptive fast multiple reference frames selection algorithm for H.264/AVC. In: IEEE International Conference on Acoustics, Speech and Signal Processing, 2008. ICASSP 2008, pp. 1017–1020. 31 March–4 April (2008)
Heitz, F., Mémin, E., Bouthemy, P.: Markov random fields and paralell algoritms for 2D motion analysis. In: Imacs World Congress on Computation and Applied Mathematics, Dublin, Irlande, Junary (1991)
JM10.2.zip. http://iphome.hhi.de/suehring/tml/. Accessed 15 June 2008
Zhang, J.: Mean field theory in EM procedures for MRFs. IEEE Trans. Signal Processing 40, 2570–2583 (1992)
Geiger, D., Girosi, F.: Parallel and deterministic algorithms for MRFs: surface reconstruction. IEEE Trans. Pan. Anal. Mach. Intell. 13, 410–412 (1991)
Aach, T., Kaup, A., Mester, R.: Statistical model-based change detection in moving video. Signal Processing 31(2), 165–180 (1993)
Luthon, F., Caplier, A., Liévin, M.: Spatiotemporal approach to video segmentation: application to motion detection and lip segmentation. Signal Processing 76(1), 61–80 (1999)
Dumontier, C., Luthon, F., Charras, J-P.: Real-time DSP implementation for MRF-based video motion detection. IEEE Trans. Image Process. 8(10), 1341–1347 (1999)
Caplier, A., Luthon, F., Dumontier, C.: Real-time implementations of an MRF-based motion detection algorithm. Real Time Imaging 4, 41–54 (1998)
Lohier, F., Garda, P., Lacassagne, L.: Procédé et dispositif de traitement de sequences dimages avec masquage. National Patent N FR 62060 L, 3 February 2000, France. International extension pending
Lohier, F., Lcassagne, L., Garda, P.: Masked-Motion-JPEG2000: a new reduced-complexity video sequence compression scheme based on a MRF-motion detection algorithm towards interframe masking, ICSPAT2000. In: International Conference on Signal Processing Application and Technology, Dallas, 16–19 October (2000)
Faura, D., Garda, P.: Masked Motion JPEG 2000: a New Video Compression Scheme Based on JPEG2000, ISCE 2004, London, 1–3 September (2004)
Nios II Embedded Design Suite, http://www.altera.com. Accessed 14 May 2008
ADSP-BF533 EZ-KIT Lite Evaluation System Manual. http://www.analog.com/en/prod/0,2877,BF533-HARDWARE,00.html. Accessed 8 Sept 2008
OV6620 single chip CMOS CIF Color difital camera, http://www.ovt.com/pdfs/pb_6120_6620.pdf. Accessed 5 April 2008
Lancelot User Manual, Product Brochure, Microtronix, http://www.microtronix.com/product_lancelot.html. Accessed 8 Sept 2008
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hachicha, K., Faura, D., Romain, O. et al. Accelerating the multiple reference frames compensation in the H.264 video coder. J Real-Time Image Proc 4, 55–65 (2009). https://doi.org/10.1007/s11554-008-0101-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-008-0101-1